Passionate about prototyping? You’re not alone. Prototyping is crucial to minimise risk and validate your product or feature idea before fully committing. After all, who wants to bet the bank on a solution that doesn't solve a problem? But, prototyping techniques often fall short when AI, Machine Learning (ML) & a bunch of data come into the picture. Fear not, this blog post is here to prove that it is possible. So let’s dive into 5 key steps to effectively prototype and test your machine learning solution.

Step 1: Data solution sketching‍

While data solution sketching may not be the newest concept, it remains a crucial step in ensuring that challenges are well-defined and solutions can be validated. However, for features that involve machine learning, you need to shift your focus to the data side of things. 

This involves identifying your essential data sources, determining techniques to transform data into insights, and understanding how stakeholders or users will use the insights to take action and create value. 

Our preferred approach is to capture the full scope of your next feature in a Data Value Canvas. A dead easy way that allows you to quickly verify you’re creating something valuable in a matter of hours.

‍
An example of a filled-out Data Value Canvas, trying to understand the scope of an automated voicemail transcription.
‍

Step 2: The (false) clickable prototype

To create a successful ML model, it’s key to make the process concrete by considering the user’s data inputs, automatic processing and desired accuracy. And now we add some trickery to the mix. You don’t need a working ML system to validate these questions. Instead, we use an interactive prototype to show test users mocked results, which they believe to be real ML predictions. 

This method also allows you to play around with the quality of the output and investigate happy flows, and sad flows, mimic edge cases or test out design variations. By focusing on the outcome over the model accuracy, you’ll much better reflect real-life performance and it will help you overcome the first technical limitations you might encounter.

The clickable prototype in our voicemail example revealed how users handle transcription errors, report issues, and validate results.

Step 3: Technical Proof of Concept

Whereas steps 1 and 2 answered questions regarding desirability, business viability and usability, step 3 is all about technical feasibility. This is where a proof of concept comes in to tackle the biggest potential technical risks and uncover all the nitty-gritty implementation details. 

The type of risks you’ll uncover will depend a lot on the project and the previous experience of your team. Nevertheless, we can distinguish 3 main categories:

• Data: do you have the data? All data sources should be available, in the right format and labelled correctly.
• Model performance: Do you reach acceptable performance from your ML models? Pre-define your acceptance criteria! 
• Quality atributes: Does the implementation fit into the eventual architecture? Does it adhere to scaling, monitoring and legal requirements? 

This step is where you get down and dirty with the tech to make sure your product can really take off.

In our voicemail use case, it was crucial to validate that the Whisper model could run in the cloud and test response times. Acceptance criteria required the ability to call the API with an audio payload and receive a sentence-by-sentence transcription breakdown.

Step 4: In Vitro testing

Step 4 of your prototyping journey is where things get real. This is where you test your technical solution with actual users. The key here is to deploy a simple & effective interactive prototype and get user validation as soon as possible. 

The best way to create interactive prototypes quickly is by using low-code tools like Streamlit or Gradio. Deploying these prototypes is a breeze using cloud solutions like Vertex AI or SageMaker. Once your prototype is live, it's time to gather as much user feedback as you can. This will help you fine-tune your product to make sure it's a hit with users.

For our use case of transcribing voice mail, we created a simple Streamlit web app to test the transcription quality.

Step 5: Real-life testing

At this point, you’re past prototyping and it’s time to get ready and integrate your ML solution into the real world. This is where things get expensive, as you'll need the whole gang involved - product manager, ML engineer, cloud developers, front-end developers, you name it. But, if you thoroughly followed the previous steps, no major surprises should pop up - saving you tons of money. 

Of course, it doesn’t end there. Once your ML product is live, you want to keep testing on a large scale by tracking user behaviour, analysing usage logs, and doing some AB testing to keep improving your product incrementally. From there, your product will only grow for the better. 

‍Never not prototype your ML solution

So what’s the takeaway here? In conclusion, prototyping for ML features turns out essential for any digital product. You can validate the feasibility, value and usability of your ML solution before investing in expensive development. You identify and solve problems early on, avoiding costly mistakes down the road. And it’s perfectly doable if you know what to do and where to look at. So, in essence, prototyping is a must-do for anyone seeking to deliver a successful digital product.