This is a story about drifting away from people and their problems. It’s about gaining distance from facts and phenomena that you’d normally die to understand and be close to. I’d like to invite you to follow me on this little bit of my machine learning journey, not only to submerge in my personal retro- and prospectives but also with my deepest hope to produce something for you to take away from my musings.

Artificial intelligence to solve non-artificial problems

AI startups make up a grand part of all new ventures in deep tech space. I myself, for a long time wanted to build a product that solves a real world problem. A problem of the important ones-kind. One that is worth solving from an essentially human point of view. However, I’ve never thought it had to be something that had to do with machine learning.

The problem appeared before me in the first weeks of the pandemic. Like many others, I find it hard and at times annoying, to choose sizes of apparel when I’m shopping online. Looking at pictures of models wearing different sizes. Being the skinny-fat legend I am, I knew I had to make it fit right. Choosing the right pair of jeans, in the right size turned out to be nearly impossible. There were a lot of different styles I liked but finally, I turned to my go-to-brand and ordered another identical item to what I already had in my closet. The only difference came with wash and colour.

Machine Learning hides a solution to any problem

Conventional machine learning has answers to all kinds everyday problems. Some of the situations we’re trying to deal with where we think about employing ML can’t however be found on lists typical ML tasks and state-of-the-art models built to cope with them.

Sometimes you have to turn to unconventional mental models to deal with a specific issue. At the very beginning of our journey, we had to realise a most suitable solution to the problem posed. And we didn’t care if it’s going to be a machine learning model.

First, there were facts. Facts about our users and the reality that we so badly wanted to mend. We knew that people would like to feel more confident when making apparel purchases online. We knew that people aren’t good at measuring themselves - because it’s hard to do it correctly - but also, it’s unpleasant for a significant group of us, online fashion shoppers. Another fact we very quickly uncovered was that almost all of us, our friends and family members already had some clothes they considered their favourites. Those two truths were the main vectors in search for an adequate solution proposal.

Image credit Sindre Strøm

Validating Machine Learning approach early

As a startup co-founder, you have to think agile. It sounds cliche but actually have to prove that your product opportunity is real and your concept on how you’re going to solve the problem for your customers is sound and feasible. In small steps. Along the way there are plenty of decisions you’ll have to make and switching direction is something you should be outmost used to.

With our truths in consideration, we started sketching on a solution design. Fashion is a very visual matter - that one was clear to us. From the users’ perspective, is there a better way to tell a solution about their favourite clothes than to visually show them?

Our design was simple. The users would take a picture of their favourite apparel. Using augmented reality functionality supported by most of the smartphones we own, we would measure the objects - the clothes and extract some other basic information about them. All from just one image. Then, we’d compare the users’ favourites with apparel available for purchase and recommend the ones that would make the best fit.

Before we jumped into coding of a sophisticated deep learning computer vision net, we decided that we needed to confirm two things:

• that the solution is actually capable of producing reliable size recommendations

• and that this kind of interaction is intuitive enough for the users

Image credit Noelle Otto

Show that your prediction could be valuable

As far as the PoC goes, the most we should have accomplished to showcase our solution concept was - in our case - to have three major components clickable:

• mobile app that takes images with accurate frame size in cm and sends the them to the brain of our system

• something that processes the images and extract valuable features of the objects captured

• comparer that takes one ground truth and tries to find another fitting item amongst available size variants

Instead of going for hefty neural networks, we opted to give traditional computer vision algorithms a try. We went for GrabCut and watershed to heuristically segment the objects out. We then multiplied the number of pixels between primitively extracted key points and started comparing observations.

That has enabled us to have a fidel resemblance of what the product would look like very early on. The next thing was to ensure that we have got proper object measurements. Using some basic ARKit tricks has given us somewhat accurate physical dimensions of the objects segmented.

Lastly, we needed to have a program that would run this all - handle communication between the very potatoey mobile app and the brain. And guess what. That turned out to be the greatest challange of our little PoC.

The machine learning model deployment 😱

What about Heroku? Heroku is cool but using it would most surely imply that we’d have to start form scratch once we’ve validated that our PoC makes sense. Dedicated MLOps platforms would on the other hand provide extensible possibilities but we’d have to spend time on getting knowledge of that blossoming area and even consider hiring a person who’d be dedicated to DevOps and MLOps stuff. At that time we haven’t even thought about paying ourself any salaries, let alone having somebody else on our payroll.

And then there was our very specific problems. After a look around at state-of-the-art computer vision nets it was clear that there wasn’t anything out there that we could just take out of the box and place in the middle of our product. That implied that we can’t depend on the data science puzzle bits offered by AWS, GCP or Azure.

We had to build something entirely proprietary and find an effective way to bring it out to our customers. And we wanted our customers to see our product at all stages of its development so we’d never have to worry if there’s any valuable feedback we could be missing.

Spend time on wrong things and you’ll find yourself misaligned

When I’m reading about the Machine Learning lifecycle, I always find the conceptual drift to be the most fearsome turn of events. The risk of conceptual drift amplifies when you focus on other things than making your model serve a prediction that provides valuable information to the recipient.

Forgive me, product management god, for I have sinned. Our PoC was a hyper primitive showcase of what the produced information for customers might look like. All we focused on at that time is to present the answers we generate in the most interpretable, suitable and usable way. We secured some funding and then suddenly, we wanted to start to make serious software development. What it really meant was that we lost our focus. We no longer cared the most about the utility of our predictions and the UX. We started caring about all the best practices observed in bigger organizations with significantly bigger wallets than ours.

a typical, confusing black box. Image credit Abode of Chaos

Don’t fall in love with the MLOps, DevOps and other -Opses, until it’s time to

We went full DevOps, we went full MLOps. It sounds a bit overkill now but back at that time, we thought that publishing model artefacts to a Nexus repo was a great idea for a startup of five people. We believed that without scalability, we had nothing to show. All this at a time when we failed to be able to show our predictions to users frequently and gather their feedback and observe how they get them to gain insights and make our product aligned with their problems and their everyday lives.

We have spent over a month on building the finest machine learning pipelines. We took pride in how well our Kubernetes cluster is configured and how effectively our nodes scale. The only problem is that during all that time, we should be devoted to something else.

The awakening came drastically. One time, a close friend insisted on looking at “what we’ve got” since “we’ve been working on it for some time” and haven’t shown anything to people. I’ve let her go through the process of inquiring on an apparel size. She didn’t understand what’s going on. It wasn’t only the UX of the app. She simply couldn’t grasp where the information we present to her comes from and how her actions have contributed to this exact prediction. I panicked. We’ve booked another couple of user test sessions and the results were the same - people didn’t have any confidence in our predictions because we were showing them something that was strange to the indications they’d normally get in fashion. Neglecting keeping aligned to our customers, their customs and problems could have costed us our grand product opportunity.

Summary of our business, machine learning and MLops journey

• Keep yourself aligned with your problem space when developing machine learning models

• Talk to your customers often. Try to show them your solution frequently. Watch them use it, react to it and listen to their feedback

• Deploy your machine learning models even more often and test them with your customers frequently

• Try to find a balance between not accumulating too much technical debt and deploying your machine learning models often

• Don’t fall in love with the idea of building sophisticated software business - that might be the next step once you’ve confirmed that your solution solves a problem

• Find solutions that allow you to prototype so that you can conceptually improve your ML model and how the predictions are served to the users

• Don't introduce the Ops part (MLOps, DevOps etc.) too quickly in your business - that might be an overkill

• Make your artificial intelligence as interpretable and human as possible. People won’t feel comfortable using a black box to get valuable information