• Luke Hallinan

How to Automate Retail Field Force Management using Mobile Computer Vision Technology

A Truly Scalable Image Recognition Software for Fast Moving Consumer Goods

Three mobile phones with supermarket shelves that have bounding boxes around their products on the screens.
An automated mobile approach to retail FFM requires a robust technological backbone.
 

Reality Check

Automating the task of shelf auditing for Consumer Packaged Goods (CPG) and Fast Moving Consumer Goods (FMCG) across thousands of retail locations around the globe is a monumental task. Despite the best efforts of Field Force Management (FFM), even the most sophisticated mobile, automated solutions for in-store retail execution are limited by one key factor, the availability of high quality product data.


The conventional approach has been to deploy image recognition techniques with traditional Computer Vision (CV). This involves the tedious collection and labelling of an eye-watering mass of real images of each Stock-Keeping Unit (SKU). These images are then used to train the software to detect the products that need to be detected.


The time and cost required to reach a reliable real-world deployment of this technology should be sobering for any retail technologist that is serious about the cost-benefit of automation.


The simple truth is that the most common form of image recognition deployed in retail today is extremely limited by the technology that underlies it.


The good news is that a much better solution to the problem has emerged, one that is completely transforming the shape of CPG merchandising and the effectiveness of Field Force Management in retail.

Supermarket shelves packed with vibrantly coloured products.
The data required to effectively automate in supermarket environments is onerous, time-consuming, and costly.

Virtual Perception

Neurolabs has developed a novel approach to shelf monitoring for supermarkets, the benefits of which are easily transferrable to mobile shelf auditing via FFM.


For the shelf monitoring solution, we use fixed, in-store cameras to collect SKU data in real time. That data is then analysed by our SKU-detection software to empower each store to optimise inventory and boost sales.



A GIF of a supermarket fridge with products being detected in real time.
Unparalleled, real-time SKU-level detection using the most advanced form of Computer Vision on the market.

The true game-changer here is the way in which our SKU-detection software is trained and deployed.


Instead of relying on copious amounts of real data for each SKU, Neurolabs uses Synthetic Data in the form of virtual, 3D versions of each supermarket product. These 3D, digital recreations of each CPG product are used to train CV models in a much more efficient and scalable way. We call this technology Synthetic Computer Vision (SCV) and have been successfully deploying it for retailers in our product “ReShelf” which uses fixed cameras in retail environments.

A GIF of a 3D virtual recreation of a supermarket product used for Synthetic Computer Vision
Synthetic Data enables an image recognition technique that is both flexible yet robust.

The great thing about using Synthetic Data is that it is extremely adaptable and the data from one use case such as shelf monitoring can easily be reused for many different ones.


This is exactly what we have found from our real world deployments in supermarkets and retail stores. The advantages of Synthetic Computer Vision are entirely transferable to a mobile solution for use by Field Force Management in CPG shelf auditing

A synthetic recreation of a supermarket fridge including individual products
Creating virtual recreations of each product makes it simple to transfer automation power to mobile.

Knowledge Transfer

For shelf auditing, we started with the most important element first, the data. In one day we generated complete SKU and scene data for a fixed camera use case.


A comparison of two images taken on a fixed camera, one synthetic and one real, used for product detection in supermarkets.
A synthetic scene with SKU models used to train the SCV model vs its real-world, fixed-camera counterpart.

These datasets were perfect for training our SCV models to detect CPG products.


The ability to manipulate each SKU’s virtual model at will removes the need for collecting countless real images of each SKU in different environments and conditions. Instead, our AI technology enables the procedural generation of endless different variations of each virtual CPG product in order to train the SCV models to account for real world variability at a speed and scale that traditional CV simply cannot match.


We were very happy with the detections for a fixed setup so we turned our attention to a different form of deployment, mobile.


A comparison of two images taken on a mobile device, one synthetic and one real, used for product detection in supermarkets.
A synthetic scene with SKU models used to train the SCV models for mobile vs its real-world counterpart.

One of the most striking benefits of using Synthetic Data for Computer Vision is the ease of which the effort spent on one use case transfers to another. After the initial one day it took to generate Synthetic Data for fixed camera shelf auditing, we were pleasantly surprised that it only took three hours to develop a CPG product detector on a mobile device.


Taking the conventional approach to Computer Vision would have made this timeline impossible. With Synthetic Data, we were able to execute and deploy at speed, testing and validating our assumptions of the mobile shelf auditing use case in a truly cost-efficient and scalable way.


As a result, our new mobile product detection software, ReShelf Go, which was purpose built with Field Force Management in mind, came to be.


Using ReShelf Go as your SKU recognition software of choice for your FFM mobile app empowers your field agents to get the job done more effectively and provides your CPG clients with trustworthy, real-time information to compute their most important retail execution KPIs.

An image taken on a mobile device of a supermarket fridge. There are bounding boxes around each product that is being detected.
Real-time shelf insights in the palm of your hand.

Handheld Hero

Using SCV we have achieved 96% accuracy for SKU-level product recognition as a baseline from day 1. Accuracy continues to improve as the SCV model carries out more detections. This level of precision, combined with the adaptability of the training data, is crucial for a successful automated shelf auditing deployment given the ever-changing nature of the retail environment and the inconsistency that comes from images captured on mobile devices.


For FFM companies, a superior approach to image recognition, using a mobile device for data capture, can be tested in less than a week. This reduces the cost and risk associated with switching to a new technology as you can rapidly verify its effectiveness before considering implementing the software into your native mobile application.


All of our datasets are available in the Neurolabs platform and deploying one of our custom-trained Synthetic Computer Vision models is as simple as making a call to our API endpoint.

A screenshot of the Neurolabs platform and its catalog of virtual supermarket products used for Synthetic Computer Vision.
Any CPG product can be created in virtual reality to take SKU recognition to the next level.

Any CPG product can be created in virtual reality to take SKU recognition to the next level.


No longer held back by the limitations of traditional image recognition, using Synthetic Computer Vision, you can achieve unprecedented benefits such as:

  1. Speed: A real-world deployment can be implemented in less than one week.

  2. Scale: Access to image recognition datasets for over 100,000 SKUs through the ReShelf platform.

  3. Quality: Achieve 96% accuracy for SKU-level product recognition from day 1.

With such vast improvements to your image recognition workflows, why not try it for yourself?


Three mobiles devices each with an image of a supermarket fridges with bounding boxes around each product.
Transferring product detection capabilities from fixed cameras to mobile is a game-changer for FFM
 

Written by Luke Hallinan, Product Marketing Manager at Neurolabs. Retailers worldwide lose a mind-blowing $634 Billion annually due to the cost of poor inventory management with 5% of all sales lost due to Out-Of-Stocks alone. 🤯 Neurolabs helps optimise in-store retail execution for FFM companies and CPG brands using a powerful combination of computer vision and synthetic data, improving customer experience and increasing revenue. 🤖 🛒