A trip to the store is increasingly being replaced by a visit to an e-commerce retailer for many consumers. The convenience of e-commerce has turned over a quarter of the world’s population into at least occasional online shoppers. This trend has been a huge boon to online retailers, but it has also brought with it many logistical issues that have not yet been fully solved. Front and center among these issues are the problems related to warehousing and shipping all of the products in inventory. This can be especially challenging when dealing with fragile items. As you have almost certainly noticed, overworked warehouse staff and delivery drivers do not always give every package the white-glove treatment.
It has been estimated that somewhere in the neighborhood of 10 percent of all packages are in some way damaged by the major couriers. Replacing a damaged product can cost an e-commerce retailer up to 17 times the original shipping cost. Ouch! And it is not just shipping incidents that cause costs to pile up — inappropriate handling and storage of products in the warehouse is also a source of significant financial losses for retailers. Proving that there might be a relatively simple solution to this costly problem, engineer Manivannan Sivan has built a prototype device that monitors packages to ensure that they are handled and stored safely.
Sivan’s device is attached to a fragile package where it runs a machine learning algorithm developed with Edge Impulse Studio that can determine its three-dimensional positioning. If that position information is not consistent with safe handling or storage of the item, it wirelessly sends an alert message that allows action to be taken before losses are incurred. The tiny yet powerful Silicon Labs Thunderboard Sense 2 Advanced IoT Kit was selected as the hardware platform for this project. This board is very inexpensive and energy efficient, yet it also comes standard with an Arm Cortex-M4 MCU with 256 KB of RAM and 1024 KB of flash memory that is perfect for running inferences against a highly-optimized Edge Impulse machine learning pipeline. An integrated Bluetooth Low Energy radio provides the needed wireless connectivity and makes this development kit an all-in-one solution.
To determine the position of a package, the accelerometer on the Thunderboard Sense 2 was leveraged to collect measurements of motion. Based on past experience, Sivan knew that a neural network classifier should be able to distinguish between the measurements when the package is in different positions to determine its orientation. But to teach that neural network what each orientation looked like, sample data would first need to be supplied. So, Sivan got to work by flashing Edge Impulse firmware to the hardware and linking it to his Edge Impulse project. From that point on, he could capture accelerometer measurements at will and the data would automatically be uploaded to Edge Impulse.
After shaking and tilting a package equipped with his device for about two and a half minutes in each of four different orientations (correct, upside down, tilted forward, tilted backward), there was enough information for the model to learn from. An impulse was then created to process this data, beginning with steps to split the measurements into three second segments and then extract the most significant features from the dataset. These features were then forwarded into a neural network classifier. After a little fine-tuning of the model’s hyperparameters, the training process was started with the click of a button.
After 200 training cycles had completed, the training statistics were presented which showed an accuracy rate of 100% had been achieved. Of course this is an excellent result, but to confirm that the model did not overfit to the data, Sivan ran another test on a dataset that was not included in the training process. This test also showed that an accuracy rate of 100% had been achieved. As a final validation that the model was in fact performing as expected, the Feature Explorer tool was checked. This showed very good, consistent separation between samples from each class. Taken in this light, it is not concerning that the model accuracy looks almost too good to be true — the signal is very clear and strong, so 100% accuracy is believable.
With no further room for improvement of the model, it was deployed to the hardware. Since the Thunderboard Sense 2 is fully supported by Edge Impulse, it was as simple as downloading a custom firmware image containing the entire machine learning classification pipeline and flashing it to the board. At that point, the device was ready for real world testing. Sivan installed an app on his phone to enable him to see the Bluetooth messages generated by the device, then put the instrumented package into various orientations. With each change in position, it was found that the model reported accurate results.
Sivan’s prototype was another example of how tiny algorithms and tiny hardware can come together to solve big problems. These techniques are simple to implement and do not require a high level of technical knowledge. You can get up to speed quickly by reading the project documentation, then you will be well on your way to bringing your own ideas to life.
Want to see Edge Impulse in action? Schedule a demo today.