Inventory management and quality control may not be the most exciting topics in the world of manufacturing, but they are fundamental to the success of any production operation. Properly managing inventory ensures that materials and products are available when needed, avoiding costly delays and disruptions in the production line. Similarly, strict quality control helps maintain high standards and consistency, preventing defects and customer dissatisfaction. Together, these practices contribute to efficient, cost-effective manufacturing processes and improved customer satisfaction, which are crucial for long-term success and to gain a competitive advantage.
At their core, inventory management and quality control depend a lot on keeping accurate counts — counts of raw materials sitting on shelves, items whizzing by on an assembly line, and completed products in a warehouse. This may seem simple enough, but considering the speed and scale of a large manufacturing operation, it quickly becomes very complicated to keep accurate counts of items at various stages of production. As such, all but the smallest of organizations now rely on some type of automated counting system.
With speed becoming such a crucial factor in the industry today, these systems are tuned to work fast. As such, low-resolution sensors, like color and proximity sensors, are often incorporated into automated counters. Since the data is very coarse, it can be processed very rapidly. However, that speed comes at a price — they generally cannot distinguish between small objects that are close to one another, or differentiate between multiple types of objects. On a fast-moving assembly line, that can be a big problem.
Speed and resolution
Engineer Jallson Suryo, a member of the Edge Impulse Expert Network, recently built a proof of concept automated counting device that offers the best of both worlds — speed and high-resolution sensing. By using a highly-optimized object detection algorithm developed with Edge Impulse and a low-cost edge computing platform, Suryo demonstrated that it is possible to get an accurate count of even tiny, closely-spaced objects as they zip by on a conveyor belt. While this work is still in its early stages, it is easy to imagine a future version of it enhancing existing manufacturing processes.
The hardware is centered around an NVIDIA Jetson Nano Developer Kit. With a 128-core NVIDIA Maxwell architecture GPU, this inexpensive single board computer is capable of performing 472 gigaflops to accelerate AI inferences at the edge. Suryo paired this with a Logitech C922 USB webcam for capturing images, and an optional small LCD display to provide information about the objects that have been counted by the system. The webcam was positioned above a small conveyor belt for development and testing purposes.
Before building the machine learning pipeline, Suryo needed to collect some data for it to learn from. For this purpose, he captured a number of images of bolts with the device’s webcam. They were distributed randomly along the conveyor belt, in differing orientations and quantities, to assist the model in generalizing to a wide range of conditions. The images were then uploaded to an Edge Impulse project.
Object detection algorithms also require that the images be annotated by drawing bounding boxes around the objects of interest and assigning them labels. Fortunately, Edge Impulse has automated tools to speed up this process. For Developer accounts, the Labeling Queue tool will attempt to draw a box around each object, which must be manually confirmed (and potentially adjusted) for each image. (For Enterprise accounts, on the other hand, the Auto-Labeler tool completely automates the process with a more advanced algorithm.)
Model design in Edge Impulse
After running the Auto-Labeler, Suryo was ready to design the impulse, which defines how the images are processed from the time they are captured until an object count is produced. In this case, the images are first resized to 720 x 720 pixels. Typically, images would be further reduced in size to decrease the computational workload, but given the need to detect small objects, and the power of the Jetson Nano, that was unnecessary in this scenario. The images were then forwarded into Edge Impulse’s ground-breaking FOMO object detection model. FOMO is 30 times faster than MobileNet SSD, so it is ideal for this application where speed is critical.
Suryo tweaked a few hyperparameters, then intiated the model training process with the click of a button. Upon completion, metrics were displayed to help assess the model’s performance. An accuracy score of better than 96 percent was observed, so things were certainly looking good. It is always a good idea to do another check using data that was not included in the training process, however. That can be accomplished using the Model Testing tool, which Suryo ran to reveal an accuracy score of 100 percent.
Those results did not leave much room for improvement, so the impulse was deployed to the Jetson Nano hardware. The Deployment tool has a TensorRT library option, which is designed to make use of the powerful GPU available on the Jetson. After downloading this library, there are just a few commands that need to be executed to get the model up and running on the hardware, which Suryo details in the project write-up.
With inference times of approximately 90 milliseconds, Suryo’s automated counter definitely qualifies as a high speed system.
Keeping count
To wrap up the project, Suryo leveraged the Edge Impulse Python SDK to build an application that keeps a running count of all of the bolts detected by FOMO as they pass by on the conveyor belt. At present, the results are shown on an LCD display for demonstration purposes, but of course they could just as easily be fed into any other system.
With inference times of approximately 90 milliseconds, Suryo’s automated counter definitely qualifies as a high speed system. If it were hooked into an organization’s inventory management software, it could potentially be used to upgrade existing processes without much of a disruption.
Maybe bolts are not your thing, but do you have something that you need to get a better handle on with an automated counting system? You can get started by cloning the public Edge Impulse project, collecting a new dataset, and retraining the FOMO model. If you need a few tips to help you along the way, be sure to check out the project write-up.