Some professions carry with them much more inherent risk than do others. Office workers rarely need to worry about much more than repetitive motion injuries or eye strain, whereas workers on a construction site or in a manufacturing facility may often find themselves at risk of losing life or limb. The US Bureau of Labor Statistics reports that 4,764 fatal work-related injuries occurred in the United States in 2020. This is in addition to 2.7 million nonfatal workplace injuries that took place in the same year. These numbers are truly concerning, and one cannot help but wonder if there is any way to better protect these workers.
The fact of the matter is that some jobs simply have dangerous aspects to them that cannot be avoided. However, there is often protective gear available that can help to prevent injuries when accidents do happen. Unfortunately, in the course of a normal day’s work, these measures are often forgotten or even intentionally not taken. Manivannan Sivan recently came up with an idea that he believes can increase worker compliance in wearing protective equipment, and in turn prevent injuries and loss of life. He rapidly made his idea a reality by creating a machine learning algorithm with Edge Impulse, and deploying it to an NVIDIA Jetson Nano development kit.
Sivan planned to use these tools to create a machine vision-powered device that can be placed at the entrance of a worksite. As workers enter the worksite, the device would determine if they are wearing all of the required types of protective gear. To make this happen, a webcam was hooked up to the Jetson Nano to capture images of anyone that passes by. The Jetson’s 128 core NVIDIA Maxwell GPU was then put to work running an object detection algorithm built with Edge Impulse. For the initial prototype, this algorithm was trained to recognize the presence of helmets, safety goggles, and reflective jackets. This would allow Sivan’s device to raise an alert when a worker is attempting to begin work without all of the necessary safety precautions in place.
To ensure that inferences run fast enough that all workers can be assessed in real-time, Edge impulse’s ground-breaking FOMO algorithm was selected to handle object detection. FOMO is 30 times faster than MobileNet SSD, and is optimized for use on resource-constrained edge computing devices. With a relatively powerful platform like the Jetson Nano, running inferences at 60 frames per second is easily achievable.
To use FOMO for his purposes, Sivan first needed to train the model to recognize the objects of interest. Towards this end, he collected a dataset consisting of 230 images. These images contained various combinations of helmets, goggles, and safety vests at differing angles and distances from the camera. These images were uploaded to Edge Impulse Studio, then bounding boxes were drawn around the objects to label them. The labeling queue tool has an easy to use interface that helps the user to draw these boxes.
The impulse was then created, with an image preprocessing step that feeds its results into the FOMO object detection model. At this point, the model was trained with the example images that Sivan had collected and uploaded. Validation of the trained model showed that an average accuracy of 83% had been achieved. It was noted that detection of helmets brought the overall accuracy rate down. This was probably due to the fact that the helmets were white, as was the background in most of the training images. Collecting a larger, more diverse dataset would be expected to significantly improve detection of helmets.
The model was performing better than necessary to prove the concept, however, so the model was deployed to the Jetson using the Edge Impulse for Linux CLI. Edge Impulse’s Python SDK was then used to run the object detection model directly on the local hardware. Within minutes the device was ready to start checking for safety gear. Sivan ran some live tests to verify that the system was working end to end, and found it to be performing quite well.
Of course this system can detect more than just helmets, googles, and reflective jackets. One would simply need to collect images representing the objects they wish to detect and retrain the model in Edge Impulse Studio. Sivan has made that easy by making his project public — just clone his project, do a quick photo shoot with your safety gear, then click “Start training” and you are well on your way to improving safety conditions in your workplace (and you probably thought this would be difficult! Psssh). For all the tips and tricks, we also recommend that you take a look at the project’s documentation.
Want to see Edge Impulse in action? Schedule a demo today.