For many people, what goes on underneath the hood of their car is a mystery far greater than anything that happens in the Bermuda Triangle, or at a clandestine hangar at Area 51. This means that recognizing a problem early before it becomes serious can be a challenge, and even if a problem is noticed, it is difficult to communicate that problem clearly to a mechanic so that they can troubleshoot the correct system. After all, one person’s “click-whine-click” is another person’s “clang-thud-clang,” and that can make all the difference in locating the source of the problem.
Frustrated by this present difficulty in monitoring and diagnosing the failure of mechanical components in vehicles, machine learning enthusiast Manivannan S. devised a plan to improve the situation. He used Edge Impulse and a tiny microcontroller to keep watch over the sounds that a car is making, and provide warnings when something appears to be a bit off kilter.
Manivannan chose to build a device around the diminutive yet powerful Silicon Labs Thunderboard Sense 2 development board. The board features an Arm Cortex-M4 core processor with 256 kB RAM and 1024 kB flash memory. It also comes with a slew of built-in sensors, including an inertial measurement unit, light sensor, and a digital microphone. The Thunderboard also has transceivers for Bluetooth communication and cloud connectivity. With everything Manivannan needed already on-board, no additional hardware was needed.
Since the goal was to detect mechanical problems from the sounds that they make, Manivannan needed examples of these sounds in order to train a machine learning model. Fortunately, he was able to work with a mechanic that could record audio samples from cars exhibiting certain problems as they came into the shop. For the initial prototype, focus was placed on detecting three different conditions — brake pad noise, a faulty drive shaft, and normal operation. The Thunderboard is compatible with Edge Impulse, so it was possible to link the board to an existing account, and have all collected data automatically uploaded to the Data Acquisition tool within Edge Impulse.
Designing an impulse to preprocess this data and classify it with a neural network took just a few clicks. With a full data analysis pipeline having been created, the model was trained with audio samples from the previously collected training dataset. The classification accuracy was found to be excellent, however, if it had needed some refinement, it is possible to tweak a number of hyperparameters within Edge Impulse to improve the results.
With the concept fully proven, it was time to deploy the model to the Thunderboard so that model inferences can happen directly on that hardware while the monitored car is on the road. The necessary files can be downloaded from the Deployment tab in Edge Impulse, after which they can be drag-and-dropped onto the Thunderboard’s storage.
This project is a great step towards making vehicle predictive maintenance both more widely available, and more friendly. In the future it would be possible to add examples of many more abnormal sounds to the model to expand its utility in a wider range of situations. Be sure to check out Manivannan’s full project write-up for all the details.
Want to see Edge Impulse in action? Schedule a demo today.