The inexpensive production of high-quality steel through the Bessemer process is the key invention that made skyscrapers possible, but without elevators to move between their many floors, high-rise buildings would be impractical. The nearly one million elevators in the United States serve to maintain a swift flow of people and goods through multi-floor buildings, whether it is just a few floors or many dozen. With so many elevators in constant service, keeping them properly maintained for safe operation is a big challenge. According to the U.S. Bureau of Labor Statistics, elevators kill 30 — and seriously injure about 17,000 — people each year. In addition to these serious occurrences, elevators that go out of service due to maintenance issues slow commerce and impact those with disabilities and the elderly.
Engineer Ivan Arakistain thought that if elevator maintenance issues could be discovered before problems actually occurred, it would present an opportunity to save lives, time, and money. Towards that end, he built a tiny, low-cost device with Edge Impulse and a microcontroller development kit that is well suited to large-scale deployments of the sort that would be needed to monitor elevators around the world. His device uses machine learning to recognize abnormal sounds made during operation of an elevator, then report those anomalies wirelessly so that technicians can be dispatched.
Arakistain chose the low-power Seeed Studio XIAO BLE Sense with a Nordic Semiconductor nRF52840 ARM Cortex-M4 CPU running at 64 MHz and with 256 kB of RAM. With an onboard microphone, no additional sensors needed to be included in the design. The 5 μA ultra-low power sleep mode of the CPU allows the board to keep running for extended periods of service in the field. To keep a network of devices in contact, a Blues Wireless Notecard was connected to the XIAO BLE Sense to transfer data to a cloud-based dashboard via cellular networks.
To recognize the different sounds that an elevator makes, a neural network classification pipeline was constructed with Edge Impulse Studio. As it turns out, abnormal sounds made by elevators in a state of disrepair are not so easy to come by for the average person, so Arakistain instead tracked down some audio datasets of more common sounds (dog barking, glass breaking, gunshots, background noise) to prove the concept. These audio samples were uploaded to Edge Impulse using the data acquisition tool.
With the data in place, it was time to square away the rest of the processing pipeline. A Mel-frequency cepstral coefficient (MFCC) preprocessing step was selected to extract the most relevant features from the audio and reduce the computation complexity of subsequent processing steps. Next, a neural network classifier was added and tuned to the task at hand. The model was then trained and evaluated. Using a test dataset, it was found that the model had achieved a classification accuracy rate of nearly 80%.
The entire processing pipeline was exported as an Arduino library, after which it was a simple process to deploy it to the XIAO BLE Sense. Edge Impulse produced an optimized model that fits well within the constraints of the XIAO, requiring only 5 KB of RAM and 35 KB of flash memory. Impressively, inference times are only about 5 milliseconds when running on the target hardware. To complete the package, a simple case was 3D-printed to house the hardware and provide a bracket to mount the device to an elevator.
Arakistain envisions a network of these devices monitoring geographically dispersed elevators and reporting potential issues to a centralized Tableau dashboard to aid preventative maintenance efforts. As a hint of things to come, Arakistain also experimented with voice command recognition (e.g. “Up”, “Down”, “Floor 4”) which could eventually allow us to directly tell the elevator where we want to go, rather than asking the person standing nearest to the control panel to press the button.
Get the information you need to build your own audio classifier with some tips from the project page.
Want to see Edge Impulse in action? Schedule a demo today.