With the meteoric rise of embedded machine learning hardware and algorithms, glass window break detectors are becoming an increasingly common component of complete security systems for both commercial and residential buildings. These devices are designed to detect the sound of breaking glass and alert the owner of the property of a potential intruder.
The newfound popularity of these systems should come as no surprise — according to a recent study, homes with glass break detectors are 30% less likely to be burglarized than homes without them. As an additional layer of protection against burglars that can also help to deter potential intruders, more and more people are considering these devices to be a worthwhile investment.
Glass window break detectors are not without a few flaws, however. False positives, or scenarios in which an alarm is triggered without a window actually having been broken, are too frequent with existing devices. These false positives can be triggered by any number of ambient sounds, like traffic noises or people talking. Even the wind and thunder that accompanies a storm can be enough to confuse many systems. On the flip side, it is also much too common that actual broken glass is not detected at all because the sound is simply not loud enough to be picked up by the device’s sensors.
If you think that these problems sound suspiciously like they could be remedied with better algorithms and sensors, then you are not alone. The engineering team at Zalmotek had the same thought, and decided to put their soldering irons where their mouths are (figuratively speaking, of course!) and design a better glass break detection system. And with the help of a powerful IoT prototyping platform and Edge Impulse Studio, it was quite a bit simpler to build than you might expect.
The Nordic Thingy:53 was selected as the hardware platform. With a high-quality MEMS microphone onboard, the system was already a step ahead of others in its ability to detect faint sounds. And with a 128 MHz Arm Cortex-M33 CPU with 1 MB of Flash memory and 512 KB of RAM (and a second Arm Cortex-M33 CPU running at 64 MHz, to boot!), this device would have no problem running machine learning algorithms that have been highly optimized by Edge Impulse for use on resource-constrained computing platforms.
Zalmotek’s plan involved creating a neural network classifier to learn the distinctive sound patterns associated with breaking glass. This, they thought, should be capable enough to distinguish between background noises like weather, and actual breaking glass, which would eliminate the problem of false positives. And that combined with the high-quality sensor of the Nordic Thingy:53 might just come together to equal a better mousetrap.
But before we get ahead of ourselves, Zalmotek first needed to collect an audio dataset to train the model and understand how well it would perform. Breaking a lot of glass is probably not something that most people want to do. Zalmotek certainly did not, so fortunately, they were able to locate a few preexisting datasets that were publicly available and extract about 15 minutes worth of samples of breaking glass sounds. A background sound dataset was similarly acquired, and consisted of common noises like car horn honks, people talking loudly, and doors closing. The entire dataset was then uploaded to an Edge Impulse Studio project for further analysis.
Within the project, an impulse was created, which defines the flow of data all the way from the sensor to the final result. This impulse started with a preprocessing step that splits audio samples into one second segments, then another step that calculates and extracts the most informative features from the data. This not only produces a more accurate end result, but it also reduces the amount of downstream computational resources that are needed, which is critical when working on constrained hardware platforms. Finally, the features were forwarded into a neural network classifier that was configured to distinguish between the sound of breaking glass and normal background sounds.
The neural network’s hyperparameters looked reasonable to Zalmotek, so they left the default values unchanged and initiated the training process with a button click. After about a minute, training had already finished and metrics were displayed to help understand how well the model was doing its intended job. The average accuracy rate was reported as having reached 99.8% on the first attempt, so it was looking like the idea was working even better than could have been hoped.
Since the performance cannot be improved much, the team pivoted to the next phase of the project, which involved deploying the machine learning analysis pipeline to the Nordic Thingy:53 hardware. This was accomplished with the help of the Nordic nRF Edge Impulse smartphone app. After a quick setup process that links the app to both the Nordic Thingy:53 and an Edge Impulse Studio project, model deployment was as simple as tapping a button on the screen. As a side note, this app is also very useful if you need to collect your own dataset — it can stream device sensor data directly to an Edge Impulse project.
Right out of the box, the deployed model will transmit its predictions wirelessly via Bluetooth Low Energy messages. These can be picked up by the Nordic nRF Edge Impulse app for some quick testing, but they can also be leveraged by external applications to do any number of things, like send an alert to a property owner at the first sign of a broken window, for example.
It is amazing how quickly such an accurate system can be created. Going from idea to finished prototype can be accomplished in an afternoon. And that efficiency has given Zalmotek the time to dream even bigger. Now they are considering expanding the system to leverage additional types of sensors, like cameras. They are also considering expanding the scope of the device to include door opening detection as well as intruder detection.
After reading about Zalmotek’s success with this project, what do you plan to build the next time you find yourself with a free afternoon? If it involves audio classification, you will want to be sure to read up on this project’s documentation first so you can pick up some useful advice. You can even cheat a bit by cloning Zalmotek’s public project to get a headstart. They will not mind — they made it public for just that reason.
Want to see Edge Impulse in action? Schedule a demo today.