Obstructive sleep apnea (OSA) is a sleep disorder characterized by repetitive pauses in breathing during sleep, caused by a partial or complete obstruction of the upper airway. The obstruction often occurs when the soft tissues at the back of the throat collapse and block the airway, leading to decreased airflow or even temporary cessation of breathing. These breathing disruptions can last for a few seconds to minutes and may happen multiple times throughout the night, disrupting the normal sleep pattern and preventing the person from reaching restorative deep sleep stages.
OSA is a relatively common condition, affecting people of all ages, but it is more prevalent among adults. It is estimated that approximately 2% to 9% of adults suffer from moderate to severe OSA, with the risk increasing with age. Men are generally more prone to developing OSA than women, although the risk for women increases if they are overweight or obese.
The health concerns associated with untreated OSA can be significant. During the frequent disruptions in breathing, oxygen levels in the blood drop, leading to an increased strain on the cardiovascular system. As a result, individuals with OSA have a higher risk of developing hypertension, heart disease, strokes, and other cardiovascular issues. The disrupted sleep patterns can also cause excessive daytime sleepiness, fatigue, and impaired cognitive function, affecting daily activities, work performance, and even posing a risk of accidents, such as drowsy driving.
Fortunately, this condition is highly treatable. Lifestyle changes like weight loss, quitting smoking, and reducing alcohol consumption can help in mild cases. Continuous Positive Airway Pressure therapy is the most common and effective treatment for moderate to severe OSA. But in order to receive treatment, a person has to be aware that they have the condition. The gold standard for diagnosing OSA is a sleep study. Unfortunately, sleep studies can be both expensive and inconvenient, not to mention that they will not even be recommended unless there is already a very strong suspicion that the condition exists.
Better, more practical diagnostic tools are sorely needed in this area. And they may be on the way — a device recently created by engineer Naveen Kumar may prove to be a blueprint for that next generation of tools. Kumar’s solution leverages inexpensive, off-the-shelf hardware and an audio classification pipeline built with Edge Impulse Studio to listen for the sound of snoring during sleep. If detected, the device will gently wake the snorer with a haptic feedback mechanism to alert them of their condition.
This new system is powered by an Arduino Nicla Voice, with a Syntiant NDP120 Neural Decision Processor and an Arm Cortex-M4 microcontroller to make short work of running machine learning algorithms on-device, while requiring a minimal amount of energy for operation. The Nicla Voice also comes standard with a high performance microphone that is ideal for capturing sound during sleep. A haptic motor controller and an ERM vibration motor round out the bill of materials for this project.
To prepare the hardware for use, Kumar installed both the Arduino CLI and the Edge Impulse CLI. Finally, the Nicla Voice was updated with a custom firmware image provided by Edge Impulse. These steps make it simple to deploy models to the device, and also maximize the use of the available hardware resources, like the NDP120.
With the hardware ready to go, the next step was to acquire a dataset to train the audio classification model. A publicly available dataset was located, which contained the sound of snoring, and also normal background sounds that are frequently encountered at night while sleeping, like rain, ticking clocks, and crying babies. The Edge Impulse Uploader was then used to upload these audio samples to an Edge Impulse Studio project, and automatically apply labels to them via a command line option.
Kumar then got to work building the impulse, which defines the flow of data from raw sensor inputs to predictions from the machine learning algorithm. The first steps of the impulse slice the incoming audio into short segments, then compute log Mel-filterbank energy features which serve to highlight the most informative aspects of the data. These features are then forwarded into a neural network classifier that is optimized to run on the NDP120 neural processing unit. This model is configured to predict whether or not the sound of snoring was recognized in each sample.
The architecture of the 2D convolutional neural network was adjusted, then a few model hyperparameters were tweaked before the training process was initiated. After a short time, the training was complete and a dashboard was presented to help assess how well the model was performing. The classification accuracy reached 99.4% right off the bat, and the Data Explorer tool showed excellent separation between classes.
Everything appeared to be going according to plan, but it is always important to run another more stringent test that uses data that was not included in the training process. This can be achieved using the Model Testing tool. In this case, the results showed an average classification accuracy of 94.62%, which indicates that the model was in fact performing quite well and showing a good degree of generalization to different environments.
Not too many people would be thrilled with the idea of sending a constant stream of audio from their bedroom to the cloud, so next Kumar deployed the model to the Nicla Voice such that the entire classification pipeline could run entirely on-device. This was accomplished by downloading an NDP120 library from the Deployment tool, then running a few commands with the CLIs that were previously installed.
But before flashing the new firmware to the board, Kumar made a few modifications to it. This enabled him to set thresholds for what would be considered snoring (e.g. three detections within the previous ten audio samples), then trigger the haptic feedback device as appropriate. When finished, the device was installed on a pillow where it could collect clear audio samples, and also provide haptic feedback to the user.
This simple yet powerful creation is a great demonstration of the potential of machine learning-powered devices to replace expensive and cumbersome tools with inexpensive, transparent ones that can have a far greater impact. The project write-up and Edge Impulse Studio project are available to help you get your own ideas off the ground in no time. What problems are you going to solve?
Want to see Edge Impulse in action? Schedule a demo today.