You may have heard it said that you should listen to your body for better health, but sometimes that listening is much more literal than you might expect. In the case of chronic obstructive pulmonary disease (COPD), which causes airflow blockage and breathing-related problems, the sound of a cough can tell volumes about the health of a patient. If, for example, the intensity or frequency of coughs increases, it can be an indicator that the present treatment plan is inadequate. Accordingly, it makes good sense to continually monitor the coughs of a COPD patient, not only to relieve immediate symptoms, but to also ensure better long-term outcomes by treating problems early before they grow into much larger concerns.
Unfortunately, the state of the art in tracking patient coughing is shockingly outdated. Commonly, an audio recording device is used to capture a day’s worth of data, then a person laboriously listens to the recording and counts all of the coughs that they hear. Needless to say, this is highly impractical, time consuming, expensive, and unusable outside of a clinical setting. So much for continuous tracking! A better, automated solution is clearly needed. Engineer Eivind Holt knew from his past work in machine learning that there was a tremendous opportunity for improvement, so he built a proof of concept device with the hope of transforming the care of COPD patients. He leveraged the Edge Impulse machine learning development platform to build a classifier that can recognize the unique signature of a coughing sound.
Holt started with an Arduino Nano 33 BLE Sense development board. It is tiny, inexpensive, and powerful enough to run a highly optimized machine learning model generated by Edge Impulse. It has an onboard microphone, so all of the essential components are built into this single board. And with the help of the Arduino community and development tools, getting a prototype up and running quickly on this platform is possible. A few other odds and ends like a LiPo battery and a 3D-printed case were also included to complete the build.
The critical function that this device must be able to perform is detecting coughs, and distinguishing them from any other noises with a high degree of accuracy. Towards this end, Holt first sought out a dataset that could be used to train the model. While public datasets containing coughing sounds do exist, Holt found that they had issues with mislabeled samples and very subtle coughing sounds, so he made the decision to build his own dataset. He recorded about 400 audio samples containing intense coughing, and about 400 more containing various background sounds like speech, sneezing, and other environmental sounds. He personally recorded all of the sounds, so before this device could be generally used, he notes that a larger, more diverse dataset would need to be collected.
The dataset was uploaded to Edge Impulse Studio using the data acquisition tool, which assigned each sample a label automatically, and split the samples into training and test sets. Next, an impulse was created that first preprocessed the audio data by converting it into a spectrogram, which gives a concise snapshot of an audio sample, and also allows the data to be analyzed by convolutional neural networks. The next step in the pipeline was a convolutional neural network that classifies a spectrogram as being either the result of a cough, or random background noise.
Using the simple, point-and-click interface, Holt found it easy to experiment with the neural network architecture and to tweak the model’s hyperparameters. After a few trials, he found a structure that yielded an average classification accuracy rate of over 99%. To be sure this excellent result was not just a symptom of overfitting, the model testing tool was used to validate it against a dataset that was held out of the training process. This showed that the classification accuracy was over 95%, so an excellent result had indeed been achieved.
To protect patient privacy and minimize latency, it was important to run this model on-device, so Holt set about deploying this trained model to the Arduino board. The EON Tuner was used to create a model that was highly optimized for this tiny platform, but still highly accurate. Since the Arduino Nano 33 BLE Sense is fully supported by Edge Impulse, it would be possible to download a custom firmware image containing the full machine learning classification pipeline to flash to the board, but since Holt wanted to customize the logic to send Bluetooth messages based on the inference results, he downloaded the pipeline as an Arduino library, which offers this flexibility.
The Arduino was programmed to transmit a Bluetooth Low Energy packet to a Nordic Semiconductor nRF52840 Dongle when a cough was detected. The dongle then relayed that message over the internet to the Nordic nRF Cloud, which exposes several APIs that allowed Holt to build a dashboard based on the coughing data that, in theory, could be used by healthcare providers to fine-tune a patient’s treatment plan. In a later revision of the device, Holt has plans to explore using LoRaWAN or NB-IoT for data transmission to avoid having a secondary device that relays messages.
In the future, Holt would like to explore using a Syntiant Neural Decision Processor to reduce the energy consumption of the device. It was also found that the cough detector could be triggered by nearby people, rather than just the patient being monitored. It may be possible to address this issue by also leveraging an accelerometer to detect chest movements, and filter cough detections to only those that correspond with movement of the chest.
The entire project has been written up in detail, and it is well worth the read. You can also check out the public Edge Impulse project to see exactly how the machine learning pipeline was configured.
Want to see Edge Impulse in action? Schedule a demo today.