.svg)
Problem: Hyfe, a respiratory wellness company that tracks and monitors coughs for actionable insights, needed a solution for streamlining their ML models for fully on-device processing.
Solution: Hyfe imported the world’s largest cough dataset into Edge Impulse. Within hours, Hyfe was able to set up a pipeline in the Edge Impulse platform to extract features and generate a cough detection ML model capable of fitting on an Arm® Cortex® M33.
Result: With Edge Impulse, Hyfe was able to accelerate delivery of their ML models from upwards of two years to 2–3 months. Those models subsequently extended battery runtime in Hyfe test devices from seven hours for initial Hyfe-created models to eighty hours for Edge Impulse-optimized models. When the model needs to be updated, Hyfe can iterate through models quickly by changing the dataset, processing block, or learning block. The target device can also be changed with a click of a button so that the same ML pipeline flow can be deployed to a target other than the Arm Cortex M33.
We’ve all been there: What starts as a tickle in the throat quickly becomes a concerning coughing spell. Someone remarks “you should get that looked at,” so you meet with your doctor, who asks various questions, then peers into your mouth with a tongue depressor and listens to your lungs with an icy stethoscope. Medicine may be prescribed, along with the advice to get some rest.
This has been how so many of our healthcare interactions have transpired, with cough being a symptom that something is wrong, but not an indicator of what, specifically, that might be. The founders of Hyfe.ai, however, felt that there could be valuable and actionable data stored in the sound of our coughs, from the time of occurrence, to frequency and pattern, to the actual audio signal of the cough itself, so they set out to create a system that could listen for, log, and analyze our coughs, unlocking insights that could help address illness more directly.
And they’ve enlisted Edge Impulse to help them with the machine learning models that they use to do this.
Hyfe CEO Joe Brew and Chief Medical Officer Peter Small both come from a background in global health, where tuberculosis (TB) — a disease often marked by persistent coughing — was their primary focus. Their experience revealed a significant gap: While cough is a crucial symptom for diagnosing and managing TB, its quantitative analysis was largely neglected. This realization led them to explore the potential of harnessing cough data in healthcare.
With the advent of the COVID-19 pandemic in 2020, the team at Hyfe recognized an opportunity to expand their focus. The initial lockdowns and the absence of diagnostic tests for COVID-19 sparked an idea: Could cough patterns serve as a proxy for tracking the spread of the virus? This initiative led to a breakthrough with another group — chronic coughers.
People suffering from refractory chronic cough, along with conditions like asthma, chronic obstructive pulmonary disease, and allergies, found the Hyfe application provided something previously unavailable — a way to quantify their coughs. The Hyfe app quickly became a resource for those sufferers, highlighting the importance of cough as a data point in managing and understanding various respiratory conditions. Hyfe quickly carved out its identity and market, and the company evolved its mission to address respiratory health challenges.
.avif)
Central to Hyfe's product offerings are two innovative solutions. First is their Software Development Kit (SDK), designed primarily for cough detection. This SDK is versatile, finding its application in a range of products from their own wearables and apps to integration with third-party devices. The second pivotal product is their Insights API, which delves deeper into the data collected by the SDK. This API interprets cough data to provide valuable insights, aiding in disease screening, triage, prediction, and monitoring.
The transition to on-device processing has been a key development for Hyfe. This shift is particularly notable in their smartphone applications for both iPhone and Android. “It used to be that the applications would take snippets of audio and send them up to a server for classification,” explains Hyfe CEO Joe Brew. Now, the classification occurs directly on the device. This advancement not only streamlines the process but also enhances user privacy and reliability, ensuring functionality even in situations where internet connectivity is limited or unavailable.
.avif)
To make on-device processing viable and efficient, Hyfe needed to refine its machine learning models to make them smaller, lighter, more efficient, and adaptable to a variety of settings and devices. For this, they enlisted Edge Impulse to process their massive dataset of cough and impulse events into compact, edge-suitable models.
.avif)
The collaboration with Edge Impulse is instrumental in this phase, as it involves tackling the complex task of simplifying and optimizing AI models for specific hardware without compromising their efficiency. This optimization is a daunting task, potentially taking years if Hyfe had to develop it independently.
With Edge Impulse’s expertise, Hyfe is successfully creating AI models that are not only effective in cough detection but also compatible and efficient on various devices, including those with limited resources. Hyfe targeted a Nordic nRF5340 chip (Cortex-M33, 128MHz) to prove the viability of its Edge Impulse-generated models on high performance hardware with low latency and small memory footprints. With that success, they are now supplying the Hyfe SDK to the Nordic NRF52-powered ActiGraph LEAP wearable, among other devices.
The partnership between Hyfe and Edge Impulse underscores a significant advancement in making AI more accessible and practical in everyday healthcare scenarios.
We have shifted our focus from the problem of cough detection, which we feel like we have solved, to making the models which do that smaller and lighter and more efficient, and more deployable in different settings. And that's why we have this partnership now with Edge Impulse, because that's a non-trivial thing to do.
—Joe Brew, CEO, Hyfe
One of the key areas where Edge Impulse has been instrumental is in the management of data pipelines and testing frameworks. Brew explains, "We had been building systems very similar to what Edge Impulse had, but in a far less professional way." This realization came early in their collaboration, as Edge Impulse's platform offered a more organized, efficient approach to handling data. It addressed common challenges in machine learning projects, such as data segregation, cleanliness, and documentation, which are often time-consuming for data scientists to set up from scratch."It was an immediate boost to our efficiency and organization, and ability to disseminate and share experimental results," Brew says.
Another significant advantage Hyfe gained from Edge Impulse is the ability to test on various target chips and devices virtually. This capability has substantially accelerated their development timelines, as Brew highlights the speed and efficiency of this process. He shares a specific example where they observed a remarkable improvement in battery life for a new wearable device running their model after implementing Edge Impulse. “If we had done this on our own, we would have spent literally years achieving that increase and computational efficiency,” he says. “That has been our experience with Edge Impulse.”
Edge Impulse does the things that need to be done. We have some unique problems, because of the way we run our software. We're one of the more complex kinds of users in terms of audio, in terms of the amount of data and the kind of pre-processing steps. And Edge Impulse has been really responsive to helping us understand how to tackle those unique problems.
—Joe Brew, CEO, Hyfe
.avif)
Hyfe Signal Processing and Machine Learning Engineer George Kafentzis details the extensive effort put into creating a uniquely diverse dataset, essential in refining their cough detection technology. This dataset comprises over 3 million impulsive events, including coughs and actions that mirror coughs in their temporal and spectral characteristics. "This is actually the most diverse and rich dataset that exists worldwide on cough plus similar impulsive events," Kafentzis says. By including over 3 million such events, Hyfe has developed a model capable of distinguishing between coughs and other similar sounds accurately.
The collection process spanned the globe, utilizing Hyfe’s research app during clinical trials. This global approach ensured a rich diversity in the dataset, capturing sounds from different cell phones, various acoustic environments, distances from microphones, and a range of speakers. A team of dedicated individuals was assigned to label these sounds meticulously, a process that was rigorously verified by Hyfe’s medical experts.
The extensive and varied nature of the dataset is crucial to the high performance of Hyfe’s models. "We get such good performance in our models because it's coughs all around the world in so many different environments, on so many different microphones," Kafentzis notes. Training the AI on such a diverse range of data ensures that the models are well-adapted to real-world scenarios, enhancing their reliability and accuracy.
Despite the size of this dataset, Edge Impulse’s “Importing” tool made it feasible for Hyfe to move their data, hosted on Google Cloud, into the platform. These are now connected with active data pipelines for daily checks, which continue to add any new data in the Google Bucket to Hyfe’s Edge Impulse project.
Hyfe’s shift to embedded devices required making their AI models more compact to fit into the hardware constraints of devices like the ARM Cortex chips. One of the key elements in this process was feature extraction based on spectrograms, especially suitable for audio data, which is central to Hyfe’s technology. The task was to find the right balance between model size, accuracy, and other performance metrics. Kafentzis explains that not long after implementing Edge Impulse, they had successfully developed a couple of models that were well-suited for their target devices.
A significant aspect of their process was using spectrograms, a feature readily available in Edge Impulse, to transform these audio signals into visual representations. This technique is crucial in analyzing the unique characteristics of coughs. In addition to this, Hyfe experimented with various convolutional neural networks (CNNs), including custom models and those pre-built within the platform. This experimentation allowed them to tailor the models according to their specific needs, adjusting the size and scope to fit their project’s requirements.
.avif)
One of the primary benefits of using Edge Impulse, as highlighted by Kafentzis, was the availability of pre-defined architectures for audio processing. “The fact that the platform already proposes some architectures for audio is very, very convenient,” he says. This feature of the platform significantly streamlined their experimental process, allowing them to quickly test and iterate on different models to find the most effective solution.
I was amazed by the software, how easy it is to be used by someone who was working with code, and not with platforms. It made our lives easier, and we managed to reach our goals very, very quickly, within two or three months.
—George Kafentzis, Hyfe Signal Processing and ML Engineer
Cough Study PDF Reports:
With Edge Impulse’s expertise, Hyfe is successfully creating AI models that are not only effective in cough detection but also compatible and efficient on various devices, including those with limited resources.
Contact us to discuss how Edge Impulse can help bring your solutions to market.
