Not so long ago, there were many more purpose-built portable electronic devices than there are today. You might have had a phone for calls and texting, an iPod for music, a digital camera for taking photos, a GPS device for navigation, a handheld game console for entertainment, and a separate e-reader for reading digital books. But then along came the era of smartphones and watches, and they quickly gobbled up the functions of all of those standalone devices.
Condensing all of these capabilities into a single small package is a fantastic idea in theory, but there are some problems in practice. An Apple Watch, for instance, can do just about anything you might want it to do. However, that does not mean that it can do everything especially well. Purpose-built devices, on the other hand, wouldn’t be around for long if they were not really good at the one thing they were designed for.
The growing realization that kitchen-sink devices are sorely lacking when it comes to some of the applications that matter most to us has led to a resurgence of interest in more purpose-built hardware (see the release of the Pebble watch source code, for example). But, alas, few such devices have hit the shelves just yet. Fortunately, that does not mean we are stuck with one-size-fits-all electronics designed for the “average” consumer. As Thomas Vikström cleverly demonstrated in a recent project, if we use the right tools, customizing our own hardware does not have to be especially difficult.
Some time after getting back into lifting weights for the health benefits it offers, Vikström realized that he was going to need some motivation to keep up with the daily grind. Something that Vikström finds to be motivational is keeping a log to track progress over time. If things are going up and to the right, that knowledge can be enough to help many of us push through those last few reps. But after trying some commercial smartwatch options for fitness tracking, Vikström found that they did quite a poor job of identifying specific exercises, making the tracking capabilities spotty at best.
After thinking this situation over, Vikström hatched a plan. He had a Bangle.js 2 smartwatch on hand (or rather, on wrist), which is designed to be hackable. It is not super powerful with a Nordic Semiconductor 64 MHz nRF52840 Arm Cortex-M4 processor and 256 KB of RAM, but with a highly-optimized machine learning model built with Edge Impulse, it should be adequate to deploy a classification model that can recognize exercises better than existing devices that toss this sort of functionality in as an afterthought.
With an accelerometer already onboard, the watch itself had all the hardware necessary for the project. So as the next step, Vikström hit the gym to collect some data to train the classification model. A program was loaded onto the watch that collected accelerometer data while Vikström did pull ups, push ups, rows, some running, and a few other common exercises. The data was then downloaded as a CSV-format file.
That made loading the data into an Edge Impulse project a breeze. With a few clicks, the CSV Wizard had imported enough data to help the model learn to characterize Vikström’s motions when exercising.
Next up, it was time to build an impulse. An impulse specifies exactly how data will be processed, from the time it is produced by a sensor until a prediction is made by the machine learning algorithm. For this application, the impulse started with preprocessing steps that first sliced the time-series accelerometer data into segments of uniform length before extracting the most informative features from the data. The features were then fed into a neural network classifier that was designed to identify six different exercise-related actions.
Even with a fairly small training dataset, a classification accuracy of better than 95% was achieved after training the model. Eat your heart out, commercial smartwatches! It is always a good idea to validate the result by using a dataset that was excluded from the training process before getting too excited, however. That can be done with the Model Testing tool in Edge Impulse, and in this case, it reported an accuracy level of nearly 90%. Looking great for a prototype! This accuracy level would be expected to climb higher yet by simply supplying the model with a larger training dataset, if the need arises.
The model still needed to be deployed to the watch before it was of much use, however. As it turns out, one can download Tensorflow Lite models from Edge Impulse, and the Bangle.js 2 knows how to run them. It’s like a match made in Heaven! Vikström uploaded the model to the watch with the Espruino IDE, then it was ready for use.
Vikström spent some time testing the watch, and found it to work quite well, as would be expected from the accuracy results previously obtained. At this stage of the game, workout logs are downloaded from the watch in CSV format, but that could be updated in the future. Perhaps a custom app — or a wireless connection to a phone app — could be developed for a nicer user interface. Aside from that, Vikström is contemplating increasing the sampling frequency of the accelerometer to see if that might bump up the performance of the watch just a bit.
The work may have been done on a Bangle.js 2 smartwatch in this project, but any reasonably customizable watch should work fine. If you have something matching that description that you would like to try this system out on, then feel free to grab the source code from GitHub. Vikström has also made the Edge Impulse project public to save you some more time, although you might want to upload your own accelerometer data to fine tune the model to your own unique workout style. And while you are at it, be sure to take a look at the project write-up for all the details you need to get you over the finish line.