Together with Hackster, we hosted a webinar on getting started with TinyML, which was a fantastic success with almost 500 people present and over 100 questions asked! If you’ve missed it you can watch the video above which consists of a half-hour introduction to TinyML, a 20 minutes demo where we explore two ML models, and 10 minutes Q&A.
We’ve also taken the courtesy to write out the most frequently asked questions in this blog post.
Q&A
Is Edge Impulse compatible with X?
You can collect data from any device through the ingestion API. Just grab the raw sensor data and send it in. The inferencing library that is generated in the deployment step is standard C++11, without any external dependencies, and can be compiled for any development board, gateway or desktop environment (see Running your impulse locally) - and you can optionally link to it from a C application.
Where possible we use hardware optimizations - like vector extensions - to speed inferencing up, and we’ve implemented these for all Arm Cortex-M devices that have an FPU. If you’re using a different architecture then you can make the inferencing faster by implementing these for your platform (see our inferencing SDK). Typically any Cortex-M4F with 128K RAM and up will work fine out of the box. We also have some first-party support for new boards coming, including a board from Eta Compute that has great low power parts.
Is it actually possible to run TinyML models continuously on battery-powered devices for over a year?
Typically yes, but like with everything in embedded systems: it depends on how often you run the model. The ST board that we used in the webinar will automatically go into deep sleep mode between inferences, and it drops down to 60 uA power consumption (measured with a QOITECH Otii), then when you capture data you can do this at a low power setting, and then burst when you need to run your machine learning model - in this case only 10 ms. every second. So you still need to do duty cycling and power management, but it’s a lot more efficient than driving the radio.
Specialized SoCs like Eta Compute can even achieve less than this, and DSP and NN accelerators will make this even more efficient, like the Arm Ethos-55. Arm states that they can do object detection on still images on a coin cell every five seconds for a year before the battery runs out.
What if I have specialized data - like an ultrasound radar, can I use Edge Impulse for that?
Mostly, yes. If you have a novel sensor, for example, something that uses ultrasound, you can plug in your own processing blocks in any programming language (we have some examples available in Python) into Edge Impulse so then we can do the feature extraction from there. However we don’t have automatic code conversion to run you on a device then, so it requires a little bit of work there, but you can do the initial full onboarding on that sensor data from it. There’s some background in this blog post - which also links to a tutorial on writing your own signal processing block.
Is there any info about EdgeImpulse’s pricing?
Edge Impulse is free for individual developers (including for commercial use), and all models that you generate are open-source, royalty-free, and require no permission to deploy. You also get all the source code (no statically linked binaries here!). We have an enterprise subscription for large-scale production use, where teams of engineers are working on multiple algorithms in production. Contact us for details.
Do you interact with other tools like iPython notebooks or Sagemaker?
We have a couple of extension points so we don’t lock any data in. Any data that we generate, from the output of your signal processing pipeline to the TensorFlow models that we generate, you can pull out, either as files from your projects dashboard or through the API.
If you’d rather work with your own data science tools, you can go to the Neural Network classifier page in Edge Impulse, click on the three dots, and select Edit as an iPython notebook. This will give you a Python notebook with all data already pulled in, and you can start experimenting with different architectures or data sanitization. We’ve put the code that is interesting for us in a single cell, so as long as you keep everything in there you can just copy and paste the architecture back into Edge Impulse.
Wouldn’t FPGA’s be better for inferencing?
Moore’s law has caught up with MCUs quickly! Today’s Cortex-M4 or M7 can execute most TinyML inference in software in real-time. This is further accelerated with DSP or Neural Network acceleration cores. Often those are part of the SoC but could be implemented in FPGA (at a power cost).
I already have a dataset of audio files, is it possible to upload them to Edge Impulse?
Yes it is! We have an upload tool for WAV, JSON, and other files: Edge Impulse CLI.
What about the application of TinyML in industrial or automotive industries?
Yes, lots of industrial applications, e.g. predictive maintenance is one of the major applications people work on. Automotive has many examples, from advanced tire monitoring, to control systems, to crash detection and road surface condition analysis.
Can you use Bluetooth Low Energy for data ingestion?
Data ingestion really just requires moving binary JSON or CBOR blobs to our REST interface in the cloud. You could definitely do that over BLE but would need to write an app or add that support to an existing app for example.
Will you do anything with cameras or images? Is the same technique of TinyML even applicable to images, and where does the boundary end if it’s possible?
Not at the moment with Edge Impulse, but definitely something we’ll be adding in the coming year. Whether you’ll get success with it mostly depends on the complexity of the task at hand. Classifying 10 different types of objects in low-resolution images? Yes, totally going to work. But you won’t have enough processing power to run full ImageNet on your device. Full resolution video is also not going to work, so you need to work with the constraints that you have. For still images, like people counting or animal spotting, that is definitely something that’s going to be in the pipeline.
If you do need bigger models, then you can also run the same algorithms on higher power computers. There is nothing that limits this to an MCU, you could deploy on a Cortex-A processor or a Raspberry Pi and you can deploy larger models.
How to differentiate between right and wrong data? E.g. how do you know when a sensor is faulty?
Labels are the key. When collecting data you label what data it is (e.g. ’normal’). That can then be used to train ML to detect right vs. wrong or classification. You can also use anomaly detection to see when something deviates from the norm, which requires less faulty data.
Are conventional ML techniques used in TinyML, or just neural networks?
We believe strongly in using both classical ML and NNs where appropriate! Our ML blocks support the implementation of both, and we already today provide K-Means for Anomaly detection in addition to neural networks. If we see other interesting algorithms we’re happy to add them.
Can you use TinyML for deep learning algorithms?
Yes, but models are typically smaller if you use a smart feature extraction pipeline combined with e.g. a small multi-layer perceptron instead of a deep convolutional neural network. But nothing is stopping you from building it.
What I like about classic ML is that, most often, I can explain what happens. If you’re going to deploy 100s of thousands of devices into the wild that need to run without any supervision it’s scary to say: here’s a neural network black box, trust me that it’ll work. So for us we often deploy a classical ML on the side as kind of a safety guard. A neural network might perform much better, but we like the certainty of actually knowing when something goes wrong so you can flag that and send it back.
How did the development board in the demo send the data back to the cloud?
In this case the development board sent data back to the computer, and then the serial daemon picked up the data and relayed data back to the cloud. You can also use WiFi on the ST IoT Discovery Kit.
Is it correct to assume that the firmware flashed on the device needs to run the specific Edge Impulse application which collects the data (so somehow it needs to be set within the firmware) and the daemon running on the host? Are both parts Open Source?
For the full experience: yes. But as stated above you can just send data directly to the ingestion service. The daemon can be found here: edge-impulse-cli and the firmware here: firmware-st-b-l475e-iot01a.
Is data for Edge Impulse stored in US data centers or outside?
Data is stored primarily in Ireland at the moment, but backups might be stored elsewhere.
How can I increase my model accuracy?
Typically: add extra data. See some extra tips here: increasing your model performance.
After exporting, do you also calculate the features, or do you need to manually calculate them?
For built-in processing blocks, all code is included, including feature generation. Where possible we use the vector extensions (e.g. through CMSIS-DSP) to make this all fast.
Can you create custom neural network architectures?
Yes, just click the three dots on the neural network algorithm page, and click Switch to Keras (expert) mode. You then have full freedom in your neural network architecture.
If you collect data from a phone, will the model work on a microcontroller?
To a part, yes. There are differences in sensor performance between phone and microcontroller, so it’ll never be as good, but you’ll get an idea of how well it works.
During training, does everything run on your backend? Is there any upper limit for resources here?
Yes, everything runs on our servers. There’s a compute limit per project, which you can see from your projects dashboard.
How does Edge Impulse compare to the Matlab ML + Embedded code generation toolbox? We’re used that in the past with good success using SVM and Bagged Trees Models. The price was less good though...
Edge Impulse really helps with the lifecycle management of your TinyML deployment. Collecting data from many sources, verifying that the model keeps working when you add new data, deploying back to the device and keeping versions, etc. For a successful deployment, you’ll need much more than a one-off model deployed from Matlab. But Matlab definitely has its place. We have customers doing their initial feature engineering there and then moving to Edge Impulse to go to production.