Meet the Renesas RA8D1: First Cortex M85 MCU That Runs Edge Impulse and NVIDIA TAO

We recently launched a groundbreaking integration with NVIDIA to allow professional developers to access best-in-class NVIDIA TAO models within Edge Impulse — and, for the first time ever, deploy them to any edge hardware under the sun, everything from GPUs to battery powered MCUs such as the Arm Cortex M85-based RA8D1, from Renesas. Renesas is the world's largest supplier of microcontrollers (MCUs), which are found in many devices, including cars, appliances, and smoke alarms. 

The RA8D1 is the first Cortex-M85 microcontroller in the market. The Cortex-M85 is also one of the first to support Arm Helium™ technology, which delivers a significant performance uplift for machine learning (ML) and digital signal processing (DSP) algorithms. With the combination of Arm Helium technology along with NVIDIA TAO, Edge Impulse is able to create sophisticated low-power computer vision models capable of running on the RA8D1. The RA8D1 is fully supported in Edge Impulse, which makes it a perfect target for deploying the new TAO models. Professional developers can choose from the following set of NVIDIA TAO model architectures, namely RetinaNet, SSD, YOLOV3, and YOLOV4. 

Adding synthetic data

In addition, our integration with NVIDIA Omniverse can be leveraged to generate synthetic data for enhancing existing computer vision datasets and training optimized computer vision models that run on the RA8D1. The full workflow is shown below, which we used to train a custom pallet detection model using the NVIDIA TAO YOLOV3 pre-trained model that was integrated into Edge Impulse.

Target options

Once the RA8D1 has been selected as a target within Edge Impulse and the computer vision model has been trained (in this case for pallet detection), many important performance metrics become available and inform the developer on device performance and resource usage. This is essential for the developer to understand how to balance the performance and resource budgets for their particular chip.

Using only the TensorFlow Lite model, the inference performance is already impressive, netting an FPS of about 6.7 at 224x224 input resolution to the model. However a large amount of RAM is utilized at 1.4MB. Since the RA8D1 has 1MB of internal SRAM, this model wouldn’t be able to run as-is. This is where Edge Impulse’s EON compiler (RAM optimized) comes in, saving nearly 54% of RAM usage and thereby allowing NVIDIA TAO models to execute on MCUs such as the RA8D1. 

It’s easy to evaluate the model’s performance on the Renesas RA8D1 since Edge Impulse already supports the Renesas EK-RA8D1 development kit out-of-the-box. The EK-RA8D1 kit provides all the hardware one needs to develop low-power computer vision applications using Edge Impulse and Renesas (including an OV3640 camera module with an M12 lens mount). Select the EK-RA8D1 as a deployment option and flash the kit with the model and runtime application provided as a pre-built binary. 

Grab your EK-RA8D1 kit and use this end-to-end tutorial to start building your next generation low-power computer vision applications using Edge Impulse!