Cars are no longer engines with radios. These days, many models are loaded with more tech than Tony Stark's basement. Among these options, there are well-equipped cars, and then there are really well-equipped cars. Tesla vehicles, in particular, are famous for the innovative technologies that fill every available nook and cranny. But given the limitations of present state-of-the-art batteries, and an inadequate infrastructure for recharging them in many areas, electric vehicles are not suitable for everyone.
When it comes to getting their hands on the latest toys, technophiles will go to some pretty extreme lengths. Solomon Githu is very excited about the security features available in Teslas, but he did not want to have to replace his car with a pricey electric vehicle to get them. So as a compromise, he developed a portable computer vision system that can easily be installed in any car to unlock similar capabilities.
And that is a big deal, because theft costs vehicle owners billions of dollars each year, and almost half of all stolen cars are never recovered. The development of any measures that can prevent vehicle theft, aid in property recovery, or identify the perpetrators — and be widely deployed — would be very welcome.
Githu’s plan involved installing a camera inside a car to autonomously watch out for suspicious activities. When anything suspicious is detected, the system would be able to remotely alert the vehicle owner so that immediate action could be taken.
Of course building a system that meets these specifications is easier said than done. Automobile manufacturers have large professional engineering teams working on these types of features, after all. So what is the likelihood of Githu’s success when going it solo? Extremely high, actually, because of the hardware and software tools he chose to work with.
The trickiest part of all is developing the intelligent computer vision algorithms necessary for recognition of suspicious activities. Fortunately there is a major shortcut for exactly these types of use cases — the Edge Impulse platform. Using Edge Impulse, Githu was able to simplify data collection and annotation, model development and optimization, and deployment to the physical hardware with an intuitive, point-and-click interface.
As far as the hardware is concerned, it must be powerful enough to run a computer vision algorithm, but it also must be energy-efficient enough for mobile use. It really would not do to have the security system draining the car’s battery or shutting itself down when it is most needed due to a lack of available power. In much the same way that Edge Impulse solves the software problems, the Arduino Portenta H7 with a Vision Shield solves the hardware problems. With an Arm Cortex-M7 processor running at 480 MHz, a whopping (for a microcontroller!) 8 MB of RAM, and a high-quality camera, this combination has the right stuff to make this project work. And since it slowly sips power, it will keep right on working.
To prove the concept, Githu chose to initially look for three conditions:
- No threat
- Potential tire theft
- Potential break-in through the window
The Portenta H7 is fully supported by Edge Impulse, so data collection was as easy as installing the latest firmware on the board, then capturing images while acting like a crook that is trying to steal the car. When the device is linked to an Edge Impulse project, those images are automatically uploaded and waiting when you are ready to move on to the next step in the development process.
With the data collection all squared away, Githu was ready to build the model. In Edge Impulse, that is achieved by developing an impulse, which defines how data is handled from the time a sensor measurement is captured until a prediction is made. There are a lot of options for preprocessing and analyzing the data, but fortunately, you no longer have to choose just one set. Using the Experiments feature, you can create multiple impulses within the same project to make it easy to compare the performance of each.
In this case, Githu tested out a several models, including some with visual anomaly detection. They all performed with near-perfect accuracy, so it made little difference which was ultimately selected. However, for a real-world application with a much larger dataset and many more classes to learn, it is likely that some impulse versions would perform better than others, and that is where the Experiments feature really adds value.
Whichever model one decides to ultimately use, it can be deployed to the physical hardware — the Arduino Portenta H7 with a Vision Shield in this case — by using the Deployment tool. This tool takes advantage of the EON compiler and a number of other optimization options to enable powerful algorithms to fit within the limits of resource-constrained hardware platforms while maintaining high levels of accuracy. After optimization, the impulse can be delivered as custom firmware, or in a more generic format, like C++ code, that allows for application-specific customization.
Githu downloaded his model as an OpenMV IDE library, which gave him the flexibility to add custom code that sends an email to a designated address whenever suspicious activity is detected. This could, of course, also be a phone notification, text message, or whatever else a given application calls for.
So what do you think? Are you ready to supercharge your old bucket of bolts with cutting-edge technology? For starters, you should read through the project write-up to prepare yourself for the task at hand. After that, you can clone Githu’s public Edge Impulse project and upload some images that represent the suspicious activities that you would like to detect. From there, you can pretty well copy-and-paste your way to project completion. Don’t worry, we don’t mind if you borrow! The last step is the most important — enjoy your brand new vehicle security system that only looks like it cost a big chunk of change.