In the heart of urban landscapes, a hidden source of energy waste continues to undermine sustainability efforts and deepen the strain on both wallets and the environment. Poor management of resources in unoccupied rooms of buildings is a major problem that has steep costs. One of the main offenders is the inefficient operation of HVAC systems. These systems often continue to operate at full capacity even when rooms are empty, resulting in unnecessary energy consumption. Similarly, inadequate control of lighting, and other systems, can result in energy being wasted in unoccupied spaces.
The scale of the problem is staggering, with the US Department of Energy estimating that Americans spend about $100 billion every year on energy that is wasted due to inefficient heating and cooling practices. These funds, squandered in empty corridors and vacant rooms, represent a missed opportunity for investment in more constructive ventures while amplifying the already considerable energy demand. In a world where the pursuit of sustainability and fiscal prudence intersect, the price of overlooking this issue underscores the urgency of implementing intelligent solutions to curb such wasteful practices.
Smart building technologies, which are driven by precise sensors, data analysis, and automation, offer a promising solution to this energy problem. These innovations orchestrate an intricate dance of efficiency by deploying occupancy sensors that detect vacant spaces. HVAC settings change in response to occupancy patterns, and lights dim when rooms are unoccupied. Additionally, cutting-edge building management systems take into account occupancy trends, upcoming weather conditions, and other variables to fine-tune HVAC operations and only illuminate spaces when necessary. These advancements appear to hold the key to significant energy savings and a reduced environmental impact.
However, despite the promise of savings, we all know that adopting such a system is likely to entail a significant initial investment, ongoing maintenance costs, and a steep learning curve. Factors such as these are enough to keep many from even considering a new smart building system, especially smaller businesses. But, does it need to be this costly and complicated? An engineer by the name of Jallson Suryo says that it does not, and moreover, he built a prototype smart building controller to prove it.
Suryo focused on control of an HVAC system in building a scaled-down prototype that monitors occupancy of several zones in a model office building. A machine learning object detection model developed with Edge Impulse Studio was used to locate and count people such that the speed of the fans (standing in for HVAC zones) could be controlled intelligently. The images were acquired by, and the machine learning algorithm ran on, a low-cost, off-the-shelf edge computing device that is highly accessible. This effort resulted in the development of an inexpensive smart building controller that could detect room occupancy with a high degree of accuracy and respond accordingly.
The Sony Spresense main board was chosen as the hardware platform. It has a six-core Arm Cortex-M4F processor, 1.5 MB of SRAM, and 8 MB of flash memory that make it well-suited for running image processing algorithms at the edge. It is also loaded with available GPIO pins that can be used to interface with any arbitrary external system, like Suryo’s HVAC controller. The main board was paired with a Spresense camera board that can capture high-resolution images at up to five megapixels.
An Arduino Nano was also included in the design to interface with a pair of motor drivers that control the speed of the four DC fans. A small OLED display was also added to the project to display information about the state of the system to the user. These components were attached to a custom PCB that was produced to keep the circuit nice and tidy.
To make the most of the hardware resources, Suryo captured images of the model building from above, such that a single camera could cover all four of the zones that he defined. A number of images were captured from above, with different configurations of occupants, and under different lighting conditions, to be used as training data for the object detection model. The variety in the training data would be expected to help the model to generalize well such that it can perform with accuracy under many different real-world conditions. After the data was collected, it was uploaded to a project in Edge Impulse Studio.
Each image was labeled by drawing a bounding box (with assistance from AI that helps to draw them) using the Labeling Queue tool. Then Suryo pivoted to the creation of the impulse, which is what defines the flow of data from the time it is captured by the sensor all the way until a prediction is made by the model. The impulse began with preprocessing steps to first shrink the images and convert them to grayscale, then extract the most informative features — all critical steps that reduce the computational complexity of downstream processing steps. The features are then fed into Edge Impulse’s ground-breaking FOMO object detection algorithm that is highly optimized for accuracy with minimal resource utilization (typically less than 200 KB of RAM is required).
The default model hyperparameters were kept, then the training process was initiated. After it was complete, metrics were displayed to help assess how well the model was performing. In this case, an accuracy of 100% had been achieved. While this is excellent, it can also come across as too good to be true, so the more stringent Model Testing tool was also utilized to ensure that the previous result did not arise from overfitting the model to the training data. This tool also revealed an accuracy of 100%, so everything was good to go!
Suryo was then ready to deploy the model to the Sony Spresense hardware. Doing so eliminates the need for an Internet connection during operation, and also sidesteps the privacy issues that arise when transferring a constant stream of images to the cloud for processing. The model was exported as an Arduino library because this option offers a lot of flexibility. It allowed Suryo to add his own logic to trigger the fan controller based on the inference results, then flash the code to the Sony Spresense main board.
A number of tests were conducted, during which it was shown that the fans increased or decreased their speed reliably as people were moved in and out of the different zones of the model building. While this prototype does only control fans in a model building, it is fairly easy to envision such a system being scaled up to cover larger areas, detect real people, and control an HVAC system with only relatively minor modifications.
Are you ready to save some big money and help the environment without laying down a huge pile of cash? I thought you might be. Check out Suryo’s project write-up and public Edge Impulse Studio project to get you on the fast path to energy (and money!) savings.
Want to see Edge Impulse in action? Schedule a demo today.