We are proud to announce a significant enhancement to our machine learning capabilities with the integration of the learned optimizer feature using VeLO (Versatile Learned Optimizers). This new feature, aimed at simplifying and optimizing the machine learning process, is now available for enterprise users.
Advantages of Using VeLO
VeLO's primary advantage is its ability to adapt across various scenarios with minimal tuning, compared to traditional optimizers like Adam. However, it's important to note that VeLO comprises a large LSTM model, which may require more computational resources, especially for GPU-intensive models like those used in vision.
The Challenge of Selecting Learning Rates and Training Cycles
A common challenge in machine learning is the selection of appropriate learning rates and the number of training cycles. These decisions, often based on trial and error, can significantly impact the efficiency and effectiveness of model training. With the introduction of VeLO, these issues are addressed.
VeLO is a foundational model developed by Google that employs a machine learning model to dynamically determine the optimal learning rate, eliminating the need for manual selection. This represents a major step forward in automating and enhancing the training process.
The integration of VeLO aligns with our strategy to adapt to the surge in large machine learning models. While running large language models (LLMs) is not currently feasible on microcontroller units (MCUs), the potential of leveraging foundational models that excel in various tasks without retraining is immense. The learned optimizer feature (using VeLO), following our Auto-labeling feature (using "Segment Anything"), exemplifies our commitment to leveraging these advanced models to assist users in building more accurate and smaller models more efficiently.
How to use the learned optimizers in Edge Impulse Studio?
The learned optimizer can be enabled in Edge Impulse as an option on the training page. For our more advanced users, you can also enable it using the expert mode. See the documentation for more information and project examples: Learned Optimizer (VeLO).
We encourage you to maximize batch sizes (adjustable in "Advanced training settings") and consider GPU training for optimal results. This approach leverages the computational power and memory capacity necessary for VeLO's functionality.
As always, we look forward to seeing what you will build and welcome feedback & suggestions on our forum.
Working on a professional project and need some help? Send us a note!