We are excited to announce a new way to generate synthetic data at Edge Impulse, using the hyper-realistic FLUX-pro tools from Black Forest Labs to provide suitable images for training machine learning models. Per Black Forest Labs, FLUX-pro offers “state-of-the-art image generation with top of the line prompt following, visual quality, image detail and output diversity.”
The FLUX-pro model API is available through different AI platforms, including FAL, Mystic, and Together; for our integration we use the one hosted on Replicate, so you’ll need to set up an account there and obtain your API Token for this. Replicate offers integration with GitHub, which might be easier for some users if you already have billing information set up there. (Each image does cost 0.055 USD, so keep that in mind while you curate your dataset.)
If you're not yet familiar with customizing our transformation blocks, jump into this doc to get the overview.
Use cases
Particularly for computer vision, there are many ideal applications for this type of synthetic data. A couple examples:
- Anything where obtaining data in a manner that respects privacy would be difficult, such as training medical AI assistants to ensure all instruments used during operation are accounted for before finishing the procedure.
- Any bespoke security monitoring use cases in which there is only limited data available of the subjects in the setting being monitored. For example, it is challenging if not impossible to find publicly available nighttime photos of raccoons taken with active IR lighting, which is necessary for creating a humane, AI-driven raccoon-deterrent wildlife camera that protects outdoor pets.
For this second example, we used the Replicate Synthetic Data block to generate ~500 images using the prompt, “A distant and grainy realistic image of a large and feral raccoon set in a backyard patio at night as if taken by a security camera with an infrared filter.” The qualifiers ‘distant’ and ‘grainy’ as well as ‘large and feral’ were critical to get the images correct and not look cartoonish.
Here are some guidelines for using our FLUX-pro integration.
Getting started with Replicate
Before using the model in Edge Impulse to generate batches of data, it’s a good idea to use Replicate’s intuitive user interface to experiment with some of the hyperparameters to tailor the output to your specific needs and get an idea for how they affect the results. One of the neat features that the FLUX-pro model offers is the ability to set a Seed value for reproducible and deterministic generation so that the two most critical hyperparameters, Guidance and Interval, can be finely tuned for your application. However, Seed is not currently utilized in the Synthetic Data generation block on Edge Impulse, meaning that each run of the block produces a new set of outputs each time, but it may be added in the future.
Let’s look at these two hyperparameters interact with a request:
Guidance (minimum: 2, maximum: 5, default: 3)
“Controls the balance between adherence to the text prompt and image quality/diversity. Higher values make the output more closely match the prompt but may reduce overall image quality. Lower values allow for more creative freedom but might produce results less relevant to the prompt.”
Interval (minimum: 1, maximum: 4, default: 2)
“Interval is a setting that increases the variance in possible outputs letting the model be a tad more dynamic in what outputs it may produce in terms of composition, color, detail, and prompt interpretation. Setting this value low will ensure strong prompt following with more consistent outputs, setting it higher will produce more dynamic or varied outputs.”
Image generation
Now that you have a notion of what your hyperparameters should be, it’s time to start generating large batches of images to use for model training. After you’ve generated a few dozen or hundred images and labeled them manually or with our AI labeling tools, you can then use them to train and test any image classification or object detection model available on Edge Impulse for your chosen hardware. We’ve included some examples below to demonstrate just a few of the possibilities that you can unlock by combining Black Forest Labs’ state-of-the-art text-to-image generation FLUX-pro model with the industries’ leading platforms for developing complete edge AI solutions.
See more here in the custom block repository to use the block in your own Edge Impulse organization and modify it to suit your own purposes and applications!
Edge Impulse supports multiple tools for synthetic data generation; visit our docs to explore even more options.
Prompt: A distant and grainy realistic image of a large and feral raccoon set in a backyard patio at night as if taken by a security camera with an infrared filter
Prompt: A realistic top-down image from the top level of a mall during the holidays with many people shopping
Prompt: A realistic image of a busy toy store during the holidays with many people shopping
Prompt: A realistic image of a busy store during the holidays with many people shopping
Prompt: A realistic image of a parking lot with parked vehicles as well as cars coming and going
Prompt: A realistic image of a busy city intersection with pedestrians, vehicles, buses, and cyclists
Prompt: A realistic image of various surgical instruments and tools laid out on a cart in an operating room
Prompt: A realistic top-down image of bunch of small and fully assembled printed circuit boards on a flat conveyor belt
Prompt: A realistic top-down image of several fully assembled printed circuit boards on an empty electronics workbench
Prompt: A realistic top-down image of bunch of small and fully assembled printed circuit boards on a conveyor belt
Prompt: A realistic image of a forklift carrying multiple empty pallets across the floor of a large warehouse
