CyberFreezeDev
Loading
We test whether shrunk models still work on small devices next to the camera or sensor.
We measure on real Jetson, Raspberry Pi and Coral boards whether a job belongs on the device, a building server or the cloud.
For a given job, such as counting people at a shop entrance, spotting a defect on a production line, or flagging a motor that has started to vibrate differently, should the model run on the device, on a server in the building, or in the cloud?
The answer depends on five things: how fast a decision is needed, how much data the sensor produces, what power is available, whether the raw data may leave the site, and who will maintain the device after installation. This track measures those trade-offs on real boards rather than spec sheets. Edge hardware is where our team already does hands-on work, with NVIDIA Jetson, Raspberry Pi and Google Coral boards and with IoT devices and sensors.
Findings feed our edge AI and on-device AI and computer vision services. For choosing a board, see our Jetson vs Raspberry Pi comparison.
A device that sends events instead of video needs almost no upload.
A single camera streaming video all day is a lot of upload. A device that sends only events ("12 people entered between 10:00 and 10:15", "part 4471 rejected") needs almost none.
Reject gates and overheating alarms must answer on the spot, even offline.
A reject gate on a conveyor or an alarm on overheating equipment needs an answer on the spot, even when the internet is down.
Counting on the device without storing footage leaves far less personal data to protect.
If a camera counts people on the device and never stores or sends the footage, there is far less personal data to protect under the DPDP Act or GDPR.
Dust, heat, power cuts and no technician: devices must recover on their own.
Dust, heat, power cuts and nobody technical on site. A device that needs a person to restart it after every outage is not finished.
Three classes of edge hardware, plus the lab GPU for training and converting models before they go to a device.
Each task gets the same model family exported to each board, run on the same recorded test data.
| Metric | How we measure it | What counts as success |
|---|---|---|
| Latency per inference | Time from frame or reading in to decision out, on the device, median and slowest runs | Faster than the process needs: a conveyor sets a much tighter limit than a footfall count |
| Sustained throughput | Frames or readings per second over a long run inside its enclosure, not on an open bench | Holds steady once the board is warm |
| Accuracy after shrinking | Precision and recall on our labelled test set, before and after quantization and export | Loses little enough that the business result does not change |
| False alarms | Alerts raised on recordings where nothing was wrong, per day of data | Low enough that staff keep paying attention to alerts |
| Power draw | Measured at the supply during idle and full load | Fits the power available on site, including battery or solar where relevant |
| Recovery | Pull the power mid-run and watch what happens when it returns | Back to work with no human help and no lost configuration |
| Update path | Deploy a new model version remotely and roll it back | Works without a site visit |
Test tasks: people counting from a fixed camera, visual defect detection on sample parts, and anomaly detection on recorded vibration and temperature data.
The first two points come from vendor documentation. The rest are hypotheses we are testing.
Coral needs 8-bit quantized TensorFlow Lite models; unsupported operations fall back to the CPU.
Per the Coral documentation, models must be TensorFlow Lite with 8-bit quantization and compiled with the Edge TPU compiler. Operations the TPU does not support fall back to the CPU, which can erase the speed gain. Model choice has to start from what the accelerator supports.
TensorRT engines are compiled per board and software version, so they need rebuilding.
On Jetson, models are usually converted with TensorRT, which compiles a network for the specific GPU. An engine built for one board and software version generally has to be rebuilt for another.
Hypothesis: most value comes from sending counts and flags, not video or raw readings.
We expect most of the business value of edge AI to come from what is not sent: counts, flags and summaries instead of video and raw readings.
Hypothesis: remote updates and power-cut recovery may matter more than a little extra accuracy.
A model that is slightly less accurate but updates remotely and recovers from power cuts will likely beat a better model that needs a technician. Our recovery and update tests exist to check this.
One object detection model runs people counting on Jetson, Raspberry Pi and Coral.
Export one object detection model to Jetson, Raspberry Pi and Coral, and run the people counting task on all three with the same recordings.
Simple thresholds compared with small learned models on vibration and temperature data.
Compare simple statistical thresholds with small learned models on vibration and temperature data, since the simple method is often enough.
Runs repeated inside closed enclosures and through power interruptions.
Repeat the runs inside closed enclosures and with power interruptions, the way devices actually live on a factory floor or in a warehouse.
Notes will be published in the R&D / Labs notes section of our blog. Recordings used as test data are our own.
| Note | Published when | What it contains |
|---|---|---|
| Method note | Before any scores | The three test tasks, the recorded datasets, how latency and power are measured, and the export path to each board |
| One detector, three boards | When the people counting task has run on Jetson, Raspberry Pi and Coral | Median and slowest latency, sustained throughput, precision and recall before and after export, and power at idle and load, with board, OS image and runtime versions |
| Sensor anomaly baseline | After the vibration and temperature runs | False alarms per day of data for statistical thresholds against small learned models |
| Field conditions | After runs in enclosures with power cuts | Throughput once warm, recovery after power loss, and whether a remote model update and rollback worked |
Running an AI model on a device near where the data is produced, such as a camera box or a sensor gateway, instead of sending the data to a server. The device sends results, not raw data.
Jetson suits heavier vision models and several camera streams; Coral suits small 8-bit models at low power; Raspberry Pi suits light models and sensor work where cost matters most. This track exists to make that choice with measurements instead of rules of thumb.
We work hands-on with edge boards, IoT devices and sensors, and we build GPU servers for heavier workloads through GPU servers and AI hardware builds.
Mainly manufacturing (defect checks, machine monitoring), logistics (counting and tracking at docks and warehouses) and retail (footfall).
Last updated