CyberFreezeDev
Loading
Running AI models on devices near the data, like a camera board or phone.
The device runs the model on raw data locally and sends only the result onward. Training usually still happens on bigger GPU machines.
Edge AI is running AI inference on hardware at the "edge" of a network, close to sensors and users, rather than in a central data centre. The device receives raw data (video frames, vibration readings, audio), runs a model locally, and sends only the result onward: "defect at 14:02", "door open", "temperature trend abnormal".
Training usually still happens on bigger GPU machines. The edge device runs a smaller, often quantized copy of the trained model.
Small modules with an onboard GPU, suited to several cameras and heavier vision models.
Small modules with an onboard GPU and CUDA support. Good for several camera streams and heavier vision models. See NVIDIA Jetson.
A cheap Linux board for light models, sensor hubs and gateways.
A cheap Linux board for light models, sensor hubs and gateways. Pairs with add-on accelerators for faster inference.
An Edge TPU accelerator for TensorFlow Lite models, as USB sticks or modules.
An Edge TPU accelerator for TensorFlow Lite models, available as USB sticks and modules. See Coral.
Tiny chips running very small models, such as keyword spotting, on battery power.
Tiny chips that run very small models for keyword spotting or anomaly detection on battery power.
A reject arm on a conveyor cannot wait for a cloud round trip.
A reject arm on a conveyor cannot wait for a round trip to a cloud region.
Sending results instead of video saves upload costs, especially on 4G at remote sites.
Sending results instead of continuous video saves upload costs, especially on 4G links at remote sites.
Frames with faces are processed and discarded on the device, which helps with data law.
Frames with faces can be processed and discarded on the device, which helps with the DPDP Act 2023 and UK GDPR.
Local decisions continue on patchy internet, and results sync when the link returns.
A site with patchy internet still gets local decisions, and results sync when the link returns.
A Jetson board watches freezer doors and aisles in a cold store and triggers a WhatsApp alert, with no video leaving the building.
A cold storage warehouse wants an alert when a freezer door stays open or a pallet blocks an aisle. A Jetson board reads two camera feeds and runs a computer vision model locally. Temperature sensors report to the same board. When a rule fires, the board posts a small JSON message to a webhook, which triggers a WhatsApp alert to the shift lead. No video leaves the building.
Test the model first, then size the device. Cloud-sized models may not fit.
Pick and test the model first, then size the device. A model that fits in a cloud GPU may not fit in 4 or 8 GB of shared memory.
Field devices need new models, logs and security patches without a site visit.
Devices in the field need a way to receive new model versions, logs and security patches without a site visit.
Hot factory enclosures throttle or fail. Plan cooling and a stable power supply.
Small enclosures in hot factories throttle or fail. Plan for cooling and a stable power supply.
They overlap but differ in scale. On-premise AI usually means servers in your office or data centre running large models. Edge AI means small devices spread out next to sensors and cameras, each running a compact model.
Small language models can, after quantization, on boards with enough memory such as higher-end Jetson modules. Large models with tens of billions of parameters need server GPUs.
Through a device management setup: devices check a central server for new model files and configuration, download them, verify them and switch over, with a way to roll back if the new version misbehaves.
Last updated