If you’re building a robot that needs to make decisions in real time, sending every frame to the cloud isn’t going to cut it. Latency kills. Bandwidth costs add up. And the moment your internet drops, your robot goes blind. That’s why edge AI for robotics inference isn’t optional in 2026 — it’s the only practical approach for any robot that moves through the real world.
Let me walk you through what’s actually working on real robots today, from tiny microcontrollers to beefy edge computers.
TinyML vs Edge AI: Know the Difference
Before we dive into models, let’s clear up a distinction that matters for hardware selection. TinyML runs on microcontrollers — think Arduino, ESP32, STM32. We’re talking kilobytes of RAM, milliwatts of power. Edge AI runs on single-board computers like Raspberry Pi, Jetson, or Coral — megabytes or even gigabytes of RAM, watts of power.
For robotics inference in 2026, you’ll use both layers most of the time. TinyML handles sensor processing (accelerometer data, simple keyword spotting, basic obstacle detection with PIR sensors). Edge AI handles the heavy lifting — object detection, SLAM, natural language commands, scene understanding.
Here’s my rule of thumb: if your task can run on a microcontroller, put it there. It saves power, reduces latency, and frees up the main processor for harder work. But for anything involving camera frames, LiDAR data, or LLM inference, you’re looking at edge AI hardware.
Specific Models That Work for Robotics
YOLO-NAS — The Real-Time Champion
For object detection on a robot, YOLO-NAS is my default pick in 2026. Deci AI’s neural architecture search produced a model that’s noticeably faster than YOLOv8 at the same accuracy level. On a Jetson Orin Nano, YOLO-NAS runs at 60+ FPS — comfortably fast enough for a mobile robot to avoid obstacles and track targets.
What I love about YOLO-NAS for robotics: the architecture includes a “repVGG” block that’s efficient at inference without sacrificing training-time complexity. Practically, this means better accuracy per milliwatt than any previous YOLO variant. I’ve deployed it on a differential-drive robot doing person-following, and it handles occlusions and lighting changes well — much better than the MobileNet-SSD I was using two years ago.
Quantized to INT8 using TensorRT, YOLO-NAS drops to about 30 MB and runs at 90+ FPS on the Jetson Orin. That’s the sweet spot for real-time robotics.
MobileNetV4 and EfficientNet-Lite — For Resource-Constrained Robots
Not every robot carries a Jetson. For Raspberry Pi 5 or Coral TPU-based robots, MobileNetV4 is the practical choice. Google’s latest iteration pushes the efficiency frontier further — MobileNetV4 runs at 30+ FPS on a Pi 5 for 224×224 input, which is fast enough for simple collision avoidance or line-following tasks.
EfficientNet-Lite is my pick when accuracy matters more than raw speed on constrained hardware. It’s the mobile-optimized version of EfficientNet, designed without the Squeeze-and-Excite blocks that hurt on-device performance. On a Coral TPU, EfficientNet-Lite4 hits around 45 FPS with better accuracy than MobileNetV4 on fine-grained classification tasks — useful if your robot needs to distinguish between similar objects (e.g., different types of screws in a bin-picking application).
Quantized LLMs for On-Device Language Commands
This is the area that’s changed most dramatically in 2025-2026. Running LLMs on a robot for natural language commands was science fiction a couple of years ago. Today, it’s practical.
Phi-3-mini (4-bit quantized) fits in about 2.5 GB of RAM. That’s tight for a Jetson Orin Nano (8 GB), but workable if you’re careful about memory management. I’ve run it at around 10-15 tokens/second — slow for chat, but fast enough for parsing structured commands like “pick up the red cube on the left” and translating them into action sequences.
Qwen2.5-0.5B is my go-to for smaller robots. At 0.5B parameters, it quantizes down to roughly 350 MB with 4-bit quantization. It runs comfortably on a Raspberry Pi 5 with 8 GB RAM at 20+ tokens/second. The smaller size means reasoning quality drops compared to Phi-3, but for simple command parsing and object referencing, it’s more than adequate.
For deployment, I use llama.cpp with the GGUF format. The quantization workflow is straightforward: download the full model, run `llama-quantize` with Q4_K_M, and load it on the robot using the llama.cpp server or a Python binding like llama-cpp-python. ONNX Runtime is also an option for the Phi-3 models since Microsoft provides ONNX exports directly.
Hardware Platforms That Make It Happen
| Platform | RAM | AI Compute | Best For | Price |
|---|---|---|---|---|
| Nvidia Jetson Orin Nano | 8 GB | 40 TOPS | Heavy inference, LLM on-device | ~$299 |
| Nvidia Jetson Orin NX | 16 GB | 100 TOPS | Multi-model pipelines, full SLAM | ~$599 |
| Raspberry Pi 5 (8 GB) | 8 GB | ~0.5 TOPS (CPU only) | Light inference, sensor fusion | ~$80 |
| Google Coral TPU | – | 4 TOPS | Edge TPU acceleration | ~$60 |
| Intel Neural Compute Stick 2 | – | ~1 TOPS | USB-based acceleration | ~$70 |
The Jetson Orin lineup is the workhorse for serious robotics inference. The Nano handles most single-model workloads. The NX lets you run multiple models simultaneously — say YOLO-NAS for detection, Depth Anything for depth estimation, and a quantized LLM for command parsing, all at the same time.
For truly low-power robots (drones, small rovers), the Raspberry Pi 5 + Coral TPU combo is surprisingly capable. The TPU handles vision model inference at 30+ FPS while the Pi’s CPU handles SLAM and control. Total power draw: under 15 watts.
Model Optimization Techniques That Actually Matter
Quantization
INT8 quantization is table stakes in 2026. Going from FP16 to INT8 gives you roughly 2x speedup and 50% memory reduction. TensorRT handles this automatically for NVIDIA hardware. For Coral TPU, use the Edge TPU Compiler which converts TensorFlow Lite models to INT8 quantized TFLite format.
I’ve found that detection models (YOLO-NAS, MobileNet-SSD) lose about 1-2% mAP after INT8 quantization — barely noticeable in practice. Segmentation and depth estimation models lose more, around 3-5%. For those, I sometimes keep FP16 on the Orin NX where memory is less constrained.
Pruning
This is more work but the payoff is real. Structured pruning removes entire channels or layers rather than individual weights. I used TensorRT’s pruning API on a YOLO-NAS model and removed about 30% of channels with negligible accuracy loss. The result: 40% faster inference on the same hardware.
For robotics, the key insight is that you can prune specifically for your environment. A warehouse robot doesn’t need to detect penguins. Prune away classes your robot will never encounter and you free up compute for the ones it does.
ONNX Runtime
If you’re not tied to a specific hardware vendor, ONNX Runtime is the universal deployer. Export your model to ONNX format, and ONNX Runtime optimizes the computation graph for whatever hardware it’s running on. I’ve used it to deploy the same model across Jetson, Raspberry Pi, and x86 test machines with identical code — the optimization happens at runtime.
import onnxruntime as ort
session = ort.InferenceSession('model.onnx', providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'])
# Falls back gracefullyPractical Benchmarks for Robotics Use Cases
Object Detection (for pick-and-place, obstacle avoidance)
- YOLO-NAS (INT8, TensorRT) on Jetson Orin Nano: 90 FPS at 640×640, 68% mAP
- MobileNetV4 (INT8, Coral TPU) on Pi 5: 55 FPS at 320×320, 52% mAP
- EfficientNet-Lite4 (INT8, Coral TPU) on Pi 5: 45 FPS at 320×320, 55% mAP
Monocular Depth Estimation (for navigation)
- Depth Anything V2 (FP16, TensorRT) on Jetson Orin NX: 30 FPS at 512×512
- MiDaS 3.1 (INT8, ONNX) on Jetson Orin Nano: 25 FPS at 384×384
Natural Language Commands (for human-robot interaction)
- Qwen2.5-0.5B (Q4_K_M, llama.cpp) on Pi 5: 22 tok/s, 350 MB RAM
- Phi-3-mini (Q4_K_M, llama.cpp) on Jetson Orin Nano: 14 tok/s, 2.5 GB RAM
Putting It All Together
Here’s what a practical robotics inference stack looks like in 2026 on a Jetson Orin Nano robot:
- Camera input: 640×480 RGB at 30 FPS
- Object detection: YOLO-NAS INT8 via TensorRT (20 ms per frame)
- Depth estimation: Depth Anything V2 FP16 (40 ms per frame, every 3rd frame)
- SLAM: ORB-SLAM3 on CPU (30 ms per frame)
- Language: Phi-3-mini Q4_K_M (idle until voice command detected)
- Total inference latency: ~60 ms per frame, well under the 100 ms target for real-time control
The trick is pipelining — overlapping inference with rendering and control. TensorRT’s async inference API is excellent for this. Your robot can be processing the current frame while the previous frame’s detection results are already driving motor commands.
Edge AI for robotics in 2026 isn’t about choosing one model or one platform. It’s about designing a pipeline that fits your robot’s compute budget, power constraints, and latency requirements. Start with the YOLO-NAS + Depth Anything combo on a Jetson Orin Nano, add a quantized LLM for language capabilities, and optimize until it fits. That stack will handle 90% of autonomous navigation and manipulation tasks out of the box.