The Hailo-10H beats the Coral TPU on speed and watts

The Hailo-10H is a real successor to the Google Coral Edge TPU for Frigate NVR and Home Assistant builders who need more than a handful of object-detection streams. It posts 40 TOPS of INT4 compute, or 20 TOPS INT8, inside a 2.5W typical power envelope. That works out to about 8 TOPS/W on INT8 against Coral’s ~2 TOPS/W, and it slots into a Raspberry Pi 5 M.2 HAT or a cheap N100 mini PC. The “up to 10B-parameter generative AI” line is real but heavily caveated. You can run quantized Llama-3 or Phi-3 via Hailo’s Model Zoo , but token rates sit in the single digits, and the 2.5 GB/s memory bandwidth is the choke point. It is a vision accelerator that dabbles in small LLMs, so do not buy it as an LLM box.

What the Hailo-10H is, and how it compares to Coral, Hailo-8, and Kinara

Edge AI silicon in 2026 is no longer a two-horse race between Coral and Jetson. The Hailo-10H sits in a new mid-tier next to Kinara Ara-2 and Axelera Metis. Place it on that map before you pick a chip for your NVR, robot, or retrofit build.

Here are the core specs from the official Hailo-10H M.2 product page . You get 40 TOPS INT4 or 20 TOPS INT8 on a second-generation Hailo neural core, with 4 GB or 8 GB of on-module LPDDR4/4X. The link is PCIe Gen3 x4, the body is M.2 Key M in either 2242 or 2280, and typical power is 2.5W. It runs at industrial temps, -40 to 85C. An automotive AEC-Q100 Grade 2 variant is lined up for 2026 production.

Here is how the edge AI lineup compares for a Frigate-style vision workload:

The design gap that shows up in benchmarks is dataflow vs. dispatch. Hailo maps each layer of the graph onto physical compute tiles on the chip. The compiler bakes in a fixed schedule, which keeps those tiles busy on newer models like YOLOv8 and YOLOv9. Coral’s TPU does well on the MobileNet family it was built around, then drops off hard on transformer-heavy detectors. That is why a single Hailo-8 often beats four stacked Corals on YOLOv8n, and why the Hailo-10H stretches the lead at higher resolution.

Hailo-8 is still sold next to the 10H. It fits a single-sided M.2 2230 body, draws under 2.5W, and costs less. If your job is pure computer vision and fits inside its 26 TOPS, there is no reason to pay for the 10H. The 10H earns its price when you need the 8 GB of LPDDR4 for generative work, or when you want headroom for YOLOv8m and YOLOv9s at 4K.

The “10B-parameter generative AI” claim on Hailo’s marketing deck has a plainer reading. It means up to 10B parameters in 4-bit weight-only quantization, with external LPDDR4 paging. FP16 is out of reach at that size, and so is anything close to chat speed. The LLM section below covers the real-world numbers.

Hardware and form factors: M.2 2242/2280, power, and PCIe

The shape of the 10H decides which hosts you can plug it into. The module ships in two versions. The M.2 2242 B+M key card has no on-module DRAM, and it aims at classic vision work where the host feeds data over PCIe. The M.2 2280 M key card puts 4 or 8 GB of LPDDR4 right next to the accelerator for generative work. The 2280 is thicker because of that memory stack, so check the height before you plan a slim case.

The PCIe interface is Gen3 x4 and falls back as far as Gen2 x1. That fallback is why the chip works on the Raspberry Pi 5’s single-lane M.2 HAT. Lane count caps model load time, not inference throughput, because HailoRT streams activations in small bursts once the HEF sits in device memory. On Gen4 hosts like the AMD Ryzen Embedded 8000 series or Intel N305 boards, load time roughly halves. That helps if your Frigate container restarts often.

Heat is easier to handle than the 40 TOPS number implies. Full-load draw on the 10H sits around 3-4W, and the 2.5W typical figure reflects mixed duty cycles. A passive aluminum heatsink is enough up to about 45C ambient. Above that, you want airflow. The Raspberry Pi AI HAT+ 2 uses a thermal pad and leans on the Pi 5 fan. Mounts from Pineboards and Pimoroni add a machined heatsink that holds the junction temperature in the 55-65C band under steady Frigate load.

Hosts that work include the Raspberry Pi 5 with the Pimoroni NVMe Base or the AI HAT+ 2, the Radxa X4 (N100), Intel N100 and N305 mini PCs, the Khadas VIM4, and any x86 laptop with a free NVMe slot. The clumsiest fit is the Pi 5 itself. The wiring is fine, but the HAT stack is tall, so plan for that before you order a case. The same board can anchor very different builds, including this accelerator rig and a portable Kali Linux lab running on battery.

The Raspberry Pi AI HAT+ 2 board at $130 is the cheapest way into 10H ownership. Bare M.2 modules from Revinetech and regional sellers sit in the $220-$280 band, based on whether you pick 4 GB or 8 GB of LPDDR4. That is well above the $40 Coral M.2 street price. The cost only pays off if you use the extra compute or need the on-module memory.

The HailoRT SDK and Dataflow Compiler: ONNX to HEF

The Coral toolchain takes a TFLite file and produces an EdgeTPU build in one step. Hailo splits that work in two. You compile a model offline with the Dataflow Compiler into a .hef binary, then run that binary at deploy time through HailoRT. The extra stage is the price of the chip-specific scheduling that keeps the tiles busy.

Install HailoRT from the hailort GitHub repo . Debian and Ubuntu get .deb packages. They cover the hailo_pci kernel module, the user-space runtime, and the hailortcli tool. Check the card with:

sudo modprobe hailo_pci
hailortcli scan
hailortcli fw-control identify

A healthy 10H reports a firmware version, HAILO10H as the device architecture, and a serial number. If the scan returns nothing, suspect the kernel module or a PCIe lane issue on the host before you suspect the card.

The Dataflow Compiler ships as a Docker image from the Hailo Developer Zone, behind a sign-up wall. A typical conversion from an ONNX model looks like this:

hailo parser onnx yolov8s.onnx \
    --net-name yolov8s --har-path yolov8s.har
hailo optimize yolov8s.har \
    --calib-set-path ./calib_images/ \
    --output-har-path yolov8s_opt.har
hailo compile yolov8s_opt.har \
    --hw-arch hailo10h \
    --output-hef-path yolov8s.hef

The .har file, short for Hailo Archive, lets you tune the quantization config without parsing the graph again each time. Quantization happens in the optimize stage. It reads 64 to 1024 sample images and gives you an INT8 model, or INT4 weight-only for LLMs.

Python inference with HailoRT runs async by design. That is why YOLOv8n hits 400+ FPS on a single stream. The pipeline overlaps the transfer in, the compute, and the transfer out instead of doing them in turn.

from hailo_platform import VDevice, HEF, InferVStreams, \
    ConfigureParams, HailoStreamInterface

hef = HEF("yolov8s.hef")
with VDevice() as target:
    configure_params = ConfigureParams.create_from_hef(
        hef, interface=HailoStreamInterface.PCIe)
    network_group = target.configure(hef, configure_params)[0]
    with InferVStreams(network_group,
                       input_vstream_params,
                       output_vstream_params) as pipeline:
        results = pipeline.infer({"yolov8s/input_layer1": frames})

The Hailo Model Zoo ships ready-made HEFs for YOLOv5/6/8/9/10, YOLOX, CenterNet, RetinaFace, PP-OCR, SAM-tiny, Whisper-tiny, and a growing LLM set. Pulling one of those skips the sign-up dance, as long as you never bring your own model.

Two traps show up when you compile your own model. ONNX opset 17+ with dynamic shapes needs --input-shape forced at parse time, or the compiler rejects the graph. And Resize ops with align_corners=True silently fall back to the CPU, which guts throughput. Export with align_corners=False, or patch the op before you parse.

For full video pipelines, Hailo ships the TAPPAS framework . It provides GStreamer plugins for decode, inference, and overlay chains. Frigate’s Hailo detector sits on the same runtime parts underneath.

Frigate and Home Assistant: real-world FPS and config

Most Home Assistant users buy an AI accelerator to run Frigate security camera monitoring. Frigate 0.14 added first-class Hailo detector support, and it settled down across the 0.15 and 0.16 lines. The Frigate detector docs cover Hailo-8 and Hailo-8L. The same config works for the 10H once the runtime sees the newer card.

A minimal config.yml block for a Hailo-10H with YOLOv8s at 640x640 looks like:

detectors:
  hailo:
    type: hailo8l
    device: PCIe

model:
  width: 640
  height: 640
  input_tensor: nhwc
  input_pixel_format: rgb
  input_dtype: int
  model_type: yolo-generic
  path: /config/model_cache/hailo/yolov8s.hef
  labelmap_path: /labelmap/coco-80.txt

The detector type still reads hailo8l, and that is not a typo. Frigate kept the old name because the runtime interface is the same. It detects the real hardware and picks defaults to suit.

The Docker side is simple once the host is ready. Load hailo_pci on the host, mount /dev/hailo0 into the container, and pull the Hailo build of the Frigate image (ghcr.io/blakeblackshear/frigate:stable-h8l, and watch release notes for a dedicated 10H tag). Model files land under /config/model_cache/hailo/ on first run.

Throughput is where the 10H earns its price. Real-world numbers on a Pi 5 with the HAT+ 2 come out roughly like this:

A typical Frigate setup with eight 4K cameras at 5 FPS motion detection needs about 40 inferences per second in total. One Hailo-10H leaves roughly 95 percent of its budget free, so you can add a second classifier, license plate OCR, or a sharper primary model. The old “stack two Corals” trick adds up to 8 TOPS shared across two PCIe lanes, and those pairs often hit firmware quirks that a single Hailo avoids.

One caveat: the Hailo detector expects a set YOLO output layout. Export custom-trained YOLOv8 models with NMS fused into the graph. Skip that and the detector runs but draws no boxes. Model Zoo HEFs ship with NMS already fused, so retrain against a Model Zoo shape rather than bring a plain ultralytics export.

Routing Frigate events into Home Assistant works the way it always has. MQTT events flow into binary_sensor.frigate_* entities. The new twist with the 10H is that you can pair object detection with a local scene description model running on the same card.

Generative AI on the edge: small LLMs on 40 TOPS INT4

Hailo’s 2025 pivot was adding generative AI support. By 2026 their Model Zoo carries compiled HEFs for LLMs under 10B parameters. Results on a Pi 5 with the Hailo-10H 2280 land in the middle. The card is great for vision, fine for small LLMs, and nowhere near a modern GPU for anything bigger.

Supported LLMs include Llama-3.2-1B and 3B, Phi-3-mini (3.8B), Qwen2.5-3B, Qwen2-1.5B, and an experimental Llama-3.1-8B. They run with INT4 weight-only quantization plus INT8 activations and KV cache, using Hailo’s QuaROT + GPTQ pipeline .

Memory math sets the real limit. 8 billion parameters at 0.5 bytes each is 4 GB of weights. That leaves roughly 3.5 GB of the 8 GB LPDDR4 for KV cache, or about 2k of context. You run out of context window before you run out of parameters. Qwen2-1.5B fits in 1.2 GB total with a 2048-token KV cache.

Token rates from third-party reviews on the Pi 5 AI HAT+ 2 will let you down if you expect GPU numbers:

The awkward number in the CNX-Software AI HAT+ 2 review is that the CM5’s CPU matches or beats the Hailo-10H on DeepSeek-R1 1.5B at 9.0 tok/s. The cause is memory bandwidth. 2.5 GB/s is tight for generation that reads the whole weight matrix for every token. The Hailo still wins on power, roughly 7.5W for the system against 10.5W CPU-only, and it frees up CPU cycles. On very small models, though, it gives you no raw speed gain.

The bandwidth ceiling bites hardest on the wait for the first token. A 1024-token prompt takes several seconds to prefill on 8B-class models, so live chat feels sluggish. The good jobs are the ones where a delay is fine: offline intent parsing for Home Assistant voice assistants, local RAG over a small document set, image captioning for Frigate snapshots, and Whisper-tiny transcription. Coding assistants, long-context summaries, and anything that needs FP16 accuracy will frustrate you.

The 10H pays you back on power. An always-on box running a 3B model at 2.1W average draw is a different animal from a mini PC burning 25W for the same job. For a battery-backed voice assistant or a 24/7 Home Assistant scene describer, the Hailo-10H sits in a tier Coral cannot reach.

Who should buy one

Buy a Hailo-10H if you run Frigate with four or more 4K cameras, if you want YOLOv8m or YOLOv9 headroom instead of cutting down to YOLOv5n, or if you want a taste of on-device small-LLM work without paying for a Jetson Orin. The Pi 5 AI HAT+ 2 at $130 is the cleanest entry point. Bare M.2 modules suit you better if you already own an N100 mini PC with a free slot.

A plain Hailo-8 is the better buy when your workload fits inside 26 TOPS, you won’t touch LLMs, and you want to keep the single-sided 2230 body. A Coral still wins on tight budgets with one or two cameras at 480p motion detection. It is the cheapest path to good enough at that size.

Look at a Jetson Orin Nano Super instead if you need to train or fine-tune on the edge, if you want CUDA, or if your models are odd enough that Hailo’s compiler will push half of them to the CPU. The 10H is a specialist for vision and small-LLM inference. The Jetson is the generalist that wins when you cannot guess the workload up front.

The 10H is the most interesting edge accelerator to land since the original Coral. It is the first card that makes Coral’s decade-long efficiency lead look beatable by someone other than Nvidia. It still will not replace a GPU or drive a local Copilot. If you need to host a 70B model at home, a Strix Halo mini PC with 128GB of unified memory is the box for that job. The 10H will run a stack of vision pipelines and a small LLM side by side on a 5W budget, which nothing else in its class can do.