On an RTX 5090 with 32 GB of VRAM, Llama 4.0 70B runs at roughly 28 tokens per second using 4-bit GGUF quantization through llama.cpp or Ollama . Mid-range cards like the RTX 5070 Ti with 16 GB hold around 11 tokens per second on the same model. This guide covers the install, the VRAM math, and the benchmark numbers.

What Is Llama 4.0? Architecture and What Changed

Llama 4.0 marks a real architectural shift, and that shift directly affects VRAM use and speed. The biggest change is a move to a Mixture-of-Experts (MoE) layout, with some variants using a hybrid dense-MoE design. In a dense model like Llama 3, every parameter fires for every token. In a MoE model, the network splits into many “expert” sub-networks, and a routing layer picks only a few of them per token. So a 70 billion parameter Llama 4.0 model might fire just 13 billion of them on any forward pass. The upshot: a 70B Llama 4.0 model often runs at speeds closer to a 13B dense model, while keeping the reasoning depth of a much larger network.