Quantization

The best open-weight coding model you can run on a 24 GB GPU in 2026 is Qwen3.6-27B at Q4. It scores 77.2 on SWE-bench Verified while fitting in about 17 GB, the highest coding skill per gigabyte you can actually load at home. DeepSeek V4 wins the leaderboard, but no consumer card can hold it.

Why Capability Per GB Beats the Leaderboard

Most 2026 roundups rank coding models by the score of a flagship variant that needs a multi-GPU server. For anyone running models at home, that number is a fantasy. The only figure that counts is how much coding skill fits in the VRAM you actually own.

DeepSeek V4 is a 1.6 trillion parameter open-weight Mixture-of-Experts model. It reads 1M tokens at once. It uses 27% of V3.2’s inference FLOPs and 10% of its KV cache. The DeepSeek V4 tech report credits three moves: hybrid CSA plus HCA attention, Manifold-Constrained Hyper-Connections, and the Muon optimizer in place of AdamW.

DeepSeek V4 at a Glance

The official launch announcement on April 24, 2026 framed the release as “the era of cost-effective 1M context length.” It shipped two checkpoints under the MIT license. DeepSeek-V4-Pro runs at 1.6T total and 49B active parameters. DeepSeek-V4-Flash runs at 284B total and 13B active. Both models read 1M tokens at once. Both ship as open weights on Hugging Face . The routed expert weights use FP4 math, and most other weights use FP8.

You can run DeepSeek R1 ’s distilled reasoning models on an RTX 5080 with 16 GB of VRAM. Use Ollama or llama.cpp with 4-bit quantization. The 14B distilled variant (Q4_K_M) fits in about 10 GB of VRAM. It shows visible <think> reasoning traces that rival cloud quality on math, coding, and logic. The full 671B model needs multi-GPU rigs, but the distilled models give you 80-90% of the quality for far less hardware.

Qwen 2.5 Coder 7B is the most accurate of the three for Python and TypeScript completions. Phi-4 Mini (3.8B) uses the least VRAM and runs nearly twice as fast. Pick it when memory or latency counts more than raw accuracy. Gemma 3 4B sits in the middle. It is the best choice when you need one model for code, commit messages, docs, and error explanations. Below are the benchmark numbers, the test method, and how to set up each model in VS Code or Neovim.

MiniMax M2.7 , released in April 2026, is a 230B-parameter open-weights reasoning model (Mixture-of-Experts, 10B active, 8 of 256 experts routed per token) that scores 50 on the Artificial Analysis Intelligence Index. That lands it on par with Sonnet 4.6 across coding and agent benchmarks and within a couple of points of Claude Opus 4.6. Weights are on HuggingFace at MiniMaxAI/MiniMax-M2.7 , the hosted API runs $0.30 / $1.20 per million input/output tokens (roughly a tenth of Opus), and if you have a 128GB-unified-memory Mac Studio, an AMD Strix Halo box, or an NVIDIA DGX Spark , you can run it offline with zero token bills. Two big asterisks: the M2.7 license is not the permissive M2.5 license (commercial use is restricted), and there is no multimodal support. For homelabbers and agent builders who are text-only and non-commercial, M2.7 is the best locally runnable Opus-class option shipped so far.

The short answer is no, the Gemma 4 26B MoE model will not fit entirely in 8 GB of VRAM at standard Q4_K_M quantization - the weights alone require roughly 16-18 GB. But with the right approach, you can run it on budget hardware and get usable interactive performance. The three practical strategies are aggressive quantization (IQ3_XS brings weights under 10 GB), GPU-CPU layer offloading (split 15-20 of 30 layers to GPU, rest on system RAM), and multi-GPU setups (two cheap 8 GB cards via tensor parallelism). Each involves different trade-offs between quality, speed, and hardware requirements.