Llm

Prompt caching lets you skip re-processing identical prefix tokens across LLM API calls, cutting costs by up to 90% and reducing latency by 50-80% on requests that share long system prompts, few-shot examples, or document context. Anthropic’s Claude offers prompt caching with explicit cache_control breakpoints, OpenAI’s GPT-4o supports automatic prefix caching, and local inference servers like vLLM and SGLang implement prefix caching natively. The rule: put your static, reusable prompt content first and the variable user query last.

Aider is the open-source AI pair programming tool that shipped before Claude Code , Codex CLI , and Gemini CLI . It is still the only major AI coding assistant that lets you pick whichever language model you want. Claude, GPT-5, Gemini, DeepSeek, Grok, a local model through Ollama : Aider connects to all of them. The project sits at 42K GitHub stars, 5.7 million pip installs, and 15 billion tokens per week. It ships under Apache 2.0, so the tool itself costs nothing. You only pay for API tokens at provider rates, which runs $30 to $60 per month for most developers.

You can build a multi-modal RAG pipeline that searches text, diagrams, and screenshots at once. The trick is to mix CLIP-based image embeddings with text embeddings in one shared vector space. Store them in a ChromaDB or Qdrant collection. Route queries through a retrieval layer that returns both passages and images. Feed it all to an LLM. With OpenCLIP ViT-G/14 for images plus a self-hosted Llama 4 Scout as the LLM, the whole pipeline runs offline on an RTX 5070 or better.

For most local AI workloads in 2026, the RTX 5080 with 16 GB of GDDR7 is the better buy. It delivers 40-60 tokens per second on quantized 7B-13B parameter models at roughly half the price of the RTX 5090. The RTX 5090’s 32 GB of GDDR7 only justifies the premium if you regularly run 30B+ parameter models or full-precision fine-tuning jobs that cannot fit in 16 GB of VRAM. If either of those describes you, the 5090 earns its keep. If not, you are paying $1,000 extra for headroom you will not use.

Gemma 4 shipped on April 2, 2026 with four model variants under the Apache 2.0 license. The 31B dense model ranks third on the Arena AI text leaderboard with a score of 1452. The 26B MoE model scores 1441 while firing only 3.8B of its 26B total parameters per forward pass. So what design choices make this possible? Three of them break from the standard transformer recipe: Per-Layer Embeddings (PLE), Shared KV Cache, and Dual RoPE. Each one shifts the math for inference cost, memory use, and fine-tuning. The rest of this post covers those three, plus the Mixture-of-Experts layer and the multimodal encoders.

Google’s Gemma 4 is not one model - it is a family of four, each targeting different hardware and different use cases. The smallest runs on a Raspberry Pi. The largest ranks #3 on LMArena across all models, open and closed. All four ship under the Apache 2.0 license, a first for the Gemma family. This guide walks through installing each variant with Ollama (currently at v0.20.2), benchmarks them on real consumer hardware, and helps you decide which one fits your setup.