Fine-Tuning

OpenAI’s Whisper is one of the best open-source speech models around. Out of the box, whisper-large-v3-turbo hits about 8% word error rate (WER) on general English tests like LibriSpeech. But point it at radiology reports, esports commentary, court audio, or factory SOPs and that number can spike to 30-50%. The model just hasn’t seen enough of those niche terms in training.

You can fix this. Fine-tuning Whisper on a small set of domain audio, as little as one to three hours, with LoRA adapters cuts domain-term WER by 30-60%. The full training run fits on a single consumer GPU with 12-16 GB of VRAM. It takes a couple of hours and yields an adapter file under 100 MB. Below is the full path from data prep to deployment.

Gemma 4 shipped on April 2, 2026 with four model variants under the Apache 2.0 license. The 31B dense model ranks third on the Arena AI text leaderboard with a score of 1452. The 26B MoE model scores 1441 while firing only 3.8B of its 26B total parameters per forward pass. So what design choices make this possible? Three of them break from the standard transformer recipe: Per-Layer Embeddings (PLE), Shared KV Cache, and Dual RoPE. Each one shifts the math for inference cost, memory use, and fine-tuning. The rest of this post covers those three, plus the Mixture-of-Experts layer and the multimodal encoders.

Google’s Gemma 4 family covers the 2.3B E2B, 4.5B E4B, 26B MoE, and 31B dense variants. It delivers strong open-weight performance across text, vision, and audio. But general-purpose models still struggle with narrow tasks. You often need a fixed output format, special terms, or facts that weren’t in the training data. Fine-tuning fixes this. Unsloth makes it work on a single consumer GPU. Its custom CUDA kernels cut VRAM by up to 60% and double training speed next to a standard Hugging Face plus PEFT setup.

You can clone your own voice with Coqui TTS using just 5 minutes of recorded audio, all on your own hardware. The steps are simple. Record clean audio. Turn it into a training set. Fine-tune an XTTS v2 or VITS model. Export the result for real-time use. On a modern GPU like the RTX 5070 with 12 GB of VRAM, fine-tuning takes 2 to 4 hours. The output sounds natural and keeps the target voice’s timbre, pacing, and accent.

The best way to fine-tune Stable Diffusion XL 2.0 is with Low-Rank Adaptation (LoRA) : a small adapter that injects your style or subject without touching the base weights. Instead of retraining the full model, LoRA trains a tiny side network next to the frozen base. The result is a 50 to 300 MB file you can load, swap, and stack at inference, trained on a 12 GB GPU in an afternoon.