Basal 2.2 decision model learns from your own data

The Basal 2.2 decision model learns from your own data. It picks an answer from your list and gives it a confidence score. Version 2.2 retunes those scores on your own examples, on your computer. On JevBench, its largest model scores best among open models in Jev’s price range, but drops to third once speed and price count.

What is the Basal decision model?

Basal is an open decision engine by Remigiusz “Remek” Kinas, with code on GitHub and models on Hugging Face. It never writes text. You give it an input, called the state, such as an email, a contract or a web page as JSON. You add a question and up to 255 allowed answers. Basal returns a probability for each answer in a single step.

For example, it can read a customer email, send it to complaints, payments or service, and flag whether it’s urgent. The answer is always one of your options, with a confidence score attached.

Basal handles six types of question:

• choice picks one option from a list.
• noul answers yes or no.
• score places an input on a scale, such as urgency from 0 to 3.
• multi, act and tool pick several labels, the cheapest action from a table of costs, or an AI agent’s next tool.

Basal is an open alternative to TypeSafe’s closed Jev, which we covered in the Jev model and what TypeSafe won’t disclose. It works in Polish and English. It was trained on about 640,000 example decisions, from contracts and deadlines to routing and agents’ tool choices.

Earlier versions were basal-1.0, which has a technical report on Zenodo, and basal-1.5. The current basal-2.0 family builds on Polish Bielik v3 models. A separate basal-omni line builds on Google’s Gemma 4 and also accepts images.

Basal 2.2 calibrates the decision model on your own data

A decision model handles the answers it’s sure about and sends the rest to a person. That cut-off, the threshold, only works if stated confidence matches real accuracy. If the model says “95% sure” but is right 80% of the time, bad decisions get through.

Basal ships with a calibration file that turns raw model output into confidence numbers. Kinas tuned it on his own data. On your data, the model can be just as accurate, but its confidence scores can be off. Kinas announced the new tool on X, in Polish:

Dziś wydaję silnik basal 2.2 🇵🇱 z autokalibracją - model sam uczy się na Waszych prywatnych danych 🤯

Model decyzyjny jest użyteczny tylko wtedy, gdy „90% pewności” znaczy 90% na Twoich danych. Kalibrację dostarczaną z modelami dopasowaliśmy na naszych danych. Na Twoich model… pic.twitter.com/KeWZTjcsCT

— Remek Kinas (@KinasRemek) October 19, 2026

What basal-calibrate does

The new basal-calibrate tool doesn’t train the model. You label a few hundred of your own requests with the right answers, and the tool writes a calibration file fitted to them. Then you start the server with that file:

basal-calibrate my_labelled.jsonl --model Remek/basal-omni-1.0-max --mode eager
basal-serve --model Remek/basal-omni-1.0-max --mode eager --calibration CALIBRATION.custom.json

Your data stays on your machine, and the model itself doesn’t change. A few dozen labels already shift the settings. The --report-only flag shows how the shipped file does on your data before you replace it. A hosted decision API offers no such step.

Results on data the model never saw

Kinas tested the tool with the small basal-omni-1.0-mini on a public dataset it never trained on, fitting on one part and testing on another. The shipped file’s strictest setting promises at most 1% wrong answers. On this data, it would have let the model decide about one case in eight alone, and 8% of those would have been wrong, eight times the promised rate.

After calibration, calibration error on choice questions fell from 0.268 to 0.083. Calibration error is the average gap between stated confidence and real accuracy, so it shrank from about 27 percentage points to 8. It halved for score questions and fell about 30% for yes/no.

With the new file, the model decides far less alone, and that’s intended. The small model is often wrong on this data, so at the 1% target every case goes to a person. At 5%, it decides 3.7% of cases alone, all correctly, where the shipped file decided 54.3% and got one in five wrong. Checking claims was the exception. The model is 95% accurate there and decided 54.8% of claims alone at 1%, with no errors.

What calibration can’t fix

A thread in r/LocalLLaMA, a Reddit forum on running AI locally, looked at H2O-Lightning-4B, a rival decision model. u/retumbler pointed out a limit of methods like temperature scaling. These push all confidence numbers up or down, which moves where your threshold falls. They can’t help the model tell its right answers from its wrong ones.

The H2O developer agreed that the gap to a larger model was “separation, not scale.” The larger model was better at giving right answers more confidence than wrong ones. On a measure of this called AUC, where 1.0 is perfect, the 4B model scored 0.965 and the larger one 0.992. The same limit applies to Basal. basal-calibrate makes its confidence match real accuracy on your data, but it can’t make the model better at telling right from wrong.

Basal 2.2 runs on Macs and smaller GPUs

Version 2.2 adds compressed, text-only basal-omni versions that need less memory:

FP8 and GGUF cover basal-omni-1.0-mini and max. MLX covers those two plus the new basal-omni-1.1-max, an experimental 12B model for tasks like checking claims and extracting data.

Kinas compared each FP8 and GGUF version with the full-size bf16 original. Over 10,606 test decisions on one RTX PRO 6000 card, they gave the same answer 96% to 99% of the time. Accuracy changed by at most 0.35 points.

The text-only basal-2.0 models already came in all these formats, in three sizes: mini at 1.5 billion parameters (1.5B), 4.5B and max at 11B.

Usage is still small. Last month, basal-2.0-mini had 441 downloads, the 4.5B model 423 and max 331.

Did Basal top the JevBench leaderboard?

Two days before 2.2, Kinas announced the 31B basal-omni-1.0-ultra on X, calling it “TOP1 in the world” among open models and second among all models, open and closed. He also said it beat systems from OpenAI, Microsoft and TypeSafe’s own Jev. The post drew 265,000 views and 3,700 likes.

How JevBench scores a decision model

JevBench is an independent leaderboard from Benchmark Heaven, not TypeSafe. Each system answers 1,500 test decisions, and 184 open models are ranked. Intelligence measures how well a model picks the right answer. Calibration measures how well its confidence matches reality. Capability averages the two. Its ranking only includes models that cost at most twice as much as Jev and are at most twice as slow.

On this ranking, the claim is correct. Basal beats the closed Jev 1.13.0 by 5.4 points, almost all on Intelligence (74.1 against 63.6), with Calibration nearly equal. Three open models outside the price limit score slightly higher but cost 6 to 17 times as much as Jev.

OpenAI’s and Microsoft’s hosted services are on a separate board, where OpenAI Decisions scores 73.5 on Capability and Microsoft-Decision-1 about 70.8, both below Basal.

Quyet-1.0-Large, another Gemma 4 31B model by a different author, is only 0.8 points behind. On yes/no questions, Basal gave a firm yes or no 92% of the time and was right on 94% of those.

Why Basal ranks third on the official score

JevBench’s official ranking weighs four scores equally: Intelligence, Calibration, Speed and Cost. A low score on any one of them pulls the total down hard, and Capability plays no part. On this ranking, Basal is third:

Two 4B models win because they cost half as much per decision. Basal’s estimated $0.045 per 1,000 decisions is 1.38 times Jev’s price, within the board’s limit but enough to lower its Cost score. At a median 0.102 seconds per decision on the board’s GPU, Basal is fast, but the 4B models score a little higher on Speed.

The “TOP2 among closed and open” claim refers to a combined board of open and hosted models. The hosted model kettei-fat ranks first there. Ranks also move: when the board first listed Basal, in release 1.6.28, it was fourth of 45 on both rankings. The basal-2.0 models aren’t on JevBench at all.

What is basal-omni-1.0-ultra?

Ultra is built on Google’s Gemma 4 31B. Kinas trained it with LoRA, which trains a small set of extra weights instead of the whole model, then merged them in. Training stopped at 10,000 of 12,741 planned steps, when the test score stopped improving at 0.909. The earlier basal-1.5 supplied extra training answers.

Ultra came out five days after the smaller omni models. It accepts images, but only through Gemma’s built-in image reader, which Kinas never trained. It rejects audio, which the smaller omni models accept.

Kinas compared it with the 12B basal-omni-1.0-max on his own tests (higher is better):

Werdykt is his private Polish and English benchmark. Ultra is weakest on rules (0.550), contracts (0.822) and long documents (0.854). Its only English task below max is FinancialPhraseBank, at 0.947 against 0.967.

On one H200 GPU, ultra answers in 80 ms at the median and within 188 ms 95% of the time. With 32 requests at once, it handles 8.6 decisions per second. The full-size bf16 weights need about 60 GB. An 80 GB GPU handles inputs up to about 10,000 tokens (word pieces), and an H200 about 32,000. A text-only FP8 version cuts the weights to 33 GB and runs through vLLM.

How to run Basal on your own GPU

Ultra needs a data-center GPU. On a gaming GPU, start with the 4.5B basal-2.0 model or the 6.2 GB Q6_K GGUF version of basal-omni-1.0-mini. On an NVIDIA card, install PyTorch for CUDA 12.8, then the Basal 2.2 engine:

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.2.0.tar.gz"
basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --port 8000

The first start in fast mode compiles the model, which takes several minutes. On an Apple Silicon Mac, the MLX version skips CUDA:

uv venv --python 3.12 --allow-existing ~/basal-mlx && source ~/basal-mlx/bin/activate
uv pip install "basal[mlx] @ https://github.com/rkinas/basal/archive/refs/tags/v2.2.0.tar.gz"
basal-serve --model Remek/basal-2.0-4.5B-MLX-8bit --mode mlx --port 8000

Both servers answer at /v1/systemone. Send the input as state plus named questions, which are answered together:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Customer: since yesterday I cannot log in to online banking, it says my password is wrong.",
  "questions": {
    "dept":   {"type": "choice", "instructions": "Which team should handle this ticket?",
               "criteria": {"cards": "Card complaints", "online": "Online banking support", "loans": "Loans"}},
    "urgent": {"type": "noul", "instructions": "Is the customer unable to use the service?"}}}'

Each answer gives a probability per option, the chosen option and a confidence value. To send doubtful cases to a person, add "max_error": 0.05, the error rate you accept, to an act or tool question. When confidence is too low for that rate, the answer comes back marked as deferred.

Start at 5% rather than 1%, and run basal-calibrate on your own labels first. For AI agents, install the mcp extra and run basal-serve --mcp on. A coding agent can then call Basal as a tool through the Model Context Protocol.

Where Basal falls short

JevBench tests only decisions. It says nothing about how well Basal writes or reasons.

Kinas calls the basal-omni models, ultra included, experimental, since they were trained on text decisions only. All sizes share these limits:

• Arithmetic and checks like “is this amount over the limit?” are weak spots, so do them in code.
• Option order can change the answer. The --orders 2 setting reduces this but halves the speed.
• Tool questions with no fitting tool usually get “no suitable tool”, but not always.
• Long states fail with an error when the GPU runs out of memory.
• Serious decisions with legal, financial or human stakes still need a person to sign off.

The Apache 2.0 license and self-hosting keep customer data in-house, which suits EU companies wary of US-hosted decision services. Treat the JevBench win as narrow and disputed, and the calibration numbers as one author’s test until someone repeats it on other data.