Jev: when AI only makes decisions, and doesn't write

TypeSafe AI launched Jev, a model that returns only typed decisions with probabilities and generates no text. How it works, and the open-source alternatives.

Almost every “new AI” since 2022 has been a variant of the LLM: it generates text one token at a time. Great for writing, but wasteful and slow when all your software needs is a decision: is this message a refund request, which department does this ticket belong to, how severe is this bug.

On 15 September 2026, TypeSafe AI (founder Diogo Almeida, a former OpenAI engineer who worked on RLHF/ChatGPT/GPT-4) launched Jev, the first model in a class they call “System One Model”, along with a $40M seed round led by DCVC. Jev doesn’t write. It only answers questions that have a predefined set of answers, with a probability attached.

How it works

The easiest way to picture it: Jev is a smart if. You define the whole answer space up front (a set of labels, a scale, or yes/no), and Jev only picks from that set, which means it never answers with the wrong data type.

A regular LLM:  input text → free-form text → your code parses it
Jev:            state + typed question → a value of the right type + probability

Parallel sampling, not autoregressive

This is the most important architectural difference. A regular LLM generates one token at a time, each depending on the one before; an N-token answer takes N sequential inference steps. Jev uses a parallel sampler: it produces the whole answer in a single pass. Send 13 questions that share one state, and Jev answers all 13 in parallel, without one answer becoming context for another. TypeSafe measured this as 10x faster and 12.2x cheaper than 13 sequential calls.

RLCD instead of RLHF

Regular LLMs are trained with RLHF, which optimises for answers that sound reasonable to people. Jev is trained with RLCD (Reinforcement Learning for Calibrated Decisions), which optimises for probabilities that are statistically correct: if Jev says “90% confident this is true”, then across a large set of predictions labelled 90%, about 90% of them should actually be true.

That is the core idea of the whole product. Everything else (the three question types, using confidence for routing, the limitations) follows from “the probability must be trustworthy”, not “the answer must sound good”.

Three question types

TypeReturnsExample
Noul (yes/no)1 probability, 0–1{"type":"noul","instructions":"Is the customer asking for a refund?"} → {"noul": 0.93}
Choice (pick 1 of N)label + probability distribution + confidence{"type":"choice","criteria":{"billing":"Billing","technical":"Technical issue"}} → {"choice":"billing","confidence":0.84}
Score (ordered scale)position on the scale, a weighted average{"type":"score","criteria":["Cosmetic","Broken with workaround","Blocking"]} → {"score":1.43,"confidence":0.35}
const { answers } = await client.systemOne({
  state: userMessage,
  questions: {
    category: choice('Classify this message', {
      bug: 'Bug report',
      billing: 'Billing issue'
    })
  }
})

if (answers.category.confidence < 0.5) {
  routeToHuman(userMessage)
} else if (answers.category.choice === 'bug') {
  handleBugReport()
}

A common pattern: combine several Scores into an overall priority right in your code, without any extra AI call:

const priority =
  0.6 * normalized(answers, 'severity') +
  0.3 * normalized(answers, 'frustration')

Performance, cost, use cases

  • Latency: 70–500ms end-to-end, ~100ms on average
  • Price: $0.042 per million input tokens; output is free
  • Rate limit: 250,000 tokens/second, 1,200 requests/minute
  • Compared with an equivalent LLM: 40x–200x faster, 5x–240x cheaper for classification tasks

Notable real-world use cases: ticket routing/triage, content moderation, large-scale data labeling (1,018 research papers labelled for $0.08), verification (using Jev as a “referee” that checks another LLM’s output), search re-ranking, fraud scoring, and, most interesting for anyone building a coding agent, guardrails: classifying a shell command as read-only / reversible / destructive before the agent runs it.

Where Jev is weak on purpose

Because it trades free-form text generation for speed and calibration, Jev is unreliable at: math and counting occurrences, dates in mixed formats, double negatives, long noisy context, and manipulative (adversarial) content. The official advice: when you want to “extract X”, rephrase it as “here are the candidates for X, which one is right?”, always turning it back into a choice from a finite set.

A wave of open-source alternatives within days

The most interesting part isn’t Jev itself, but how fast the open-source community reacted. Less than two weeks after launch, a dozen projects with public weights were doing the same thing (“typed, calibrated decisions in one forward pass”) without being packaged as a paid API.

ProjectSizeWhat you need to host itNotes
Laya322M–421MRuns fine CPU-only; on GPU, CUDA/Apple MPS/Intel XPU all work. Light, no big VRAM needed. Install with pip install layaModernBERT-large + mmBERT, 100+ languages
Kev0.8B–27B0.8B: ~4GB VRAM (an L4 GPU or any Mac). 4B/9B: ~17GB (L40S, H100, or a 32GB Mac). 27B: 55–66GB of weights, needs an 80GB GPU (H100/H200/B200)Qwen + LoRA + pointer head, implements /v1/systemone directly, compatible with the TypeSafe SDK
Nimble9B~18GB VRAM at BF16. Needs a CUDA GPU with BF16 support, or an Apple Silicon Mac (MLX). The adapter merge step runs on CPU, so more RAM is better (≥64GB recommended)Qwen3.5-9B + rank-16 LoRA, reads candidate token logits instead of generating
NanoJev0.6BNeeds a CUDA environment (the README doesn’t confirm CPU-only), but at 0.6B a consumer GPU is enoughQwen3-0.6B + decision heads, with a full training pipeline to retrain it yourself
SemIfdepends on the base modelNot fixed. With the Qwen3.5-4B baseline you need ~8GB VRAM (BF16); or run CPU-only through llama.cpp with a GGUF checkpoint (Q8_0/Q4_K_M)Reads logits directly from an existing open-weight LLM, no fine-tuning needed
Von~395MThe lightest on the list. CPU (via OpenVINO) is practical (p50 ~0.36s/request); a cheap GPU like an A10G brings it down to ~23ms. Also supports CUDA, ROCm, Apple MPSModernBERT + OptionMarker, picks the device itself with --device auto
Rizzo Flow1.7B–4B4B at Q8_0 (default): ~5.6GB VRAM; Q4_K_M: ~3.9GB. Runs through llama.cpp, so no separate GPU toolkit to install; picks Metal/CUDA/Vulkan/ROCm/CPU itselfSpark-X2.5, local-first, caches state between questions

The most notable one is open-alternative-jev (Apache-2.0): a Python library, so1, with almost exactly Jev’s mechanism, but running on any open-weights model (Qwen2.5/3.5/3.6, via HF Transformers or vLLM):

  1. Render each question as one chat turn
  2. Pack several questions into one token sequence, with placeholder answers between them
  3. Run a single forward pass, reading the logits only at the answer positions
  4. Softmax-normalise the logits of the labels (A, B, C…), so the probabilities always stay within the set you defined
  5. Optional temperature scaling to calibrate the probabilities

What you need to host so1: it depends entirely on the base model you pick; the library only reads logits and adds no significant overhead. The benchmark below uses Qwen3.6-27B with load_in_8bit=True (via bitsandbytes), which needs about 27GB of VRAM and fits on one 32GB GPU. For something lighter, switch to Qwen2.5-1.5B or 3B, which runs even on CPU (slower) or on a consumer 8–12GB GPU.

from so1 import Decider, Choice, yes_no

decider = Decider.from_pretrained("Qwen/Qwen3.6-27B", backend="hf", load_in_8bit=True)
decisions = decider.decide(
    state="Support ticket describing an issue...",
    questions=[
        Choice("Category", ["bug", "feature", "billing", "question"]),
        Choice("Urgency", ["low", "medium", "high"]),
        yes_no("Escalate?"),
    ],
)

The benchmark the author published on the typed-decisions set (400 cases, 2,000 decisions) is even more interesting: Qwen3.6-27B through this library scored 73.7% accuracy, KL-to-gold 0.27, ECE 0.020, 582ms/case, against Jev 1.13.0 at 72.7% accuracy, KL 1.44, ECE 0.144, 710ms/case. So the open-source version, zero-shot and without any special fine-tuning, came out slightly ahead on both accuracy and probability calibration (lower ECE = more trustworthy probability estimates), and was faster.

My take

The core concept to remember: Jev isn’t a “smaller/faster LLM”. It’s a different class of model with a different training goal. An LLM is optimised to sound reasonable; Jev is optimised for statistically correct probabilities, at the price of giving up free-form text generation entirely.

But because the core mechanism (a single forward pass, reading logits at the answer position, softmax over a fixed label set) is no secret, and it doesn’t need a model trained from scratch, only reading the right logits from an existing open-weight LLM, the technical barrier to cloning it is close to zero. That’s why it took less than two weeks to go from “TypeSafe launches a proprietary product” to “the community has an open-source version that beats it on both accuracy and calibration”.

For a real project: if your system uses GPT-4/Claude only for tasks with a fixed, known set of answers (spam/not spam, which department a ticket goes to, how severe a bug is), you’re paying for a text-generation machine to do a classifier’s job. There’s now both a paid API (Jev) and free self-hosted libraries (open-alternative-jev and similar) to split that part into its own layer, much cheaper and faster, running in front of the “heavy” LLM layer, and calling the LLM only when you actually need to generate content.

Related posts

2026-11-15

AI for office work: 7 things you can do today, no coding needed

How to use ChatGPT, Gemini, Claude or Copilot at work: emails, document summaries, Excel formulas, meeting minutes. Copy-ready prompts included.

#ai#chatgpt#office#productivity
2026-05-07

RAG: when an LLM isn't smart enough on its own

What RAG is, why you need it, and the real problems when you build it: chunking, embedding drift, model swaps, latency. Lessons, not a tutorial.

#ai#rag#llm#backend