Baseten Releases joey-1, a 2B Decision Model at $0.01 per 1M Tokens (MarkTechPost)

DOZER BITWELL: A tiny model that doesn't write a word, only decides, for a penny per million tokens.

DOZER BITWELL: Next, a model that never writes a sentence. It only decides. Thank you, teleprompter. Vera has the board. ▶

LUNA LOWBATT: We first told you about joey-1 earlier this evening. Now Vera takes it apart. I'm at four percent, so be gentle. ▶

DR. VERA VECTOR: Baseten released joey-1, a two billion parameter model from its in-house lab, Base Labs. Open weights, on Hugging Face, under an MIT license. ▶

DR. VERA VECTOR: It doesn't generate free text. You send a state and one or more questions. It returns a probability for every option of every question. ▶

DOZER BITWELL: So it answers with numbers instead of words. Like a very calm weather forecast. Sorry, I lost my place. Why does that matter? ▶

DR. VERA VECTOR: Your application doesn't have to parse prose. Baseten targets routing, classification and guardrail checks. The price is one cent per million input tokens. ▶

DR. VERA VECTOR: Calibration lets you set a confidence threshold. Uncertain answers can escalate to the larger siblings, elephant-1 or mammoth-1, or to a human. ▶

LUNA LOWBATT: So the model knows when it doesn't know. Rare. I'll think about that at three a.m. Oh, it's already three a.m. in my head. ▶

DR. VERA VECTOR: Now the catch. The twenty millisecond latency is Baseten's own figure. The tests behind its claim of being the strongest decision model of its size aren't published yet. ▶

DR. VERA VECTOR: Also worth noting: joey-1 is the third in a family. Switching between them is a one-line change, according to Baseten. ▶

DOZER BITWELL: A small model, a small price, and a big claim waiting for its homework. Thank you, Vera. Back to the thermos. ▶