Baseten has released joey-1, a 2B-parameter decision model post-trained from Qwen3.5-2B by its Base Labs division, offering calibrated probabilities and labels instead of generated text for routing, classification, and guardrail tasks. The model processes text and images with a 16,384-token context window on Model APIs (262k on dedicated deployments) using a block-bidirectional attention mask and shared-prefix prefill to answer multiple questions in a single forward pass. Trained on 6.21M decision examples (58.3% core skills, 16.1% safety, 16.1% judging/expert domains, 9.5% public data) for one epoch over 43,390 steps at a 1e-5 peak learning rate in bf16 on 16 H100 GPUs, the objective combines cross-entropy with a Brier calibration term and a trained "I don't know" option. joey-1 achieves a Jev Decision Index 0.4 public split score of 63.30, ranking #37 of 230 models and leading the sub-4B category, though it trails its 9B sibling elephant-1 (72.24) by 8.94 points. It scores 62.9 on Baseten's Artificial Analysis Decision Index replica versus mammoth-1's 65.9. Baseten reports a P50 end-to-end latency of 20 ms for average 2,000-token requests—approximately 2.5x faster than elephant-1's 49 ms—at $0.01 per million input tokens (output and cached input free), compared to $0.035 for elephant-1 and $0.10 for mammoth-1. MIT-licensed weights are available on Hugging Face for self-hosting on a single CUDA GPU via a PyTorch 2.8/CUDA 12.8 setup, with deployment options including Baseten Model APIs (/v1/systemone), dedicated deployments, and OpenRouter.

Read original