Fixing Tool-Call Accuracy in vLLM
hn

Fixing Tool-Call Accuracy in vLLM

u/coffeemug 2026-10-26

Ubicloud addressed tool-call schema accuracy issues when serving open-weight models like Kimi K2.6 via vLLM. Initial testing with vLLM 0.28 yielded only 92% schema adherence, meaning 2 out of every 25 tool calls failed, …

→ View original source
Speculative Diffusion is a Tale of Two Tradeoffs

Speculative Diffusion is a Tale of Two Tradeoffs

Aaron Zorel, Guy Van den Broeck, Aditya Grover, Daniel Israel 2026-10-21

The paper introduces speculative decoding as a method to accelerate autoregressive LLM inference by using a faster draft model, traditionally requiring lossless acceptance relative to a verifier distribution, whereas rec…

→ View original source
Loading more articles...