Orthrus is a hybrid autoregressive‑diffusion model that speeds up language‑model inference by generating multiple tokens in parallel with a frozen autoregressive backbone, claiming lossless speculative decoding through an intra‑model consensus mechanism. The authors independently reproduce Orthrus and evaluate this claim under various numerical precisions, including BF16 inference.
Read original
huggingface/daily-papers