Soup (v0.76.0) is a Python CLI tool that simplifies LLM fine-tuning and post-training via a single YAML config and one command, eliminating SSH and complex setup. Built on PyTorch, Transformers, and PEFT, it supports SFT, DPO, GRPO, PPO, KTO, and more, with automatic batch sizing, quantization (QLoRA), and GPU detection. A key feature is layer streaming (opt-in, beta), which trains 8B models on 4GB GPUs by loading frozen base layers from host RAM sequentially. Measured on an RTX 3050 Laptop (4GB): Llama-3.1-8B-Instruct with NF4 achieved 119.6 tok/s at 952 MHz with 3.32GB peak VRAM (v0.72.2, pre-repair); post-repair results on another RTX 3050 showed 208.6 tok/s at 1935 MHz in 2.40GB (not directly comparable). These figures were independently replicated on an H100 at 113.00 tok/s. Layer streaming was verified as bit-exact against resident runs. The paper “Exact Layer Streaming” (Zenodo, v3) details the method, correcting prior claims about bandwidth bottlenecks and documenting a silent gradient defect fixed in v0.73.0, validated against 32B/72B models. New commands include `soup bench`, `soup data clean`, `soup recipes verify`, and `soup ab` (A/B testing). Config validation now rejects unknown or ineffective keys at load time. Installation supports pipx, pip, or Docker, requiring Python 3.10–3.12.

Read original