Mini-AGI is a byte-level continual learning language model designed to train from scratch on hardware with as little as 8 GB of VRAM. To overcome memory constraints, the architecture utilizes a disk-paging mechanism where weights are stored as ordinary files and only a "working set" of experts is resident in VRAM at any given time. The model employs a recurrent architecture consisting of two dense prelude blocks followed by up to 24 recurrent applications, each utilizing a top-8 soft routing mechanism to select experts from a shared pool. To enable adaptive computation, it incorporates a PonderNet-inspired halting head that terminates processing once further iterations fail to change the output. To prevent catastrophic forgetting during single-stream learning, the system uses a specialized "trunk" (embeddings, attention, routers, and halting head) trained at 0.1x the learning rate of the experts. In testing with a 1.04-million-character PG19 domain shift, this mitigated configuration achieved a 97.30% retention rate of previous knowledge, significantly outperforming unmitigated settings. The model dynamically grows its capacity by recombining existing hidden units into new experts and prunes unused ones to manage disk space. Current benchmarks show a held-out loss of 0.7644 nats at 483.4M characters, with a perplexity of 2.15 bits/char.