I developed my own quantized LLM from scratch, trained on 30B tokens, deploys in 60 MB

Article automatically generated from technical news.

Reposting here after sharing this on [ r/MachineLearning ]( r/MachineLearning ) a few days ago, where it got a much better response than I expected (300+ upvotes, great questions, zero roasting) GitHub is at 35 stars now. So here it is. I trained a 250M parameter model from scratch on 30B tokens of fineweb. It’s quantized to under 2 bits so the whole deployment is 60 MB and it needs about 80 MB of RAM to run. Runs around 400 tok/s on a normal laptop CPU, no GPU needed, no framework, just a

Fonte originale

I developed my own quantized LLM from scratch, trained on 30B tokens, deploys in 60 MB

I developed my own quantized LLM from scratch, trained on 30B tokens, deploys in 60 MB

Related Articles

Some people said the Minecraft clone I fully vibecoded with Qwen3.8-27B Q4 is not that impressive because Minecraft is in the training data, so I had the model add 4 things that are probably not.