The author trained a 102 M‑parameter recursive BitNet‑v2 model from scratch, employing ternary weights, shared transformer layers, and hashed n‑gram embeddings. The model handles a 64K token context and was trained on less than 5 billion tokens with a budget of about €100. Weights, inference code, model card, and evaluation results are hosted on Hugging Face at the provided link.
Read original
reddit/r/LocalLLaMA