The author built a human‑token‑per‑second (TPS) counter to measure how fast they can manually type tokens, achieving about 2 TPS. This performance exceeds that of a 70 B parameter model running on a laptop CPU but is roughly 76× slower than an 8 B model on an RTX 4090. They invite others to try the counter and share results via the provided web page.
Read original
reddit/r/LocalLLaMA