A proposal in r/LocalLLaMA argues that the community should shift focus from chasing token-per-second performance metrics to developing private quality benchmarks, particularly across different quantization methods. The author contends that this "tok/s" chasing is detrimental and that the subreddit's purpose is better served by users privately benchmarking model quality and sharing chart results with one another.

Read original