A Reddit user compared how Qwen 35B A3B and Gemma 26B A4B tokenize the same 330-line HTML/JS code, finding that Qwen tokenized it into 1,609 tokens while Gemma produced 4,258 tokens. This significant difference in tokenization suggests Qwen processes code more efficiently, potentially explaining its stronger performance on coding tasks compared to Gemma's language-focused strengths. The observation highlights how tokenization strategies can influence model behavior across different input types.

Read original