Meta researchers trained small language models (~1B parameters) to process raw byte sequences instead of tokenized input, using knowledge distillation from Llama 3 8B as a teacher model. The core challenge involved aligning the teacher's token-level representations with the student's byte-level processing through a learned translation mechanism. While byte-level models initially underperform token-based counterparts, the paper predicts they ultimately achieve superior scaling and generalization.