To enable real-time musical autocomplete on mobile hardware like the iPhone 15, a 125M-parameter transformer model was developed to process MIDI data through a specialized tokenization strategy. Moving away from inefficient event-based or grammar-enforced sequences—which often caused "drift" or required up to four transformer passes per note—the architecture utilizes a novel single-note composite token. This approach represents a complete musical note as one unit containing five categorical fields: event_type, pitch_id, delta_id, duration_id, and velocity_id. Each field is assigned an individual embedding, and the final token is the sum of these embeddings. To maintain efficiency, the model employs separate output heads and a small nested decoder that conditions subsequent fields on previously predicted ones, avoiding the heavy computational cost of multiple backbone passes. Additionally, sustain pedal effects are preprocessed directly into note durations to simplify the prediction space. This engineering optimization allows the model to achieve a generation speed of approximately 108 notes per second on a mobile device, providing high-fidelity, real-time performance capabilities far exceeding the requirements for live human interaction.

Read original