MixTok introduces a hybrid visual tokenizer that merges a compact 1D global token stream with a set of sparse 2D tokens to refine locally complex image regions. A content‑adaptive router evaluates candidate spatial locations, assigns them a refinement priority score, and selects how many 2D tokens to allocate per image while respecting a prescribed average token budget. At the locations with the highest priority scores, the 2D token features supersede the spatial features that would otherwise be reconstructed from the 1D representation; a shared decoder then ingests the combined 1D and 2D feature map to produce the final image reconstruction. Experiments on ImageNet at 256 × 256 resolution show that MixTok attains competitive fidelity: with an average budget of roughly 128 content tokens per image it achieves an rFID of 1.195, and with a budget of about 131 tokens it reaches a gFID of 1.95. These results demonstrate that the adaptive allocation of sparse 2D refinements alongside a global 1D backbone yields a favorable trade‑off between token efficiency and preservation of fine‑grained visual detail.

Read original