Researchers have developed Isometric Associative Memory (ISOM), an open-source architecture that maintains flat inference memory usage regardless of context length, addressing the transformer memory wall where Key-Value caches grow indefinitely. ISOM achieves O(1) inference memory even at 128K context windows, with performance verified on NVIDIA Tesla T4 GPUs across model sizes ranging from 130M to 40B parameters. The approach enables processing of long documents and repository-scale code without the memory constraints that limit standard attention mechanisms.

Read original