Researchers have introduced Activity Frames, a deterministic, zero-model compilation pipeline designed to transform passive screen-capture streams into structured memory for computer-use agents. Unlike traditional methods that rely on expensive LLM summarization, this method segments local captures into "activity frames"—bounded episodes containing application data, website information, timing, and input volume. This approach produces byte-identical, cacheable outputs that can be replayed deterministically without model inference. Testing on a single-user corpus of 128,756 frames over 51 days demonstrated that the compiler reduces raw capture data into prompt-ready context blocks 86x smaller in only 68 ms. When used as context, an agent achieved 98.4% accuracy in answering daily queries compared to a 66-80% accuracy rate for LLM-generated summaries. Furthermore, the framework provides new metrics for agent cost modeling: the Routine Overhead Ratio (R), which ranged from 60x to 343x, and routine recurrence (h), measured at 9.0% in-sample and 7.7% out-of-sample. By quantifying these parameters, the researchers suggest a realistic token ceiling for all-fleet deployments near 8%. The system allows for "zero model token" replays on guard-matched hits, effectively eliminating the need for frontier inference when re-deriving previously performed human routines.

Read original