The paper identifies a "phantom progress" failure in memory-augmented vision-language-action (VLA) policies on RoboMME counting tasks. Policies uniformly sample frames across episodes and index memory by time step, causing elapsed time to serve as a proxy for task progress. Introducing brief null-event pauses or wrist fidgets — which alter no task-relevant state — reduces success rates from 47/50 to 16/50 and 44/50 to 10/50 on two tasks. Withholding these frames restores performance, confirming the failure originates in memory content rather than the interruption. Rescaling time stamps alone induces early stopping (fast clock) or repetition (slow clock), demonstrating a temporal shortcut. Removing time encoding fails to resolve the issue because null-event frames still occupy fixed slots in time-sampled memory. Vulnerability correlates with memory indexing strategy across RoboMME designs. Training with injected pauses does not eliminate the failure and degrades counting accuracy. In contrast, re-indexing memory by events rather than time recovers over half the performance loss without retraining, supporting the conclusion that memory should count events, not clock ticks.