To address the limitations of Markovian policies in non-Markovian, long-horizon robotic manipulation, the authors propose MaP-WAM, a "Memory-as-Plans" framework. Unlike traditional methods that rely on growing visual windows or language summaries, MaP-WAM decomposes tasks into memory-grounded planning and plan-conditioned execution. The system represents episodic memory as completed segment records containing language instructions and sparse visual context, which are then transformed into compact plans consisting of segment-level language plans and visual guidance. Execution is handled by a World-Action-Progress (WAP) model that simultaneously predicts action chunks and execution progress. This allows for adaptive segment transitions through plan-observation alignment, ensuring the executor remains closed-loop without requiring the full historical context. By utilizing structured attention for key-value caching and maintaining a fixed context length for the executor, the framework ensures that inference latency remains approximately constant even as task history grows. MaP-WAM demonstrates superior performance, achieving a state-of-the-art 83.3% success rate on RMBench and a 78.0% success rate on real-robot tasks, effectively balancing long-term memory coverage with execution efficiency.