EnvACE introduces a novel agentic reinforcement learning framework designed to mitigate the high costs and scaling limitations associated with external environment interactions during LLM agent training. Instead of relying on costly real-world or synthesized environments, the method employs "world rehearsal," where the policy alternates between acting and simulating the environment. During this process, the agent generates a tool call and then simultaneously assumes the role of the environment to produce the corresponding response. By jointly optimizing both the acting and environmental roles end-to-end using task-success rewards, the policy internalizes environment dynamics directly into its parameters, effectively creating an integrated world model. This approach allows for private rehearsal during test time, enabling the agent to simulate outcomes before committing to actual execution. Empirical evaluations across several benchmarks, including BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, demonstrate that EnvACE achieves superior and transferable performance compared to standard environment-scaling baselines. Furthermore, controlled studies confirm that the world rehearsal mechanism consistently enhances policy learning across various model scales, establishing a scalable pathway for training long-horizon tool-use agents without the constraints of external simulators or executable environment verification.
Read original
huggingface/daily-papers