Baidu's ERNIE Evolve automated research system employs a dual-loop architecture where a Director agent selects research directions from an Experience Bank and Knowledge Base, while parallel lanes execute code modifications via Selector and Executor agents; changes must pass a hidden Evaluator with a noise threshold and a Reviewer to prevent reward hacking. Tested on three public benchmarks using 1×NVIDIA B200 (NanoChat, SOL-ExecBench) and 8×H100 (Speedrun), ERNIE Evolve achieved a 0.8763 validation BPB on NanoChat AutoResearch within a 5-minute budget by adding n-gram and document-retrieval branches with a learned gate, 38.9 seconds on NanoGPT Speedrun Track 1 (vs. 39.9s record) and 2,590 steps on Track 3 (vs. 2,690) via weight decomposition and EMA-Nesterov extrapolation, and a 0.819 mean SOL on SOL-ExecBench across 235 kernels, trailing the 0.827 leaderboard best. Applied to ERNIE-5.5-Flash inference on Kunlunxin P800 and undisclosed GPUs, the system delivered 623.6% baseline generation speed after four rounds—enabling fused mHC, speculative decoding via multi-token prediction, host overhead reduction, and DeepEP low-latency dispatch—while deterministic throughput under SGLang rose 33.7% (output token time -26%, TTFT -12%) via a Triton deterministic sampling kernel, and single-request GPU decoding improved 14.9% using FP16 for tiny FP8 matmuls and a two-level MoE expert scheduler. The system remains closed-source with only results and logs released.
Read original
marktechpost