The authors present Hebero, a GPU‑parallel benchmark built on Isaac Lab that enables joint training and evaluation of a single reinforcement‑learning policy across 40 heterogeneous manipulation tasks. By scaling the number of parallel replicas per task under a fixed wall‑clock budget, they show that higher parallelism yields improved task success rates. To address sparse rewards and limited demonstration data, they introduce Demonstration‑Guided Policy Optimization (DGPO), which repurposes expert demonstrations to generate dense tracking rewards and employs an asymmetric value‑learning scheme within a Proximal Policy Optimization (PPO) framework. Within DGPO, the IW‑ABC algorithm combines adaptive behavior cloning (ABC) with a per‑task learning‑progress signal that relaxes demonstration guidance as the agent improves, and importance weighting (IW) that amplifies updates for lagging tasks during PPO steps. Experiments with 50 demonstrations per task report a state‑input mean success of 90.1% for IW‑ABC, exceeding the strongest baseline FAMO‑ABC by 7.8 percentage points; the visual variant attains 93.5% mean success. Finally, a policy trained entirely in simulation is transferred to a physical Piper robot, where it successfully executes four distinct real‑world tasks, validating the sim‑to‑real transfer capability of the proposed multi‑task RL framework.