Abstract
Learning dynamic whole-body motions for legged robots through reinforcement learning (RL) remains challenging due to the high risk of failure, which hinders efficient exploration and often leads to unstable learning. In this paper, we propose External Force-Guided Curriculum Learning (EFGCL), a guided RL approach based on the principle of physical guidance, in which external assistive forces are introduced during training. Inspired by spotting in artistic gymnastics, EFGCL enables agents to physically experience successful motion executions without relying on task-specific reward shaping or reference trajectories. Experiments on a quadrupedal robot performing the Jump, Backflip, and Lateral-Flip tasks demonstrate that EFGCL accelerates learning of the Jump task by approximately a factor of two and enables the acquisition of complex whole-body motions that conventional RL methods fail to learn. We further show that the learned policies can be deployed on a real robot, reproducing motions consistent with those observed in simulation. These results indicate that physically guided exploration, which allows agents to experience success early in training, is an effective and general strategy for improving learning efficiency in dynamic whole-body motion tasks.
How EFGCL Works
1. Assistive Force
EFGCL begins by applying strong external "assistive" forces to the robot early in the training process. This physical guidance is analogous to a coach spotting a gymnast, allowing the robot to successfully complete the motion from the very beginning. By frequently experiencing these successful trajectories, the agent rapidly learns which states are valuable, accelerating the learning of the critic's value function.
2. Curriculum Decay
The initial assistance is not permanent. As the robot's performance improves, EFGCL gradually reduces the magnitude of the assistive forces. This "curriculum decay" is managed automatically based on the agent's success rate. This adaptive process ensures that the robot does not become overly dependent on external assistance and learns to perform the skill autonomously. By the end of the training, the assistance is completely removed, and the robot can execute the dynamic motion independently.
Simulation Results
Jump
Backflip
Lateral-Flip
Efficient Learning with Simple Rewards
EFGCL significantly simplifies the acquisition of dynamic whole-body motions. By providing physical guidance through external assistive forces, agents can learn complex acrobatic skills using only simple and sparse reward functions. This approach eliminates the need for the intricate, task-specific reward shaping or reference trajectories typically required in standard reinforcement learning. Our experiments demonstrate that while conventional methods often struggle with sparse rewards in complex tasks, EFGCL enables stable and rapid convergence by allowing the agent to experience success early in the training process.
From Simulation to the Real World
The ultimate test is real-world performance. The policies learned entirely in simulation were transferred directly to our quadrupedal robot, KLEIYN. The robot was able to successfully reproduce all three dynamic motions—Jump, Backflip, and Lateral-Flip—demonstrating effective and robust sim-to-real transfer.
Accelerating Value Estimation
A key reason for EFGCL's success is its ability to accelerate the learning of the critic's value function. By experiencing successful motions early, the value function quickly learns to assign high values to desirable states. With EFGCL, the value estimates converge to the final distribution as early as 200 iterations. In contrast, the baseline requires more than 1,000 iterations to reach a comparable level, showing that physical guidance significantly stabilizes and accelerates the entire learning process.
Video Presentation
BibTeX
@article{yoneda2026efgcl,
author={Keita Yoneda and Kento Kawaharazuka and Kei Okada},
journal={IEEE Robotics and Automation Letters},
title={EFGCL: Learning Dynamic Motion through Spotting-Inspired External Force-Guided Curriculum Learning},
year={2026},
volume={11},
number={5},
pages={5907-5913},
keywords={Learning from Experience; Bioinspired Robot Learning; Legged Robots},
doi={10.1109/LRA.2026.3675955}
}