Model‑free reinforcement learning can acquire contact‑rich robotic manipulation skills through trial‑and‑error interaction, yet it often forces the policy to learn both task strategy and low‑level motion generation. In this setting, the action representation is crucial because it determines how policy outputs are converted into robot motion, shaping exploration and physical execution. Direct Cartesian command interfaces require the policy to generate motion at every decision step, coupling task‑level adaptation with continuous low‑level control and increasing the learning burden. We propose the PA‑RL framework, which uses artificial potential fields as the action representation. Instead of commanding motion directly, the policy adapts parameters of an energy‑like potential field; the field yields a state‑dependent guidance direction executed via a Cartesian impedance controller. We evaluate PA‑RL on peg‑in‑hole insertion, a representative contact‑rich task with nonlinear dynamics and discontinuous contact transitions. In simulation, PA‑RL is compared with Cartesian velocity, Cartesian pose, and variable‑impedance action spaces using the same RL algorithm. PA‑RL is the only method to achieve a 100% success rate within the allotted training time, while the best baseline reaches 92.6%. It also reduces joint‑torque variation by 55.4% and Cartesian acceleration variation by 70.8% relative to the best baseline, without explicit motion‑quality penalties in the reward. The simulation‑trained policy further completes 9/9 real‑robot insertions without fine‑tuning, demonstrating the deployment feasibility of the learned potential‑field interface.
Review