Recent work investigates how to exploit pre‑collected datasets to boost policy performance and sample efficiency in reinforcement learning (RL). A promising recipe follows a two‑stage pipeline: first, diverse low‑level skills are extracted from offline data; second, a high‑level policy is trained to solve a target task. Conventional skill extraction relies on unsupervised methods such as trajectory VAEs, making the low‑level policy’s quality highly sensitive to dataset fidelity. To remedy this, we introduce QDOS (Quality‑Diversity Offline Skill learning), a unified offline‑to‑online framework. QDOS incorporates an Advantage‑Weighted Quality‑Diversity pretraining objective that weights each trajectory segment by its estimated advantage, thereby balancing diversity with value and yielding a rich yet high‑utility skill space. We further adopt a dual dataset reuse scheme: the same offline data serves both skill pretraining and populates the online replay buffer via pseudo‑labeling, enhancing exploration without extra data collection. Empirical results on structured manipulation and unstructured locomotion benchmarks show that QDOS markedly outperforms strong baselines, confirming its ability to accelerate exploration and improve final returns in sparse‑reward domains.
Blogger's Review: QDOS cleverly injects advantage information into the quality‑diversity objective, shifting offline skill extraction from blind diversity toward high‑value behaviors. The dual‑reuse strategy bridges offline and online learning, offering a more robust transfer pipeline for robotics and game AI. Future work could investigate finer‑grained advantage estimators and cross‑task skill sharing.