Effective decision‑making in human‑centric settings such as Intelligent Tutoring Systems (ITS) and clinical care demands policies that are adaptive, data‑efficient, and ethically deployable. Apprenticeship Learning (AL) recovers the underlying reward function from expert demonstrations, enabling strictly offline policy inference without any environment interaction. This work tackles two core challenges—reward modeling and demonstration quality—through a series of offline AL frameworks.
For reward modeling, EM‑EDM clusters demonstrations by latent reward functions and derives cluster‑specific policies, while THEMES partitions trajectories into temporally coherent sub‑trajectories guided by a hierarchically learned reward regulator. Across one educational and three clinical datasets, THEMES consistently outperforms flat and hierarchical AL baselines.
Regarding demonstration quality, HALIDE incorporates continuous quality signals into hierarchical policy induction, allowing principled reuse of mixed‑quality data under distribution shift. CCHAL extends this by assigning Bayesian credit to individual decisions, employing dual‑arm criticality scoring to contrast expert and non‑expert behavior, and using a Bradley‑Terry contrastive IRL objective on failure trajectories. On three clinical EHR datasets and one ITS dataset, CCHAL surpasses baselines including HALIDE.
A real‑world classroom study of StuEDM—a policy induced by EDM that intervenes only on disagreement while preserving student agency—shows significant learning gains over an expert baseline and a higher pre‑test‑adjusted post‑test score than a CQL policy trained on a far larger corpus, while overriding students less than half as often.
Together, these results establish offline AL as a practical, data‑efficient, and ethically viable paradigm for human‑centric environments.
Review