Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions. This paper advocates for a shift from static and emulative alignment to interactive and complementary alignment, where preferences emerge through interaction and alignment is defined not merely by satisfying preferences.
We first formalize this gap by contrasting existing alignment with a trajectory-level view in which human and model behavior co-evolve over time. Because these interaction dynamics have not been adequately captured within existing ML formulations, we ground this perspective in insights from an interdisciplinary workshop.
Drawing on lessons from social-science accounts of human-human collaboration, we argue that human-AI systems amplify these dynamics, introducing new asymmetries that complicate reasoning about uncertainty and create new coordination challenges. Based on these lessons and new challenges, we conclude by outlining a research agenda for developing AI systems that align with humans in interaction, requiring an interdisciplinary synthesis of machine learning and the social and decision sciences.
Blogger's Review: This paper's proposed approach to dynamic human-AI alignment offers a profound critique of existing AI limitations, emphasizing the importance of interactivity. This research direction not only enriches the theoretical framework for AI-human collaboration but also provides fresh perspectives for future AI system designs, making it worthy of attention.