As AI systems interact more with people and make decisions about them, grasping human interpretations becomes essential for building human‑centered AI. Conventional machine learning assumes a single definitive ground truth, handling annotation differences by aggregation or treating them as noise. Yet many human‑centered tasks are inherently ambiguous: the same input can admit several reasonable and valid interpretations. Collapsing this ambiguity into one target hides valuable information about the diversity of human perception, judgment, and experience.
This position paper advocates shifting toward modeling the space of plausible human judgments and clearly distinguishing meaningful ambiguity from annotation noise. Such a perspective should guide AI systems across representation, learning, evaluation, deployment, and governance, fostering human‑centered AI that truly reflects the richness of human interpretation.
Key points
- Recognize that a "single ground truth" is not a valid assumption for all tasks.
- Treat annotation diversity as a resource, not merely noise.
- Incorporate probabilistic or distributional representations of the interpretation space in model design.
- Use evaluation metrics that capture coverage of multiple valid answers rather than single‑answer accuracy.
- Provide transparent ambiguity disclosures during deployment and governance to help users understand AI uncertainty.
Adopting this approach enables AI systems to better accommodate human cognitive diversity, improving interpretability and fairness.
Review