NeFut Logo NeFut
中 Admin Login

[CS.AI] A Battery for AI Propensity Measurement: Rubrics, Synthetic Item Banks, and Annotations

Published at: 2026-10-28 22:00 Last updated: 2026-10-29 00:58
#AI #Machine Learning #Open Source

We release a battery of 28 AI research propensities equipped with expert‑written rubrics, synthetic item banks and annotated demand intervals, built following the method of Romero‑Alvarado et al. (2026).

The propensities cover generic factors that may shape the behavior of AI systems and humans, drawing on psychometric, psychological and behavioral literature so that results can be interpreted within familiar human‑behavior constructs.

The rubrics enable annotation of existing benchmark items to indicate relevance to one or more propensities. The synthetic item banks are constructed to be sensitive to a single propensity, allowing rapid profiling by selecting the appropriate dataset.

Annotated demand intervals support evaluation of models under varying intensity levels, and we outline future research directions such as expanding the propensity set, multimodal assessment and linking propensities to real‑world tasks.

Review

Original Source: https://arxiv.org/abs/2610.34614

[h] Back to Home