A Battery for AI Propensity Measurement: Rubrics, Synthetic Item Banks, and Annotations
Peter Romero, Haotian Li, Hugo Save, Luning Sun, Zachary R. Tidler, Kazuya Horibe, Joe Watson, Daniel Romero-Alvarado 等 11 位作者
We release a battery of 28 propensities for AI research, with expert-written rubrics, synthetic datasets, and annotated demand intervals, built with the method of Romero-Alvarado et al. (2026). We selected a wide collection of generic propensities that may influence the behavior of AI systems and humans. They are inspired by psychometric, psychological, and behavioral research, so that experimental results can be understood in the context of familiar constructs previously used also for human behavior research. The rubrics are useful for annotating existing benchmarks whose items may be relevant or not for one or more propensities. For easier profiling, however, the synthetic datasets are defined in such a way that they should be sensitive to one propensity only, allowing for a quick prototyping of propensity profiles using the battery of datasets. We also outline directions for future propensity research.
尚无 AI 解读，展示原文摘要