NeFut Logo NeFut
Admin Login

[CS.AI] Model Recognition in LLMs: Introducing NameRank

Published at: 2026-07-15 22:00 Last updated: 2026-07-17 08:46
#AI #LLM #Open Source

Abstract

The recall of a frontier model about a person or tool from its own weights often shapes the first description a human sees, making the presence of that parametric corpus a measurement problem. Citations explain about one-third of whether a model recognizes a researcher; we target the residual and build NameRank, a [0,1] recognition score: each of 4,685 entities in 54 cohorts is probed with one open-ended question across 36 models, and an independent judge returns a binary verdict against a curated gold—did the model state a specific, non-guessable fact about this exact entity?—thus hallucination, context echo, and guesses earn nothing. Synthetic-null entities hold the floor near zero, and verdicts track the entity, not the model.

One thesis organizes the findings: recognition is paid to named, indexable artifacts, not to credentials or titles. Every Olympic-style credential sits below a working-researcher baseline, because no named artifact ships with the medal; yet the ranking inverts at the marquee tier, where Nobel, Turing, and Fields laureates saturate the panel. For independent creators, the tool out-ranks its maker, and the credential that does propagate is a named method or awarded paper. Being one of many named contributors to a celebrated artifact, by contrast, earns almost nothing—the authors listed on a flagship model report or system card sit near the recognition floor—because recognition attaches to the artifact's own distinctive name, not to the roster behind it. No bibliometric predicts recognition well; top-density institutions out-recognize peers at matched citations; and on 258 news events, recognition loads on peak salience, not persistence. A self-report probe shows introspection reads a corpus prior, not its own knowledge.

Blogger's Review: This study reveals the complexities of how large language models recognize specific entities, highlighting the significance of named artifacts in model cognition. This has important implications for understanding the limitations of AI and how to leverage these tools more effectively.

Original Source: https://arxiv.org/abs/2607.12520

[h] Back to Home