We investigate whether the output embeddings of language models inherently contain structured temporal and geographic information. To this end we introduce a simple projection‑based method: given a handful of seed examples with known timestamps or locations, we define an axis in the embedding space and rank any text or entity by its projection onto that axis. The approach is fully black‑box and model‑agnostic, requiring only the embeddings themselves—no access to model weights, internal activations, auxiliary probes, or further training—making it applicable to modern embedding services that expose only an API.
We evaluate the method on public temporal corpora and geo‑annotated datasets. The projections consistently recover meaningful chronological orderings (e.g., early news articles receive lower projection scores than later ones) and spatial gradients that align with latitude/longitude. These findings demonstrate that language‑model output embeddings encode signals relevant to both time and space.
Practically, the technique serves two purposes: it offers a lightweight interpretability tool for probing whether a model captures specific structured knowledge, and it can be directly employed in downstream tasks such as temporal ordering, geographic ranking, or automatic tagging without additional fine‑tuning.
Review