In a closed world setting, AI is expected to have specialized competence and to generate the iid data assumed by theory: a fixed criterion drives example collection until the task is covered. By contrast, an open world demands versatility—taking on tasks the machine was not built for, from only a few examples and with minimal human prior—while providing data that is grouped rather than shuffled. Because tasks are unknown beforehand, a machine must learn at inference time, which forms the first requirement.
Modeling tasks as sharing latent factors and decomposing the training objective yields two additional requirements: a rich representation that retains more information than iid generalization needs, and a predictive disentanglement that aligns inputs and outputs into matched low‑dimensional groups. The analysis further predicts the need for a consistent inner learner and the benefit of long contexts spanning related tasks.
We test these claims on language models up to 8 B parameters, asking whether a model possesses a property (qualitative) rather than how much it wins (quantitative). Models that satisfy the three requirements improve as data from related tasks accumulates; those that do not degrade, and neither scaling nor layer‑type interleaving repairs this.
Review