A hypernetwork that maps a context directly to a LoRA adapter lets a large language model embed that context in its weights. Previous studies demonstrated this only on base models up to 14 B parameters. The Internalizer is a state‑of‑the‑art, portable Context‑to‑Parameter Mapping hypernetwork that produces document‑specific LoRA adapters for the frozen 284 B‑parameter DeepSeek v4 Flash, two orders of magnitude larger than any prior work. Most of its parameters reside in a model‑agnostic trunk with only thin entry and exit layers per base model, allowing cheap training on small models before porting to the large one. On unseen documents up to 4096 tokens, the generated adapters achieve 84.9% top‑1 and 97.8% top‑5 teacher‑forced accuracy, compared with 63.4% and 83.5% for the base model, while the context window contains only a three‑word instruction. Once the hypernetwork is trained, a single forward pass turns any document into an adapter for such a model, which can be served alone for speed or alongside the document in the window to raise accuracy further.
Review