Real‑world agents need persistent non‑parametric knowledge for dynamic reasoning, i.e., long‑term memory and retrieval‑augmented generation. Sparse graphs provide reliable structured evidence but limit machine readability and semantic density required by complex agentic workflows. To address this, the industry is shifting toward LLM Wiki—an agent‑native knowledge representation that couples dense document contexts with markdown files containing multi‑layer topological links. Encoding such rich semantics with traditional sparse‑graph embeddings remains difficult, and learning LLM Wiki with existing graph encoders incurs heavy distributed‑system overhead, hindering large‑scale deployment. We therefore introduce the Wiki Foundation Model (WFM), tailored for scalable, agent‑native representation and retrieval. We first formalize a Wiki Graph schema that bridges fine‑grained structures with dense contexts, preserving explicit topologies alongside continuous semantics. Next, we design a query‑conditioned attentive aggregation for rich wiki message passing, adding explicit attention variance regularization for stability. Finally, we engineer an NCCL boundary exchange protocol that lifts static partition indices and uses fixed‑shape GPU‑to‑GPU collectives, bypassing CPU serialization and memory‑copy bottlenecks. Extensive evaluation on five long‑term memory and multi‑hop reasoning benchmarks shows that WFM markedly improves performance while achieving a 10.5× training speedup on distributed clusters.
Review