In real deployments, LLM systems are shifting from single‑model calls to heterogeneous setups where different‑scale models are invoked on demand. Reusing the same context across models traditionally requires each newly selected model to re‑process the context, incurring heavy compute costs. ACCT (Asymmetric Cross‑LLM Context Transfer) addresses this by enabling KV‑cache transfer between models of varying sizes within the same family. It converts the KV cache produced by a source model into sparse KV states that a target model can consume directly, decoupling context computation from downstream inference. ACCT consists of three lightweight modules: the Cartographer identifies query‑relevant context tokens for each target layer and KV head; the Allocator decides how many tokens each layer and head retains under a shared KV budget; the Compiler transforms the selected source KV into sparse target KV. Evaluations on five general‑capability benchmarks and six highly context‑dependent tasks show that ACCT matches or exceeds the original inference performance even with a 75% reduction in KV cache size. At roughly 19K tokens, KV conversion speeds up target pre‑fill by 6.5‑21.2×, demonstrating the feasibility of cross‑model context transfer under limited KV budgets.
Review: ACCT offers a practical way to share context computation across heterogeneous LLMs, delivering substantial speedups and resource savings without sacrificing accuracy.