Local-cloud collaboration is a practical way to deploy large language models under resource constraints, but existing methods often rely on trained routers or collaboration-aware finetuning that tie routing behavior to a particular operating regime.
This work shows that such training may be unnecessary: the local model's own inference-time agreement across sampled responses provides a strong signal for deciding when to trust local execution and when to offload to a stronger cloud model.
We propose CARGO, a training-free routing framework that estimates this agreement through prompt-varied sampling, applies Bayesian early stopping for sample-efficient uncertainty control, and supports arbitrary target collaboration ratios through lightweight deployment-time calibration.
Across diverse reasoning and question-answering tasks, multiple local LLM families and scales, and both pretrained and finetuned local models, CARGO consistently outperforms other training-free baselines and in several settings surpasses supervised learned routers.
These results suggest that effective and adaptable local-cloud collaboration can emerge directly from the local model's intrinsic response behavior, without requiring an additional trained router.
Blogger's Review: The introduction of CARGO challenges the conventional approach of relying on trained models for routing, leveraging the self-feedback mechanism of local models. This framework's flexibility and efficiency provide new insights into how to effectively utilize cloud computing resources in resource-constrained environments, making it a noteworthy development in LLM deployment.