NeFut Logo NeFut
Admin Login

[CS.AI] Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration

Published at: 2026-09-12 22:00 Last updated: 2026-09-15 01:15
#AI #Machine Learning #optimization

Heterogeneous model collaboration aims to exploit complementary strengths of diverse models to keep predictive performance while cutting inference cost. Existing solutions either train routers, tying routing decisions to a fixed task and model pool, or use raw‑confidence cascades whose thresholds lack a common reliability meaning across models, resulting in poor adaptation to changing pools or budgets. We introduce Calibration‑Aware Uncertainty Cascades (CAUC), a simple post‑hoc framework that first calibrates each model’s confidence independently and then selects deployment policies on validation data. The calibrated scores provide a shared reliability scale for deciding whether to accept an early prediction, invoke a stronger model, or selectively combine outputs. This unified criterion decouples policies from any specific model set or budget. Theoretically, calibration gives confidence thresholds an explicit selective‑risk interpretation, whereas uncalibrated scores offer no comparable guarantee. Experiments on six language benchmarks show CAUC improves relative accuracy by 1.9% on average over strong‑model‑only inference while avoiding about 47% of strong model calls; on image classification benchmarks it maintains or improves performance while reducing measured GFLOPs by up to 57%.

Review

Original Source: https://arxiv.org/abs/2609.11446

[h] Back to Home