In the application of Large Language Model (LLM) ensembles, combining predictions from multiple LLMs has become increasingly common to enhance reliability. However, existing aggregation methods typically assume that all models are equally trustworthy, overlooking differences in uncertainty quality. This assumption is poorly suited for heterogeneous LLMs, which exhibit significant variability in reliability and capability, making naive aggregation vulnerable to unreliable or adversarial experts.
In this work, we formulate multi-LLM aggregation as a problem of uncertainty-aware trust estimation. We adapt structured expert judgment from decision theory, using context-aware calibration questions to assess expert reliability based on the quality of their probabilistic predictions. Specifically, we employ Cooke-style log weighting, which penalizes overconfident incorrect predictions while favoring well-calibrated experts.
We evaluate our approach on MMLU and MMLU-Pro datasets across homogeneous, heterogeneous, and contaminated expert panels. Results show that while aggregation methods perform similarly in homogeneous settings, Cooke weighting becomes critical under heterogeneity and contamination. It achieves a superior accuracy-reliability balance and remains robust when unreliable experts are introduced. These findings suggest that Multi-LLM aggregation requires not just combining predictions, but calibrating trust under uncertainty.
Blogger's Review: The proposed uncertainty-aware trust estimation method presents an innovative approach for aggregation in multi-LLM systems, highlighting the importance of dynamically assessing expert reliability in heterogeneous environments. This method holds significant potential for real-world applications, especially when facing complex and uncertain prediction tasks. By effectively calibrating trust, the system can better tackle challenges posed by unreliable information.