Reliable uncertainty quantification (UQ) is a prerequisite for deploying neural networks in scientific and high‑stakes settings, yet full Bayesian inference over all network parameters is computationally prohibitive. We introduce a low‑rank generalized Laplace approximation that leverages a small set of data‑driven curvature directions. Starting from a generalized Bayesian posterior defined by an empirical loss, we build a local Gaussian around pretrained weights confined to an active curvature subspace. The posterior variances within this subspace admit closed‑form expressions, and the prior variance is calibrated via an empirical Bayes step. The generalized Bayesian formulation lets us compare two posterior scalings: the standard scaling tied to the summed negative log‑likelihood, and a mean‑loss scaling that normalizes the empirical loss by the number of samples $n$. A key observation is that the standard scaling induces a data‑size dependent contraction of posterior variance along the leading directions. In regression problems this forces the low‑rank model to retain additional weak‑curvature directions to achieve nominal coverage on calibration data, but propagating these directions through the nonlinear network can degrade the coherence of predictive intervals and shift the posterior predictive mean away from the pretrained model. By contrast, the mean‑loss scaling yields a more stable, lower‑dimensional active subspace and produces calibrated, coherent confidence intervals. Experiments demonstrate that generalized Laplace active subspaces provide a practical, scalable route to reliable UQ in neural networks.
Review