ArXiv

Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification

2026-05-12 · 14:46 UTC ·Julian Rodemann, Alexander Marquard, Thomas Augustin... ·1 min read
Authors
Julian Rodemann, Alexander Marquard, Thomas Augustin...
Categories
stat.ML, cs.AI, cs.LG, stat.CO
arXiv
https://arxiv.org/abs/2605.12208v1
PDF
https://arxiv.org/pdf/2605.12208v1

Brief

The paper proposes Self-Supervised Laplace Approximation (SSLA), which sidesteps parameter posteriors and directly approximates the posterior predictive by refitting on model self-predictions; a faster approximate variant (ASSLA) avoids costly refits. The authors provide theoretical analysis and experiments on Bayesian linear models through Bayesian neural networks, showing better predictive calibration than classical Laplace methods while keeping computation efficient.

Why it matters

Self-Supervised Laplace Approximation (SSLA) approximates the posterior predictive directly by refitting the model on its own self-predicted data, producing a deterministic, sampling-free posterior-predictive approximation and supporting prior-sensitivity analysis via a modular prior interface.

Key details

  • The paper introduces an approximate, cheaper variant (ASSLA) to avoid expensive refitting; the authors prove properties of (A)SSLA and evaluate them on regression problems from Bayesian linear models to Bayesian neural networks, reporting improved predictive calibration over classical Laplace approximations while remaining computationally efficient.
  • Work by Rodemann, Marquard, Augustin, and Caprio (arXiv:2605.12208v1) was posted 2026-05-12 and accepted to TMLR.
Source evidence

Abstract

Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model's predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.

Comment: Accepted for publication in TMLR (https://openreview.net/forum?id=T8w8L2t3JG)
Journal: Transactions on Machine Learning Research (TMLR). ISSN 2835-8856 (2026)