Optimal Conformal Prediction Under Epistemic Uncertainty

MCML Authors

Alireza Javanmardi

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Eyke Hüllermeier

Prof. Dr.

Core PI

Artificial Intelligence and Machine Learning

Abstract

Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees. In practice, CP is typically applied on top of probabilistic classifiers, which are able to express aleatoric but not epistemic uncertainty. In this paper, we consider the question of how to optimally employ CP on top of a more expressive formalism, namely credal sets, which can express both aleatoric and epistemic uncertainty. More specifically, we propose probabilistic {Bernoulli} prediction sets and derive a variant that achieves conditional coverage for valid credal sets while remaining minimal in expected size. We then address the more realistic scenario in which the validity of the credal sets is not guaranteed. Assuming access to calibration data with ground-truth distributions over labels, we apply conformal risk control to BPS and derive a PAC-style guarantee: with high probability over the data, the achieved conditional coverage is at least the desired level. We validate our theoretical findings empirically over various datasets.

inproceedings JZT+26a

---

UAI 2026

42nd Conference on Uncertainty in Artificial Intelligence. Amsterdam, The Netherlands, Aug 17-21, 2026.

---

Authors

A. Javanmardi • S. H. Zargarbashi • S. M. A. R. Thies • W. Waegeman • A. Bojchevski • E. Hüllermeier

---

Links

URL

---

Research Area

 A3 | Computational Models