Conformal Calibration for Multi-modal Regression with Missing Modalities
PMLR, COPA
Published
Abstract
Prediction intervals for multi-modal regression with tabular variables, text, images, or other input sources are difficult to calibrate when those sources disagree or one is missing. A single global quantile averages these regimes together instead of calibrating to the modality pattern observed at test time. We address this through a modality-aware conformal calibration layer. The layer trains or reuses one predictor per modality, computes a disagreement score from the modality-specific predictions, and uses that score in split conformal calibration under a strict split protocol. We use the score in two complementary ways. First, a continuous weighted method reallocates interval width across examples while preserving the usual marginal split-conformal guarantee. Second, a Mondrian (stratified) method calibrates within disagreement or modality-availability groups fixed before final calibration, giving group guarantees under joint exchangeability of the final-calibration and test examples. Across four multi-modal datasets, the weighted layer matches or improves the marginal conformal baseline in 59 of 60 paired interval continuous ranked probability score (CRPS) comparisons and 52 of 60 width comparisons while keeping empirical coverage near the 95% target. In missing-modality stress tests, regime-Mondrian calibration recovers up to 20.2 percentage points of coverage in the hardest masked regimes. The result is a simple, model-agnostic reliability layer for multi-modal regression systems.