On the Global and Local Calibration of Graph Neural Networks
1University of Trento · 2Fondazione Bruno Kessler · 3UiT The Arctic University of Norway · 4University of Pisa · 5Aalborg University
Towards Trustworthy Predictions: Theory and Applications of Calibration for Modern AI at AISTATS 2026 · Tangier, Morocco · 2026
In one sentence. This study shows that carefully tuned classical GNNs can be globally well-calibrated, while an embedding-space Local Calibration Error uncovers substantial miscalibration hidden by aggregate metrics.
Problem
Reliable graph neural networks should produce confidence scores that match empirical correctness frequencies. Prior node-classification studies report under-confident GNNs using Expected Calibration Error (ECE), motivating graph-specific calibrators. This paper asks whether the apparent problem reflects weak hyperparameter choices and whether global ECE conceals calibration errors within local regions of the learned representation space.
Main contributions
- It re-evaluates graph-specific calibration methods against a carefully tuned GCN baseline.
- It shows that tuned classical GNNs obtain global ECE comparable to specialized calibration approaches on Planetoid benchmarks.
- It provides a systematic local GNN calibration analysis using Local Calibration Error (LCE).
- It defines graph-data locality through distances between node embeddings learned by the GNN and examines the effective sample size of local estimates.
Method
The baseline is a Graph Convolutional Network selected by validation grid search over 16–64 hidden dimensions, one to four layers, and dropout from 0.1 to 0.5. Global calibration uses 20-bin ECE. Locally, a radial kernel weights nodes by distance in GNN embedding space; LCE uses 10, 15, and 20 bins and bandwidth γ from 0.1 to 5. Effective sample size (ESS) measures local support.
Experimental setting
- Task
- Transductive semi-supervised node classification.
- Datasets
- Cora, CiteSeer, and PubMed citation networks with fixed Planetoid splits.
- Metrics
- Expected Calibration Error (ECE), Local Calibration Error (LCE), and effective sample size (ESS); lower ECE and LCE are better.
- Comparisons
- CaGCN, Graph Attention Temperature Scaling (GATS), Adaptive and Universal Label Smoothing (AU-LS), and SCAR.
Key results
The tuned GNN reaches low global calibration error without a specialized calibrator. SCAR has the lowest mean ECE on all three datasets, but the tuned GNN is close on Cora and PubMed. The local analysis changes the picture: on Cora with 20 bins, GNN LCE is 11.99 ± 0.77 at γ = 2 and 6.62 ± 0.81 at γ = 5, versus global ECE of 3.54 ± 0.39. Bandwidths γ ≤ 1 generally have insufficient ESS, while γ ≥ 2 gives more reliable support.
| Dataset | Tuned GNN | Best specialized method | Difference |
|---|---|---|---|
| Cora | 3.54 ± 0.39 | SCAR: 3.35 ± 0.53 | +0.19 |
| CiteSeer | 6.31 ± 0.27 | SCAR: 3.43 ± 0.58 | +2.88 |
| PubMed | 3.87 ± 0.27 | SCAR: 3.81 ± 0.47 | +0.06 |
When this work is relevant
This work is relevant for evaluating confidence calibration in GNN node classifiers, choosing between graph-specific calibration methods and tuned GCN baselines, detecting local uncertainty failures hidden by ECE, and designing calibration studies based on node-embedding similarity or larger graph benchmarks.
Limitations
The experiments cover only node classification on three small Planetoid citation graphs. Their few labeled nodes per class can make calibration bins unstable, and strongly local LCE estimates have very low effective sample sizes. Larger benchmarks such as OGB, other graph tasks, and the computational cost of LCE are not evaluated.
Citation
Persistent identifier: OpenReview FHsEi0qz7t