Residual Connections Mitigate Oversmoothing in Deep Graph Neural Networks
In deep graph neural networks, repeated neighbourhood aggregation drives every vertex's feature vector toward a common limit ('oversmoothing'), collapsing the network's expressive power. Modeling a deep GNN as a product of random weight matrices and applying the multiplicative ergodic theorem yields explicit asymptotic convergence rates for a normalized vertex-similarity measure, and shows that residual (skip) connections slow or prevent this collapse across broad families of random weight distributions.
2501.00762
Deep graph neural networks suffer from 'oversmoothing': as layers are stacked, each vertex's feature vector converges toward the others so that the vertices become nearly indistinguishable and the ne…