NeuralDocs

Types of Neural Networks / Graph Neural Networks

Graph Neural Networks

Advanced

GNNs learn on graph-structured data by passing messages between connected nodes.

Message passing

vabcde
In one message-passing layer, node v gathers messages from its neighbours N(v) = {a, b, c, d}, aggregates them (sum, mean or max), and combines the result with its own state to produce its new embedding.
hv(k+1)=σ ⁣(W(k)AGG⁡u∈N(v)hu(k)+B(k)hv(k)+b(k))\boldsymbol{\mathbf{h}_v^{(k+1)}=\sigma\!\left(W^{(k)}\operatorname{AGG}_{u\in\mathcal{N}(v)}\mathbf{h}_u^{(k)}+B^{(k)}\mathbf{h}_v^{(k)}+\mathbf{b}^{(k)}\right)}

This is one message-passing layer, not a universal formula for every GNN. k is the layer index, N(v) excludes the node v itself, and AGG is permutation-invariant over neighbor features, including repeated values (for example, sum or mean). B preserves a separate self-node contribution. For isolated nodes, define the aggregate as the zero vector.

Models

  • GCN — graph convolution
  • GraphSAGE — sampling & aggregation
  • GAT — attention over neighbours

Applications

  • Molecule property prediction
  • Social networks & recommendation
  • Traffic and fraud detection

Interview questions & answers

  1. Q1. Why must neighbor aggregation be permutation invariant?

    Answer: A node's neighbors form an unordered collection. Sum, mean or max aggregation gives the same result regardless of their listing order. With shared node updates, this supports permutation-equivariant node representations; graph-level predictions also need an invariant readout.

  2. Q2. How do GCN, GraphSAGE and GAT differ?

    Answer: GCNs typically aggregate using degree-normalized graph connectivity. GraphSAGE uses neighborhood sampling and learned aggregation for scalable, inductive representations. GAT learns attention weights for neighboring nodes. Their best use depends on graph size, features and the task.

  3. Q3. What is oversmoothing in a GNN?

    Answer: Repeated message passing can make connected nodes' representations increasingly similar, reducing their ability to distinguish node roles or classes. Residual connections, normalization and limiting depth can help. Oversmoothing is different from oversquashing, where too much distant information must fit into a fixed-size representation.

← Previous

Transformers

Next →

Transfer Learning & Fine-tuning