KAN: Rethinking the Building Blocks of Neural Networks
For decades, the multi-layer perceptron has been the universal approximator of choice — the default building block for everything from transformers to diffusion models. But a landmark paper from Liu et al. at MIT (accepted to ICLR 2025) proposes a fundamental rethinking of what a neural network layer should look like. Kolmogorov-Arnold Networks (KANs) replace the fixed activations on neurons with learnable activation functions on edges, parameterized as B-splines. The result is a network that achieves dramatically better parameter efficiency, possesses provably faster neural scaling laws, and — perhaps most importantly — can be visualized and interpreted in ways that MLPs simply cannot match.