Browse Teach Curate

Mixture of Experts in Deep Learning

A neural-network design in which many specialized expert sub-networks are combined and a gating mechanism routes each input to a subset of them, increasing model capacity without a proportional increase in per-input computation. Students learn how conditional computation and expert specialization let a single model hold diverse domain-specific knowledge.

jeremy Video

Mixture of Experts Router Architecture in Large Language Models

Visual walkthrough of MoE layers: replacing the dense feedforward block with routed experts, top-k softmax gating, load-balancing losses, and why sparse activation decouples parameter count from per-…