Conceptual
Login

Dense Parameter Scaling Where Every Token Activates Every Weight

the assumption behind the original scaling curves, and the one that stopped being the efficient frontier

Why this Concept is marked deprecated

Superseded at the frontier by sparse mixture-of-experts: routing decouples parameter count from per-token FLOPs, so dense-only scaling is no longer the compute-efficient frontier.

Read the source