Browse Teach Curate

Benchmarking Gradient-Free Zeroth-Order Optimization for Catastrophic Forgetting

A benchmark and set of findings showing that continual-learning models can resist catastrophic forgetting using only forward passes, via zeroth-order optimization that estimates update directions from function values instead of gradients. Students learn why gradient-free updates are needed when backpropagation is banned (black-box APIs, non-differentiable or memory-constrained systems) and how forward-pass methods trade off forgetting, task conflict, and memory cost.

B9 Text

ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Think Tao Feng 1 Wei Li 1 * Didi

Backpropagation-based optimizers like SGD and Adam are the usual tools for overcoming catastrophic forgetting in continual learning, but many real settings impose 'gradient bans' where gradients are …