Conceptual

Chain Rule

The derivative of a composition is the product of the derivatives of its stages. Long products of stage derivatives are the direct cause of vanishing and exploding gradients in deep and recurrent networks.