Layer Normalization Gradient Through Mean and Variance in Backpropagation

… more less For \( \hat{x} = (x-\mu)/\sigma \), the gradient is \( \frac{1}{\sigma}\big(g - \bar{g} - \hat{x}\,\overline{g\hat{x}}\big) \): the upstream gradient with its mean and its projection on \( \hat{x} \) removed.