← Open in Shiplog

Dynamical isometry

theme machine learning selected

Conditions under which signal propagation in deep networks preserves norms and gradients; spectral analysis of layerwise Jacobians and Fisher information.

Findings (1)

Papers (1)
- The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry2021· AISTATS

Connections

This topic …

usesOrthogonal initialization, Asymptotic freeness

Referenced by

Interpretability and training dynamics · contrasts with (undirected)
Self-attention spectra · extends