500

IGPM500.pdf January 2020 
TITLE 
Kinetic Theory for Residual Neural Networks 
AUTHORS 
Michael Herty, Torsten Trimborn, Giuseppe Visconti

ABSTRACT 
Deep residual neural networks (ResNet) are performing very well for many data science
applications. We use kinetic theory to improve understanding and existing methods. A
microscopic simplified residual neural network (SimResNet) model is studied as the limit of
infinitely many inputs. This leads to kinetic formulations of the SimResNet and we analyze
those with respect to sensitivities and steady states. Aggregation phenomena in the case of a
linear activation function are also studied. In addition the analysis is validated by numerics.
In particular, results on a clustering and regression problem are presented.

KEYWORDS 
Keywords Residual neural network, continuous limit, mean field equation, kinetic equation, machine learning application
