500	RWTH Publication No: 775908        2020        IGPM500.pdf
TITLE	Kinetic Theory for Residual Neural Networks
AUTHORS	Michael Herty, Torsten Trimborn, Giuseppe Visconti
ABSTRACT	Deep residual neural networks (ResNet) are performing very well for many data science applications. We use kinetic theory to improve understanding and existing methods. A microscopic simplified residual neural network (SimResNet) model is studied as the limit of infinitely many inputs. This leads to kinetic formulations of the SimResNet and we analyze those with respect to sensitivities and steady states. Aggregation phenomena in the case of a linear activation function are also studied. In addition the analysis is validated by numerics. In particular, results on a clustering and regression problem are presented.
KEYWORDS	Residual neural network, continuous limit, mean field equation, kinetic equation, machine learning application