|| IGPM500.pdf January 2020 |
|| Kinetic Theory for Residual Neural Networks |
||Michael Herty, Torsten Trimborn, Giuseppe Visconti
||Deep residual neural networks (ResNet) are performing very well for many data science
applications. We use kinetic theory to improve understanding and existing methods. A
microscopic simplified residual neural network (SimResNet) model is studied as the limit of
infinitely many inputs. This leads to kinetic formulations of the SimResNet and we analyze
those with respect to sensitivities and steady states. Aggregation phenomena in the case of a
linear activation function are also studied. In addition the analysis is validated by numerics.
In particular, results on a clustering and regression problem are presented.
|| Keywords Residual neural network, continuous limit, mean field equation, kinetic equation, machine learning application