Activities per year
Abstract
We introduce exact macroscopic on-line learning dynamics of two-layer neural networks with ReLU units in the form of a system of differential equations, using techniques borrowed from statistical physics. For the first experiments, numerical solutions reveal similar behavior compared to sigmoidal activation researched in earlier work. In these experiments the theoretical results show good correspondence with simulations. In ove-rrealizable and unrealizable learning scenarios, the learning behavior of ReLU networks shows distinctive characteristics compared to sigmoidal networks.
Original language | English |
---|---|
Journal | ArXiv |
Publication status | Published - 18-Mar-2019 |
Keywords
- cs.LG
- cond-mat.dis-nn
- stat.ML
-
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning
Michael Biehl (Attendee)
24-Apr-2019 → 26-Apr-2019Activity: Organising and attending an event › Attending an event › Academic
-
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning
Michiel Straat (Speaker)
24-Apr-2019 → 26-Apr-2019Activity: Talk and presentation › Academic presentation › Academic