You Already Have It: A Generator-Free Low-Precision DNN Training Framework using Stochastic Rounding
TimeTuesday, July 12th6pm - 7pm PDT
LocationLevel 2 Lobby
DescriptionTo generate random numbers for use in the stochastic rounding technique, a large number of random number generators are used in conventional designs.
Therefore, extra hardware costs are introduced inevitably.
In this paper, we innovatively propose to employ the stochastic property of the neural network training process and directly extract random numbers from neural networks in a self-sufficient manner.
We explore the quality of the obtained random numbers from different sources and methods, and propose a generator-free framework for low-precision neural network training.
We validate the effectiveness of our framework on a variety of deep learning tasks.