- Wei Hu, Lechao Xiao, Jeffrey Pennington:
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks. CoRR abs/2001.05992 (2020) - Jiri Hron, Yasaman Bahri, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein:
Exact posterior distributions of wide Bayesian neural networks. CoRR abs/2006.10541 (2020) - Wei Hu, Lechao Xiao, Ben Adlam, Jeffrey Pennington:
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks. CoRR abs/2006.14599 (2020) - Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein:
Finite Versus Infinite Neural Networks: an Empirical Study. CoRR abs/2007.15801 (2020) - Ben Adlam, Jeffrey Pennington:
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization. CoRR abs/2008.06786 (2020) - Atish Agarwala, Jeffrey Pennington, Yann N. Dauphin, Samuel S. Schoenholz:
Temperature check: theory and practice for training models with softmax-cross-entropy losses. CoRR abs/2010.07344 (2020) - Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington, Jasper Snoek:
Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit. CoRR abs/2010.07355 (2020) - Ben Adlam, Jeffrey Pennington:
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition. CoRR abs/2011.03321 (2020) - 2019
- Krzysztof Choromanski, Aldo Pacchiano, Jeffrey Pennington, Yunhao Tang:
KAMA-NNs: Low-dimensional Rotation Based Neural Networks. AISTATS 2019: 236-245 - Jeritt G. Thayer, Jeffrey M. Miller, Jeffrey W. Pennington:
Fault-Tolerant, Distributed, and Scalable Natural Language Processing with cTAKES. AMIA 2019 - Aaron J. Masino, Daniel Forsyth, Heather Nuske
, John Herrington, Jeffrey W. Pennington, Yelena Kushleyeva, Christopher P. Bonafide:
m-Health and Autism: Recognizing Stress and Anxiety with Machine Learning and Wearables Data. CBMS 2019: 714-719 - Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein:
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. ICLR (Poster) 2019 - Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington:
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. NeurIPS 2019: 8570-8581 - Dar Gilboa, Bo Chang, Minmin Chen, Greg Yang, Samuel S. Schoenholz, Ed H. Chi, Jeffrey Pennington:
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs. CoRR abs/1901.08987 (2019) - Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Jascha Sohl-Dickstein, Jeffrey Pennington:
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. CoRR abs/1902.06720 (2019) - Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, Samuel S. Schoenholz:
A Mean Field Theory of Batch Normalization. CoRR abs/1902.08129 (2019) - Ben Adlam, Jake Levinson, Jeffrey Pennington:
A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning. CoRR abs/1912.00827 (2019) - Lechao Xiao, Jeffrey Pennington, Samuel S. Schoenholz:
Disentangling trainability and generalization in deep learning. CoRR abs/1912.13053 (2019) - 2018
- Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli:
The emergence of spectral universality in deep networks. AISTATS 2018: 1924-1932 - Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein:
Deep Neural Networks as Gaussian Processes. ICLR (Poster) 2018 - Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein:
Sensitivity and Generalization in Neural Networks: an Empirical Study. ICLR (Poster) 2018 - Minmin Chen, Jeffrey Pennington, Samuel S. Schoenholz:
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks. ICML 2018: 872-881 - Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington:
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. ICML 2018: 5389-5398 - Jeffrey Pennington, Pratik Worah:
The Spectrum of the Fisher Information Matrix of a Single-Hidden-Layer Neural Network. NeurIPS 2018: 5415-5424 - Roman Novak, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein:
Sensitivity and Generalization in Neural Networks: an Empirical Study. CoRR abs/1802.08760 (2018) - Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli:
The Emergence of Spectral Universality in Deep Networks. CoRR abs/1802.09979 (2018) - Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington:
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. CoRR abs/1806.05393 (2018) - Minmin Chen, Jeffrey Pennington, Samuel S. Schoenholz:
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks. CoRR abs/1806.05394 (2018) - Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein:
Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. CoRR abs/1810.05148 (2018)