Stochastic Gradient Descent Articles

Page 1 of 1 (3 articles)

4/23/2021 • EN

On Information Theoretic Bounds for SGD

Explores how mutual information and KL divergence can be used to derive information-theoretic generalization bounds for Stochastic Gradient Descent (SGD).

Generalization Bounds Information Theory Kl Divergence Machine Learning Stochastic Gradient Descent

Ferenc Huszár

4/1/2021 • EN

Notes on the Origin of Implicit Regularization in SGD

Explores how Stochastic Gradient Descent (SGD) inherently prefers certain minima, leading to better generalization in deep learning, beyond classical theory.

Deep Learning Generalization Implicit Regularization Optimization Algorithms Stochastic Gradient Descent

Ferenc Huszár

9/5/2019 • EN

Evolution Strategies

An introduction to Evolution Strategies (ES) as a black-box optimization alternative to gradient descent, with applications in deep reinforcement learning.

Black Box Optimization Deep Reinforcement Learning Evolution Strategies Evolutionary Algorithms Stochastic Gradient Descent

Lilian Weng

Stochastic Gradient Descent Articles

On Information Theoretic Bounds for SGD

Notes on the Origin of Implicit Regularization in SGD

Evolution Strategies

Select Language