
Stochastic Polyak Stepsize with a Moving Target
We propose a new stochastic gradient method that uses recorded past loss...
read it

Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization
We introduce MADGRAD, a novel optimization method in the family of AdaGr...
read it

Dual Averaging is Surprisingly Effective for Deep Learning Optimization
Firstorder stochastic optimization methods are currently the most widel...
read it

Understanding the Role of Momentum in NonConvex Optimization: Practical Insights from a Lyapunov Analysis
Momentum methods are now used pervasively within the machine learning co...
read it

On the convergence of the Stochastic Heavy Ball Method
We provide a comprehensive analysis of the Stochastic Heavy Ball (SHB) m...
read it

Factorial Powers for Stochastic Optimization
The convergence rates for convex and nonconvex optimization methods dep...
read it

EndtoEnd Variational Networks for Accelerated MRI Reconstruction
The slow acquisition speed of magnetic resonance imaging (MRI) has led t...
read it

MRI Banding Removal via Adversarial Training
MRI images reconstructed from subsampled data using deep learning techn...
read it

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
Purpose: To advance research in the field of machine learning for MR ima...
read it

Offset Masking Improves Deep Learning based Accelerated MRI Reconstructions
Deep learning approaches to accelerated MRI take a matrix of sampled Fou...
read it

GrappaNet: Combining Parallel Imaging with Deep Learning for MultiCoil MRI Reconstruction
Magnetic Resonance Image (MRI) acquisition is an inherently slow process...
read it

Scaling Laws for the Principled Design, Initialization and Preconditioning of ReLU Networks
In this work, we describe a set of rules for the design and initializati...
read it

Methods of interpreting error estimates for grayscale image reconstructions
One representation of possible errors in a grayscale image reconstructio...
read it

On the Curved Geometry of Accelerated Optimization
In this work we propose a differential geometric motivation for Nesterov...
read it

Controlling Covariate Shift using Equilibrium Normalization of Weights
We introduce a new normalization technique that exhibits the fast conver...
read it

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning
The application of stochastic variance reduction to optimization has sho...
read it

fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measuremen...
read it

A Simple Practical Accelerated Method for Finite Sums
We describe a novel optimization method for finite sums (such as empiric...
read it

New Optimisation Methods for Machine Learning
A thesis submitted for the degree of Doctor of Philosophy of The Austral...
read it

NonUniform Stochastic Average Gradient Method for Training Conditional Random Fields
We apply stochastic average gradient (SAG) algorithms for training condi...
read it

A Comparison of learning algorithms on the Arcade Learning Environment
Reinforcement learning agents have traditionally been evaluated on small...
read it

SAGA: A Fast Incremental Gradient Method With Support for NonStrongly Convex Composite Objectives
In this work we introduce a new optimisation method called SAGA in the s...
read it
Aaron Defazio
is this you? claim profile