Webinar Series: Mathematical Foundations of Data Science

Optimization with Momentum: Dynamical, Variational, and Symplectic Perspectives

Abstract

We analyze momentum-based optimization algorithms from dynamical systems and Hamiltonian points of view.  We show that a continuous-time perspective provides insight into the role of geometry in determining the convergence rates of optimization algorithms.  We discuss a generalization of symplectic integration to dissipative Hamiltonian systems that allows us to translate continuous-time rates of convergence into discrete time, with the error controlled by a backwards error analysis. Finally, we also show how to formulate lower bounds ala Nemirovskii in continuous time.  

Joint work with Michael Muehlebach, Guilherme Franca and Rene Vidal.

Bio

Michael I. Jordan is the Pehong Chen Distinguished Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics at the University of California, Berkeley. He received his Masters in Mathematics from Arizona State University,  and earned his PhD in Cognitive Science in 1985 from the University of  California, San Diego.  He was a professor at MIT from 1988 to 1998. His research interests bridge the computational, statistical, cognitive and biological sciences.  Prof. Jordan is a member of the National Academy of Sciences, a member of the National Academy of Engineering and a member of the American Academy of Arts and Sciences.  He is a Fellow of the American Association for the Advancement of Science. He has been named a Neyman Lecturer and a Medallion Lecturer by the Institute of Mathematical Statistics.  He was a Plenary Lecturer at the International Congress of Mathematicians in 2018.  He received the IEEE John von Neumann Medal in 2020, the IJCAI Research Excellence Award in 2016, the David E. Rumelhart Prize in 2015 and the ACM/AAAI Allen Newell Award in 2009.  He is a Fellow of the AAAI, ACM, ASA, CSS, IEEE, IMS, ISBA and SIAM.

In 2016, Professor Jordan was named the "most influential computer scientist" worldwide in an article in Science, based on rankings from the Semantic Scholar search engine.