Webinar Series: Mathematical Foundations of Data Science

Derivative-Free Optimization of Noisy Functions

Speaker

Jorge Nocedal, (Northwestern)

Abstract

The most successful method for derivative-free optimization developed by the optimization community in the last 2 decades was proposed by Powell. It creates quadratic models using a minimum change principle in conjunction with function interpolation, and computes a step  by minimizing the model inside a trust region. This approach works surprisingly well when functions contain noise. Nevertheless, we claim that a more flexible approach is based on approximating derivatives of noisy functions (using smoothing techniques) and using them within a quasi-Newton method. To ensure stability we propose a noise-tolerant quasi-Newton updating technique and provide its supporting theory. The efficiency of the method is demonstrated on unconstrained, nonlinear least squares, and constrained optimization problems with noise.

Bio

Nocedal was born and raised in Mexico. He obtained a B.Sc. in physics from the National University of Mexico in 1974. From 1974 to 1978, Nocedal studied at Rice University where he obtained a PhD in mathematical sciences under the supervision of Richard A. Tapia. Prior to joining Northwestern University in 1983, Nocedal spent three years (1978-1981) as an assistant professor at the National University of Mexico and two years (1981-1983) as a research assistant at the Courant Institute of Mathematical Sciences at NYU. Nocedal joined the Electrical Engineering and Computer Sciences department at Northwestern University in 1983. He held this appointment until 2012, before joining the Industrial Engineering and Management Sciences department, where he served as the David and Karen Sachs Professor and Chair from 2013 to 2017.

Nocedal has won numerous awards in the fields of nonlinear optimization, applied mathematics and operations research. In 1998, he was the invited speaker to the International Congress of Mathematicians in Berlin.[9][10] He was named an ISI Highly Cited Researcher in 2004. He received the George B. Dantzig Prize in 2012[11] and the Charles Broyden Prize in 2009.[12] He was also named a SIAM Fellow in 2010.[13] In 2017, he received the INFORMS John Von Neumann Theory Prize.[14] More recently, he was elected to the National Academy of Engineering in 2020.