Newton step methods for AD of an objective defined using implicit functions

作     者：Bell, Bradley M. Kristensen, Kasper 

作者机构：Univ Washington IHME Seattle WA 98195 USA Tech Univ Lyngby DTU Aqua Lyngby Denmark 

出 版 物：《OPTIMIZATION METHODS & SOFTWARE》 (最优化方法与软件)

年 卷 期：2018年第33卷第4-6期

页      面：907-923页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 07[理学] 070105[理学-运筹学与控制论] 0835[工学-软件工程] 0701[理学-数学] 

基　　金：Bill & Melinda Gates Foundation 

主　　题：implicit functions automatic derivatives higher order derivatives Newton step optimal control nonlinear mixed effects 

摘      要：We consider the problem of computing derivatives of an objective that is defined using implicit functions;i.e., implicit variables are computed by solving equations that are often nonlinear and solved by an iterative process. If one were to apply Algorithmic Differentiation (AD) directly, one would differentiate the iterative process. In this paper we present the Newton step methods for computing derivatives of the objective. These methods make it easy to take advantage of sparsity, forward mode, reverse mode, and other AD techniques. We prove that the partial Newton step method works if the number of steps is equal to the order of the derivatives. The full Newton step method obtains two derivatives order for each step except for the first step. There are alternative methods that avoid differentiating the iterative process;e.g., the method implemented in ADOL-C. An optimal control example demonstrates the advantage of the Newton step methods when computing both gradients and Hessians. We also discuss the Laplace approximation method for nonlinear mixed effects models as an example application.