检索结果-内蒙古大学图书馆

REDUCTION OF THE RANDOM ACCESS MEMORY SIZE IN ADJOINT ALGORITHMIC DIFFERENTIATION BY OVERLOADING

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Naumann, Uwe Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Germany

Adjoint algorithmic differentiation by operator and function overloading is based on the interpretation of directed acyclic graphs resulting from evaluations of numerical simulation programs. The size of the computer system memory required to store the graph grows proportional to the number of floating-point operations executed by the underlying program. It quickly exceeds the available memory resources. Naive adjoint algorithmic differentiation often becomes infeasible except for relatively simple numerical simulations. Access to the data associated with the graph can be classified as sequential and random. The latter refers to memory access patterns defined by the adjacency relationship between vertices within the graph. Sequentially accessed data can be decomposed into blocks. The blocks can be streamed across the system memory hierarchy thus extending the amount of available memory, for example, to hard discs. Asynchronous i/o can help to mitigate the increased cost due to accesses to slower memory. Much larger problem instances can thus be solved without resorting to technically challenging user intervention such as checkpointing. Randomly accessed data should not have to be decomposed. Its block-wise streaming is likely to yield a substantial overhead in computational cost due to data accesses across blocks. Consequently, the size of the randomly accessed memory required by an adjoint should be kept minimal in order to eliminate the need for decomposition. We propose a combination of dedicated memory for adjoint L-values with the exploitation of remainder bandwidth as a possible solution. Test results indicate significant savings in random access memory size while preserving overall computational efficiency. Copyright © 2022, The Authors. All rights reserved.

关键词： Bandwidth

Differential invariants

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Naumann, Uwe Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Germany

Validation is a major challenge in differentiable programming. The state of the art is based on algorithmic differentiation. Consistency of first-order tangent and adjoint programs is defined by a well-known first-order differential invariant. This paper generalizes the approach through derivation of corresponding differential invariants of arbitrary order. © 2021, CC BY-NC-ND.

关键词：

Numerical stability of tangents and adjoints of implicit functions

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Naumann, Uwe Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Germany

We investigate errors in tangents and adjoints of implicit functions resulting from errors in the primal solution due to approximations computed by a numerical solver. Adjoints of systems of linear equations turn out to be unconditionally numerically stable. Tangents of systems of linear equations can become instable as well as both tangents and adjoints of systems of nonlinear equations, which extends to optima of convex unconstrained objectives. Sufficient conditions for numerical stability are derived. © 2021, CC BY.

关键词： Nonlinear equations

On the computational complexity of the chain rule of differential calculus

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Naumann, Uwe Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University AachenD-52056 Germany

Many modern numerical methods in computational science and engineering rely on derivatives of mathematical models for the phenomena under investigation. The computation of these derivatives often represents the bottleneck in terms of overall runtime performance. First and higher derivative tensors need to be evaluated efficiently. The chain rule of differentiation is the fundamental prerequisite for computing accurate derivatives of composite functions which perform a potentially very large number of elemental function evaluations. Data flow dependences amongst the elemental functions give rise to a combinatorial optimization problem. We formulate Chain Rule Differentiation and we prove it to be NP-complete. Pointers to research on its approximate solution are *** Codes 26B05, 68Q17 © 2021, CC BY-NC-ND.

关键词： Differentiation (calculus)

Algorithmic Differentiation of Numerical Methods: Second-order Adjoint Solvers for Parameterized Systems of Nonlinear Equations

学校读者我要写书评

暂无评论

Procedia Computer Science 2016年 80卷 2231-2235页

作者： Niloofar Safiran Johannes Lotz Uwe Naumann LuFG Informatik 12: Software and Tools for Computational Engineering RWTH Aachen Germany

Adjoint mode algorithmic (also know as automatic) differentiation (AD) transforms implementations of multivariate vector functions as computer programs into first-order adjoint code. Its reapplication or combinations with tangent mode AD yields higher-order adjoint code. Second derivatives play an important role in nonlinear programming. For example, second-order (Newton-type) nonlinear optimization methods promise faster convergence in the neighborhood of the minimum through taking into account second derivative information. The adjoint mode is of particular interest in large-scale gradient-based nonlinear optimization due to the independence of its computational cost on the number of free variables. Part of the objective function may be given implicitly as the solution of a system of n parameterized nonlinear equations. If the system parameters depend on the free variables of the objective, then second derivatives of the nonlinear system's solution with respect to those parameters are required. The local computational overhead as well as the additional memory requirement for the computation of second-order adjoints of the solution vector with respect to the parameters by AD depends on the number of iterations performed by the nonlinear solver. This dependence can be eliminated by taking a symbolic approach to the differentiation of the nonlinear system.

关键词： Second-Order Adjoint Nonlinear Solver Algorithmic Differentiation

A Case Study in Adjoint Sensitivity Analysis of Parameter Calibration

学校读者我要写书评

暂无评论

Procedia Computer Science 2016年 80卷 201-211页

作者： Johannes Lotz Marc Schwalbach Uwe Naumann LuFG Informatik 12: Software and Tools for Computational Engineering RWTH Aachen Germany Von Karman Institute for Fluid Dynamics Belgium

Adjoint sensitivity computation of parameter estimation problems is a widely used technique in the field of computational science and engineering for retrieving derivatives of a cost functional with respect to parameters efficiently. Those derivatives can be used, e.g. for sensitivity analysis, optimization, or robustness analysis. Deriving and implementing adjoint code is an error-prone, non-trivial task which can be avoided by using Algorithmic Differentiation (AD) software. Generating adjoint code by AD software has the downside of usually requiring a huge amount of memory as well as a non-optimal run time. In this article, we couple two approaches for achieving both, a robust and efficient adjoint code: symbolically derived adjoint formulations and AD. Comparisons are carried out for a real-world case study originating from the remote atmospheric sensing simulation software JURASSIC developed at the Institute of Energy and Climate Research – Stratosphere, Research Center Jülich. We show, that the coupled approach outperforms the fully algorithmic approach by AD in terms of run time and memory requirement and argue that this can be achieved while still preserving the desireable feature of AD being automatic.

关键词： Adjoints Algorithmic Differentiation Optimization C++

Second-order Tangent Solvers for Systems of Parameterized Nonlinear Equations

学校读者我要写书评

暂无评论

Procedia Computer Science 2015年 51卷 231-238页

作者： Niloofar Safiran Johannes Lotz Uwe Naumann LuFG Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Germany

Forward mode algorithmic differentiation transforms implementations of multivariate vector functions as computer programs into first directional derivative (also: first-order tangent) code. Its reapplication yields higher directional derivative (higher-order tangent) code. Second derivatives play an important role in nonlinear programming. For example, second-order (Newton-type) nonlinear optimization methods promise faster convergence in the neighborhood of the minimum through taking into account second derivative information. Part of the objective function may be given implicitly as the solution of a system of n parameterized nonlinear equations. If the system parameters depend on the free variables of the objective, then second derivatives of the nonlinear system's solution with respect to those parameters are required. The local computational overhead for the computation of second-order tangents of the solution vector with respect to the parameters by Algorithmic Differentiation depends on the number of iterations performed by the nonlinear solver. This dependence can be eliminated by taking a second-order symbolic approach to differentiation of the nonlinear system.

关键词： Nonlinear System Second-Order Tangent Hessian Algorithmic Differentiation Symbolic Differentiation

Towards tangent-linear GPU programs using OpenACC

学校读者我要写书评

暂无评论

Towards tangent-linear GPU programs using OpenACC

4th Symposium on Information and Communication Technology, SoICT 2013

作者： Minh, Bui Tat Förster, Michael Naumann, Uwe Bangkok Thailand LuFG Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Aachen Germany

ISBN: (纸本)9781450324540

Recently, Graphics Processing Units(GPUs) have emerged as a very promisingly powerful resource in scientific computing. Algorithmic Differentiation is a technique to numerically evaluate first and higher derivatives of a function specified by a computer program efficiently up to machine precision. Derivative programs which are used to compute derivatives of functions are so-called tangent-linear program and adjoint program. This paper aims to offload any particular independent loop in tangent-linear program to GPUs. The proposed technique is OpenACC APIs for annotating an independent loop to be executed in parallel on GPUs. Our case study for OpenACC tangent-linear code shows an enormous speedup. OpenACC shows its simplicity of accelerating tangent-linear code by hiding the data movement between CPU and GPU memory. Copyright © 2013 ACM.

关键词： Program processors

Discrete adjoints of PETSc through dco/c++ and adjoint MPI

学校读者我要写书评

暂无评论

Discrete adjoints of PETSc through dco/c++ and adjoint MPI

19th International Conference on Parallel Processing, Euro-Par 2013

作者： Lotz, Johannes Naumann, Uwe Sagebaum, Max Schanen, Michel LuFG Informatik 12: Software and Tools for Computational Engineering RWTH Aachen University Germany Department of Mathematics Center for Computational Engineering Science RWTH Aachen University Germany

ISBN: (纸本)9783642400469

PETSc's [1] robustness, scalability and portability makes it the foundation of various parallel implementations of numerical simulation codes. We formulate a least squares problem using a PETSc implementation as the model function and rely on adjoint mode Algorithmic Differentiation (AD) [2] for the accumulation of the derivative information. Various AD tools exist that apply the adjoint model to a given C/C++ code, while none is able to differentiate MPI [3] enabled code. We solved this by combining dco/c++ and the Adjoint MPI library, leading to a fully discrete adjoint implementation of PETSc. We want to underline that this work differs from accumulating derivative information through AD for PETSc algorithms (see e.g. [4]). We compute derivative information of PETSc itself opening up the possibility of an enclosing optimization problem (as needed, e.g., by [5]). © 2013 Springer-Verlag.

关键词： Codes (symbols)