NAG Library Manual, Mark 27.1

Interfaces: FL CL CPP AD

NAG AD Library Introduction

E04 (Opt) Chapter Contents

E04 (Opt) Chapter Introduction (FL)

e04gb: FL CL CPP AD

NAG AD Library
e04gb_a1w_f (lsq_uncon_quasi_deriv_comp_a1w)

Note: a1w denotes that first order adjoints are computed in working precision; this has the corresponding argument type nagad_a1w_w_rtype. Also available is the t1w (first order tangent linear) mode, the interface of which is implied by replacing a1w by t1w throughout this document. Additionally, the p0w (passive interface, as alternative to the FL interface) mode is available and can be inferred by replacing of active types by the corresponding passive types. The method of codifying AD implementations in the routine name and corresponding argument types is described in the NAG AD Library Introduction.

Keyword Search:

NAG Library Manual, Mark 27.1

Interfaces: FL CL CPP AD

NAG AD Library Introduction

E04 (Opt) Chapter Contents

E04 (Opt) Chapter Introduction (FL)

e04gb: FL CL CPP AD

▸▿ Contents

2 Specification

▸▿ 3 Description

▸▿ 3.1 Symbolic Adjoint

3.1.1 Mathematical Background

3.1.2 Usable Adjoints

4 References

5 Arguments

6 Error Indicators and Warnings

8 Parallelism and Performance

9 Further Comments

▸▿ 10 Example

10.1 Adjoint mode (a1w)

10.2 Tangent mode (t1w)

10.3 Passive mode (p0w)

© The Numerical Algorithms Group Ltd. 2020

1 Purpose

e04gb_a1w_f is the adjoint version of the primal routine e04gbf. Depending on the value of ad_handle, e04gb_a1w_f uses algorithmic differentiation or symbolic adjoints to compute adjoints of the primal.

2 Specification

Fortran Interface

Subroutine e04gb_a1w_f (

ad_handle, m, n, selct, lsqfun, lsqmon, iprint, maxcal, eta, xtol, stepmx, x, fsumsq, fvec, fjac, ldfjac, s, v, ldv, niter, nf, iw, liw, w, lw, iuser, ruser, ifail)

Integer, Intent (In)	::	m, n, selct, iprint, maxcal, ldfjac, ldv, liw, lw
Integer, Intent (Inout)	::	iw(liw), iuser(*), ifail
Integer, Intent (Out)	::	niter, nf
Type (nagad_a1w_w_rtype), Intent (In)	::	eta, xtol, stepmx
Type (nagad_a1w_w_rtype), Intent (Inout)	::	x(n), fjac(ldfjac,n), v(ldv,n), w(lw), ruser(*)
Type (nagad_a1w_w_rtype), Intent (Out)	::	fsumsq, fvec(m), s(n)
Type (c_ptr), Intent (Inout)	::	ad_handle
External	::	lsqfun, lsqmon

C++ Header Interface

#include <nagad.h>

void e04gb_a1w_f_ (

void *&ad_handle, const Integer &m, const Integer &n, const Integer &selct,
void (NAG_CALL lsqfun)(void *&ad_handle, Integer &iflag, const Integer &m, const Integer &n, const nagad_a1w_w_rtype xc[], nagad_a1w_w_rtype fvec[], nagad_a1w_w_rtype fjac[], const Integer &ldfjac, Integer iuser[], nagad_a1w_w_rtype ruser[]),
void (NAG_CALL lsqmon)(void *&ad_handle, const Integer &m, const Integer &n, const nagad_a1w_w_rtype xc[], const nagad_a1w_w_rtype fvec[], const nagad_a1w_w_rtype fjac[], const Integer &ldfjac, const nagad_a1w_w_rtype s[], const Integer &igrade, const Integer &niter, const Integer &nf, Integer iuser[], nagad_a1w_w_rtype ruser[]),
const Integer &iprint, const Integer &maxcal, const nagad_a1w_w_rtype &eta, const nagad_a1w_w_rtype &xtol, const nagad_a1w_w_rtype &stepmx, nagad_a1w_w_rtype x[], nagad_a1w_w_rtype &fsumsq, nagad_a1w_w_rtype fvec[], nagad_a1w_w_rtype fjac[], const Integer &ldfjac, nagad_a1w_w_rtype s[], nagad_a1w_w_rtype v[], const Integer &ldv, Integer &niter, Integer &nf, Integer iw[], const Integer &liw, nagad_a1w_w_rtype w[], const Integer &lw, Integer iuser[], nagad_a1w_w_rtype ruser[], Integer &ifail)

The routine may be called by the names e04gb_a1w_f or nagf_opt_lsq_uncon_quasi_deriv_comp_a1w. The corresponding t1w and p0w variants of this routine are also available.

3 Description

e04gb_a1w_f is the adjoint version of the primal routine e04gbf.

e04gbf is a comprehensive quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of

m

nonlinear functions in

n

variables

(m \geq n)

. First derivatives are required. The routine is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities). For further information see Section 3 in the documentation for e04gbf.

3.1 Symbolic Adjoint

e04gb_a1w_f can provide symbolic adjoints by setting the symbolic mode as described in Section 3.2.2 in the X10 Chapter introduction. Please see Section 4 in the Introduction to the NAG AD Library for API description on how to use symbolic adjoints.

In comparison to the algorithmic adjoint, the user-supplied primal and adjoint callbacks need specific implementation to support symbolic adjoint computation. Please see Section 4.5 in the Introduction to the NAG AD Library and recall what primal and adjoint callbacks need to calculate in the case of an algorithmic adjoint.

Assuming the original user-supplied function evaluates

(z, g) = (f (x, p), \nabla_{x} f (x, p)),

(1)

where

p

is given by the w or by use of COMMON globals. The variables

x

,

z

and

g

correspond to xc, fvec and fjac of lsqfun. The symbolic adjoint of e04gbf then also requires the following capability / modes:

(a)Function value evaluation only.
(b)Function value evaluation and adjoint computation w.r.t. xc only (corresponds to $x$ in the following equation), i.e.,
$x_{(1)} + = {[\nabla_{x} f (x, p)]}^{T} z_{(1)} + {[\nabla_{x}^{2} f (x, p)]}^{T} g_{(1)}$ (2)
(c)Function value evaluation and adjoint computation w.r.t. $p$ only, i.e.,
$p_{(1)} + = {[\nabla_{p} f (x, p)]}^{T} z_{(1)} + {[\nabla_{x, p}^{2} f (x, p)]}^{T} g_{(1)} .$ (3)

Here $p$ is a placeholder for any user variable either passed via the user segment of w or via COMMON global variables.

3.1.1 Mathematical Background

To be more specific, the symbolic adjoint solves

[\nabla_{x}^{2} F (x, p)] z = - x_{(1)}

(4)

followed by an adjoint projection through the user-supplied adjoint routine

p_{(1) k} = \sum_{j = 1}^{n} \frac{\partial^{2} F (x, p)}{\partial x_{j} \partial p_{k}} z_{j} = 2 \sum_{j = 1}^{n} \sum_{i = 1}^{m} [\frac{\partial f_{i}}{\partial p_{k}} \frac{\partial f_{i}}{\partial x_{j}} z_{j} + f_{i} \frac{\partial^{2} f_{i}}{\partial x_{j} \partial p_{k}} z_{j}] .

(5)

The Hessian

\nabla_{x}^{2} F (x, p)

as well as the mixed derivative tensor

\frac{d^{2} F (x, p)}{d x_{j} d p_{k}}

is computed using the user-supplied adjoint routine.

Please see Du Toit and Naumann (2017), Naumann et al. (2017) and Giles (2017) for reference.

3.1.2 Usable Adjoints

You can set or access the adjoints of output arguments x, fvec, fjac and fsumsq. The adjoints of all other output arguments are ignored.

e04gb_a1w_f increments the adjoints of the variable

p

, where

p

is given by the argument w or by use of COMMON globals (see (1)).

4 References

Du Toit J, Naumann U (2017) Adjoint Algorithmic Differentiation Tool Support for Typical Numerical Patterns in Computational Finance

Giles M (2017) Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation

Gill P E and Murray W (1978) Algorithms for the solution of the nonlinear least squares problem SIAM J. Numer. Anal. 15 977–992

Naumann U, Lotz J, Leppkes K and Towara M (2017) Algorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations

5 Arguments

In addition to the arguments present in the interface of the primal routine, e04gb_a1w_f includes some arguments specific to AD.

A brief summary of the AD specific arguments is given below. For the remainder, links are provided to the corresponding argument from the primal routine. A tooltip popup for all arguments can be found by hovering over the argument name in Section 2 and in this section.

1: ad_handle – Type (c_ptr) Input/Output

On entry: a handle to the AD configuration data object, as created by x10aa_a1w_f. Symbolic adjoint mode may be selected by calling x10aa_a1w_f with this handle.

2: m – Integer Input

3: n – Integer Input

4: selct – Integer Input

selct enables you to specify whether the linear minimizations (i.e., minimizations of

F (x^{(k)} + α^{(k)} p^{(k)})

with respect to

α^{(k)}

) are to be performed by a routine which just requires the evaluation of the

f_{i} (x)

(

selct = 1

), or by a routine which also requires the first derivatives of the

f_{i} (x)

(

selct = 2

).

5: lsqfun – Subroutine External Procedure

The specification of lsqfun is:

Fortran Interface

Subroutine lsqfun (

ad_handle, iflag, m, n, xc, fvec, fjac, ldfjac, iuser, ruser)

Integer, Intent (In)	::	m, n, ldfjac
Integer, Intent (Inout)	::	iflag, iuser(*)
Type (nagad_a1w_w_rtype), Intent (In)	::	xc(n)
Type (nagad_a1w_w_rtype), Intent (Inout)	::	fjac(ldfjac,n), ruser(*)
Type (nagad_a1w_w_rtype), Intent (Out)	::	fvec(m)
Type (c_ptr), Intent (Inout)	::	ad_handle

C++ Header Interface

#include <nagad.h>

void lsqfun (

void *&ad_handle, Integer &iflag, const Integer &m, const Integer &n, const nagad_a1w_w_rtype xc[], nagad_a1w_w_rtype fvec[], nagad_a1w_w_rtype fjac[], const Integer &ldfjac, Integer iuser[], nagad_a1w_w_rtype ruser[])

1: ad_handle – Type (c_ptr) Input/Output: On entry: a handle to the AD configuration data object.
2: iflag – Integer Input/Output
3: m – Integer Input
4: n – Integer Input
5: xc – Type (nagad_a1w_w_rtype) array Input
6: fvec – Type (nagad_a1w_w_rtype) array Output
7: fjac – Type (nagad_a1w_w_rtype) array Output
8: ldfjac – Integer Input
9: iuser( $*$ ) – Integer array User Workspace
10: ruser( $*$ ) – Type (nagad_a1w_w_rtype) array User Workspace

6: lsqmon – Subroutine External Procedure

The specification of lsqmon is:

Fortran Interface

Subroutine lsqmon (

ad_handle, m, n, xc, fvec, fjac, ldfjac, s, igrade, niter, nf, iuser, ruser)

Integer, Intent (In)	::	m, n, ldfjac, igrade, niter, nf
Integer, Intent (Inout)	::	iuser(*)
Type (nagad_a1w_w_rtype), Intent (In)	::	xc(n), fvec(m), fjac(ldfjac,n), s(n)
Type (nagad_a1w_w_rtype), Intent (Inout)	::	ruser(*)
Type (c_ptr), Intent (Inout)	::	ad_handle

C++ Header Interface

#include <nagad.h>

void lsqmon (

void *&ad_handle, const Integer &m, const Integer &n, const nagad_a1w_w_rtype xc[], const nagad_a1w_w_rtype fvec[], const nagad_a1w_w_rtype fjac[], const Integer &ldfjac, const nagad_a1w_w_rtype s[], const Integer &igrade, const Integer &niter, const Integer &nf, Integer iuser[], nagad_a1w_w_rtype ruser[])

1: ad_handle – Type (c_ptr) Input/Output: On entry: a handle to the AD configuration data object.
2: m – Integer Input
3: n – Integer Input
4: xc – Type (nagad_a1w_w_rtype) array Input
5: fvec – Type (nagad_a1w_w_rtype) array Input
6: fjac – Type (nagad_a1w_w_rtype) array Input
7: ldfjac – Integer Input
8: s – Type (nagad_a1w_w_rtype) array Input
9: igrade – Integer Input
10: niter – Integer Input
11: nf – Integer Input
12: iuser( $*$ ) – Integer array User Workspace
13: ruser( $*$ ) – Type (nagad_a1w_w_rtype) array User Workspace

7: iprint – Integer Input

8: maxcal – Integer Input

9: eta – Type (nagad_a1w_w_rtype) Input

10: xtol – Type (nagad_a1w_w_rtype) Input

11: stepmx – Type (nagad_a1w_w_rtype) Input

12: x(n) – Type (nagad_a1w_w_rtype) array Input/Output

13: fsumsq – Type (nagad_a1w_w_rtype) Output

14: fvec(m) – Type (nagad_a1w_w_rtype) array Output

15: fjac(ldfjac, n) – Type (nagad_a1w_w_rtype) array Output

16: ldfjac – Integer Input

17: s(n) – Type (nagad_a1w_w_rtype) array Output

18: v(ldv, n) – Type (nagad_a1w_w_rtype) array Output

19: ldv – Integer Input

20: niter – Integer Output

21: nf – Integer Output

22: iw(liw) – Integer array Input/Output

Workspace.

23: liw – Integer Input

24: w(lw) – Type (nagad_a1w_w_rtype) array Input/Output

Workspace.

25: lw – Integer Input

26: iuser( $*$ ) – Integer array User Workspace

User workspace.

27: ruser( $*$ ) – Type (nagad_a1w_w_rtype) array User Workspace

User workspace.

28: ifail – Integer Input/Output

6 Error Indicators and Warnings

e04gb_a1w_f preserves all error codes from e04gbf and in addition can return:

$ifail = - 89$: An unexpected AD error has been triggered by this routine. Please contact NAG.
See Section 4.8.2 in the NAG AD Library Introduction for further information.

$ifail = - 899$: Dynamic memory allocation failed for AD.
See Section 4.8.1 in the NAG AD Library Introduction for further information.

In symbolic mode the following may be returned:

7 Accuracy

Not applicable.

8 Parallelism and Performance

e04gb_a1w_f is not threaded in any implementation.

9 Further Comments

None.

10 Example

The following examples are variants of the example for e04gbf, modified to demonstrate calling the NAG AD Library.

10.1 Adjoint mode (a1w)

Language	Source File	Data	Results
Fortran	e04gb_a1w_fe.f90	e04gb_a1w_fe.d	e04gb_a1w_fe.r
C++	e04gb_a1w_hcppe.cpp	e04gb_a1w_hcppe.d	e04gb_a1w_hcppe.r

10.2 Tangent mode (t1w)

Language	Source File	Data	Results
Fortran	e04gb_t1w_fe.f90	e04gb_t1w_fe.d	e04gb_t1w_fe.r
C++	e04gb_t1w_hcppe.cpp	e04gb_t1w_hcppe.d	e04gb_t1w_hcppe.r

10.3 Passive mode (p0w)

Language	Source File	Data	Results
Fortran	e04gb_p0w_fe.f90	e04gb_p0w_fe.d	e04gb_p0w_fe.r
C++	e04gb_p0w_hcppe.cpp	e04gb_p0w_hcppe.d	e04gb_p0w_hcppe.r

NAG Library Manual, Mark 27.1

Interfaces: FL CL CPP AD

NAG AD Library Introduction

E04 (Opt) Chapter Contents

E04 (Opt) Chapter Introduction (FL)

e04gb: FL CL CPP AD

© The Numerical Algorithms Group Ltd. 2020