NAG AD Library
e04gb (lsq_uncon_quasi_deriv_comp)

Settings help

AD Name Style:


AD Specification Language:

1 Purpose

e04gb is the AD Library version of the primal routine e04gbf. Based (in the C++ interface) on overload resolution, e04gb can be used for primal, tangent and adjoint evaluation. It supports tangents and adjoints of first and second order. The parameter ad_handle can be used to choose whether adjoints are computed using a symbolic adjoint or straightforward algorithmic differentiation. In addition, the routine has further optimisations when symbolic expert strategy is selected.

2 Specification

Fortran Interface
Subroutine e04gb_AD_f ( ad_handle, m, n, selct, lsqfun, lsqmon, iprint, maxcal, eta, xtol, stepmx, x, fsumsq, fvec, fjac, ldfjac, s, v, ldv, niter, nf, iuser, ruser, ifail)
Integer, Intent (In) :: m, n, selct, iprint, maxcal, ldfjac, ldv
Integer, Intent (Inout) :: iuser(*), ifail
Integer, Intent (Out) :: niter, nf
ADTYPE, Intent (In) :: eta, xtol, stepmx
ADTYPE, Intent (Inout) :: x(n), fjac(ldfjac,n), v(ldv,n), ruser(*)
ADTYPE, Intent (Out) :: fsumsq, fvec(m), s(n)
Type (c_ptr), Intent (Inout) :: ad_handle
External :: lsqfun, lsqmon
Corresponding to the overloaded C++ function, the Fortran interface provides five routines with names reflecting the type used for active real arguments. The actual subroutine and type names are formed by replacing AD and ADTYPE in the above as follows:
when ADTYPE is Real(kind=nag_wp) then AD is p0w
when ADTYPE is Type(nagad_a1w_w_rtype) then AD is a1w
when ADTYPE is Type(nagad_t1w_w_rtype) then AD is t1w
when ADTYPE is Type(nagad_a1t1w_w_rtype) then AD is a1t1w
when ADTYPE is Type(nagad_t2w_w_rtype) then AD is t2w
C++ Interface
#include <dco.hpp>
#include <nagad.h>
namespace nag {
namespace ad {
template <typename LSQFUN_T, typename LSQMON_T>
void e04gb ( handle_t &ad_handle, const Integer &m, const Integer &n, const Integer &selct, LSQFUN_T &&lsqfun, LSQMON_T &&lsqmon, const Integer &iprint, const Integer &maxcal, const ADTYPE &eta, const ADTYPE &xtol, const ADTYPE &stepmx, ADTYPE x[], ADTYPE &fsumsq, ADTYPE fvec[], ADTYPE fjac[], const Integer &ldfjac, ADTYPE s[], ADTYPE v[], const Integer &ldv, Integer &niter, Integer &nf, Integer &ifail)
}
}
The function is overloaded on ADTYPE which represents the type of active arguments. ADTYPE may be any of the following types:
double,
dco::ga1s<double>::type,
dco::gt1s<double>::type,
dco::gt1s<dco::gt1s<double>::type>::type,
dco::ga1s<dco::gt1s<double>::type>::type
Note: this function can be used with AD tools other than dco/c++. For details, please contact NAG.

3 Description

e04gb is the AD Library version of the primal routine e04gbf.
e04gbf is a comprehensive quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of m nonlinear functions in n variables (mn). First derivatives are required. The routine is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities). For further information see Section 3 in the documentation for e04gbf.

3.1 Symbolic Adjoint

3.1.1 Symbolic Strategy

Symbolic strategy may be selected by calling ad_handle.set_strategy(nag::ad::symbolic) prior to calling e04gb. No further changes are needed compared to using the algorithmic strategy.

3.1.2 Symbolic Expert Strategy

Symbolic expert strategy may be selected by calling ad_handle.set_strategy(nag::ad::symbolic_expert) prior to calling e04gb. In contrast to the symbolic strategy, in symbolic expert strategy the user-supplied primal callback needs a specific implementation to support symbolic computation, but this can improve overall performance. See the example e04gb_a1_sym_expert_dcoe.cpp for details.

3.1.3 Mathematical Background

To be more specific, the symbolic adjoint solves
[x2F(x,p)]z=-x(1) (1)
followed by an adjoint projection through the user-supplied adjoint routine
p(1)k= j=1 n 2F(x,p)xjpkzj=2 j=1 n i=1 m [fipkfixjzj+fi2fixjpkzj]. (2)
The Hessian x2F(x,p) as well as the mixed derivative tensor d2F(x,p)dxjdpk is computed using the user-supplied adjoint routine.
Please see Du Toit and Naumann (2017), Naumann et al. (2017) and Giles (2017) for reference.

3.1.4 Usable Adjoints

You can set or access the adjoints of output arguments x, fvec, fjac and fsumsq. The adjoints of all other output arguments are ignored.
e04gb increments the adjoints of the variable p, where p is the parameter used in the function callback lsqfun.

4 References

Du Toit J, Naumann U (2017) Adjoint Algorithmic Differentiation Tool Support for Typical Numerical Patterns in Computational Finance
Giles M (2017) Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation
Gill P E and Murray W (1978) Algorithms for the solution of the nonlinear least squares problem SIAM J. Numer. Anal. 15 977–992
Naumann U, Lotz J, Leppkes K and Towara M (2017) Algorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations

5 Arguments

In addition to the arguments present in the interface of the primal routine, e04gb includes some arguments specific to AD.
A brief summary of the AD specific arguments is given below. For the remainder, links are provided to the corresponding argument from the primal routine. A tooltip popup for all arguments can be found by hovering over the argument name in Section 2 and in this section.
1: ad_handlenag::ad::handle_t Input/Output
On entry: a configuration object that holds information on the differentiation strategy. Details on setting the AD strategy are described in AD handle object and AD Strategies in the NAG AD Library Introduction.
2: m – Integer Input
3: n – Integer Input
4: selct – Integer Input
selct enables you to specify whether the linear minimizations (i.e., minimizations of F(x (k) +α (k) p (k) ) with respect to α(k)) are to be performed by a routine which just requires the evaluation of the fi(x) (selct = 1), or by a routine which also requires the first derivatives of the fi(x) (selct = 2).
5: lsqfun – Callable Input
lsqfun needs to be callable with the specification listed below. This can be a C++ lambda, a functor or a (static member) function pointer. If using a lambda, parameters can be captured safely by reference. No copies of the callable are made internally.
The specification of lsqfun is:
Fortran Interface
Subroutine lsqfun ( ad_handle, iflag, m, n, xc, fvec, fjac, ldfjac, iuser, ruser)
Integer, Intent (In) :: m, n, ldfjac
Integer, Intent (Inout) :: iflag, iuser(*)
ADTYPE, Intent (In) :: xc(n)
ADTYPE, Intent (Inout) :: fjac(ldfjac,n), ruser(*)
ADTYPE, Intent (Out) :: fvec(m)
Type (c_ptr), Intent (Inout) :: ad_handle
C++ Interface
auto lsqfun = [&]( const handle_t &ad_handle, Integer &iflag, const Integer &m, const Integer &n, const ADTYPE xc[], ADTYPE fvec[], ADTYPE fjac[], const Integer &ldfjac)
1: ad_handlenag::ad::handle_t Input/Output
On entry: a handle to the AD handle object.
2: iflag – Integer Input/Output
3: m – Integer Input
4: n – Integer Input
5: xcADTYPE array Input
6: fvecADTYPE array Output
7: fjacADTYPE array Output
8: ldfjac – Integer Input
*: iuser(*) – Integer array User Workspace
*: ruser(*)ADTYPE array User Workspace
6: lsqmon – Callable Input
lsqmon needs to be callable with the specification listed below. This can be a C++ lambda, a functor or a (static member) function pointer. If using a lambda, parameters can be captured safely by reference. No copies of the callable are made internally.
If a null pointer is used as the argument, then a NAG supplied routine will be used as the argument for this parameter (C++ only).
For the Fortran interface, the NAG supplied routine e04fd_AD_z may be used as the actual argument for this parameter.
The specification of lsqmon is:
Fortran Interface
Subroutine lsqmon ( ad_handle, m, n, xc, fvec, fjac, ldfjac, s, igrade, niter, nf, iuser, ruser)
Integer, Intent (In) :: m, n, ldfjac, igrade, niter, nf
Integer, Intent (Inout) :: iuser(*)
ADTYPE, Intent (In) :: xc(n), fvec(m), fjac(ldfjac,n), s(n)
ADTYPE, Intent (Inout) :: ruser(*)
Type (c_ptr), Intent (Inout) :: ad_handle
C++ Interface
auto lsqmon = [&]( const handle_t &ad_handle, const Integer &m, const Integer &n, const ADTYPE xc[], const ADTYPE fvec[], const ADTYPE fjac[], const Integer &ldfjac, const ADTYPE s[], const Integer &igrade, const Integer &niter, const Integer &nf)
1: ad_handlenag::ad::handle_t Input/Output
On entry: a handle to the AD handle object.
2: m – Integer Input
3: n – Integer Input
4: xcADTYPE array Input
5: fvecADTYPE array Input
6: fjacADTYPE array Input
7: ldfjac – Integer Input
8: sADTYPE array Input
9: igrade – Integer Input
10: niter – Integer Input
11: nf – Integer Input
*: iuser(*) – Integer array User Workspace
*: ruser(*)ADTYPE array User Workspace
7: iprint – Integer Input
8: maxcal – Integer Input
9: etaADTYPE Input
10: xtolADTYPE Input
11: stepmxADTYPE Input
12: x(n) – ADTYPE array Input/Output
Please consult Overwriting of Inputs in the NAG AD Library Introduction.
13: fsumsqADTYPE Output
14: fvec(m) – ADTYPE array Output
15: fjac(ldfjac, n) – ADTYPE array Output
16: ldfjac – Integer Input
17: s(n) – ADTYPE array Output
18: v(ldv, n) – ADTYPE array Output
19: ldv – Integer Input
20: niter – Integer Output
21: nf – Integer Output
*: iuser(*) – Integer array User Workspace
User workspace.
*: ruser(*) – ADTYPE array User Workspace
User workspace.
22: ifail – Integer Input/Output

6 Error Indicators and Warnings

e04gb preserves all error codes from e04gbf and in addition can return:
ifail=-89
An unexpected AD error has been triggered by this routine. Please contact NAG.
See Error Handling in the NAG AD Library Introduction for further information.
ifail=-199
The routine was called using a strategy that has not yet been implemented.
See AD Strategies in the NAG AD Library Introduction for further information.
ifail=-444
A C++ exception was thrown.
The error message will show the details of the C++ exception text.
ifail=-899
Dynamic memory allocation failed for AD.
See Error Handling in the NAG AD Library Introduction for further information.

7 Accuracy

Not applicable.

8 Parallelism and Performance

e04gb is not threaded in any implementation.

9 Further Comments

None.

10 Example

The following examples are variants of the example for e04gbf, modified to demonstrate calling the NAG AD Library.
Description of the primal example.
This example finds least squares estimates of x1, x2 and x3 in the model
y=x1+t1x2t2+x3t3  
using the 15 sets of data given in the following table.
y t1 t2 t3 0.14 1.0 15.0 1.0 0.18 2.0 14.0 2.0 0.22 3.0 13.0 3.0 0.25 4.0 12.0 4.0 0.29 5.0 11.0 5.0 0.32 6.0 10.0 6.0 0.35 7.0 9.0 7.0 0.39 8.0 8.0 8.0 0.37 9.0 7.0 7.0 0.58 10.0 6.0 6.0 0.73 11.0 5.0 5.0 0.96 12.0 4.0 4.0 1.34 13.0 3.0 3.0 2.10 14.0 2.0 2.0 4.39 15.0 1.0 1.0  
Before calling e04gb, the program calls e04ya to check lsqfun. It uses (0.5,1.0,1.5) as the initial guess at the position of the minimum.

10.1 Adjoint modes

Language Source File Data Results
Fortran e04gb_a1t1w_fe.f90 e04gb_a1t1w_fe.d e04gb_a1t1w_fe.r
Fortran e04gb_a1w_fe.f90 e04gb_a1w_fe.d e04gb_a1w_fe.r
C++ e04gb_a1_algo_dcoe.cpp None e04gb_a1_algo_dcoe.r
C++ e04gb_a1_sym_dcoe.cpp None e04gb_a1_sym_dcoe.r
C++ e04gb_a1_sym_expert_dcoe.cpp None e04gb_a1_sym_expert_dcoe.r
C++ e04gb_a1t1_algo_dcoe.cpp None e04gb_a1t1_algo_dcoe.r
C++ e04gb_a1t1_sym_dcoe.cpp None e04gb_a1t1_sym_dcoe.r
C++ e04gb_a1t1_sym_expert_dcoe.cpp None e04gb_a1t1_sym_expert_dcoe.r

10.2 Tangent modes

Language Source File Data Results
Fortran e04gb_t1w_fe.f90 e04gb_t1w_fe.d e04gb_t1w_fe.r
Fortran e04gb_t2w_fe.f90 e04gb_t2w_fe.d e04gb_t2w_fe.r
C++ e04gb_t1_dcoe.cpp None e04gb_t1_dcoe.r
C++ e04gb_t2_dcoe.cpp None e04gb_t2_dcoe.r

10.3 Passive mode

Language Source File Data Results
Fortran e04gb_p0w_fe.f90 e04gb_p0w_fe.d e04gb_p0w_fe.r
C++ e04gb_passive_dcoe.cpp None e04gb_passive_dcoe.r