NAG Library Chapter Introduction

This chapter is concerned with utility operations used in conjunction with the NAG AD Library. Details specific to the NAG AD Library can be found in the NAG AD Library Introduction.

All routines in the NAG AD Library have, as first argument in their interfaces, a handle to a configuration data object which is consistently named throughout the NAG AD Library as ad_handle. This argument stores configuration details used in the calculation of algorithmic derivatives. Further details and examples on the use of ad_handle can be found in the NAG AD Library Introduction. Chapter X10 provides facilities for:

(i)	Creating or removing a configuration data object
(ii)	Setting or getting particular configuration details
(iii)	Creating a callback object
(iv)	Setting or getting callback object configuration details
(v)	Inserting a callback location into a callback object
(vi)	Reading from and writing to a callback object

There are times when information that affects the form of computation performed by NAG AD Library routines or supplied procedure arguments needs to be communicated through the routine and supplied procedure interfaces.

The configuration data object, ad_handle is used, in particular, to pass the AD computational mode to be used (see Section 2.2). This argument always appears as the first argument to NAG AD Library computational routines.

When a routine contains a procedure argument that has active type arguments in its interface, ad_handle will also be the first argument of that interface. In addition to computational mode, ad_handle may also contain the form of operation to be performed by a procedure argument.

There is one other data object used by the NAG AD Library. This is the callback data object, which is used to: pass data from a procedure argument to a fixed interface companion callback; insert the location of the companion callback; and force the internal execution of the companion callback at the appropriate point in the evaluation of adjoints. This object is consistently named throughout the NAG AD Library as cb_handle.

There are two basic types of AD computational mode.

Other modes are possible, for example the following two cases.

(i)	hybrid mode This refers to some combination of algorithmic and symbolic modes, in which some part (though not all) of the differential calculation is performed using symbolic code.
(ii)	robust mode Here the intrumented algorithmic code has been refined by hand to improve various aspects of the computation. This can include: performance improvements through knowledge of the algorithm and through profiling; and reducing the memory footprint through knowledge of the algorithm and checkpointing. Some symbolic calculations may replace some algorithmic steps.

There are three aspects to the algorithmic type:

(i)	Direction This basically means whether the tangent linear derivatives or adjoints are being calculated for active variables.
(ii)	Order The order of differentiation to be performed, i.e., first, second or higher order.
(iii)	Precision Differentiation can be performed in working precision, reduced precision or higher precision. These terms are relative to the precision used in the primal calculation.

There is a separate type for each combination of these three aspects. For example, the algorithmic type codified as a1w refers to the type that calculates adjoints (a) to first order (1) in working precision (w).

Usually this is a very straightforward change of type in the translation from callback in the corresponding primal routine of the NAG Library to the NAG AD Library callback. Overloading / extension of basic arithmetic operations and intrinsic functions means that algorithmic differentiation is automatically performed on simple code containing only arithmetic operations, intrinsic evaluations, or calls to other routines in the NAG AD Library.

Where the operations performed within a supplied procedure argument (callback) are more complex than those described above, the differentiation will have to be performed in a staged manner: primal calculation, differentiation of active output variables with respect to active input (state) variables, and differentiation with respect to user supplied variables (e.g., problem parameters stored in ruser array). The stage to be performed by the supplied procedure is supplied through ad_handle. If some or all of the operations of a stage need to avoid automatic algorithmic differentation, such operations are performed within a companion callback, which takes the single argument cb_handle. Data required for such operations should be passed, from the procedure argument, through cb_handle, to the companion callback; results of the operations need to be inserted into the global data object for evaluating adjoints via primitive function calls to set values and increment derivatives.

In symbolic mode, the operations performed by a procedure argument are always staged: primal computation, differentiation with respect to active input arguments, differentiation with respect to supplied active variables (e.g., in the array ruser) and, in combining the previous two, differentiation with respect to both state and supplied active variables. The operations performed by a stage must be performed through companion callbacks (see Section 2.4.1) to avoid automatic algorithmic differentiation.

An ad_handle, which is a pointer to an internal data structure, is passed among the routines in the NAG AD Library. It holds configuration details about the AD approach to be used. Routines for creating and removing this ad_handle are as follows.

x10aa_a1w_f

Create a configuration data object.

x10ab_a1w_f

Remove a conguration data object.

The computational modes available are routine dependent. All have the default mode of nagad_algorithmic, some have the additional mode of nagad_symbolic, and other modes may become available in the future. Routines for setting and getting the computational modes are as follows.

x10ac_a1w_f

Set the computational mode.

x10ad_a1w_f

Get the computational mode.

This is the default mode that will be set when the configuration data object is initialized. In this mode the differentials (e.g., first order adjoints) are calculated by using the following: code created by converting and instrumenting the primal algorithmic code; intrinsic and operator extensions to work with special data types; and dco to perform the differentiation (see Section 5 in NAG AD Library Introduction).

For computational routines in the NAG AD Library where the symbolic computational mode is available, this mode may be selected by calling x10ad_a1w_f with mode set to nagad_symbolic prior to calling the computational routine.

The symbolic mode of computation is where code has been written to perform the particular form of differentiation (e.g., first order adjoints) of the primal algorithm.

The list of computational routines at Mark 26.2 for which the symbolic mode is available is:

• c05ay_a1w_f
• e04gb_a1w_f
• f07ca_a1w_f
• f08kd_a1w_f

Where such a routine contains a procedure argument (callback), then the method of evaluating and providing primal values and their differentials is complicated and involves writing special fixed interface companion callbacks whose location is inserted into a callback data object. The description of this is best described by example (see Section 10 in c05ay_a1w_f and Section 10 in e04gb_a1w_f).

There is currently only one algorithmic type, first order adjoints in working precision (a1w). This type is coded as ‘a1w’ and this code will appear in the short and long names of the routines in this chapter and in the remainder of the NAG AD Library Manual.

For routines with one or more supplied routine arguments, the data passed to and from the callback is managed via a callback data object. For each callback supplied, a companion callback is also required to perform the algorithmic differentiation via the callback data object. This companion callback is always a subroutine and always has just one argument: a pointer to the object. In Fortran the interface looks like:

Subroutine adjoint_callback(mode, cb_handle)
    Integer (Kind=c_int) :: mode
    Type (c_ptr), Value :: cb_handle
End Subroutine

Note that the mode argument is not the same as the callback mode held in the computational data object. It is not currently used, but will become important when further computational modes and further facilities are introduced. When the callback data object is created, it is used to communicate data between the primal callback and the differential callback. Utility routines are provided to read data from and write data to the callback data object as required. The full set of utility routines for managing callbacks are as follows.

x10ba_a1w_f

Create a callback data object

x10bb_a1w_f

Insert the location of the AD callback into the callback data object

x10bc_a1w_f

Set the callback algorithmic mode.

x10bd_a1w_f

Get the callback algorithmic mode.

x10be_a1w_f

Write an integer scalar to the callback data object.

x10bf_a1w_f

Write a real (working precision) scalar to the callback data object.

x10bj_a1w_f

Write a scalar of type nagad_a1w_w_rtype to the callback data object.

x10ce_a1w_f

Read an integer scalar from the callback data object.

x10cf_a1w_f

Read a real (working precision) scalar from the callback data object.

x10cj_a1w_f

Read a scalar of type nagad_a1w_w_rtype from the callback data object.

None.

Dunford N, Schwartz J T, Bade W G and Bartle R G (1971) Linear Operators Wiley Interscience, New York

Giles M (2008) Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation Springer

Griewank A and Walther A (2008) Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiations (2nd Edition) SIAM

Hascoët L, Naumann U and Pascual V (2005) ‘To be Recorded’ Analysis in Reverse-Mode Automatic Differentiation Future Generation Computer Systems 21(8) 299–304 Elsevier

Naumann U (2012) The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation SIAM

Naumann U, Lotz J, Leppkes K, Towara M (2015) Algorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations ACM Trans. Math. Softw.