X10 Chapter Introduction : NAG AD Library, Mark 27

All routines in the NAG AD Library have, as first argument in their interfaces, a handle to a configuration data object which is consistently named throughout the NAG AD Library as ad_handle. This argument stores configuration details used in the calculation of algorithmic derivatives. Further details and examples on the use of ad_handle can be found in the Introduction to the NAG AD Library. Chapter X10 provides facilities for:

(i)Creating or removing a configuration data object
(ii)Setting or getting particular configuration details
(iii)Creating a callback object
(iv)Setting or getting callback object configuration details
(v)Inserting a callback location into a callback object
(vi)Reading from and writing to a callback object

2 Background to the Problems

2.1 Configuration Data Objects

There are times when information that affects the form of computation performed by NAG AD Library routines or supplied procedure arguments needs to be communicated through the routine and supplied procedure interfaces.

The configuration data object, ad_handle is used, in particular, to pass the AD computational mode to be used (see Section 2.2). This argument always appears as the first argument to NAG AD Library computational routines.

When a routine contains a procedure argument that has active type arguments in its interface, ad_handle will also be the first argument of that interface. In addition to computational mode, ad_handle may also contain the form of operation to be performed by a procedure argument.

There is one other data object used by the NAG AD Library. This is the callback data object, which is specific to the calculation of adjoints and is used to: pass data from a procedure argument to a fixed interface companion callback; insert the location of the companion callback; and force the internal execution of the companion callback at the appropriate point in the evaluation of adjoints. This object is consistently named throughout the NAG AD Library as cb_handle.

2.2 AD Computational Modes

There are two basic types of AD computational mode.

Other modes are possible, for example the following two cases.

(i)hybrid mode
This refers to some combination of algorithmic and symbolic modes, in which some part (though not all) of the differential calculation is performed using symbolic code.
(ii)robust mode
Here the intrumented algorithmic code has been refined by hand to improve various aspects of the computation. This can include: performance improvements through knowledge of the algorithm and through profiling; and reducing the memory footprint through knowledge of the algorithm and checkpointing. Some symbolic calculations may replace some algorithmic steps.

2.3 Algorithmic Types

There are three aspects to the algorithmic type:

(i)Direction
This basically means whether the tangent linear derivatives or adjoints are being calculated for active variables.
(ii)Order
The order of differentiation to be performed, i.e., first, second or higher order.
(iii)Precision
Differentiation can be performed in working precision, reduced precision or higher precision. These terms are relative to the precision used in the primal calculation.

There is a separate type for each combination of these three aspects. For example, the algorithmic type codified as a1w refers to the type that calculates adjoints (a) to first order (1) in working precision (w).

2.4 Supplied Procedure Arguments (Callbacks)

2.4.1 Callbacks in algorithmic mode

Usually this is a very straightforward change of type in the translation from callback in the corresponding primal routine of the NAG Library to the NAG AD Library callback. Overloading / extension of basic arithmetic operations and intrinsic functions means that algorithmic differentiation is automatically performed on simple code containing only arithmetic operations, intrinsic evaluations, or calls to other routines in the NAG AD Library.

Where the operations performed within a supplied procedure argument (callback) are more complex than those described above, the differentiation will have to be performed in a staged manner: primal calculation, differentiation of active output variables with respect to active input (state) variables, and differentiation with respect to user supplied variables (e.g., problem parameters stored in ruser array). The stage to be performed by the supplied procedure is supplied through ad_handle. If computing adjoints and some or all of the operations of a stage need to avoid automatic algorithmic differentation, such operations are performed within a companion callback, which takes the single argument cb_handle. Data required for such operations should be passed, from the procedure argument, through cb_handle, to the companion callback; results of the operations need to be inserted into the global data object for evaluating adjoints via primitive function calls to set values and increment derivatives.

2.4.2 Callbacks in symbolic mode

In symbolic mode, the operations performed by a procedure argument are always staged: primal computation, differentiation with respect to active input arguments, differentiation with respect to supplied active variables (e.g., in the array ruser) and, in combining the previous two, differentiation with respect to both state and supplied active variables. When computing the adjoints the operations performed by a stage must be performed through companion callbacks (see Section 2.4.1) to avoid automatic algorithmic differentiation.

3 Recommendations on Choice and Use of Available Routines

3.1 Creating and Removing Configuration Data Objects

An ad_handle, which is a pointer to an internal data structure, is passed among the routines in the NAG AD Library. It holds configuration details about the AD approach to be used. Routines for creating and removing this ad_handle are as follows.

x10aa_a1w_f: Create a configuration data object.
x10ab_a1w_f: Remove a conguration data object.

3.2 Setting and Getting the Computational Mode

The computational modes available are routine dependent. All have the default mode of nagad_algorithmic, some have the additional mode of nagad_symbolic, and other modes may become available in the future. Routines for setting and getting the computational modes are as follows.

x10ac_a1w_f: Set the computational mode.
x10ad_a1w_f: Get the computational mode.

3.2.1 The algorithmic computational mode

This is the default mode that will be set when the configuration data object is initialized. In this mode the differentials (e.g., first order adjoints) are calculated by using the following: code created by converting and instrumenting the primal algorithmic code; intrinsic and operator extensions to work with special data types; and dco to perform the differentiation (see Section 5 in the Introduction to the NAG AD Library).

3.2.2 The symbolic computational mode

For computational routines in the NAG AD Library where the symbolic computational mode is available, this mode may be selected by calling x10ad_a1w_f with mode set to nagad_symbolic prior to calling the computational routine.

The symbolic mode of computation is where code has been written to perform the particular form of differentiation (e.g., first order adjoints) of the primal algorithm.

The list of computational routines at Mark 27 for which the symbolic mode is available is:

Where such a routine contains a procedure argument (callback), then the method of evaluating and providing primal values and their differentials is complicated and involves writing special fixed interface companion callbacks whose location is inserted into a callback data object. The description of this is best described by example (see Section 10 in c05ay_a1w_f and Section 10 in e04gb_a1w_f).

3.3 Algorithmic Type

There are currently two algorithmic types, first order adjoints in working precision (a1w) and first order tangents in working precision (t1w). The type coded ‘a1w’ will appear in the short and long names of the routines in this chapter and in the remainder of the NAG AD Library Manual. Four routines in this chapter have corresponding t1w versions: x10aa_t1w_f, x10ab_t1w_f, x10ac_t1w_f, and x10ad_t1w_f. In the remainder of the NAG AD Library Manual, all ‘a1w’ routines have corresponding t1w versions. Interfaces of t1w routines are implied from the documentation of the corresponding ‘a1w’ routine. Where t1w example programs exist, these will be listed in the table of example programs in the ‘a1w’ routine document.

For convenience the algorithmic type p0w refers to zeroth order passive calculations using passive data types. There are no p0w routines in this chapter; however, all routines in the remainder of the NAG AD Library Manual have corresponding p0w versions which differ only in the use of passive types corresponding to the active types of the a1w routines (as listed in Section 4.3.2 in the Introduction to the NAG AD Library).

3.4 Callback Object Utilities for Adjoints

For routines with one or more supplied routine arguments, the data passed to and from the callback is managed via a callback data object. For each callback supplied, a companion callback is also required to perform the algorithmic differentiation via the callback data object. This companion callback is always a subroutine and always has just one argument: a pointer to the object. In Fortran the interface looks like:

Subroutine adjoint_callback(mode, cb_handle)
    Integer (Kind=c_int) :: mode
    Type (c_ptr), Value :: cb_handle
End Subroutine

Note that the mode argument is not the same as the callback mode held in the computational data object. It is not currently used, but will become important when further computational modes and further facilities are introduced. When the callback data object is created, it is used to communicate data between the primal callback and the differential callback. Utility routines are provided to read data from and write data to the callback data object as required. The full set of utility routines for managing callbacks are as follows.

x10ba_a1w_f: Create a callback data object
x10bb_a1w_f: Insert the location of the AD callback into the callback data object
x10bc_a1w_f: Set the callback algorithmic mode.
x10bd_a1w_f: Get the callback algorithmic mode.
x10be_a1w_f: Write an integer scalar to the callback data object.
x10bf_a1w_f: Write a real (working precision) scalar to the callback data object.
x10bj_a1w_f: Write a scalar of type nagad_a1w_w_rtype to the callback data object.
x10ce_a1w_f: Read an integer scalar from the callback data object.
x10cf_a1w_f: Read a real (working precision) scalar from the callback data object.
x10cj_a1w_f: Read a scalar of type nagad_a1w_w_rtype from the callback data object.