NAG FL Interface
e04kff (handle_solve_bounds_foas)
Note: this routine uses optional parameters to define choices in the problem specification and in the details of the algorithm. If you wish to use default
settings for all of the optional parameters, you need only read Sections 1 to 10 of this document. If, however, you wish to reset some or all of the settings please refer to Section 11 for a detailed description of the algorithm and to Section 12 for a detailed description of the specification of the optional parameters.
1
Purpose
e04kff is a solver from the NAG optimization modelling suite for boundconstrained largescale nonlinear programming (NLP) problems. It is a firstorder activeset method (FOAS) that has low memory requirements and thus is suitable for very largescale problems.
2
Specification
Fortran Interface
Subroutine e04kff ( 
handle, objfun, objgrd, monit, nvar, x, rinfo, stats, iuser, ruser, cpuser, ifail) 
Integer, Intent (In) 
:: 
nvar 
Integer, Intent (Inout) 
:: 
iuser(*), ifail 
Real (Kind=nag_wp), Intent (Inout) 
:: 
x(nvar), ruser(*) 
Real (Kind=nag_wp), Intent (Out) 
:: 
rinfo(100), stats(100) 
Type (c_ptr), Intent (In) 
:: 
handle, cpuser 
External 
:: 
objfun, objgrd, monit 

C Header Interface
#include <nag.h>
void 
e04kff_ (void **handle, void (NAG_CALL *objfun)(const Integer *nvar, const double x[], double *fx, Integer *inform, Integer iuser[], double ruser[], void **cpuser), void (NAG_CALL *objgrd)(const Integer *nvar, const double x[], const Integer *nnzfd, double fdx[], Integer *inform, Integer iuser[], double ruser[], void **cpuser), void (NAG_CALL *monit)(const Integer *nvar, const double x[], Integer *inform, const double rinfo[], const double stats[], Integer iuser[], double ruser[], void **cpuser), const Integer *nvar, double x[], double rinfo[], double stats[], Integer iuser[], double ruser[], void **cpuser, Integer *ifail) 

C++ Header Interface
#include <nag.h> extern "C" {
void 
e04kff_ (void *&handle, void (NAG_CALL *objfun)(const Integer &nvar, const double x[], double &fx, Integer &inform, Integer iuser[], double ruser[], void *&cpuser), void (NAG_CALL *objgrd)(const Integer &nvar, const double x[], const Integer &nnzfd, double fdx[], Integer &inform, Integer iuser[], double ruser[], void *&cpuser), void (NAG_CALL *monit)(const Integer &nvar, const double x[], Integer &inform, const double rinfo[], const double stats[], Integer iuser[], double ruser[], void *&cpuser), const Integer &nvar, double x[], double rinfo[], double stats[], Integer iuser[], double ruser[], void *&cpuser, Integer &ifail) 
}

The routine may be called by the names e04kff or nagf_opt_handle_solve_bounds_foas.
3
Description
e04kff solves largescale boundconstrained nonlinear optimization problems of the form
where
$n$ is the number of decision variables,
${\ell}_{x}$,
${u}_{x}$, and
$x$ are
$n$dimensional vectors, and the nonlinear objective function
$f\left(x\right)$ is assumed to be sufficiently smooth.
The solver is a firstorder method (i.e., uses only first derivatives) that has very low memory requirements and therefore is suitable for very large boundconstrained problems. It is based on an activeset method coupled to a nonmonotone projected gradient algorithm (NPG), nonlinear conjugate gradient method (CG) and its limitedmemory variant (LCG). The activeset method is based on alternating between both solvers, the NPG step handles the box constraints and identifies a suitable search space while the CG step explores it for a solution.
For a detailed description of the algorithm see
Section 11. Under standard assumptions on the problem (smoothness of the first derivative of the objective) the algorithm converges to a local solution or to a critical point.
e04kff serves as a solver for problems stored as a handle. The handle points to an internal data structure which defines the problem and serves as a means of communication for routines in the NAG optimization modelling suite. First, the problem handle is initialized by calling
e04raf. Then some of the routines
e04ref,
e04rff,
e04rgf or
e04rhf may be called to formulate the objective and to define (simple) box constraints for the problem. Once the problem is fully described, the handle may be passed to the solver
e04kff. When the handle is no longer needed,
e04rzf should be called to destroy it and deallocate the memory held within. See
Section 3.1 in the
E04 Chapter Introduction for more details about the NAG optimization modelling suite.
The algorithm behaviour can be modified by various optional parameters (see
Section 12) which can be set by
e04zmf and
e04zpf anytime between the initialization of the handle by
e04raf and a call to the solver. Once the solver has finished, options may be modified for the next solve. The solver may be called repeatedly with various starting points and/or optional parameters. Option getter
e04znf can be called to retrieve the current value of any option.
The optional parameter
Task may be used to switch the problem to maximization, while
FOAS Estimate Derivatives can be used to complete missing elements from the gradient. Optional parameter
Verify Derivatives may help verify the correctness of the gradient vector before starting to solve a problem.
Several options may have significant impact on the performance of the solver. Even if the defaults were chosen to suit the majority of problems, it is recommended that you experiment in order to find the most suitable set of options for a particular problem, see
Sections 11 and
12 for further details.
4
References
Dai YH and Kou CX (2013) A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search SIAM J. Optim. 23(1) 296–320
Gill P E and Leonard M W (2003) LimitedMemory ReducedHessian Methods for LargeScale Unconstrained Optimization SIAM J. Optim. 14(2) 380–401
Hager W W and Zhang H (2005) A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search SIAM J. Optim. 16(1) 170–192
Hager W W and Zhang H (2006a) Algorithm 851: CG DESCENT, a Conjugate Gradient Method with Guaranteed Descent ACM Trans. Math. Software 32(1) 113–137
Hager W W and Zhang H (2006b) A New Active Set Algorithm for Box Constrained Optimization SIAM J. Optim. 17(2) 525–557
Hager W W and Zhang H (2013) The Limited Memory Conjugate Gradient Method SIAM J. Optim. 23(4) 2150–2168
Nocedal J and Wright S J (2006) Numerical Optimization (2nd Edition) Springer Series in Operations Research, Springer, New York
5
Arguments

1:
$\mathbf{handle}$ – Type (c_ptr)
Input

On entry: the handle to the problem. It needs to be initialized by
e04raf and the problem formulated by some of the routines
e04ref,
e04rff,
e04rgf and
e04rhf. It
must not be changed between calls to the NAG optimization modelling suite.

2:
$\mathbf{objfun}$ – Subroutine, supplied by the NAG Library or the user.
External Procedure

objfun must calculate the value of the nonlinear objective function
$f\left(x\right)$ at a specified point
$x$. If there is no nonlinear objective (e.g.,
e04ref or
e04rff was called to define a linear or quadratic objective function),
objfun will never be called by
e04kff and may be
the dummy routine
e04kfv (included in the NAG Library.)
The specification of
objfun is:
Fortran Interface
Integer, Intent (In) 
:: 
nvar 
Integer, Intent (Inout) 
:: 
inform, iuser(*) 
Real (Kind=nag_wp), Intent (In) 
:: 
x(nvar) 
Real (Kind=nag_wp), Intent (Inout) 
:: 
ruser(*) 
Real (Kind=nag_wp), Intent (Out) 
:: 
fx 
Type (c_ptr), Intent (In) 
:: 
cpuser 

C++ Header Interface
#include <nag.h> extern "C" {
}


1:
$\mathbf{nvar}$ – Integer
Input

On entry: $n$, the number of variables in the problem.

2:
$\mathbf{x}\left({\mathbf{nvar}}\right)$ – Real (Kind=nag_wp) array
Input

On entry: the vector $x$ of variable values at which the objective function is to be evaluated.

3:
$\mathbf{fx}$ – Real (Kind=nag_wp)
Output

On exit: the value of the objective function at $x$.

4:
$\mathbf{inform}$ – Integer
Input/Output

On entry: a nonnegative value.
In some cases, it is known beforehand that the evaluations of the objective function and its gradient are required at the same point $x$, in such cases, ${\mathbf{inform}}=1$. This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient; the objective function is always evaluated before the objective gradient. This notification parameter may be safely ignored if such optimization is not required.
On exit: may be used to indicate that the function cannot be evaluated at the requested point
$x$ by setting
${\mathbf{inform}}<0$. Returning NaN or
$\pm \infty $ in
fx has the same effect. The algorithm will try to recover if the objective cannot be evaluated; if recovery is not possible it will stop with
${\mathbf{ifail}}={\mathbf{25}}$.

5:
$\mathbf{iuser}\left(*\right)$ – Integer array
User Workspace

6:
$\mathbf{ruser}\left(*\right)$ – Real (Kind=nag_wp) array
User Workspace

7:
$\mathbf{cpuser}$ – Type (c_ptr)
User Workspace

objfun is called with the arguments
iuser,
ruser and
cpuser as supplied to
e04kff. You should use the arrays
iuser and
ruser, and the data handle
cpuser to supply information to
objfun.
objfun must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which
e04kff is called. Arguments denoted as
Input must
not be changed by this procedure.

3:
$\mathbf{objgrd}$ – Subroutine, supplied by the NAG Library or the user.
External Procedure

objgrd must calculate the values of the nonlinear objective function gradient
$\frac{\partial f}{\partial x}$ at a specified point
$x$. Every call to
objgrd is preceded by a call to
objfun at the same point, if this is known in advance, both functions will be notified via
${\mathbf{inform}}=1$. If there is no nonlinear objective (e.g.,
e04ref or
e04rff was called to define a linear or quadratic objective function),
objgrd will never be called by
e04kff
and
objgrd may be the dummy subroutine
e04kfw (included in the NAG Library.)
If the optional parameter
${\mathbf{FOAS\; Estimate\; Derivatives}}=\mathrm{YES}$, then after returning from
objgrd the gradient vector is checked for missing entries which you have not supplied. Missing entries are estimated using the finite difference method, see optional parameter
FOAS Estimate Derivatives description for more details.
The specification of
objgrd is:
Fortran Interface
Integer, Intent (In) 
:: 
nvar, nnzfd 
Integer, Intent (Inout) 
:: 
inform, iuser(*) 
Real (Kind=nag_wp), Intent (In) 
:: 
x(nvar) 
Real (Kind=nag_wp), Intent (Inout) 
:: 
fdx(nnzfd), ruser(*) 
Type (c_ptr), Intent (In) 
:: 
cpuser 

C++ Header Interface
#include <nag.h> extern "C" {
}


1:
$\mathbf{nvar}$ – Integer
Input

On entry: $n$, the number of variables in the problem.

2:
$\mathbf{x}\left({\mathbf{nvar}}\right)$ – Real (Kind=nag_wp) array
Input

On entry: the vector $x$ of variable values at which the objective function gradient is to be evaluated.

3:
$\mathbf{nnzfd}$ – Integer
Input

On entry: the number of nonzero elements in the sparse gradient vector of the objective function, as was set in a previous call to
e04rgf.

4:
$\mathbf{fdx}\left({\mathbf{nnzfd}}\right)$ – Real (Kind=nag_wp) array
Input/Output

On entry: the elements should only be assigned and not referenced.
On exit: the values of the nonzero elements in the sparse gradient vector of the objective function, in the order specified by
idxfd in a previous call to
e04rgf.
${\mathbf{fdx}}\left(\mathit{i}\right)$ will store the gradient element
$\frac{\partial f}{\partial {x}_{{\mathbf{idxfd}}\left(\mathit{i}\right)}}$.

5:
$\mathbf{inform}$ – Integer
Input/Output

On entry: a nonnegative value.
If
${\mathbf{inform}}=1$, then the previous call to
objfun was also made at the same point
$x$ with
${\mathbf{inform}}=1$. This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient. This notification parameter may be safely ignored if such optimization is not required.
On exit: may be used to inform that the gradient cannot be evaluated at the requested point
$x$ by setting
${\mathbf{inform}}<0$. Returning NaN or
$\pm \infty $ in any element of
fdx has the same effect. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with
${\mathbf{ifail}}={\mathbf{25}}$.

6:
$\mathbf{iuser}\left(*\right)$ – Integer array
User Workspace

7:
$\mathbf{ruser}\left(*\right)$ – Real (Kind=nag_wp) array
User Workspace

8:
$\mathbf{cpuser}$ – Type (c_ptr)
User Workspace

objgrd is called with the arguments
iuser,
ruser and
cpuser as supplied to
e04kff. You should use the arrays
iuser and
ruser, and the data handle
cpuser to supply information to
objgrd.
objgrd must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which
e04kff is called. Arguments denoted as
Input must
not be changed by this procedure.

4:
$\mathbf{monit}$ – Subroutine, supplied by the NAG Library or the user.
External Procedure

monit is provided to enable you to monitor the progress of the optimization and optionally to terminate the solver early if necessary. It is invoked at the end of every
$i$th iteration where
$i$ is given by the
FOAS Monitor Frequency (the default is
$0$,
monit is not called).
monit
may be the dummy subroutine
e04kfu (included in the NAG Library).
The specification of
monit is:
Fortran Interface
Integer, Intent (In) 
:: 
nvar 
Integer, Intent (Inout) 
:: 
inform, iuser(*) 
Real (Kind=nag_wp), Intent (In) 
:: 
x(nvar), rinfo(100), stats(100) 
Real (Kind=nag_wp), Intent (Inout) 
:: 
ruser(*) 
Type (c_ptr), Intent (In) 
:: 
cpuser 

C++ Header Interface
#include <nag.h> extern "C" {
}


1:
$\mathbf{nvar}$ – Integer
Input

On entry: $n$, the number of variables in the problem.

2:
$\mathbf{x}\left({\mathbf{nvar}}\right)$ – Real (Kind=nag_wp) array
Input

On entry: the vector $x$ of decision variables at the current iteration.

3:
$\mathbf{inform}$ – Integer
Input/Output

On entry: a nonnegative value.
On exit: may be used to request the solver to stop immediately. Specifically, if ${\mathbf{inform}}<0$ the solver will terminate immediately with ${\mathbf{ifail}}={\mathbf{20}}$ otherwise, the solver will proceed normally.

4:
$\mathbf{rinfo}\left(100\right)$ – Real (Kind=nag_wp) array
Input

On entry: error measures and various indicators at the end of the current iteration as described in
rinfo.

5:
$\mathbf{stats}\left(100\right)$ – Real (Kind=nag_wp) array
Input

On entry: solver statistics at the end of the current iteration as described in
stats.

6:
$\mathbf{iuser}\left(*\right)$ – Integer array
User Workspace

7:
$\mathbf{ruser}\left(*\right)$ – Real (Kind=nag_wp) array
User Workspace

8:
$\mathbf{cpuser}$ – Type (c_ptr)
User Workspace

monit is called with the arguments
iuser,
ruser and
cpuser as supplied to
e04kff. You should use the arrays
iuser and
ruser, and the data handle
cpuser to supply information to
monit.
monit must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which
e04kff is called. Arguments denoted as
Input must
not be changed by this procedure.

5:
$\mathbf{nvar}$ – Integer
Input

On entry: $n$, the number of variables in the problem.

6:
$\mathbf{x}\left({\mathbf{nvar}}\right)$ – Real (Kind=nag_wp) array
Input/Output

On entry: ${x}_{0}$, the initial estimates of the variables $x$.
On exit: the final values of the variables $x$.

7:
$\mathbf{rinfo}\left(100\right)$ – Real (Kind=nag_wp) array
Output

On exit: error measures and various indicators at the end of the final iteration as given in the table below:
$1$ 
Objective function value $\mathrm{f}\left(x\right)$. 
$2$ 
Norm of inactive gradient, the objective gradient over the current search space. If the problem is unconstrained, then elements $2$–$4$ coincide. See Section 11.4 for details. 
$3$ 
Norm of projected direction, used in (3), see Section 11.2. 
$4$ 
Norm of objective gradient. 
$5$ 
Last step size (${\alpha}_{k}$) used in (2) and (5), see Section 11.2 and Section 11.3. 
$6$ 
Progress score, a positive value that measures the progress of the solver. A low score close to zero indicates poor progress, see (8) in Section 11.4. 
$7$–$100$ 
Reserved for future use. 

8:
$\mathbf{stats}\left(100\right)$ – Real (Kind=nag_wp) array
Output

On exit: solver statistics at the end of the final iteration as given in the table below:
$1$ 
Number of function evaluations performed by NPG, see Section 11.2. 
$2$ 
Number of gradient evaluations performed by NPG. 
$3$ 
Number of function evaluations performed by CG, see Section 11.3. 
$4$ 
Number of gradient evaluations performed by CG. 
$5$ 
Number of function evaluations performed by LCG. 
$6$ 
Number of gradient evaluations performed by LCG. 
$7$ 
Number of function evaluations used by the finite difference method to estimate missing components of the gradient. 
$8$ 
Number of iterations. 
$9$ 
Total time spent in the solver (including usersupplied function calls). 
$10$ 
Total time spent in usersupplied objective function. 
$11$ 
Total time spent in usersupplied objective gradient. 
$12$–$100$ 
Reserved for future use. 

9:
$\mathbf{iuser}\left(*\right)$ – Integer array
User Workspace

10:
$\mathbf{ruser}\left(*\right)$ – Real (Kind=nag_wp) array
User Workspace

11:
$\mathbf{cpuser}$ – Type (c_ptr)
User Workspace

iuser,
ruser and
cpuser are not used by
e04kff, but are passed directly to
objfun,
objgrd and
monit and may be used to pass information to these routines. If you do not need to reference
cpuser, it should be initialized to
c_null_ptr.

12:
$\mathbf{ifail}$ – Integer
Input/Output

On entry:
ifail must be set to
$0$,
$1\text{or}1$. If you are unfamiliar with this argument you should refer to
Section 4 in the Introduction to the NAG Library FL Interface for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value
$1\text{or}1$ is recommended. If the output of error messages is undesirable, then the value
$1$ is recommended. Otherwise, because for this routine the values of the output arguments may be useful even if
${\mathbf{ifail}}\ne {\mathbf{0}}$ on exit, the recommended value is
$1$.
When the value $\mathbf{1}\text{or}1$ is used it is essential to test the value of ifail on exit.
On exit:
${\mathbf{ifail}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see
Section 6).
6
Error Indicators and Warnings
If on entry
${\mathbf{ifail}}=0$ or
$1$, explanatory error messages are output on the current error message unit (as defined by
x04aaf).
Errors or warnings detected by the routine:
Note: in some cases e04kff may return useful information.
 ${\mathbf{ifail}}=1$

The supplied
handle does not define a valid handle to the data structure for the NAG optimization modelling suite. It has not been initialized by
e04raf or it has been corrupted.
 ${\mathbf{ifail}}=2$

The problem is already being solved.
This solver does not support the model defined in the handle.
 ${\mathbf{ifail}}=4$

The information supplied does not match the value previously stored.
On entry,
${\mathbf{nvar}}=\u2329\mathit{\text{value}}\u232a$ must match the value given during initialization of the
handle, i.e.,
$\u2329\mathit{\text{value}}\u232a$.
 ${\mathbf{ifail}}=7$

The dummy
objfun routine was called but the problem requires these values.
Please provide a proper
objfun routine.
The dummy
objgrd routine was called but the problem requires these derivatives.
Please provide a proper
objgrd routine.
 ${\mathbf{ifail}}=20$

User requested termination during a monitoring step.
inform was set to a negative value in
monit.
 ${\mathbf{ifail}}=21$

The current starting point is unusable.
Either
inform was set to a negative value within the usersupplied functions
objfun,
objgrd, or an Infinity or NaN was detected in values returned from them.
 ${\mathbf{ifail}}=22$

Maximum number of iterations reached.
 ${\mathbf{ifail}}=23$

The solver terminated after the maximum time allowed was exhausted.
Maximum number of seconds exceeded. Use optional parameter
Time Limit to change the limit.
 ${\mathbf{ifail}}=24$

The solver was terminated because no further progress could be achieved.
This can indicate that the solver is calculating very small step sizes and is making very little progress. It could also indicate that the problem has been solved to the best numerical accuracy possible given the current scaling.
 ${\mathbf{ifail}}=25$

Invalid number detected in usersupplied function and recovery failed.
Either
inform was set to a negative value within the usersupplied function
objfun, or an Infinity or NaN was detected in values returned from them. Recovery attempts failed.
Invalid number detected in usersupplied gradient and recovery failed.
Either
inform was set to a negative value within the usersupplied function
objgrd, or an Infinity or NaN was detected in values returned from them. Recovery attempts failed.
 ${\mathbf{ifail}}=26$

Userprovided gradient is likely to be incorrect.
This error indicates that the usersupplied gradient vector
fdx has entries that do not match with a finitedifferences approximation of it.
 ${\mathbf{ifail}}=50$

Problem was solved to an acceptable level; full accuracy was not achieved.
This indicates that the algorithm detected a sequence of very small reductions in the objective function value and is unable to reach a point satisfying the requested optimality tolerance. This may happen if the desired tolerances are too small for the current problem, or if the objective function is badly scaled.
 ${\mathbf{ifail}}=54$

The problem seems to be unbounded and the algorithm was stopped.
This indicates that the solver found a direction where the objective function decreases arbitrarily. In the case of maximizing, the direction found increases arbitrarily the objective value. Cases where this happens are when a linear objective is provided with no constraints, or when the constraints defined do not bound the search direction.
 ${\mathbf{ifail}}=99$
An unexpected error has been triggered by this routine. Please
contact
NAG.
See
Section 7 in the Introduction to the NAG Library FL Interface for further information.
 ${\mathbf{ifail}}=399$
Your licence key may have expired or may not have been installed correctly.
See
Section 8 in the Introduction to the NAG Library FL Interface for further information.
 ${\mathbf{ifail}}=999$
Dynamic memory allocation failed.
See
Section 9 in the Introduction to the NAG Library FL Interface for further information.
7
Accuracy
The accuracy of the solution is determined by optional parameters
FOAS Stop Tolerance and
FOAS Rel Stop Tolerance. If
${\mathbf{ifail}}={\mathbf{0}}$ on exit, the returned point satisfies
(6) or
(7) to the defined accuracy.
Please refer to
Section 11.4 and the description of the particular options in
Section 12.
8
Parallelism and Performance
e04kff is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
e04kff makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the
Users' Note for your implementation for any additional implementationspecific information.
The solver can print information to give an overview of the problem and of the progress of the computation. The output may be sent to two independent streams (files) which are set by optional parameters
Print File and
Monitoring File. Optional parameters
Print Level,
Monitoring Level,
Print Solution and
Print Options determine the exposed level of detail. This allows, for example, a detailed log file to be generated while the condensed information is displayed on the screen.
By default (
${\mathbf{Print\; File}}=6$,
${\mathbf{Print\; Level}}=2$), the following sections are printed to the standard output:
 Header
 Optional parameters list (if ${\mathbf{Print\; Options}}=\mathrm{YES}$)
 Problem statistics
 Verification of derivatives (if ${\mathbf{Verify\; Derivatives}}=\mathrm{YES}$)
 Iteration log
 Summary
 Solution (if ${\mathbf{Print\; Solution}}=\mathrm{YES}$)
Header
The header is a message indicating the start of the solver. It should look like:

E04KF, Firstorder method for boundconstrained problems

Optional parameters list
The list shows all options of the solver, each displayed on one line. The output contains the option name, its current value and an indicator for how it was set. The options unchanged from the default setting are noted by ‘d’, options you set are noted by ‘U’, and options reset by the solver are noted by ‘S’. Note that the output format is compatible with the file format expected by
e04zpf. The output might look as follows:
Begin of Options
Print File = 6 * d
Print Level = 2 * U
Print Options = Yes * d
Print Solution = All * U
Monitoring File = 9 * U
Monitoring Level = 3 * U
Foas Monitor Frequency = 0 * d
Foas Print Frequency = 5 * U
End of Options
Problem statistics
If
${\mathbf{Print\; Level}}\ge 2$, statistics on the problem is printed, for example:
Problem Statistics
Number of variables 63
Number of free Variables 0
Number of fixed Variables 12
Number of vars with only lower bound 40
Number of vars with only upper bound 0
Number of vars with bounds 11
Objective type Nonlinear
Verification of derivatives
If
${\mathbf{Verify\; Derivatives}}=\mathrm{YES}$, then the usersupplied nonlinear objective gradient is verified. If
${\mathbf{Print\; Level}}<5$, then it will only print entries where the gradient entry substantially differs from a finitedifference approximation, it might look as follows:
Derivative checker
idx test fdx approx rel error
1 Fail 1.71E+02 1.74E+02 1.56E02
On the other hand, if
${\mathbf{Print\; Level}}=5$, it will print one line for each gradient entry and the output should be similar to:
Derivative checker
idx test fdx approx rel error
1 Fail 9.84E01 9.86E01 2.72E03
2 Ok 1.30E+00 1.30E+00 4.13E08
3 Skip missing  
4 Skip missing  
5 Ok 4.82E01 4.82E01 6.23E08
Iteration log
If ${\mathbf{Print\; Level}}\ge 2$, the solver prints the status of every $k$th iteration, specified using the optional parameter ${\mathbf{FOAS\; Print\; Frequency}}=k$, with default value of $k=1$. It will also print status information regarding the switch between NPG, CG and LCG.
It will also print status information regarding the switch between NPG, CG and LCG.
If
${\mathbf{Print\; Level}}=2$, the output shows the iteration number (
$0$ represents the starting point), the current objective value, and optimality measures (norm of inactive gradient and the norm of the projected direction). Note that all these values for the last iteration are also available in
rinfo. The output might look as follows:

iters  objective  optim  dir

20 7.71339E01 1.22E01 1.70E+00
25 7.06709E01 1.75E+00 1.54E+00
30 7.06709E01 1.75E+00 1.54E+00
35 6.82989E01 1.35E+00 1.35E+00
30 6.55834E01 3.26E+00 1.67E+00
If
${\mathbf{Print\; Level}}=3$, the solver also prints for each iteration the progress score, iteration type, size or type of performed step, accumulated number of objective function and gradient calls. The output takes the following form:

iters  objective  optim  dir  progrss  it step  nf ng

0 1.10354E+02 0.00E+00 6.81E+01 1.00E+00 Start 1 1
1 3.29016E+01 1.45E+01 1.45E+01 5.36E+00 NPG 1.39E02 8 2
2 3.29016E+01 1.45E+01 1.45E+01 5.36E+00 Switch CG 8 2
3 1.84090E+01 2.48E+01 2.48E+01 2.85E+00 CG 2.42E02 11 4
4 1.84090E+01 2.48E+01 2.48E+01 2.85E+00 Restart 11 4
5 9.48087E+00 1.25E+01 1.25E+01 1.45E+00 CG 1.49E02 13 5
6 4.20527E+00 8.28E+00 8.22E01 2.51E+00 CG 3.61E02 15 6
7 7.20453E01 3.08E+00 1.62E+00 9.70E+00 LCG 7.21E01 17 7
If
${\mathbf{Print\; Level}}>3$, each iteration produces more information that expands over several lines. This additional information can contain:
 The sizes of gradient in full space and subspace;
 The quadratic model prediction error and counter;
 The amount of search direction restarts;
 The extreme eigenvalues of the basis matrix for the subspace;
 Information and size of the activeset;
 Whether the orthogonal property holds;
 The action to take for the next iteration (switch solver);
 The search space size;
 The amount of memory vectors used.
The output might look as follows:
 Iteration details (CG) 
Size of gradient 2.64E+01
Size of subspace gradient 2.64E+01
Eigenvalues R range 6.80E01 6.80E01
Orthogonal property lost
Action Switch to LCG

 LCG solver details 
Search space has 2 element(s)
Memory vectors 2 ( 2)

Summary
Once the solver finishes, a detailed summary is produced:

Status: converged, an optimal solution was found

Value of the objective 3.55353E14
Norm of inactive gradient 2.53812E07
Norm of projected direction 1.68481E07
Iterations 32
Function evaluations 80
FD func. evaluations 5
Gradient evaluations 48
NPG function calls 0
NPG gradient calls 0
CG function calls 11
CG gradient calls 9
LCG function calls 69
LCG gradient calls 39

It starts with a status line of the overall result, followed by the final objective value as well as the gradient (unconstrained problem) or inactive gradient and projected direction (constrained problem) norms. If ${\mathbf{Print\; Level}}>1$ or ${\mathbf{Monitoring\; Level}}>1$, it will additionally report iteration count, objective function and gradient call information.
Optionally, if
${\mathbf{Stats\; Time}}=\mathrm{YES}$,
the timings for the usersupplied objective are displayed. It might look as follows:
Timing
Total time spent 12.54 sec
Total time in obj function 3.01 sec ( 24.0%)
Total time in obj gradient 5.28 sec ( 42.1%)
Solution
If
${\mathbf{Print\; Solution}}=\mathrm{YES}$,
the values of the solution, their bounds and the (inactive) gradient (see
Section 11.3) are printed. It might look as follows:
Variables:
idx lower bound value upper bound gradient
1 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00
2 0.00000E+00 1.00000E01 1.00000E+00 0.00000E+00
3 0.00000E+00 1.52323E04 inf 1.97514E07
4 0.00000E+00 8.49117E03 inf 1.30979E07
10
Example
In this example, we minimize in
${\mathbb{R}}^{2}$ the Rosenbrock function over simple bounds on the variables. The problem to solve is
The initial guess is ${x}_{0}={\left(1.5,1.9\right)}^{\mathrm{T}}$, and the expected solution point is ${x}^{*}={\left(0.8,0.64\right)}^{\mathrm{T}}$ with objective value $\mathrm{f}\left({x}^{*}\right)=0.04$.
10.1
Program Text
10.2
Program Results
11
Algorithmic Details
This section contains the description of the underlying algorithms used in
e04kff, a firstorder activeset (FOAS) method with very low memory requirements suitable for largescale boundconstrained nonlinear optimization problems. For further details, see
Hager and Zhang (2006b) and references therein.
11.1
ActiveSet Method and Algorithm Outline
Activeset is a useful method to tackle problems with bound constraints. It derives its name from the partitioning of the search space: elements of $x$ that are fixed at a bound (${x}_{i}={\ell}_{i}$ or ${x}_{i}={u}_{i}$), are said to be active, while the rest of the elements of $x$ are said to be inactive. The goal of this method is to estimate which variables will be active at the solution point while optimizing over the inactive components.
e04kff consists of a constrained solver, an unconstrained solver and a set of rules to switch between them. The constrained solver (nonmonotone projected gradient, NPG) has two purposes: while solving
(1) it tries to guess which components of
$x$ are active at a solution. Once a
reasonable guess is available, control is transferred to an unconstrained solver (conjugate gradient method, CG, and its limitedmemory variant, LCG) that operates only over the inactive elements of
$x$ but convergence is much faster that NPG.
The following is an outline of the implemented algorithm.
FOAS Algorithm

(i)Make initial point ${x}_{0}$ feasible.

(ii)Loop until stopping criteria are satisfied

(a)Constrained problem (NPG): start solving constrained problem (1) and try to identify a suitable inactive search space; if found, then go to (ii)(b).

(b)Unconstrained subproblem (CG): solve unconstrained problem (4) over the elements of $x$ marked inactive. If elements of $x$ become newly active or it is deemed that some active elements should be explored, then go back to (ii)(a).
11.2
Constrained Subproblem (NPG)
By setting
$\Omega =\left\{x\in {\mathbb{R}}^{n}:{\ell}_{x}\le x\le {u}_{x}\right\}$, the constrained problem
(1) can be stated as
and can be solved using NPG which is an iterative method of the form
with step size
${\alpha}_{k}$ and projection direction
${d}_{\Omega}\left(x\right)$.
The projected direction,
${d}_{\Omega}\left(x\right)$, is obtained by a projection of the objective gradient
$g\left(x\right)$ onto
$\Omega $:
with
${P}_{\Omega}$ the Euclidean projection operator over the box
$\Omega $. Note that if
$\Omega ={\mathbb{R}}^{n}$, then
${d}_{\Omega}\left({x}_{k}\right)=g\left({x}_{k}\right)$ and the method reduces to nonmonotone steepest descent.
The step size
${\alpha}_{k}>0$ is chosen to guarantee global convergence, i.e., guarantee sufficient progress at each iteration using the nonmonotone Armijo condition
where
${f}_{k}^{R}$ is a reference objective function value for iteration
$k$, possibly
${f}_{k}^{R}=f\left({x}_{k}\right)$. The step size is estimated using a nonmonotone backtracking technique. As soon as a suitable inactive set is identified (see
Hager and Zhang (2006b)), the control is transferred to CG (LCG) to accelerate the convergence.
11.3
Unconstrained Subproblem (CG)
The exploration of the inactive search space is performed using the CG method which is an iterative method to solve
and each iterate is updated using the expression
where
${d}_{k}$ is a direction of descent that combines the current gradient
$g\left(x\right)$ and a scaled version of the previous direction,
${\beta}_{k}{d}_{k1}$, with
${\beta}_{k}\in \mathbb{R}$,
The step size
${\alpha}_{k}>0$ is chosen to ensure global convergence and is achieved by satisfying the weak Wolfe conditions
with
$0<\delta <\sigma <1$ or the approximate Wolfe conditions (see
Hager and Zhang (2005))
$\epsilon \epsilon $,
with
$\delta <\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left(0.5,\sigma \right)$ and
${\epsilon}_{k}>0$. The line search performed by
e04kff will always satisfy either/or both stated conditions. The line search builds a series of nested intervals that contains a point satisfying the above sufficient decrease requirements.
It is important to note that when $\Omega \ne {\mathbb{R}}^{n}$, then the used gradient is actually the inactive gradient,
it is the gradient, $g\left({x}_{k}\right)$, but with zeros in the same positions where the elements of ${x}_{k}$
are marked as active.
The performance of the CG method can degrade when orthogonality is lost between consecutive search directions. Therefore, e04kff uses a limited number of previous search directions to detect and restore orthogonality. When the current search direction is no longer orthogonal, it is discarded and a quasiNewton variant known as limitedmemory CG (LCG) is used to build a new search direction orthogonal to the explored subspace.
The available memory is used to build an approximation of the Hessian and the new search direction is estimated using
The quasiNewton Hessian approximation
${H}_{k}$, is updated at each iteration using the BFGS update formula, see
Nocedal and Wright (2006).
Experiments in literature indicate that an adequate range for the quasiNewton (amount of vectors used to build
${H}_{k}$) lies between
$7$ and
$20$. The default value for the maximum amount of vectors used in
e04kff is
$11$ and can be changed via the
FOAS Memory.
11.4
Stopping Criteria
A point is considered a solution when there are no feasible descent directions to use. Under this circumstance the routine will stop, declaring to have found a solution.
If the problem is unconstrained, the routine declares to have found a solution and stops when the firstorder optimality condition is met within the defined absolute or relative tolerances (
${\epsilon}_{\mathrm{tol}}\ge 0$,
${\epsilon}_{\mathrm{rel}}\ge 0$),
where
$p$ can designate either the Euclidean or Infinity (default) norms.
On the other hand, if the problem is constrained, the routine characterizes to have found a solution when the firstorder condition is satisfied for the projected direction,
${d}_{\Omega}$, to the defined tolerances,
In the unconstrained case we have
$\Omega ={\mathbb{R}}^{n}$ and both stopping criteria
(6) and
(7) coincide.
The stopping tolerances can be changed using the optional parameters
FOAS Stop Tolerance and
FOAS Rel Stop Tolerance, while the norm used can be set by using the optional parameter
FOAS Tolerance Norm, see
Section 12 for details. If these parameters are set too small in relation to the complexity and scaling of the problem, the routine can terminate with
${\mathbf{ifail}}={\mathbf{22}}$,
${\mathbf{24}}$ or
${\mathbf{50}}$.
Progress of the solver towards a solution is monitored using two criteria.
The first one evaluates how
poor
the actual step has been and is estimated via
where
${\alpha}_{k}$ is given by the line search. If the above relation is consistently satisfied, then the solver stops with
${\mathbf{ifail}}={\mathbf{24}}$.
The tolerance
${\epsilon}_{\mathrm{prog}}$ is set using the optional parameter
FOAS Progress Tolerance. The second criteria monitors the rate of convergence using
As with the first criteria, if the previous relation is deemed permanent, then the solver stops with
${\mathbf{ifail}}={\mathbf{50}}$. The tolerance
${\epsilon}_{\mathrm{slow}}$ can be changed using the optional parameter
FOAS Slow Tolerance.
11.5
A Note About Lagrangian Multipliers
It is often useful to have access to the Lagrangian multipliers (dual variables) associated with the constraints if there are any defined. In the case where only simple bounds are present, the multipliers directly relate to the values of the gradient at the solution. The multipliers of the active bounds are the absolute values of the associated elements of the gradient. The multipliers of the inactive bounds are always zero.
The multipliers based on the final gradient value (or its finitedifference approximation) can be retrieved by calling
e04rxf with the command string
${\mathbf{cmdstr}}='\mathrm{Dual\; Variables}'$. The format is the same as for other routines, see
Section 3.1 in
e04svf. Note that if the problem has not fully converged, the provided approximation might be quite crude.
12
Optional Parameters
Several optional parameters in e04kff define choices in the problem specification or the algorithm logic. In order to reduce the number of formal arguments of e04kff these optional parameters have associated default values that are appropriate for most problems. Therefore, you need only specify those optional parameters whose values are to be different from their default values.
The remainder of this section can be skipped if you wish to use the default values for all optional parameters.
The optional parameters can be changed by calling
e04zmf anytime between the initialization of the handle and the call to the solver. Modification of the optional parameters during intermediate monitoring steps is not allowed. Once the solver finishes, the optional parameters can be altered again for the next solve.
The option values may be retrieved by
e04znf.
The following is a list of the optional parameters available. A full description of each optional parameter is provided in
Section 12.1.
12.1
Description of the Optional Parameters
For each option, we give a summary line, a description of the optional parameter and details of constraints.
The summary line contains:
 the keywords;
 a parameter value,
where the letters $a$, $i$ and $r$ denote options that take character, integer and real values respectively.
 the default value, where the symbol $\epsilon $ is a generic notation for machine precision (see x02ajf).
All options accept the value $\mathrm{DEFAULT}$ to return single options to their default states.
Keywords and character values are case and white space insensitive.
This special keyword may be used to reset all optional parameters to their default values. Any value given with this keyword will be ignored.
FOAS Estimate Derivatives  $a$  Default $\text{}=\mathrm{NO}$ 
This option indicates whether to check for and estimate missing entries of the usersupplied gradient vector. Since the associated cost of estimating missing elements can be high, this option should only be used if strictly necessary. In general terms, if the gradient is not provided (in its entirety or for arbitrary points) potential degradation in the progress of the solver is to be expected. Depending on the complexity of the objective, the routine may not achieve the desired optimality accuracy or even terminate with no possible further progress error
${\mathbf{ifail}}={\mathbf{24}}$, it is advisable to increase the values of
FOAS Stop Tolerance and
FOAS Rel Stop Tolerance when using this option.
Missing elements from the gradient vector are estimated by finitedifferences using the perturbation interval specified by the optional parameter
FOAS Finite Diff Interval.
If ${\mathbf{FOAS\; Estimate\; Derivatives}}=\mathrm{NO}$, the entries are not checked and all derivative elements need to be provided.
Constraint: ${\mathbf{FOAS\; Estimate\; Derivatives}}=\mathrm{YES}$ or $\mathrm{NO}$.
FOAS Finite Diff Interval  $r$  Default $=\sqrt{\epsilon}$

Specifies the relative perturbation size used to estimate a derivative using the forward (or backward) finitedifference method. Setting the value too small or too big may lead e04kff to terminated with ${\mathbf{ifail}}={\mathbf{24}}$ or ${\mathbf{25}}$.
Constraint: ${10}^{12}\le {\mathbf{FOAS\; Finite\; Diff\; Interval}}\le {10}^{1}$.
FOAS Iteration Limit  $i$  Default $={10}^{7}$ 
This parameter sets the maximum number of iterations to be performed by e04kff. Setting the option too low might lead to ${\mathbf{ifail}}={\mathbf{22}}$.
Constraint: ${\mathbf{FOAS\; Iteration\; Limit}}\ge 1$.
FOAS Memory  $i$  Default $=11$ 
This parameter specifies the maximum number of memory vectors to use in the LCG solver.
Constraint: $0\le {\mathbf{FOAS\; Memory}}\le 100$.
FOAS Monitor Frequency  $i$  Default $=0$ 
This parameter specifies the frequency on which to call the monitor routine
monit.
If zero, the monitor routine will not be called.
Constraint: ${\mathbf{FOAS\; Monitor\; Frequency}}\ge 0$.
FOAS Print Frequency  $i$  Default $=\mathrm{1}$ 
This parameter specifies the frequency with which to print information regarding each iteration to
Print File and/or
Monitoring File. By default, it will print information of every iteration.
Constraint: ${\mathbf{FOAS\; Print\; Frequency}}\ge 1$.
FOAS Progress Tolerance  $r$  Default $\text{}={\epsilon}^{\frac{3}{4}}$

Specifies the tolerance for
${\epsilon}_{\mathrm{prog}}$ (see
(8)) for which the routine characterises a poor rate of progress given that it deems to be far from a solution. If this behaviour is persistent, then the routine asserts that no substantial further progress can be achieved and the process is terminated with
${\mathbf{ifail}}={\mathbf{24}}$. Setting a high tolerance can lead to misinterpret reasonable progress for unsatisfactory progress or even issue a premature stop, see
(8) in
Section 11.4.
Constraint: $0<{\mathbf{FOAS\; Progress\; Tolerance}}<1$.
FOAS Rel Stop Tolerance  $r$  Default $\text{}={\epsilon}^{\frac{3}{4}}$

This parameter sets the value of
${\epsilon}_{\mathrm{rel}}$ which specifies the relative tolerance for the convergence measures in the stopping criteria, see
(6) and
(7) in
Section 11.4.
Constraint: $0\le {\epsilon}_{\mathrm{rel}}<1$.
FOAS Restart Factor  $r$  Default $=6.0$ 
This factor specifies the frequency ${\mathbf{nvar}}\times {\mathbf{FOAS\; Restart\; Factor}}$ with which the CG/LCG directions are replaced by the steepest descent direction (${d}_{k}={g}_{k}$). Setting the value too small can potentially slow the convergence speed.
Constraint: ${\mathbf{FOAS\; Restart\; Factor}}\ge 0$.
FOAS Slow Tolerance  $r$  Default $\text{}={\epsilon}^{\frac{1}{8}}$

Specifies the tolerance for
${\epsilon}_{\mathrm{slow}}$ (see
(9)) for which the routine characterises a slow rate of convergence. If this behaviour is deemed permanent, then the routine asserts that no substantial improvement can be achieved and the process is terminated with
${\mathbf{ifail}}={\mathbf{50}}$. Setting a large tolerance can lead to incorrectly identifying a suboptimal solution, see
(9) in
Section 11.4.
Constraint: ${\mathbf{FOAS\; Slow\; Tolerance}}>0$.
FOAS Stop Tolerance  $r$  Default $=\mathrm{max}\phantom{\rule{0.125em}{0ex}}\left({10}^{6},\sqrt{\epsilon}\right)$ 
This parameter sets the value of
${\epsilon}_{\mathrm{tol}}$ which specifies the tolerance for the convergence measures in the stopping criteria, see
(6) and
(7) in
Section 11.4.
Constraint: $0\le {\epsilon}_{\mathrm{tol}}<1$.
FOAS Tolerance Norm  $a$  Default $=\mathrm{INFINITY}$ 
This parameter specifies the norm used to measure some stopping metrics, such as optimality tolerances (see
Section 11.4). It is possible to choose between 2norm and ∞norm. Solving problems using ∞norm generally has lower computational costs than those based on 2norm.
Constraint: ${\mathbf{FOAS\; Tolerance\; Norm}}=\mathrm{INFINITY}$ or $\mathrm{TWO}$.
Infinite Bound Size  $r$  Default $\text{}={10}^{20}$ 
This defines the ‘infinite’ bound $\mathrm{bigbnd}$ in the definition of the problem constraints. Any upper bound greater than or equal to $\mathrm{bigbnd}$ will be regarded as $+\infty $ (and similarly any lower bound less than or equal to $\mathrm{bigbnd}$ will be regarded as $\infty $). Note that a modification of this optional parameter does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: ${\mathbf{Infinite\; Bound\; Size}}\ge 1000$.
Monitoring File  $i$  Default $=1$ 
If
$i\ge 0$, the
unit number
for the secondary (monitoring) output. If set to
$1$, no secondary output is provided. The following information is output to the unit:

–a listing of the optional parameters;

–problem statistics, the iteration log and the final status as set by Monitoring Level;

–the solution if set by Print Solution.
Constraint: ${\mathbf{Monitoring\; File}}\ge 1$.
Monitoring Level  $i$  Default $=4$ 
This parameter sets the amount of information detail that will be printed by the solver to the secondary output. The meaning of the levels is the same as
Print Level.
Constraint: $0\le {\mathbf{Monitoring\; Level}}\le 5$.
Print File  $i$  Default
$=\text{advisory message unit number}$

If
$i\ge 0$, the
unit number
for the primary output of the solver. If
${\mathbf{Print\; File}}=1$, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number as defined by
x04abf at the time of the optional parameter's initialization, e.g., at the initialization of the handle. The following information is output to the unit:

–a listing of optional parameters if set by Print Options;

–problem statistics, the iteration log and the final status from the solver as set by Print Level;

–the solution if set by Print Solution.
Constraint: ${\mathbf{Print\; File}}\ge 1$.
Print Level  $i$  Default $=2$ 
This parameter defines how detailed information should be printed by the solver to the primary output.
$i$ 
Output 
$0$ 
No output from the solver 
$1$ 
Only the final status and the objective value 
$2$ 
Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics 
$3$ 
As level $2$ but each iteration line is longer and includes step length and progress measure 
$4,5$ 
As level $3$ but further details of each iteration are presented 
Constraint: $0\le {\mathbf{Print\; Level}}\le 5$.
Print Options  $a$  Default $=\mathrm{YES}$ 
If ${\mathbf{Print\; Options}}=\mathrm{YES}$, a listing of optional parameters will be printed to the primary output.
Constraint: ${\mathbf{Print\; Options}}=\mathrm{YES}$ or $\mathrm{NO}$.
Print Solution  $a$  Default $=\mathrm{NO}$ 
If ${\mathbf{Print\; Solution}}=\mathrm{YES}$, the final values of the solution vector are printed on the primary and secondary outputs.
Constraint: ${\mathbf{Print\; Solution}}=\mathrm{YES}$ or $\mathrm{NO}$.
Stats Time  $a$  Default $=\mathrm{NO}$ 
This parameter allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent. This might be helpful for a choice of different solving approaches. It is possible to choose between CPU and wall clock time. Choice $\mathrm{YES}$ is equivalent to $\mathrm{WALL\; CLOCK}$.
Constraint: ${\mathbf{Stats\; Time}}=\mathrm{YES}$, $\mathrm{NO}$, $\mathrm{CPU}$ or $\mathrm{WALL\; CLOCK}$.
Task  $a$  Default $=\mathrm{MINIMIZE}$ 
This parameter specifies the required direction of the optimization. If
${\mathbf{Task}}=\mathrm{FEASIBLEPOINT}$, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found. If no objective function is set,
Task reverts to
$\mathrm{FEASIBLEPOINT}$ automatically.
Constraint: ${\mathbf{Task}}=\mathrm{MINIMIZE}$, $\mathrm{MAXIMIZE}$ or $\mathrm{FEASIBLE\; POINT}$.
Time Limit  $r$  Default $\text{}={10}^{6}$ 
This parameter specifies a limit in seconds that the solver can use to solve one problem. If during the convergence check this limit is exceeded, the solver will terminate with ${\mathbf{ifail}}={\mathbf{23}}$ error message.
Constraint: ${\mathbf{Time\; Limit}}>0$.
Verify Derivatives  $a$  Default $=\mathrm{NO}$ 
This parameter specifies whether the routine should perform numerical checks on the consistency of the usersupplied gradient function. If any discrepancies are found, ${\mathbf{ifail}}={\mathbf{26}}$ is returned.
It is recommended that such checks are enabled when first developing the formulation of the problem, however, the verification process results in a significant increase of the number of the function evaluations and thus it shouldn't be used in production code.
Constraint: ${\mathbf{Verify\; Derivatives}}=\mathrm{YES}$ or $\mathrm{NO}$.