naginterfaces.library.opt.handle_​solve_​bxnl

naginterfaces.library.opt.handle_solve_bxnl(handle, lsqfun, lsqgrd, x, nres, lsqhes=None, lsqhprd=None, monit=None, data=None, io_manager=None, spiked_sorder='C')

handle_solve_bxnl is a bound-constrained nonlinear least squares trust region solver (BXNL) from the NAG optimization modelling suite aimed for small to medium-scale problems.

Note: this function uses optional algorithmic parameters, see also: handle_opt_set(), handle_opt_get().

For full information please refer to the NAG Library document for e04gg

https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html

Parameters

handleHandle

The handle to the problem. It needs to be initialized (e.g., by handle_init()) and to hold a problem formulation compatible with handle_solve_bxnl. It must not be changed between calls to the NAG optimization modelling suite.

lsqfuncallable (rx, inform) = lsqfun(x, nres, inform, data=None)

$lsqfun$ must evaluate the value of the nonlinear residuals, $r_i\left(x\right):=y_i-\varphi (t_i;x),i=1,\dots ,n_{res}$, at a specified point $x$.

Parameters

xfloat, ndarray, shape $\left(\text{nvar}\right)$

$x$, the vector of variable values at which the residuals, $r_i$, are to be evaluated.

nresint

$n_{\text{res}}$, the current number of residuals in the model.

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns

rxfloat, array-like, shape $\left(nres\right)$

The value of the residual vector, $r\left(x\right)$, evaluated at $x$.

informint

May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting $inform$ to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with $errno$ = 25.

lsqgrdcallable inform = lsqgrd(x, nres, rdx, inform, data=None)

$lsqgrd$ evaluates the residual gradients, $\nabla r_i\left(x\right)$, at a specified point $x$.

Parameters

xfloat, ndarray, shape $\left(\text{nvar}\right)$

$x$, the vector of variable values at which the residual gradients, $\nabla r_i\left(x\right)$, are to be evaluated.

nresint

$n_{\text{res}}$, the current number of residuals in the model.

rdxfloat, ndarray, shape $\left(\text{nnzrd}\right)$, to be modified in place

On entry: the elements should only be assigned and not referenced.

On exit: the vector containing the nonzero residual gradients evaluated at $x$,

$$\nabla r\left(x\right)=[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\dots ,\nabla r_{n_{\text{res}}}\left(x\right)]\text{,}$$

where

$$\nabla r_i\left(x\right)={[\frac{\partial r_i\left(x\right)}{\partial x_1},\dots ,\frac{\partial r_i\left(x\right)}{\partial x_{n_{\text{var}}}}]}^T\text{.}$$

The elements must be stored in the same order as the defined sparsity pattern provided in the call to handle_set_nlnls().

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns

informint

May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting $inform$ to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with $errno$ = 25.

xfloat, array-like, shape $\left(\text{nvar}\right)$

$x_0$, the initial estimates of the variables, $x$.

nresint

$n_{\text{res}}$, the current number of residuals in the model.

lsqhesNone or callable inform = lsqhes(x, lamda, hx, inform, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

$lsqhes$ evaluates the residual Hessians, ${\nabla }^2{r}_i\left(x\right)$, at a specified point $x$.

By default, the option $\text{‘Bxnl Use Second Derivatives'}=\text{'NO'}$ and $lsqhes$ is never called. $lsqhes$ may be None.

This function will only be called if the option $\text{‘Bxnl Use Second Derivatives'}=\text{'YES'}$ and if the model (see Models) requires second order information.

Under these circumstances, if you do not provide a valid $lsqhes$ the solver will terminate with either $errno$ = 6 or $errno$ = 21.

Parameters

xfloat, ndarray, shape $\left(\text{nvar}\right)$

$x$, the vector of decision variables at the current iteration.

lamdafloat, ndarray, shape $\left(\text{nres}\right)$

$\lambda$, the vector containing the (weighted) residuals at $x$, ${\lambda }_i=w_i{r}_i\left(x\right)$. See [equation] and Residual Weights.

hxfloat, ndarray, shape $(\text{nvar},\text{nvar})$, to be modified in place

On entry: the elements should only be assigned and not referenced.

On exit: a dense square (symmetric) matrix containing the weighted sum of residual Hessians,

$$H\left(x\right)=\sum _{i=1}^{n_{\text{res}}}{\lambda }_i{\nabla }^2{r}_i\left(x\right)\text{,}$$

where

$$\begin{array}{c}{\nabla }^2{r}_i\left(x\right)=\left(\begin{array}{cccc}\frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_{n_{var}}}}{r}_i\left(x\right)\\ \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_{n_{var}}}}{r}_i\left(x\right)\\ ⋮& ⋮& \ddots & ⋮\\ \frac{{\partial }^2}{{\partial }_{x_{n_{var}}}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_{n_{var}}}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_{n_{var}}}{\partial }_{x_{n_{var}}}}{r}_i\left(x\right)\end{array}\right)\text{,}\end{array}$$

is also a dense square (symmetric) matrix containing the $i$th residual Hessian evaluated at the point $x$. All matrix elements must be provided: both upper and lower triangular parts.

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns

informint

May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting $inform$ to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with $errno$ = 25.

lsqhprdNone or callable (hxy, inform) = lsqhprd(x, y, hxy, inform, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

$lsqhprd$ evaluates the residual Hessians, ${\nabla }^2{r}_i\left(x\right)$, at a specified point, $x$, and performs matrix-vector products with a given vector, $y$, returning the dense matrix $[{\nabla }^2{r}_1\left(x\right)y,{\nabla }^2{r}_2\left(x\right)y,\dots ,{\nabla }^2{r}_{n_{\text{res}}}\left(x\right)y]$.

If you do not supply this function, it may be None.

Parameters

xfloat, ndarray, shape $\left(\text{nvar}\right)$

$x$, the vector of decision variables at the current iteration.

yfloat, ndarray, shape $\left(\text{nvar}\right)$

$y$, the vector used to perform the required matrix-vector products.

hxyfloat, ndarray, shape $(\text{nvar},\text{nres})$

The elements should only be assigned and not referenced.

informint

The first call to $lsqhprd$ will have a non-zero value and can be used to optimize your code in order to avoid recalculations of common quantities when evaluating the Hessians. For all other instances $inform$ will have a value of zero. This notification argument may be safely ignored if such optimization is not required.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns

hxyfloat, array-like, shape $(\text{nvar},\text{nres})$

A dense matrix of size $n_{\text{var}}\times n_{\text{res}}$ containing the following matrix-vector products,

$$H(x,y)=[{\nabla }^2{r}_1\left(x\right)y,{\nabla }^2{r}_2\left(x\right)y,\dots ,{\nabla }^2{r}_{n_{\text{res}}}\left(x\right)y]\text{.}$$

informint

May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting $inform$ to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with $errno$ = 25. The value of $inform$ returned on the first call is ignored.

monitNone or callable monit(x, rinfo, stats, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

$monit$ is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.

If no monitoring is required, $monit$ may be None.

$monit$ is called at the end of every $i$th step where $i$ is controlled by the option ‘Bxnl Monitor Frequency’ (the default value is $0$, $monit$ is not called).

Parameters

xfloat, ndarray, shape $\left(\text{nvar}\right)$

The current best point.

rinfofloat, ndarray, shape $\left(100\right)$

Best objective value computed and various indicators (the values are as described in the main argument $rinfo$).

statsfloat, ndarray, shape $\left(100\right)$

Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument $stats$).

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

dataarbitrary, optional

User-communication data for callback functions.

io_managerFileObjManager, optional

Manager for I/O in this routine.

spiked_sorderstr, optional

If $hxy$ in $lsqhprd$ is spiked (i.e., has unit extent in all but one dimension, or has size $1$), $spiked\_sorder$ selects the storage order to associate with it in the NAG Engine:

spiked_sorder = $\text{'C'}$

row-major storage will be used;

spiked_sorder = $\text{'F'}$

column-major storage will be used.

Returns

xfloat, ndarray, shape $\left(\text{nvar}\right)$

The final values of the variables, $x$.

rxfloat, ndarray, shape $\left(nres\right)$

The values of the residuals at the final point given in $x$.

rinfofloat, ndarray, shape $\left(100\right)$

Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:

$0$	Objective function value, $f\left(x\right)$.
$1$	Norm of the projected gradient at the current iterate, see PG STEP in Bound Constraints and [equation] in Stopping Criteria.
$2$	Norm of the scaled projected gradient at the current iterate, see [equation] in Stopping Criteria
$3$	Norm of the step between the current and previous iterate.
$4$	Convergence tests result. A scalar value between $0-7$ indicates whether a convergence test has passed. Specifically, $1$ indicates small objective test passed, $2$ indicates small (scaled) gradient test passed, $4$ indicates small step test passed. In the case where two or more tests passed, they are accumulated.
$5$	Norm of the current iterate $x$. If regularization is requested, then this value was used in the regularization and it might differ from $\parallel x\parallel$ if $x$ has fixed or disabled elements.
$6$–$99$	Reserved for future use.

statsfloat, ndarray, shape $\left(100\right)$

Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:

$0$	Number of iterations performed.
$1$	Total number of calls to the objective function $lsqfun$.
$2$	Total number of calls to the objective gradient function $lsqgrd$.
$3$	Total number of calls to the objective Hessian function $lsqhes$.
$4$	Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.
$5$	Number of calls to the objective function $lsqfun$ required by linesearch steps.
$6$	Number of calls to the objective gradient function $lsqgrd$ required by linesearch steps.
$7$	Number of calls to the objective function $lsqfun$ required by projected gradient steps.
$8$	Number of calls to the objective gradient function $lsqgrd$ required by projected gradient steps.
$9$	Number of inner iterations performed, see option $\text{‘Bxnl Model'}=\text{'TENSOR-NEWTON'}$.
$10$	Number of linesearch iterations performed.
$11$	Number of projected gradient iterations performed.
$12$	Total number of calls to the objective auxiliary Hessian function $lsqhprd$.
$13$–$99$	Reserved for future use.

Other Parameters

‘Defaults’valueless

This special keyword may be used to reset all options to their default values. Any value given with this keyword will be ignored.

‘Bxnl Basereg Pow’float

Default $=2.0$

This argument defines the regularization power $p$ in [equation] and for the tensor Newton subproblem (when $\text{‘Bxnl Tn Method'}=\text{'IMPLICIT'}$). Some values are restricted depending on the type of regularization specified, see ‘Bxnl Basereg Type’ for more details.

Constraint: $\text{‘Bxnl Basereg Pow'}>0$.

‘Bxnl Basereg Term’float

Default $=0.01$

This argument defines the regularization term $\sigma$ in [equation] and for the tensor Newton subproblem (when $\text{‘Bxnl Tn Method'}=\text{'IMPLICIT'}$).

Constraint: $\text{‘Bxnl Basereg Term'}>0$.

‘Bxnl Basereg Type’str

Default $=\text{'NONE'}$

This argument specifies the method used to incorporate the regularizer into [equation] and optionally into the tensor Newton subproblem (when $\text{‘Bxnl Model'}=\text{'Tensor-Newton'}$ and $\text{‘Bxnl Tn Method'}=\text{'IMPLICIT'}$).

The option $\text{‘Bxnl Basereg Type'}=\text{'EXPAND-NVAR-DOF'}$ reformulates the original problem by expanding it with $n_{\text{var}}$ degrees of freedom that is subsequently solved. For the case $\text{‘Bxnl Basereg Type'}=\text{'EXPAND-1-DOF'}$ the residual vector is extended with a new term of the form $\frac{\sigma }{p}{\parallel x\parallel }_2^p$; for this method a value of $p=3$ is recommended.

If $\text{‘Bxnl Basereg Type'}=\text{'EXPAND-NVAR-DOF'}$ then the regularization power term $p$ must be $2.0$, that is $\text{‘Bxnl Basereg Pow'}=\text{'2.0'}$. For further details see Subproblems.

Constraint: $\text{‘Bxnl Basereg Type'}=\text{'NONE'}$, $\text{'EXPAND-NVAR-DOF'}$ or $\text{'EXPAND-1-DOF'}$.

‘Bxnl Save Covariance Matrix’str

Default $=\text{'NO'}$

This argument indicates to the solver to store the covariance matrix into the handle.

If $\text{‘Bxnl Save Covariance Matrix'}=\text{'YES'}$ then the lower triangle part of the covariance matrix is stored in packed column order (see the F07 Introduction) into the handle and can be retrieved via handle_set_get_real() using $\text{cmdstr}=\text{'COVARIANCE MATRIX'}$ with $\text{lrarr}=(n_{\text{var}}\times (n_{\text{var}}+1))/2$.

In the special case where $\text{‘Bxnl Save Covariance Matrix'}=\text{'VARIANCE'}$, only the diagonal elements of the covariance matrix are stored in the handle and can be retrieved via handle_set_get_real() using $\text{cmdstr}=\text{'VARIANCE'}$ with $\text{lrarr}=n_{\text{var}}$.

Similarly, if $\text{‘Bxnl Save Covariance Matrix'}=\text{'HESSIAN'}$ then the lower triangle part of the matrix $H\left(x\right)=\nabla r\left(x\right){\nabla r\left(x\right)}^T={J\left(x\right)}^{T}J\left(x\right)$ is stored in packed column order into the handle and can be retrieved via handle_set_get_real() using $\text{cmdstr}=\text{'HESSIAN MATRIX'}$ with $\text{lrarr}=(n_{\text{var}}\times (n_{\text{var}}+1))/2$.

Limitations: If the number of enabled residuals is not greater than the number of enabled variables, or the pseudo-inverse of $H\left(x\right)$ could not be calculated, then the covariance matrix (variance vector) is not stored in the handle and will not be available.

For more information on how the covariance matrix is estimated, see lsq_uncon_covariance().

Constraint: $\text{‘Bxnl Save Covariance Matrix'}=\text{'NO'}$, $\text{'YES'}$, $\text{'VARIANCE'}$ or $\text{'HESSIAN'}$.

‘Bxnl Stop Abs Tol Fun’float

Default $={ϵ}^{\frac{1}{2}}$

This argument specifies the relative tolerance for the error test, specifically, it sets the value of ${ϵ}_{\text{abs}}^f$ of equation [equation] in Stopping Criteria. Setting ‘Bxnl Stop Abs Tol Fun’ to a large value may cause the solver to stop prematurely with a suboptimal solution.

Constraint: $\text{‘Bxnl Stop Abs Tol Fun'}>0$.

‘Bxnl Stop Abs Tol Grd’float

Default $=2.2{ϵ}^{\frac{1}{3}}$

This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of ${ϵ}_{\text{abs}}^g$ of equation [equation] in Stopping Criteria. Setting ‘Bxnl Stop Abs Tol Grd’ to a large value may cause the solver to stop prematurely with a suboptimal solution.

Constraint: $\text{‘Bxnl Stop Abs Tol Grd'}>0$.

‘Bxnl Stop Rel Tol Fun’float

Default $={ϵ}^{\frac{1}{2}}$

This argument specifies the relative tolerance for the error test, specifically, it sets the value of ${ϵ}_{\text{rel}}^f$ of equation [equation] in Stopping Criteria. Setting ‘Bxnl Stop Rel Tol Fun’ to a large value may cause the solver to stop prematurely with a suboptimal solution.

Constraint: $\text{‘Bxnl Stop Rel Tol Fun'}>0$.

‘Bxnl Stop Rel Tol Grd’float

Default $={ϵ}^{\frac{1}{2}}$

This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of ${ϵ}_{\text{rel}}^g$ of equation [equation] in Stopping Criteria. Setting ‘Bxnl Stop Rel Tol Grd’ to a large value may cause the solver to stop prematurely with a suboptimal solution.

Constraint: $\text{‘Bxnl Stop Rel Tol Grd'}>0$.

‘Bxnl Stop Step Tol’float

Default $=2ϵ$

Specifies the stopping tolerance for the step length test, specifically, it sets the value for ${ϵ}_{\text{step}}$ of equation [equation] in Stopping Criteria. Setting ‘Bxnl Stop Step Tol’ to a large value may cause the solver to stop prematurely with a suboptimal solution.

Under certain circumstances, e.g., when in doubt of the quality of the first - or second-order derivatives, in the event of the solver exiting with a successful step length test, it is recommended to verify that either the error or the gradient norm is acceptably small.

Constraint: $\text{‘Bxnl Stop Step Tol'}>0$.

‘Bxnl Reg Order’str

Default $=\text{'AUTO'}$

This argument specifies the order of the regularization $p$ in [equation] used when $\text{‘Bxnl Glob Method'}=\text{'Reg'}$.

Some values for $p$ are restricted depending on the method chosen in ‘Bxnl Nlls Method’, see Regularization for more details.

Constraint: $\text{‘Bxnl Reg Order'}=\text{'AUTO'}$, $\text{'QUADRATIC'}$ or $\text{'CUBIC'}$.

‘Bxnl Glob Method’str

Default $=\text{'TR'}$

This argument specifies the globalization method used to estimate the next step $s_k$. It also determines the class of subproblem to solve. The trust region subproblem finds the step by minimizing the specified model withing a given radius. On the other hand, when $\text{‘Bxnl Glob Method'}=REG$, the problem is reformulated by adding an aditional regularization term and minimized in order to find the next step $s_k$. See Subproblems for more details.

Constraint: $\text{‘Bxnl Glob Method'}=\text{'TR'}$ or $\text{'REG'}$.

‘Bxnl Nlls Method’str

Default $=\text{'GALAHAD'}$

This argument defines the method used to estimate the next step $s_k$ in $x_{k+1}=x_k+s_k$. It only applies to $\text{‘Bxnl Model'}=\text{'GAUSS-NEWTON'}$, $\text{'QUASI-NEWTON'}$ or $\text{'HYBRID'}$. When the globalization technique chosen is trust region ($\text{‘Bxnl Glob Method'}=TR$) the methods for ‘Bxnl Nlls Method’ available are Powell’s dogleg method, a generalized eigenvalue method (AINT) Adachi et al. (2015), a variant of Moré–Sorensen’s method, and GALAHAD’s DTRS method. Otherwise, when the globalization method chosen is via regularization ($\text{‘Bxnl Glob Method'}=REG$) the methods available are comprised by a linear system solver and GALAHAD’s DRQS method. See Subproblems for more details.

Constraint: $\text{‘Bxnl Nlls Method'}=\text{'POWELL-DOGLEG'}$, $\text{'AINT'}$, $\text{'MORE-SORENSEN'}$, $\text{'LINEAR SOLVER'}$ or $\text{'GALAHAD'}$.

‘Bxnl Model’str

Default $=\text{'HYBRID'}$

This argument specifies which model is used to approximate the objective function and estimate the next point that reduces the error. This is one of the most important options and should be chosen according to the problem characteristics. The models are briefly described in Models.

Constraint: $\text{‘Bxnl Model'}=\text{'GAUSS-NEWTON'}$, $\text{'QUASI-NEWTON'}$, $\text{'HYBRID'}$ or $\text{'TENSOR-NEWTON'}$.

‘Bxnl Tn Method’str

Default $=\text{'MIN-1-VAR'}$

This argument specifies how to solve the subproblem and find the next step $s_k$ for the tensor Newton model, $\text{‘Bxnl Model'}=\text{'TENSOR-NEWTON'}$. The subproblems are solved using a range of regularization schemes. See Tensor Newton subproblem.

Constraint: $\text{‘Bxnl Tn Method'}=\text{'IMPLICIT'}$, $\text{'MIN-1-VAR'}$, $\text{'MIN-NVAR'}$, $\text{'ADD-1-VAR'}$ or $\text{'ADD-NVAR'}$.

‘Bxnl Use Second Derivatives’str

Default $=\text{'NO'}$

This argument indicates whether the weighted sum of residual Hessians are available through the call-back $lsqhes$. If $\text{‘Bxnl Use Second Derivatives'}=\text{'NO'}$ and the specified model in ‘Bxnl Model’ requires user-suppied second derivatives, then the solver will terminate with $errno$ = 6.

Constraint: $\text{‘Bxnl Use Second Derivatives'}=\text{'YES'}$ or $\text{'NO'}$.

‘Bxnl Use Weights’str

Default $=\text{'NO'}$

This argument indicates whether to use a weighted nonlinear least square model. If $\text{‘Bxnl Use Weights'}=\text{'YES'}$ then the weights $w_i>0,i=1,\dots ,n_{\text{res}}$ in [equation] must be supplied by you via handle_set_get_real(). If weights are to be used, then all $n_{\text{res}}$ elements must be provided, see Residual Weights. If the solver is expecting to use weights but they are not provided or have non-positive values, then the solver will terminate with $errno$ = 11.

Constraint: $\text{‘Bxnl Use Second Derivatives'}=\text{'YES'}$ or $\text{'NO'}$.

‘Bxnl Iteration Limit’int

Default $=1000$

This argument specifies the maximum amount of major iterations the solver is alloted. If this limit is reached, then the solver will terminate with $errno$ = 22.

Constraint: $\text{‘Bxnl Iteration Limit'}\ge 1$.

‘Bxnl Monitor Frequency’int

Default $=0$

If $\text{‘Bxnl Monitor Frequency'}>0$, the user-supplied function $monit$ will be called at the end of every $i$th step for monitoring purposes.

Constraint: $\text{‘Bxnl Monitor Frequency'}\ge 0$.

‘Bxnl Print Header’int

Default $=30$

This argument defines, in number of iterations, the frequency with which to print the iteration log header.

Constraint: $\text{‘Bxnl Print Header'}\ge 1$.

‘Infinite Bound Size’float

Default $={10}^{20}$

This defines the ‘infinite’ bound $\text{bigbnd}$ in the definition of the problem constraints. Any upper bound greater than or equal to $\text{bigbnd}$ will be regarded as $+\infty$ (and similarly any lower bound less than or equal to $-\text{bigbnd}$ will be regarded as $-\infty$). Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.

Constraint: $\text{‘Infinite Bound Size'}\ge 1000$.

‘Monitoring File’int

Default $=-1$

If $i\ge 0$, the unit number for the secondary (monitoring) output. If $\text{‘Monitoring File'}=-1$, no secondary output is provided. The information output to this unit is controlled by ‘Monitoring Level’.

Constraint: $\text{‘Monitoring File'}\ge -1$.

‘Monitoring Level’int

Default $=4$

This argument sets the amount of information detail that will be printed by the solver to the secondary output. The meaning of the levels is the same as for ‘Print Level’.

Constraint: $0\le \text{‘Monitoring Level'}\le 5$.

‘Print File’int

Default $=\text{advisory message unit number}$

If $i\ge 0$, the unit number for the primary output of the solver. If $\text{‘Print File'}=-1$, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by ‘Print Level’.

Constraint: $\text{‘Print File'}\ge -1$.

‘Print Level’int

Default $=2$

This argument defines how detailed information should be printed by the solver to the primary and secondary output.

$i$	Output
$0$	No output from the solver.
$1$	The Header and Summary.
$2$, $3$, $4$, $5$	Additionally, the Iteration log.

Constraint: $0\le \text{‘Print Level'}\le 5$.

‘Print Options’str

Default $=\text{'YES'}$

If $\text{‘Print Options'}=\text{'YES'}$, a listing of options will be printed to the primary output and is always printed to the secondary output.

Constraint: $\text{‘Print Options'}=\text{'YES'}$ or $\text{'NO'}$.

‘Print Solution’str

Default $=\text{'NO'}$

If $\text{‘Print Solution'}=\text{'X'}$, the final values of the primal variables are printed on the primary and secondary outputs.

If $\text{‘Print Solution'}=\text{'YES'}$ or $\text{'ALL'}$, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.

Constraint: $\text{‘Print Solution'}=\text{'YES'}$, $\text{'NO'}$, $\text{'X'}$ or $\text{'ALL'}$.

‘Stats Time’str

Default $=\text{'NO'}$

This argument turns on timing. This might be helpful for a choice of different solving approaches. It is possible to choose between CPU and wall clock time. Choice ‘YES’ is equivalent to ‘WALL CLOCK’.

Constraint: $\text{‘Stats Time'}=\text{'YES'}$, $\text{'NO'}$, $\text{'CPU'}$ or $\text{'WALL CLOCK'}$.

‘Time Limit’float

Default $={10}^6$

A limit to the number of seconds that the solver can use to solve one problem. If at the end of an iteration this limit is exceeded, the solver will terminate with $errno$ = 23.

Constraint: $\text{‘Time Limit'}>0$.

Raises

NagValueError

(errno $1$)

$handle$ has not been initialized.

(errno $1$)

$handle$ does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.

(errno $1$)

$handle$ has not been initialized properly or is corrupted.

(errno $2$)

This solver does not support the model defined in the handle.

(errno $2$)

The problem is already being solved.

(errno $3$)

Unsupported option combinations.

(errno $3$)

Unsupported model and method chosen.

(errno $4$)

On entry, $\text{nvar}=⟨value⟩$, expected $value=⟨value⟩$.

Constraint: $\text{nvar}$ must match the current number of variables of the model in the $handle$.

(errno $4$)

The information supplied does not match with that previously stored.

On entry, $nres=⟨value⟩$ must match that given during the definition of the objective in the $handle$, i.e., $⟨value⟩$.

(errno $4$)

There are no decision variables. $\text{nvar}$ must be greater than zero.

(errno $6$)

Exact second derivatives needed for tensor model.

(errno $11$)

Data for residual weights not found or is invalid.

(errno $21$)

The current starting point is unusable.

Warns

NagAlgorithmicMajorWarning

(errno $18$)

Numerical difficulties encountered and solver was terminated.

(errno $19$)

Iteration limit reached while solving a subproblem.

(errno $19$)

Line Search failed.

(errno $22$)

Maximum number of iterations reached.

(errno $23$)

The solver terminated after the maximum time allowed was exceeded.

(errno $24$)

The solver was terminated because no further progress could be achieved.

(errno $25$)

Invalid number detected in user-supplied function and recovery failed.

NagCallbackTerminateWarning

(errno $20$)

User requested termination during a monitoring step.

Notes

handle_solve_bxnl computes a solution $x$ to the nonlinear least squares problem

$$\begin{array}{c}\begin{array}{cc}{\text{minimize}}_{x\in R^{n_{\text{var}}}}& f\left(x\right)=\frac{1}{2}{\sum }_{i=1}^{n_{\text{res}}}{\left[w_i{r}_i\left(x\right)\right]}^2+\frac{\sigma }{p}{\parallel x\parallel }_2^p\\ \text{subject to}& l_x\le x\le u_x\text{,}\end{array}\end{array}$$

where $r_i\left(x\right),i=1,\dots ,n_{\text{res}}$, are smooth nonlinear functions called residuals, $w_i,i=1,\dots ,n_{\text{res}}$ are weights (by default they are all defined to $1$, see Residual Weights on how to change them), and the rightmost element represents the regularization term with argument $\sigma \ge 0$ and power $p>0$ (by default the regularization term is not used, see Algorithmic Details on how to enable it). The constraint elements $l_x$ and $u_x$ are $n_{\text{var}}$-dimensional vectors defining the bounds on the variables.

Typically in a calibration or data fitting context, the residuals will be defined as the difference between the observed values $y_i$ at $t_i$ and the values provided by a nonlinear model $\varphi (t;x)$, i.e., $r_i\left(x\right):=y_i-\varphi (t_i;x)$. If these residuals (errors) follow a Gaussian distribution, then the values of the optimal parameter vector $x^{\ast }$ are the maximum likelihood estimates. For a description of the various algorithms implemented for solving this problem see Algorithmic Details. It is also recommended that you read the E04 Introduction.

handle_solve_bxnl serves as a solver for problems stored as a handle. The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. First, the problem handle is initialized by calling handle_init(). A nonlinear least square residual objective can be added by calling handle_set_nlnls() and, optionally, (simple) box constraints can be defined by calling handle_set_simplebounds(). It should be noted that handle_solve_bxnl internally works with a dense representation of the residual Jacobian even if a sparse structure was defined in the call to handle_set_nlnls(). Once the problem is fully described, the handle may be passed to the solver handle_solve_bxnl. When the handle is not needed anymore, handle_free() should be called to destroy it and deallocate the memory held within. For more information refer to the NAG optimization modelling suite in the E04 Introduction.

The algorithm is based on the trust region framework and its behaviour can be modified by various options (see Other Parameters) which can be set by handle_opt_set() and handle_opt_set_file() anytime between the initialization of the handle by handle_init() and a call to the solver. Once the solver has finished, options may be modified for the next solve. The solver may be called repeatedly with various starting points and/or options. The option getter handle_opt_get() can be called to retrieve the current value of any option.

Several options might have significant impact on the performance of the solver. Even though the defaults were chosen to suit the majority of anticipated problems, it is recommended that you experiment with the option settings to find the most suitable set of options for a particular problem, see Algorithmic Details and Other Parameters for further details.

References

Adachi, S, Iwata, S, Nakatsukasa, Y, and Takeda, A, 2015, Solving the trust region subproblem by a generalized eigenvalue problem, Technical report, METR 2015-14., Mathematical Engineering, The University of Tokyo, https://www.keisu.t.u-tokyo.ac.jp/data/2015/METR15-14.pdf

Conn, A R, Gould, N I M and Toint, Ph L, 2000, Trust Region Methods, SIAM, Philadephia

Gould, N I M, Orban, D, and Toint, Ph L, 2003, GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization, ACM Transactions on Mathematical Software (TOMS) (29(4)), 353–372

Gould, N I M, Rees, T, and Scott, J A, 2017, A higher order method for solving nonlinear least-squares problems, Technical report, RAL-P-1027-010, RAL Library. STFC Rutherford Appleton Laboratory, http://www.numerical.rl.ac.uk/people/rees/pdf/RAL-P-2017-010.pdf

Kanzow, C, Yamashita, N, and Fukushima, M, 2004, Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints, Journal of Computational and Applied Mathematics (174), 375–397

Lanczos, C, 1956, Applied Analysis, 272–280, Prentice Hall, Englewood Cliffs, NJ, USA

Nielsen, H B, 1999, Damping parameter in Marquadt’s Method, Technical report TR IMM-REP-1999-05., Department of Mathematical Modelling, Technical University of Denmark, http://www2.imm.dtu.dk/documents/ftp/tr99/tr05_99.pdf

Nocedal, J and Wright, S J, 2006, Numerical Optimization, (2nd Edition), Springer Series in Operations Research, Springer, New York

See also

naginterfaces.library.examples.opt.handle_disable_ex.main()