NAG CL Interface
e04ggc (handle_​solve_​bxnl)

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

1: $\mathbf{handle}$ – void * Input

On entry: the handle to the problem. It needs to be initialized (e.g., by e04rac) and to hold a problem formulation compatible with e04ggc. It must not be changed between calls to the NAG optimization modelling suite.

2: $\mathbf{lsqfun}$ – function, supplied by the user External Function

lsqfun must evaluate the value of the nonlinear residuals, $r_i\left(x\right)≔y_i-\varphi (t_i;x),i=1,\dots ,n_{\mathrm{res}}$, at a specified point $x$.

The specification of lsqfun is:

copy

void  lsqfun (Integer nvar, const double x[], Integer nres, double rx[], Integer *inform, Nag_Comm *comm)

1: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

2: $\mathbf{x}\left[\mathbf{nvar}\right]$ – const double Input

On entry: $x$, the vector of variable values at which the residuals, $r_i$, are to be evaluated.

3: $\mathbf{nres}$ – Integer Input

On entry: $n_{\mathrm{res}}$, the current number of residuals in the model.

4: $\mathbf{rx}\left[\mathbf{nres}\right]$ – double Output

On exit: the value of the residual vector, $r\left(x\right)$, evaluated at $x$.

5: $\mathbf{inform}$ – Integer * Input/Output

On entry: a non-negative value.

On exit: may be used to indicate that some residuals could not be computed at the requested point. This can be done by setting inform to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_USER_NAN.

6: $\mathbf{comm}$ – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to lsqfun.

user – double *

iuser – Integer *

p – Pointer 

The type Pointer will be void *. Before calling e04ggc you may allocate memory and initialize these pointers with various quantities for use by lsqfun when called from e04ggc (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

3: $\mathbf{lsqgrd}$ – function, supplied by the user External Function

lsqgrd evaluates the residual gradients, $\nabla r_i\left(x\right)$, at a specified point $x$.

The specification of lsqgrd is:

copy

void  lsqgrd (Integer nvar, const double x[], Integer nres, Integer nnzrd, double rdx[], Integer *inform, Nag_Comm *comm)

1: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

2: $\mathbf{x}\left[\mathbf{nvar}\right]$ – const double Input

On entry: $x$, the vector of variable values at which the residual gradients, $\nabla r_i\left(x\right)$, are to be evaluated.

3: $\mathbf{nres}$ – Integer Input

On entry: $n_{\mathrm{res}}$, the current number of residuals in the model.

4: $\mathbf{nnzrd}$ – Integer Input

On entry: the number of nonzeros in the first derivative matrix. If $\mathbf{isparse}=0$ in the call to e04rmc (recommended use for e04ggc) then $\mathbf{nnzrd}=\mathbf{nvar}*\mathbf{nres}$.

5: $\mathbf{rdx}\left[\mathit{dim}\right]$ – double Input/Output

On entry: the elements should only be assigned and not referenced.

On exit: the vector containing the nonzero residual gradients evaluated at $x$,

$$\nabla r\left(x\right)=[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\dots ,\nabla r_{n_{\mathrm{res}}}\left(x\right)]\text{,}$$

where

$$\nabla r_i\left(x\right)={[\frac{\partial r_i\left(x\right)}{\partial x_1},\dots ,\frac{\partial r_i\left(x\right)}{\partial x_{n_{\mathrm{var}}}}]}^{\mathrm{T}}\text{.}$$

The elements must be stored in the same order as the defined sparsity pattern provided in the call to e04rmc.

6: $\mathbf{inform}$ – Integer * Input/Output

On entry: a non-negative value.

On exit: may be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting inform to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_USER_NAN.

7: $\mathbf{comm}$ – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to lsqgrd.

user – double *

iuser – Integer *

p – Pointer 

The type Pointer will be void *. Before calling e04ggc you may allocate memory and initialize these pointers with various quantities for use by lsqgrd when called from e04ggc (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

4: $\mathbf{lsqhes}$ – function, supplied by the user External Function

lsqhes evaluates the residual Hessians, ${\nabla }^2{r}_i\left(x\right)$, at a specified point $x$. By default, the optional parameter $\mathbf{Bxnl\; Use\; Second\; Derivatives}=\mathrm{NO}$ and lsqhes is never called. lsqhes may be specified as NULLFN.

This function will only be called if the optional parameter $\mathbf{Bxnl\; Use\; Second\; Derivatives}=\mathrm{YES}$ and if the model (see Section 11.2) requires second order information. Under these circumstances, if you do not provide a valid lsqhes the solver will terminate with either $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_DERIV_ERRORS or $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_FAILED_START.

The specification of lsqhes is:

copy

void  lsqhes (Integer nvar, const double x[], Integer nres, const double lambda[], double hx[], Integer *inform, Nag_Comm *comm)

1: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

2: $\mathbf{x}\left[\mathbf{nvar}\right]$ – const double Input

On entry: $x$, the vector of decision variables at the current iteration.

3: $\mathbf{nres}$ – Integer Input

On entry: $n_{\mathrm{res}}$, the current number of residuals in the model.

4: $\mathbf{lambda}\left[\mathbf{nres}\right]$ – const double Input

On entry: $\lambda$, the vector containing the (weighted) residuals at $x$, ${\lambda }_i=w_i{r}_i\left(x\right)$. See (1) and Section 9.2.

5: $\mathbf{hx}\left[\mathbf{nvar}\times \mathbf{nvar}\right]$ – double Output

On exit: a dense square (symmetric) matrix containing the weighted sum of residual Hessians,

$$H\left(x\right)=\sum _{i=1}^{n_{\mathrm{res}}}{\lambda }_i{\nabla }^2{r}_i\left(x\right)\text{,}$$

where

$${\nabla }^2{r}_i\left(x\right)=\left(\begin{array}{cccc}\frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_1}{\partial }_{x_{n_{\mathrm{var}}}}}{r}_i\left(x\right)\\ \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_2}{\partial }_{x_{n_{\mathrm{var}}}}}{r}_i\left(x\right)\\ ⋮& ⋮& \ddots & ⋮\\ \frac{{\partial }^2}{{\partial }_{x_{n_{\mathrm{var}}}}{\partial }_{x_1}}{r}_i\left(x\right)& \frac{{\partial }^2}{{\partial }_{x_{n_{\mathrm{var}}}}{\partial }_{x_2}}{r}_i\left(x\right)& \dots & \frac{{\partial }^2}{{\partial }_{x_{n_{\mathrm{var}}}}{\partial }_{x_{n_{\mathrm{var}}}}}{r}_i\left(x\right)\end{array}\right)\text{,}$$

is also a dense square (symmetric) matrix containing the $i$th residual Hessian evaluated at the point $x$. All matrix elements must be provided: both upper and lower triangular parts.

6: $\mathbf{inform}$ – Integer * Input/Output

On entry: a non-negative value.

On exit: may be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting inform to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_USER_NAN.

7: $\mathbf{comm}$ – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to lsqhes.

user – double *

iuser – Integer *

p – Pointer 

The type Pointer will be void *. Before calling e04ggc you may allocate memory and initialize these pointers with various quantities for use by lsqhes when called from e04ggc (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

5: $\mathbf{lsqhprd}$ – function, supplied by the user External Function

lsqhprd evaluates the residual Hessians, ${\nabla }^2{r}_i\left(x\right)$, at a specified point, $x$, and performs matrix-vector products with a given vector, $y$, returning the dense matrix $[{\nabla }^2{r}_1\left(x\right)y,{\nabla }^2{r}_2\left(x\right)y,\dots ,{\nabla }^2{r}_{n_{\mathrm{res}}}\left(x\right)y]$. If you do not supply this function, it may be specified as NULLFN.

The specification of lsqhprd is:

copy

void  lsqhprd (Integer nvar, const double x[], const double y[], Integer nres, double hxy[], Integer *inform, Nag_Comm *comm)

1: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

2: $\mathbf{x}\left[\mathbf{nvar}\right]$ – const double Input

On entry: $x$, the vector of decision variables at the current iteration.

3: $\mathbf{y}\left[\mathbf{nvar}\right]$ – const double Input

On entry: $y$, the vector used to perform the required matrix-vector products.

4: $\mathbf{nres}$ – Integer Input

On entry: $n_{\mathrm{res}}$, the current number of residuals in the model.

5: $\mathbf{hxy}\left[\mathbf{nvar}\times \mathbf{nres}\right]$ – double Input/Output

Note: the $(i,j)$th element of the matrix is stored in $\mathbf{hxy}\left[(j-1)\times \mathbf{nvar}+i-1\right]$.

On entry: the elements should only be assigned and not referenced.

On exit: a dense matrix of size $n_{\mathrm{var}}\times n_{\mathrm{res}}$ containing the following matrix-vector products,

$$H(x,y)=[{\nabla }^2{r}_1\left(x\right)y,{\nabla }^2{r}_2\left(x\right)y,\dots ,{\nabla }^2{r}_{n_{\mathrm{res}}}\left(x\right)y]\text{.}$$

6: $\mathbf{inform}$ – Integer * Input/Output

On entry: The first call to lsqhprd will have a non-zero value and can be used to optimize your code in order to avoid recalculations of common quantities when evaluating the Hessians. For all other instances inform will have a value of zero. This notification parameter may be safely ignored if such optimization is not required.

On exit: may be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting inform to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_USER_NAN. The value of inform returned on the first call is ignored.

7: $\mathbf{comm}$ – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to lsqhprd.

user – double *

iuser – Integer *

p – Pointer 

The type Pointer will be void *. Before calling e04ggc you may allocate memory and initialize these pointers with various quantities for use by lsqhprd when called from e04ggc (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

6: $\mathbf{monit}$ – function, supplied by the user External Function

monit is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.

If no monitoring is required, monit may be specified as NULLFN.

monit is called at the end of every $i$th step where $i$ is controlled by the optional parameter $\mathbf{Bxnl\; Monitor\; Frequency}$ (the default value is $0$, monit is not called).

The specification of monit is:

copy

void  monit (Integer nvar, const double x[], Integer *inform, const double rinfo[], const double stats[], Nag_Comm *comm)

1: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

2: $\mathbf{x}\left[\mathbf{nvar}\right]$ – const double Input

On entry: the current best point.

3: $\mathbf{inform}$ – Integer * Input/Output

On entry: a non-negative value.

On exit: may be used to request the solver to stop immediately by setting inform to a non-zero value in which case it will terminate with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_USER_STOP; otherwise, the solver will proceed normally.

4: $\mathbf{rinfo}\left[100\right]$ – const double Input

On entry: best objective value computed and various indicators (the values are as described in the main argument rinfo).

5: $\mathbf{stats}\left[100\right]$ – const double Input

On entry: solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument stats).

6: $\mathbf{comm}$ – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to monit.

user – double *

iuser – Integer *

p – Pointer 

The type Pointer will be void *. Before calling e04ggc you may allocate memory and initialize these pointers with various quantities for use by monit when called from e04ggc (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

7: $\mathbf{nvar}$ – Integer Input

On entry: $n_{\mathrm{var}}$, the current number of decision variables, $x$, in the model.

8: $\mathbf{x}\left[\mathbf{nvar}\right]$ – double Input/Output

On entry: $x_0$, the initial estimates of the variables, $x$.

On exit: the final values of the variables, $x$.

9: $\mathbf{nres}$ – Integer Input

On entry: $n_{\mathrm{res}}$, the current number of residuals in the model.

10: $\mathbf{rx}\left[\mathbf{nres}\right]$ – double Output

On exit: the values of the residuals at the final point given in x.

11: $\mathbf{rinfo}\left[100\right]$ – double Output

On exit: objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:

$0$	Objective function value, $f\left(x\right)$.
$1$	Norm of the projected gradient at the current iterate, see PG STEP in Section 11.4 and (8) in Section 11.5.
$2$	Norm of the scaled projected gradient at the current iterate, see (8) in Section 11.5
$3$	Norm of the step between the current and previous iterate.
$4$	Convergence tests result. A scalar value between $0-7$ indicates whether a convergence test has passed. Specifically, $1$ indicates small objective test passed, $2$ indicates small (scaled) gradient test passed, $4$ indicates small step test passed. In the case where two or more tests passed, they are accumulated.
$5$	Norm of the current iterate $x$. If regularization is requested, then this value was used in the regularization and it might differ from $\Vert x\Vert$ if $x$ has fixed or disabled elements.
$6-99$	Reserved for future use.

12: $\mathbf{stats}\left[100\right]$ – double Output

On exit: solver statistics at monitoring steps or at the end of the final iteration as given in the table below:

$0$	Number of iterations performed.
$1$	Total number of calls to the objective function lsqfun.
$2$	Total number of calls to the objective gradient function lsqgrd.
$3$	Total number of calls to the objective Hessian function lsqhes.
$4$	Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.
$5$	Number of calls to the objective function lsqfun required by linesearch steps.
$6$	Number of calls to the objective gradient function lsqgrd required by linesearch steps.
$7$	Number of calls to the objective function lsqfun required by projected gradient steps.
$8$	Number of calls to the objective gradient function lsqgrd required by projected gradient steps.
$9$	Number of inner iterations performed, see option $\mathbf{Bxnl\; Model}=\mathrm{TENSOR-NEWTON}$.
$10$	Number of linesearch iterations performed.
$11$	Number of projected gradient iterations performed.
$12$	Total number of calls to the objective auxiliary Hessian function lsqhprd.
$13-99$	Reserved for future use.

13: $\mathbf{comm}$ – Nag_Comm *

The NAG communication argument (see Section 3.1.1 in the Introduction to the NAG Library CL Interface).

14: $\mathbf{fail}$ – NagError * Input/Output

The NAG error argument (see Section 7 in the Introduction to the NAG Library CL Interface).

e04ggc returns with $\mathbf{fail}\mathbf{.}\mathbf{code}=$ NE_NOERROR if the iterates have converged to a point $x$ that satisfies the convergence criteria described in Section 11.5.

6 Error Indicators and Warnings

NE_ALLOC_FAIL

Dynamic memory allocation failed.
See Section 3.1.2 in the Introduction to the NAG Library CL Interface for further information.

NE_BAD_PARAM

On entry, argument $⟨\mathit{\text{value}}⟩$ had an illegal value.

NE_DC_MATCH

Data for residual weights not found or is invalid.

Custom residual weights are required, optional parameter $\mathbf{Bxnl\; Use\; Weights}=\mathrm{YES}$, but the weights data is missing, of the wrong expected size or has invalid values. Please refer to Section 9.2.

NE_DERIV_ERRORS

Exact second derivatives needed for tensor model.

Model in the optional parameter $\mathbf{Bxnl\; Model}=\mathrm{TENSOR-NEWTON}$ requires exact second derivatives but $\mathbf{Bxnl\; Use\; Second\; Derivatives}=\mathrm{NO}$.
Provide second derivatives via lsqhes and optionally lsqhprd functions, and set optional parameter $\mathbf{Bxnl\; Use\; Second\; Derivatives}=\mathrm{YES}$.

NE_FAILED_START

The current starting point is unusable.

While trying to evaluate the starting point $x_0$, either inform was set to a non-zero value within the user-supplied functions, lsqfun, lsqgrd or lsqhes, or an Infinity or NaN was detected in values returned from them.

NE_HANDLE

The supplied handle does not define a valid handle to the data structure for the NAG optimization modelling suite. It has not been properly initialized or it has been corrupted.

NE_INTERNAL_ERROR

An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
See Section 7.5 in the Introduction to the NAG Library CL Interface for further information.

NE_NO_IMPROVEMENT

The solver was terminated because no further progress could be achieved.

This can indicate that the solver is calculating very small step sizes and is making very little progress. It could also indicate that the problem has been solved to the best numerical accuracy possible given the current scaling.

It can also indicate that a recovery procedure was interrupted due to the user-supplied function lsqgrd being incorrect.

NE_NO_LICENCE

Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library CL Interface for further information.

NE_NUM_DIFFICULTIES

Numerical difficulties encountered and solver was terminated.

This error can be caused by an ill-posed or poorly scaled problem.

NE_PHASE

The problem is already being solved.

The problem is overdetermined. The number of parameters in the problem is greater than the residual count, i.e., $n_{\mathrm{var}}>n_{\mathrm{res}}$.

Unsupported model and method chosen. The specified model in optional parameter $\mathbf{Bxnl\; Model}$ is not supported by the method specified by the optional parameter $\mathbf{Bxnl\; Nlls\; Method}=\mathrm{POWELL-DOGLEG}$.

Unsupported option combinations. The specified combination of values for optional parameters $\mathbf{Bxnl\; Nlls\; Method}$ and $\mathbf{Bxnl\; Glob\; Method}$ is not supported.

NE_REF_MATCH

On entry, $\mathbf{nvar}=⟨\mathit{\text{value}}⟩$, expected $\mathrm{value}=⟨\mathit{\text{value}}⟩$.
Constraint: nvar must match the current number of variables of the model in the handle.

The information supplied does not match with that previously stored.
On entry, $\mathbf{nres}=⟨\mathit{\text{value}}⟩$ must match that given during the definition of the objective in the handle, i.e., $⟨\mathit{\text{value}}⟩$.

There are no decision variables. nvar must be greater than zero.

NE_SETUP_ERROR

This solver does not support the model defined in the handle.

NE_TIME_LIMIT

The solver terminated after the maximum time allowed was exceeded.

Maximum number of seconds exceeded. Use optional parameter $\mathbf{Time\; Limit}$ to reset the limit.

NE_TOO_MANY_ITER

Maximum number of iterations reached.

Use optional parameter $\mathbf{Bxnl\; Iteration\; Limit}$ to reset the limit.

NE_TOO_MANY_MINOR_ITER

Iteration limit reached while solving a subproblem.

Maximum number of iterations reached while trying to solve an auxiliary subproblem.

Line Search failed.

Line Search in the projected gradient direction did not find an acceptable new iterate.

NE_USER_NAN

Invalid number detected in user-supplied function and recovery failed.

Either inform was set to a non-zero value within one of the user-supplied functions, lsqfun, lsqgrd, lsqhes, or lsqhprd, or an Infinity or NaN was detected in values returned from them and the recovery attempt failed.

NE_USER_STOP

User requested termination during a monitoring step. inform was set to a non-zero value within the user-supplied function monit.

7 Accuracy

8 Parallelism and Performance

9 Further Comments

9.1 Description of the Printed Output

• Header
• Optional parameters list (if $\mathbf{Print\; Options}=\mathrm{YES}$)
• Problem statistics
• Iteration log
• Summary
• Solution (if $\mathbf{Print\; Solution}=\mathrm{YES}$)

----------------------------------------------------------------------
 E04GG, Nonlinear least squares method for bound-constrained problems
----------------------------------------------------------------------

Begin of Options
    Bxnl Model                    =        Gauss-newton     * U
    Bxnl Nlls Method              =             Galahad     * d
    Bxnl Glob Method              =                 Reg     * U
    Bxnl Reg Order                =                Auto     * d
    Bxnl Tn Method                =           Min-1-var     * d
    Bxnl Basereg Pow              =         2.00000E+00     * d
    Bxnl Basereg Term             =         1.00000E-02     * d
    Bxnl Iteration Limit          =                1000     * d
    Bxnl Use Second Derivatives   =                 Yes     * U
End of Options

Problem Statistics
  No of variables                  4 (+2 disabled, +0 fixed)
    free (unconstrained)           3
    bounded                        1
  Objective function    LeastSquares
    No of residuals               16 (+8 disabled)

--------------------------------------------
  Iter |   error  |   optim    |  rel optim
--------------------------------------------
      0 3.6953E+01  1.30501E+01  1.51801E+00
      1 5.6464E-01  1.61396E+00  1.51877E+00
      2 7.7340E-02  7.06213E-01  1.79563E+00
      3 2.2354E-02  5.21619E-01  2.46694E+00
      4 7.0758E-03  3.11695E-01  2.62016E+00
      5 2.2392E-03  1.76206E-01  2.63307E+00

---------------------------------------------------------------------------------------------------
  Iter |   error  |   optim    |  rel optim |  Delta  |   rho   |S2IF| inn it|  step | loop|LS| tau
---------------------------------------------------------------------------------------------------
     65 2.2217E-06  3.04184E-05  1.44305E-02  4.00E+02  9.22E-01 SNR-       0 1.3E-03 gamma    1.00
     66 2.2203E-06  1.44158E-05  6.84102E-03  8.00E+02  9.82E-01 SNR-       0 2.6E-03 gamma    1.00
     67 2.2177E-06  1.49444E-06  7.09599E-04  1.60E+03  9.84E-01 SNR-       0 4.9E-03 TR Ok    1.00
     68 2.2132E-06  3.44970E-06  1.63968E-03  3.20E+03  9.71E-01 SNR-       0 9.2E-03 TR Ok    1.00
     69 2.2054E-06  1.01207E-05  4.81891E-03  6.40E+03  9.47E-01 SNR-       0 1.7E-02 TR Ok    1.00

---------------------------------------------------------------------------------------------------
  Iter |   error  |   optim    |  rel optim |  Delta  |   rho   |S2IF| inn it|  step | loop|LS| tau
---------------------------------------------------------------------------------------------------
      0 3.6953E+01  1.30501E+01  1.51801E+00  1.00E+02 -1.00E+00 ----
---------------------------------------------------------------------------------------------------
      0 3.6953E+01  2.00107E+01  2.32768E+00  1.00E+02 -1.00E+00 --I-
      1                                       3.91E-01 -1.35E-01 UNI-
      1 3.2210E+01  2.19541E+01  2.73529E+00  3.91E-01  3.15E-01 SNI-         9.6E-01 TR Ok    1.00
      2                                       1.95E-01 -6.51E-01 UYI-
      2 3.2210E+01  2.19541E+01  2.73529E+00  1.23E-30  1.00E+00 SYI-         9.2E-16 TR Ok    1.00
      3                                       3.08E-31 -2.00E+00 UYI-
      3 3.2210E+01  2.19541E+01  2.73529E+00  6.16E-31  1.00E+00 SYI-         6.5E-16 TR Ok    1.00
---------------------------------------------------------------------------------------------------
      1 1.9040E+01  9.07931E+00  1.47131E+00  1.00E+02  3.77E+00 STRC       4 9.6E-01 TR Ok    1.00
---------------------------------------------------------------------------------------------------
      0 1.9040E+01  1.59354E+01  2.58235E+00  1.00E+02 -1.00E+00 --I-
      1                                       2.50E+01 -4.67E+00 UNI-
      1 1.5631E+01  1.59186E+01  2.84704E+00  3.91E-01  4.09E-01 SNI-         6.4E-01 TR Ok    1.00
      2 1.5631E+01  1.59186E+01  2.84704E+00  9.86E-30  1.00E+00 SYI-         1.9E-15 TR Ok    1.00
      3 1.5631E+01  1.59186E+01  2.84704E+00  6.16E-31  1.00E+00 SYI-         4.8E-16 TR Ok    1.00
---------------------------------------------------------------------------------------------------
      2 8.5311E+00  6.26943E+00  1.51779E+00  1.00E+02  3.08E+00 STRC       9 6.4E-01 TR Ok    1.00

*** Solving the trust region subproblem using More-Sorensen ***
A is symmetric positive definite
iter    nd            sigma         sigma_shift
   0    3.6778E-01    0.0000E+00    0.0000E+00
nq =   7.7000E+02
   1    1.2571E-02    6.4469E-06    6.4469E-06
We're within the trust region radius
Leaving More-Sorensen
Model evaluated successfully: m_k(d) =   2.3089E-08
*** Subproblem solution found ***
Actual reduction (in cost function) =   1.2065E-09
Predicted reduction (in model) =   4.1866E-09
rho returned =   2.8819E-01
Successful step -- Delta staying at  1.2570E-02

 -------------------------------------------------------------------------------
 Status: converged, an optimal solution was found
         small (scaled) projected gradient norm
 -------------------------------------------------------------------------------
 Value of the objective             2.17328E-06
 Norm of projected gradient         1.51989E-08
 Norm of scaled projected gradient  7.29019E-06
 Norm of step                       4.98107E-04
 Iterations                                  80
   Inner iterations                           0
   LS iterations                              0
   PG iterations                              0
 Function evaluations                        81
 Gradient evaluations                        81
 Hessian evaluations (objhes)                 0
 Hessian evaluations (objhprd)                0
   LS function calls                          0
   LS gradient calls                          0
   PG function calls                          0
   PG gradient calls                          0

 Timing
   Total time spent                        2.43 sec
 -------------------------------------------------------------------------------

 Primal variables:
   idx   Lower bound       Value       Upper bound
     1   0.00000E+00    4.58516E-01    1.00000E+00
     2       -inf       3.05448E+00         inf
     3       -inf       4.65146E+00         inf
     4     Disabled         NaN          Disabled
 
 Box bounds dual variables:
   idx   Lower bound       Value       Upper bound       Value
     1   0.00000E+00    0.00000E+00    1.00000E+00    9.52218E-10
     2       -inf       4.66962E-11         inf       0.00000E+00
     3       -inf       6.55098E-11         inf       0.00000E+00
     4     Disabled         NaN          Disabled         NaN

9.2 Residual Weights

10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

11 Algorithmic Details

11.1 Trust Region Algorithm

• BXNL Algorithm
• 1. 1.Initialization
     Set $k=1$, make a feasible initial guess $x_0$ and set trust region radius ${\Delta }_0$.
  2. 2.Iteration k
     1. (i)Stop if $x_k$ is a solution point (see stopping criteria in Section 11.5).
     2. (ii)Solve the trust region subproblem with ${\Delta }_k$ and provide step $s_k$.
     3. (iii)Project the new point $x_k+s_k$ into the bound constraints.
     4. (iv)Evaluate the objective at the new point.
     5. (v)Update the trust region radius ${\Delta }_{k+1}$ based on the ratio ${\rho }_k$.
     6. (vi)If the objective has decreased sufficiently (successful step) choose $x_{k+1}=P_{\Omega }(x_k+s_k)$. Return to 2(i).
     7. (vii)Assess the severity of the projection for the new point using the ratio ${\tau }_k=\frac{\Vert P_{\Omega }(x_k+s_k)-x_k\Vert }{\Vert s_k\Vert }$.
     8. (viii)If ${\tau }_k$ is close to $1$, either return to 2(ii) with ${\Delta }_{k+1}$ or try to perform a line search in the direction $d_k^{LS}=P_{\Omega }(x_k+s_k)-x_k$. If successful set $s_k={\alpha }_k{d}_k^{LS}$ and $x_{k+1}=x_k+s_k$. Return to 2(i).
     9. (ix)Take a projected gradient step by performing a line search in the direction $d_k^{PG}=P_{\Omega }(x_k-\nabla f\left(x_k\right))-x_k$, set $s_k={\alpha }_k{d}_k^{PG}$ and $x_{k+1}=x_k+s_k$.

11.2 Models

11.3 Subproblems

11.3.1 Trust region method

11.3.2 Regularization

11.3.3 Tensor Newton subproblem

11.3.4 Incorporating the regularizer

11.4 Bound Constraints

11.5 Stopping Criteria

11.6 A Note About Lagrangian Multipliers

12 Optional Parameters

• Bxnl Basereg Pow
• Bxnl Basereg Term
• Bxnl Basereg Type
• Bxnl Glob Method
• Bxnl Iteration Limit
• Bxnl Model
• Bxnl Monitor Frequency
• Bxnl Nlls Method
• Bxnl Print Header
• Bxnl Reg Order
• Bxnl Save Covariance Matrix
• Bxnl Stop Abs Tol Fun
• Bxnl Stop Abs Tol Grd
• Bxnl Stop Rel Tol Fun
• Bxnl Stop Rel Tol Grd
• Bxnl Stop Step Tol
• Bxnl Tn Method
• Bxnl Use Second Derivatives
• Bxnl Use Weights
• Defaults
• Infinite Bound Size
• Monitoring File
• Monitoring Level
• Print File
• Print Level
• Print Options
• Print Solution
• Stats Time
• Time Limit

12.1 Description of the Optional Parameters

• the keywords, where the minimum abbreviation of each keyword is underlined;
• a parameter value, where the letters $a$, $i$ and $r$ denote options that take character, integer and real values respectively;
• the default value, where the symbol $\epsilon$ is a generic notation for machine precision (see X02AJC).