NAG Toolbox: nag_opt_lsq_lincon_solve (e04nc)

Purpose

Syntax

Description

References

Parameters

Compulsory Input Parameters

1:     $\mathrm{c}\left(\mathit{ldc},:\right)$ – double array

The first dimension of the array c must be at least $\max \left(1,\mathbf{nclin}\right)$.

The second dimension of the array c must be at least $\mathbf{n}$ if $\mathbf{nclin}>0$, and at least $1$ otherwise.

The $\mathit{i}$th row of c must contain the coefficients of the $\mathit{i}$th general constraint, for $\mathit{i}=1,2,\dots ,\mathbf{nclin}$.

If $\mathbf{nclin}=0$, c is not referenced.

2:     $\mathrm{bl}\left(\mathbf{n}+\mathbf{nclin}\right)$ – double array

3:     $\mathrm{bu}\left(\mathbf{n}+\mathbf{nclin}\right)$ – double array

bl must contain the lower bounds and bu the upper bounds, for all the constraints, in the following order. The first $n$ elements of each array must contain the bounds on the variables, and the next $n_L$ elements must contain the bounds for the general linear constraints (if any). To specify a nonexistent lower bound (i.e., $l_j=-\infty$), set $\mathbf{bl}\left(j\right)\le -\mathit{bigbnd}$, and to specify a nonexistent upper bound (i.e., $u_j=+\infty$), set $\mathbf{bu}\left(j\right)\ge \mathit{bigbnd}$; the default value of $\mathit{bigbnd}$ is ${10}^{20}$, but this may be changed by the optional parameter Infinite Bound Size. To specify the $j$th constraint as an equality, set $\mathbf{bu}\left(j\right)=\mathbf{bl}\left(j\right)=\beta$, say, where $\left|\beta \right|<\mathit{bigbnd}$.

Constraints:

• $\mathbf{bl}\left(\mathit{j}\right)\le \mathbf{bu}\left(\mathit{j}\right)$, for $\mathit{j}=1,2,\dots ,\mathbf{n}+\mathbf{nclin}$;
• if $\mathbf{bl}\left(j\right)=\mathbf{bu}\left(j\right)=\beta$, $\left|\beta \right|<\mathit{bigbnd}$.

4:     $\mathrm{cvec}\left(:\right)$ – double array

The dimension of the array cvec must be at least $\mathbf{n}$ if the problem is of type LP, QP2, QP4, LS2 or LS4, and at least $1$ otherwise

The coefficients of the explicit linear term of the objective function.

If the problem is of type FP, QP1, QP3, LS1 (the default) or LS3, cvec is not referenced.

5:     $\mathrm{istate}\left(\mathbf{n}+\mathbf{nclin}\right)$ – int64int32nag_int array

Need not be set if the (default) optional parameter Cold Start is used.

If the optional parameter Warm Start has been chosen, istate specifies the desired status of the constraints at the start of the feasibility phase. More precisely, the first $n$ elements of istate refer to the upper and lower bounds on the variables, and the next $n_L$ elements refer to the general linear constraints (if any). Possible values for $\mathbf{istate}\left(j\right)$ are as follows:

$\mathbf{istate}\left(j\right)$	Meaning
0	The constraint should not be in the initial working set.
1	The constraint should be in the initial working set at its lower bound.
2	The constraint should be in the initial working set at its upper bound.
3	The constraint should be in the initial working set as an equality. This value must not be specified unless $\mathbf{bl}\left(j\right)=\mathbf{bu}\left(j\right)$.

The values $-2$, $-1$ and $4$ are also acceptable but will be reset to zero by the function. If nag_opt_lsq_lincon_solve (e04nc) has been called previously with the same values of n and nclin, istate already contains satisfactory information. (See also the description of the optional parameter Warm Start.) The function also adjusts (if necessary) the values supplied in x to be consistent with istate.

Constraint: $-2\le \mathbf{istate}\left(\mathit{j}\right)\le 4$, for $\mathit{j}=1,2,\dots ,\mathbf{n}+\mathbf{nclin}$.

6:     $\mathrm{kx}\left(\mathbf{n}\right)$ – int64int32nag_int array

Need not be initialized for problems of type FP, LP, QP1, QP2, LS1 (the default) or LS2.

For problems QP3, QP4, LS3 or LS4, kx must specify the order of the columns of the matrix $A$ with respect to the ordering of x. Thus if column $j$ of $A$ is the column associated with the variable $x_i$ then $\mathbf{kx}\left(j\right)=i$.

Constraints:

• $1\le \mathbf{kx}\left(\mathit{i}\right)\le \mathbf{n}$, for $\mathit{i}=1,2,\dots ,\mathbf{n}$;
• if $i\ne j$, $\mathbf{kx}\left(i\right)\ne \mathbf{kx}\left(j\right)$.

7:     $\mathrm{x}\left(\mathbf{n}\right)$ – double array

An initial estimate of the solution.

Note: that it may be best to avoid the choice $\mathbf{x}=0.0$.

8:     $\mathrm{a}\left(\mathit{lda},:\right)$ – double array

The first dimension of the array a must be at least $\max \left(1,\mathbf{m}\right)$.

The second dimension of the array a must be at least $\mathbf{n}$ if the problem is of type QP1, QP2, QP3, QP4, LS1 (the default), LS2, LS3 or LS4, and at least $1$ otherwise.

The array a must contain the matrix $A$ as specified in Table 1 (see Description).

If the problem is of type QP1 or QP2, the first $m$ rows and columns of a must contain the leading $m$ by $m$ rows and columns of the symmetric Hessian matrix. Only the diagonal and upper triangular elements of the leading $m$ rows and columns of a are referenced. The remaining elements are assumed to be zero and need not be assigned.

For problems QP3, QP4, LS3 or LS4, the first $m$ rows of a must contain an $m$ by $n$ upper trapezoidal factor of either the Hessian matrix or the least squares matrix, ordered according to the kx array. The factor need not be of full rank, i.e., some of the diagonals may be zero. However, as a general rule, the larger the dimension of the leading nonsingular sub-matrix of $A$, the fewer iterations will be required. Elements outside the upper triangular part of the first $m$ rows of a are assumed to be zero and need not be assigned.

If a constrained least squares problem contains a very large number of observations, storage limitations may prevent storage of the entire least squares matrix. In such cases, you should transform the original $A$ into a triangular matrix before the call to nag_opt_lsq_lincon_solve (e04nc) and solve the problem as type LS3 or LS4.

9:     $\mathrm{b}\left(:\right)$ – double array

The dimension of the array b must be at least $\mathbf{m}$ if the problem is of type LS1 (the default), LS2, LS3 or LS4, and at least $1$ otherwise

The $m$ elements of the vector of observations.

10:   $\mathrm{lwsav}\left(120\right)$ – logical array

11:   $\mathrm{iwsav}\left(610\right)$ – int64int32nag_int array

12:   $\mathrm{rwsav}\left(475\right)$ – double array

The arrays lwsav, iwsav and rwsav must not be altered between calls to any of the functions nag_opt_lsq_lincon_solve (e04nc), nag_opt_lsq_lincon_option_string (e04ne).

Optional Input Parameters

1:     $\mathrm{m}$ – int64int32nag_int scalar

Default: the dimension of the array a.

$m$, the number of rows in the matrix $A$. If the problem is specified as type FP or LP, m is not referenced and is assumed to be zero.

If the problem is of type QP, m will usually be $n$, the number of variables. However, a value of m less than $n$ is appropriate for QP3 or QP4 if $A$ is an upper trapezoidal matrix with $m$ rows. Similarly, m may be used to define the dimension of a leading block of nonzeros in the Hessian matrices of QP1 or QP2, in which case the last $\left(n-m\right)$ rows and columns of a are assumed to be zero. In the QP case, $m$ should not be greater than $n$; if it is, the last $\left(m-n\right)$ rows of $A$ are ignored.

If the problem is of type LS1 (the default) or specified as type LS2, LS3 or LS4, m is also the dimension of the array b. Note that all possibilities ($m<n$, $m=n$ and $m>n$) are allowed in this case.

Constraint: $\mathbf{m}>0$ if the problem is not of type FP or LP.

2:     $\mathrm{n}$ – int64int32nag_int scalar

Default: the dimension of the arrays kx, x. (An error is raised if these dimensions are not equal.)

$n$, the number of variables.

Constraint: $\mathbf{n}>0$.

3:     $\mathrm{nclin}$ – int64int32nag_int scalar

Default: the dimension of the array c.

$n_L$, the number of general linear constraints.

Constraint: $\mathbf{nclin}\ge 0$.

Output Parameters

1:     $\mathrm{istate}\left(\mathbf{n}+\mathbf{nclin}\right)$ – int64int32nag_int array

The status of the constraints in the working set at the point returned in x. The significance of each possible value of $\mathbf{istate}\left(j\right)$ is as follows:

$\mathbf{istate}\left(j\right)$	Meaning
$-2$	The constraint violates its lower bound by more than the feasibility tolerance.
$-1$	The constraint violates its upper bound by more than the feasibility tolerance.
$\phantom{-}0$	The constraint is satisfied to within the feasibility tolerance, but is not in the working set.
$\phantom{-}1$	This inequality constraint is included in the working set at its lower bound.
$\phantom{-}2$	This inequality constraint is included in the working set at its upper bound.
$\phantom{-}3$	The constraint is included in the working set as an equality. This value of istate can occur only when $\mathbf{bl}\left(j\right)=\mathbf{bu}\left(j\right)$.
$\phantom{-}4$	This corresponds to optimality being declared with $\mathbf{x}\left(j\right)$ being temporarily fixed at its current value.

2:     $\mathrm{kx}\left(\mathbf{n}\right)$ – int64int32nag_int array

Defines the order of the columns of a with respect to the ordering of x, as described above.

3:     $\mathrm{x}\left(\mathbf{n}\right)$ – double array

The point at which nag_opt_lsq_lincon_solve (e04nc) terminated. If $\mathbf{ifail}=\mathbf{0}$, $\mathbf{1}$ or $\mathbf{4}$, x contains an estimate of the solution.

4:     $\mathrm{a}\left(\mathit{lda},:\right)$ – double array

The first dimension of the array a will be $\max \left(1,\mathbf{m}\right)$.

The second dimension of the array a will be $\mathbf{n}$ if the problem is of type QP1, QP2, QP3, QP4, LS1 (the default), LS2, LS3 or LS4 and $1$ otherwise.

If $\mathbf{Hessian}=\mathrm{NO}$ and the problem is of type LS or QP, a contains the upper triangular Cholesky factor $R$ of (8) (see Main Iteration), with columns ordered as indicated by kx. If $\mathbf{Hessian}=\mathrm{YES}$ and the problem is of type LS or QP, a contains the upper triangular Cholesky factor $R$ of the Hessian matrix $H$, with columns ordered as indicated by kx. In either case $R$ may be used to obtain the variance-covariance matrix or to recover the upper triangular factor of the original least squares matrix.

If the problem is of type FP or LP, a is not referenced.

5:     $\mathrm{b}\left(:\right)$ – double array

The dimension of the array b will be $\mathbf{m}$ if the problem is of type LS1 (the default), LS2, LS3 or LS4 and $1$ otherwise

The transformed residual vector of equation (10) (see Main Iteration).

If the problem is of type FP, LP, QP1, QP2, QP3 or QP4, b is not referenced.

6:     $\mathrm{iter}$ – int64int32nag_int scalar

The total number of iterations performed.

7:     $\mathrm{obj}$ – double scalar

The value of the objective function at $x$ if $x$ is feasible, or the sum of infeasibiliites at $x$ otherwise. If the problem is of type FP and $x$ is feasible, obj is set to zero.

8:     $\mathrm{clamda}\left(\mathbf{n}+\mathbf{nclin}\right)$ – double array

The values of the Lagrange multipliers for each constraint with respect to the current working set. The first $n$ elements contain the multipliers for the bound constraints on the variables, and the next $n_L$ elements contain the multipliers for the general linear constraints (if any). If $\mathbf{istate}\left(j\right)=0$ (i.e., constraint $j$ is not in the working set), $\mathbf{clamda}\left(j\right)$ is zero. If $x$ is optimal, $\mathbf{clamda}\left(j\right)$ should be non-negative if $\mathbf{istate}\left(j\right)=1$, non-positive if $\mathbf{istate}\left(j\right)=2$ and zero if $\mathbf{istate}\left(j\right)=4$.

9:     $\mathrm{lwsav}\left(120\right)$ – logical array

10:   $\mathrm{iwsav}\left(610\right)$ – int64int32nag_int array

11:   $\mathrm{rwsav}\left(475\right)$ – double array

12:   $\mathrm{ifail}$ – int64int32nag_int scalar

$\mathbf{ifail}=\mathbf{0}$ unless the function detects an error (see Error Indicators and Warnings).

nag_opt_lsq_lincon_solve (e04nc) returns with $\mathbf{ifail}=\mathbf{0}$ if $x$ is a strong local minimizer, i.e., the projected gradient (Norm Gz; see Printed output) is negligible, the Lagrange multipliers (Lagr Mult; see Definition of Search Direction) are optimal and $R_Z$ (see Main Iteration) is nonsingular.

Error Indicators and Warnings

Cases prefixed with W are classified as warnings and do not generate an error of type NAG:error_n. See nag_issue_warnings.

W  $\mathbf{ifail}=1$

x is a weak local minimum, (i.e., the projected gradient is negligible, the Lagrange multipliers are optimal, but either $R_Z$ (see Main Iteration) is singular, or there is a small multiplier). This means that $x$ is not unique.

W  $\mathbf{ifail}=2$

The solution appears to be unbounded. This value of ifail implies that a step as large as Infinite Bound Size ($\text{default value}={10}^{20}$) would have to be taken in order to continue the algorithm. This situation can occur only when $A$ is singular, there is an explicit linear term, and at least one variable has no upper or lower bound.

W  $\mathbf{ifail}=3$

No feasible point was found, i.e., it was not possible to satisfy all the constraints to within the feasibility tolerance. In this case, the constraint violations at the final $x$ will reveal a value of the tolerance for which a feasible point will exist – for example, when the feasibility tolerance for each violated constraint exceeds its Slack (see Printed output) at the final point. The modified problem (with an altered feasibility tolerance) may then be solved using a Warm Start. You should check that there are no constraint redundancies. If the data for the constraints are accurate only to the absolute precision $\sigma$, you should ensure that the value of the optional parameter Feasibility Tolerance ($\text{default value}=\sqrt{\epsilon }$, where $\epsilon$ is the machine precision) is greater than $\sigma$. For example, if all elements of $\mathbf{c}$ are of order unity and are accurate only to three decimal places, the Feasibility Tolerance should be at least ${10}^{-3}$.

   $\mathbf{ifail}=4$

The limiting number of iterations (determined by the optional parameters Feasibility Phase Iteration Limit ($\text{default value}=\max \left(50,5\left(n+n_L\right)\right)$) and Optimality Phase Iteration Limit ($\text{default value}=\max \left(50,5\left(n+n_L\right)\right)$)) was reached before normal termination occurred. If the method appears to be making progress (e.g., the objective function is being satisfactorily reduced), either increase the iterations limit and rerun nag_opt_lsq_lincon_solve (e04nc) or, alternatively, rerun nag_opt_lsq_lincon_solve (e04nc) using the Warm Start facility to specify the initial working set. If the iteration limit is already large, but some of the constraints could be nearly linearly dependent, check the monitoring information (see Description of Monitoring Information) for a repeated pattern of constraints entering and leaving the working set. (Near-dependencies are often indicated by wide variations in size in the diagonal elements of the matrix $T$ (see Definition of Search Direction), which will be printed if $\mathbf{Print\; Level}\ge 30$ ($\text{default value}=10$). In this case, the algorithm could be cycling (see the comments for $\mathbf{ifail}=\mathbf{5}$).

   $\mathbf{ifail}=5$

The algorithm could be cycling, since a total of $50$ changes were made to the working set without altering $x$. You should check the monitoring information (see Description of Monitoring Information) for a repeated pattern of constraint deletions and additions.

If a sequence of constraint changes is being repeated, the iterates are probably cycling. (nag_opt_lsq_lincon_solve (e04nc) does not contain a method that is guaranteed to avoid cycling; such a method would be combinatorial in nature.) Cycling may occur in two circumstances: at a constrained stationary point where there are some small or zero Lagrange multipliers; or at a point (usually a vertex) where the constraints that are satisfied exactly are nearly linearly dependent. In the latter case, you have the option of identifying the offending dependent constraints and removing them from the problem, or restarting the run with a larger value of the optional parameter Feasibility Tolerance ($\text{default value}=\sqrt{\epsilon }$, where $\epsilon$ is the machine precision). If nag_opt_lsq_lincon_solve (e04nc) terminates with $\mathbf{ifail}=\mathbf{5}$, but no suspicious pattern of constraint changes can be observed, it may be worthwhile to restart with the final $x$ (with or without the Warm Start option).

Note:  that this error exit may also occur if a poor starting point x is supplied (for example, $\mathbf{x}=0.0$). You are advised to try a non-zero starting point.

   $\mathbf{ifail}=6$

An input argument is invalid.

   $\mathbf{ifail}=7$

The problem to be solved is of type QP1 or QP2, but the Hessian matrix supplied in a is not positive semidefinite.

   $\mathbf{\text{Overflow}}$

If the printed output before the overflow error contains a warning about serious ill-conditioning in the working set when adding the $j$th constraint, it may be possible to avoid the difficulty by increasing the magnitude of the Feasibility Tolerance ($\text{default value}=\sqrt{\epsilon }$, where $\epsilon$ is the machine precision) and rerunning the program. If the message recurs even after this change, the offending linearly dependent constraint (with index ‘$j$’) must be removed from the problem.

   $\mathbf{ifail}=-99$

An unexpected error has been triggered by this routine. Please contact NAG.

   $\mathbf{ifail}=-399$

Your licence key may have expired or may not have been installed correctly.

   $\mathbf{ifail}=-999$

Dynamic memory allocation failed.

Accuracy

Further Comments

Scaling

Description of the Printed Output

Example

Open in the MATLAB editor: e04nc_example

function e04nc_example


fprintf('e04nc example results\n\n');

% Problem specification
n     = 9;
m     = 10;
nclin = 3;
c  = [1,    1,    1,       1,     1,    1,       1,       1,      4;
      1,    2,    3,       4,    -2,    1,       1,       1,      1;
      1,   -1,    1,      -1,     1,    1,       1,       1,      1];
x  = [1;    0.5;  0.3333;  0.25;  0.2;  0.1667;  0.1428;  0.125;  0.1111];
a  = [1,    1,    1,       1,     1,    1,       1,       1,      1;
      1,    2,    1,       1,     1,    1,       2,       0,      0;
      1,    1,    3,       1,     1,    1,      -1,      -1,     -3;
      1,    1,    1,       4,     1,    1,       1,       1,      1;
      1,    1,    1,       3,     1,    1,       1,       1,      1;
      1,    1,    2,       1,     1,    0,       0,       0,     -1;
      1,    1,    1,       1,     0,    1,       1,       1,      1;
      1,    1,    1,       0,     1,    1,       1,       1,      1;
      1,    1,    0,       1,     1,    1,       2,       2,      3;
      1,    0,    1,       1,     1,    1,       0,       2,      2 ];

b(1:m)  =  1;
bl(1:n) =  0;
bl(3)   = -1e25;
bu(1:n) =  2;
bl(n+1:n+nclin) = [        2; -1e25;    1];
bu(n+1:n+nclin) = [     1e25;     2;    4];

cvec   = [0];
istate = zeros(12, 1, 'int64');
kx     = zeros(9, 1, 'int64');

% Initialize and set no printing options
[cwsav,lwsav,iwsav,rwsav,ifail] = e04wb('e04nc');
[lwsav,iwsav,rwsav,inform] = e04ne(...
                                   'NoList', lwsav, iwsav, rwsav);
[lwsav,iwsav,rwsav,inform] = e04ne(...
                                   'Print Level = 0', lwsav, iwsav, rwsav);

% Solve
[istate, kx, x, a, b, iter, obj, clamda, lwsav, iwsav, rwsav, ifail] = ...
    e04nc(...
          c, bl, bu, cvec, istate, kx, x, a, b, lwsav, iwsav, rwsav);

fprintf('Minimum value     :  %9.4f\n\n',obj);
fprintf('Found after %3d iterations at x:\n',iter);
fprintf('    %9.4f %9.4f %9.4f\n',x);

e04nc example results

Minimum value     :     0.0813

Found after  15 iterations at x:
       0.0000    0.0415    0.5872
       0.0000    0.0996    0.0000
       0.0491    0.0000    0.3056

Algorithmic Details

Overview

Definition of Search Direction

Main Iteration

Optional Parameters

• Cold Start
• Crash Tolerance
• Defaults
• Feasibility Phase Iteration Limit
• Feasibility Tolerance
• Hessian
• Infinite Bound Size
• Infinite Step Size
• Iteration Limit
• Iters
• Itns
• List
• Monitoring File
• Nolist
• Optimality Phase Iteration Limit
• Print Level
• Problem Type
• Rank Tolerance
• Warm Start

[lwsav, iwsav, rwsav, inform] = e04ne('Print Level = 1', lwsav, iwsav, rwsav);

Description of the Optional Parameters

• the keywords, where the minimum abbreviation of each keyword is underlined (if no characters of an optional qualifier are underlined, the qualifier may be omitted);
• a parameter value, where the letters $a$, $i$ and $r$ denote options that take character, integer and real values respectively;
• the default value, where the symbol $\epsilon$ is a generic notation for machine precision (see nag_machine_precision (x02aj)).

Description of Monitoring Information