# -*- coding: utf-8 -*-
r"""
Module Summary
--------------
Interfaces for the NAG Mark 30.3 `opt` Chapter.
``opt`` - Minimizing or Maximizing a Function
This module provides functions for solving various mathematical optimization problems by solvers based on local stopping criteria.
The main classes of problems covered in this module are:
**Linear Programming (LP)** -- dense and sparse;
**Quadratic Programming (QP)** -- convex and nonconvex, dense and sparse;
**Quadratically Constrained Quadratic Programming (QCQP)** -- convex and nonconvex;
**Nonlinear Programming (NLP)** -- dense and sparse, based on active-set SQP methods or interior point methods (IPM);
**Second-order Cone Programming (SOCP)**;
**Semidefinite Programming (SDP)** -- both linear matrix inequalities (LMI) and bilinear matrix inequalities (BMI);
**Derivative-free Optimization (DFO)**;
**Least Squares (LSQ)** -- linear and nonlinear, constrained and unconstrained;
**General Nonlinear Data Fitting (NLDF)** -- nonlinear loss functions with regularization, constrained and unconstrained.
For a full overview of the functionality offered in this module, see `the functionality index <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#index>`__ or the Module Contents (submodule ``opt``).
See also other modules in the NAG Library relevant to optimization:
submodule :mod:`~naginterfaces.library.glopt` contains functions to solve **global optimization** problems;
submodule :mod:`~naginterfaces.library.mip` addresses problems arising in **operational research** and focuses on **Mixed Integer Programming (MIP)**;
submodule :mod:`~naginterfaces.library.lapacklin` and submodule :mod:`~naginterfaces.library.lapackeig` include functions for linear algebra and in particular unconstrained linear least squares;
submodule :mod:`~naginterfaces.library.fit` focuses on curve and surface fitting, in which linear data fitting in :math:`l_1` or :math:`l_{\infty }` norm might be of interest;
submodule :mod:`~naginterfaces.library.correg` offers several regression (data fitting) functions, including linear, nonlinear and quantile regression, LARS, LASSO and others.
See Also
--------
``naginterfaces.library.examples.opt`` :
This subpackage contains examples for the ``opt`` module.
See also the :ref:`library_opt_ex` subsection.
Functionality Index
-------------------
**Linear programming (LP)**
dense
active-set method/primal simplex
alternative 1: :meth:`lp_solve`
alternative 2: :meth:`lsq_lincon_solve`
sparse
interior point method (IPM): :meth:`handle_solve_lp_ipm`
simplex: :meth:`handle_solve_lp_simplex`
active-set method/primal simplex
recommended (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#variant>`__): :meth:`qpconvex2_sparse_solve`
alternative: :meth:`qpconvex1_sparse_solve`
**Quadratic programming (QP)**
dense
active-set method for (possibly nonconvex) QP problem: :meth:`qp_dense_solve`
active-set method for convex QP problem: :meth:`lsq_lincon_solve`
sparse
active-set method sparse convex QP problem
recommended (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#variant>`__): :meth:`qpconvex2_sparse_solve`
alternative: :meth:`qpconvex1_sparse_solve`
interior point method (IPM) for (possibly nonconvex) QP problems: :meth:`handle_solve_ipopt`
**Second-order Cone Programming (SOCP)**
dense or sparse
interior point method: :meth:`handle_solve_socp_ipm`
**Semidefinite programming (SDP)**
generalized augmented Lagrangian method for SDP and SDP with bilinear matrix inequalities (BMI-SDP): :meth:`handle_solve_pennon`
**Nonlinear programming (NLP)**
dense
active-set sequential quadratic programming (SQP)
direct communication
recommended (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#variant>`__): :meth:`nlp1_solve`
alternative: :meth:`nlp2_solve`
reverse communication: :meth:`nlp1_rcomm`
sparse
active-set sequential quadratic programming (SQP): :meth:`handle_solve_ssqp`
interior point method (IPM): :meth:`handle_solve_ipopt`
active-set sequential quadratic programming (SQP)
alternative: :meth:`nlp2_sparse_solve`
alternative: :meth:`nlp1_sparse_solve`
**Nonlinear programming (NLP) -- derivative-free optimization (DFO)**
model-based method for bound-constrained optimization: :meth:`bounds_bobyqa_func`
Nelder--Mead simplex method for unconstrained optimization: :meth:`uncon_simplex`
**Nonlinear programming (NLP) -- special cases**
unidimensional optimization (one-dimensional) with bound constraints
method based on quadratic interpolation, no derivatives: :meth:`one_var_func`
method based on cubic interpolation: :meth:`one_var_deriv`
unconstrained
preconditioned conjugate gradient method: :meth:`uncon_conjgrd_comp`
bound-constrained
first order active-set method (nonlinear conjugate gradient): :meth:`handle_solve_bounds_foas`
quasi-Newton algorithm, no derivatives: :meth:`bounds_quasi_func_easy`
quasi-Newton algorithm, first derivatives: :meth:`bounds_quasi_deriv_easy`
modified Newton algorithm, first derivatives: :meth:`bounds_mod_deriv_comp`
modified Newton algorithm, first derivatives, easy-to-use: :meth:`bounds_mod_deriv_easy`
modified Newton algorithm, first and second derivatives: :meth:`bounds_mod_deriv2_comp`
modified Newton algorithm, first and second derivatives, easy-to-use: :meth:`bounds_mod_deriv2_easy`
**Linear least squares, linear regression, data fitting**
constrained
bound-constrained least squares problem: :meth:`bnd_lin_lsq`
linearly-constrained active-set method: :meth:`lsq_lincon_solve`
**Data fitting**
general loss functions (for sum of squares, see nonlinear least squares): :meth:`handle_solve_nldf`
**Nonlinear least squares, data fitting**
unconstrained
combined Gauss--Newton and modified Newton algorithm
no derivatives: :meth:`lsq_uncon_mod_func_comp`
no derivatives, easy-to-use: :meth:`lsq_uncon_mod_func_easy`
first derivatives: :meth:`lsq_uncon_mod_deriv_comp`
first derivatives, easy-to-use: :meth:`lsq_uncon_mod_deriv_easy`
first and second derivatives: :meth:`lsq_uncon_mod_deriv2_comp`
first and second derivatives, easy-to-use: :meth:`lsq_uncon_mod_deriv2_easy`
combined Gauss--Newton and quasi-Newton algorithm
first derivatives: :meth:`lsq_uncon_quasi_deriv_comp`
first derivatives, easy-to-use: :meth:`lsq_uncon_quasi_deriv_easy`
covariance matrix for nonlinear least squares problem (unconstrained): :meth:`lsq_uncon_covariance`
bound constrained
model-based derivative-free algorithm
direct communication: :meth:`handle_solve_dfls`
reverse communication: :meth:`handle_solve_dfls_rcomm`
trust region algorithm
first derivatives, optionally second derivatives: :meth:`handle_solve_bxnl`
generic, including nonlinearly constrained
nonlinear constraints active-set sequential quadratic programming (SQP): :meth:`lsq_gencon_deriv`
**NAG optimization modelling suite**
initialization of a handle for the suite
initialization as an empty problem: :meth:`handle_init`
read a problem from a file to a handle: :meth:`handle_read_file`
problem definition
define a linear objective function: :meth:`handle_set_linobj`
define a linear or a quadratic objective function: :meth:`handle_set_quadobj`
define nonlinear residual functions: :meth:`handle_set_nlnls`
define a nonlinear objective function: :meth:`handle_set_nlnobj`
define a second-order cone: :meth:`handle_set_group`
define bounds of variables: :meth:`handle_set_simplebounds`
define a block of linear constraints: :meth:`handle_set_linconstr`
define a block of nonlinear constraints: :meth:`handle_set_nlnconstr`
define a structure of Hessian of the objective, constraints or the Lagrangian: :meth:`handle_set_nlnhess`
add one or more linear matrix inequality constraints: :meth:`handle_set_linmatineq`
define bilinear matrix terms: :meth:`handle_set_quadmatineq`
factor of quadratic coefficient matrix: :meth:`handle_set_qconstr_fac`
full quadratic coefficient matrix: :meth:`handle_set_qconstr`
set variable properties (e.g., integrality): :meth:`handle_set_property`
problem editing
define new variables: :meth:`handle_add_vars`
disable (temporarily remove) components of the model: :meth:`handle_disable`
enable (bring back) previously disabled components of the model: :meth:`handle_enable`
modify a single coefficient in a linear constraint: :meth:`handle_set_linconstr_coeff`
modify a single coefficient in the linear objective function: :meth:`handle_set_linobj_coeff`
modify bounds of an existing constraint or variable: :meth:`handle_set_bound`
solvers
interior point method (IPM) for linear programming (LP): :meth:`handle_solve_lp_ipm`
first order active-set method (nonlinear conjugate gradient): :meth:`handle_solve_bounds_foas`
active-set sequential quadratic programming method (SQP) for nonlinear programming (NLP): :meth:`handle_solve_ssqp`
interior point method (IPM) for nonlinear programming (NLP): :meth:`handle_solve_ipopt`
generalized augmented Lagrangian method for SDP and SDP with bilinear matrix inequalities (BMI-SDP): :meth:`handle_solve_pennon`
interior point method (IPM) for Second-order Cone programming (SOCP): :meth:`handle_solve_socp_ipm`
constrained nonlinear data fitting (NLDF): :meth:`handle_solve_nldf`
derivative-free optimisation (DFO) for nonlinear least squares problems
direct communication: :meth:`handle_solve_dfls`
reverse communication: :meth:`handle_solve_dfls_rcomm`
trust region optimisation for nonlinear least squares problems (BXNL): :meth:`handle_solve_bxnl`
model-based method for bound-constrained optimization
direct communication: :meth:`handle_solve_dfno`
reverse communication: :meth:`handle_solve_dfno_rcomm`
simplex for linear programming (LP): :meth:`handle_solve_lp_simplex`
deallocation
destroy the problem handle: :meth:`handle_free`
service routines
print information about a problem handle: :meth:`handle_print`
set/get information in a problem handle: :meth:`handle_set_get_real`
set/get integer information in a problem handle: :meth:`handle_set_get_integer`
supply option values from a character string: :meth:`handle_opt_set`
get the setting of option: :meth:`handle_opt_get`
supply option values from external file: :meth:`handle_opt_set_file`
**Service functions**
input and output (I/O)
read MPS data file defining LP, QP, MILP or MIQP problem: :meth:`miqp_mps_read`
write MPS data file defining LP, QP, MILP or MIQP problem: :meth:`miqp_mps_write`
read sparse SPDA data files for linear SDP problems: :meth:`sdp_read_sdpa`
read MPS data file defining LP or QP problem (deprecated): :meth:`qpconvex1_sparse_mps`
read a problem from a file to a handle: :meth:`handle_read_file`
derivative check and approximation
check user's function for calculating first derivatives of function: :meth:`check_deriv`
check user's function for calculating second derivatives of function: :meth:`check_deriv2`
check user's function for calculating Jacobian of first derivatives: :meth:`lsq_check_deriv`
check user's function for calculating Hessian of a sum of squares: :meth:`lsq_check_hessian`
estimate (using numerical differentiation) gradient and/or Hessian of a function: :meth:`estimate_deriv`
determine the pattern of nonzeros in the Jacobian matrix for :meth:`nlp2_sparse_solve`: :meth:`nlp2_sparse_jacobian`
covariance matrix for nonlinear least squares problem (unconstrained): :meth:`lsq_uncon_covariance`
option setting functions
NAG optimization modelling suite
supply option values from a character string: :meth:`handle_opt_set`
get the setting of option: :meth:`handle_opt_get`
supply option values from external file: :meth:`handle_opt_set_file`
:meth:`uncon_conjgrd_comp`
supply option values from external file: :meth:`uncon_conjgrd_option_file`
supply option values from a character string: :meth:`uncon_conjgrd_option_string`
:meth:`lp_solve`
supply option values from external file: :meth:`lp_option_file`
supply option values from a character string: :meth:`lp_option_string`
:meth:`lsq_lincon_solve`
supply option values from external file: :meth:`lsq_lincon_option_file`
supply option values from a character string: :meth:`lsq_lincon_option_string`
:meth:`qp_dense_solve`
supply option values from external file: :meth:`qp_dense_option_file`
supply option values from a character string: :meth:`qp_dense_option_string`
:meth:`qpconvex1_sparse_solve`
supply option values from external file: :meth:`qpconvex1_sparse_option_file`
supply option values from a character string: :meth:`qpconvex1_sparse_option_string`
:meth:`qpconvex2_sparse_solve`
initialization function: :meth:`qpconvex2_sparse_init`
supply option values from external file: :meth:`qpconvex2_sparse_option_file`
set a single option from a character string: :meth:`qpconvex2_sparse_option_string`
set a single option from an integer argument: :meth:`qpconvex2_sparse_option_integer_set`
set a single option from a real argument: :meth:`qpconvex2_sparse_option_double_set`
get the setting of an integer valued option: :meth:`qpconvex2_sparse_option_integer_get`
get the setting of a real valued option: :meth:`qpconvex2_sparse_option_double_get`
:meth:`nlp1_solve` and :meth:`nlp1_rcomm`
initialization function for :meth:`nlp1_solve` and :meth:`nlp1_rcomm`: :meth:`nlp1_init`
supply option values from external file: :meth:`nlp1_option_file`
supply option values from a character string: :meth:`nlp1_option_string`
:meth:`nlp1_sparse_solve`
supply option values from external file: :meth:`nlp1_sparse_option_file`
supply option values from a character string: :meth:`nlp1_sparse_option_string`
:meth:`lsq_gencon_deriv`
supply option values from external file: :meth:`lsq_gencon_deriv_option_file`
supply option values from a character string: :meth:`lsq_gencon_deriv_option_string`
:meth:`nlp2_sparse_solve`
initialization function: :meth:`nlp2_sparse_init`
supply option values from external file: :meth:`nlp2_sparse_option_file`
set a single option from a character string: :meth:`nlp2_sparse_option_string`
set a single option from an integer argument: :meth:`nlp2_sparse_option_integer_set`
set a single option from a real argument: :meth:`nlp2_sparse_option_double_set`
get the setting of an integer valued option: :meth:`nlp2_sparse_option_integer_get`
get the setting of a real valued option: :meth:`nlp2_sparse_option_double_get`
:meth:`nlp2_solve`
initialization function: :meth:`nlp2_init`
supply option values from external file: :meth:`nlp2_option_file`
set a single option from a character string: :meth:`nlp2_option_string`
set a single option from an integer argument: :meth:`nlp2_option_integer_set`
set a single option from a real argument: :meth:`nlp2_option_double_set`
get the setting of an integer valued option: :meth:`nlp2_option_integer_get`
get the setting of a real valued option: :meth:`nlp2_option_double_get`
For full information please refer to the NAG Library document
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html
"""
# NAG Copyright 2017-2024.
[docs]def one_var_func(funct, e1, e2, a, b, maxcal, data=None):
r"""
``one_var_func`` searches for a minimum, in a given finite interval, of a continuous function of a single variable, using function values only.
The method (based on quadratic interpolation) is intended for functions which have a continuous first derivative (although it will usually work if the derivative has occasional discontinuities).
.. _e04ab-py2-py-doc:
For full information please refer to the NAG Library document for e04ab
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04abf.html
.. _e04ab-py2-py-parameters:
**Parameters**
**funct** : callable fc = funct(xc, data=None)
You must supply this function to calculate the value of the function :math:`F\left(x\right)` at any point :math:`x` in :math:`\left[a, b\right]`.
It should be tested separately before being used in conjunction with ``one_var_func``.
**Parameters**
**xc** : float
The point :math:`x` at which the value of :math:`F` is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Must be set to the value of the function :math:`F` at the current point :math:`x`.
**e1** : float
The relative accuracy to which the position of a minimum is required. (Note that, since :math:`\mathrm{e1}` is a relative tolerance, the scaling of :math:`x` is automatically taken into account.)
:math:`\mathrm{e1}` should be no smaller than :math:`2\epsilon`, and preferably not much less than :math:`\sqrt{\epsilon }`, where :math:`\epsilon` is the machine precision.
**e2** : float
The absolute accuracy to which the position of a minimum is required. :math:`\mathrm{e2}` should be no smaller than :math:`2\epsilon`.
**a** : float
The lower bound :math:`a` of the interval containing a minimum.
**b** : float
The upper bound :math:`b` of the interval containing a minimum.
**maxcal** : int
The maximum number of calls of :math:`F\left(x\right)` to be allowed.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**e1** : float
If you set :math:`\mathrm{e1}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e1}` will be reset to the default value :math:`\sqrt{\epsilon }` before starting the minimization process.
**e2** : float
If you set :math:`\mathrm{e2}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e2}` will be reset to the default value :math:`\sqrt{\epsilon }`.
**a** : float
An improved lower bound on the position of the minimum.
**b** : float
An improved upper bound on the position of the minimum.
**maxcal** : int
The total number of times that :math:`\mathrm{funct}` was actually called.
**x** : float
The estimated position of the minimum.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
.. _e04ab-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 3`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{a} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{e2} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{b} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{a}+\mathrm{e2} < \mathrm{b}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
The maximum number of function calls, :math:`\langle\mathit{\boldsymbol{value}}\rangle`, have been performed. This may have happened simply because :math:`\mathrm{maxcal}` was set too small for the particular problem, or may be due to a mistake in the user-supplied function :math:`\mathrm{funct}`. If no mistake can be found in :math:`\mathrm{funct}`, restart ``one_var_func`` (preferably with the values of :math:`\mathrm{a}` and :math:`\mathrm{b}` given on exit from the previous call to ``one_var_func``).
.. _e04ab-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``one_var_func`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right)\quad \text{ subject to }\quad a\leq x\leq b\text{.}
It normally computes a sequence of :math:`x` values which tend in the limit to a minimum of :math:`F\left(x\right)` subject to the given bounds.
It also progressively reduces the interval :math:`\left[a, b\right]` in which the minimum is known to lie.
It uses the safeguarded quadratic-interpolation method described in Gill and Murray (1973).
You must supply a :math:`\mathrm{funct}` to evaluate :math:`F\left(x\right)`.
The arguments :math:`\mathrm{e1}` and :math:`\mathrm{e2}` together specify the accuracy
.. math::
\textit{Tol}\left(x\right) = \mathrm{e1}\times \left\lvert x\right\rvert +\mathrm{e2}
to which the position of the minimum is required.
Note that :math:`\mathrm{funct}` is never called at any point which is closer than :math:`\textit{Tol}\left(x\right)` to a previous point.
If the original interval :math:`\left[a, b\right]` contains more than one minimum, ``one_var_func`` will normally find one of the minima.
.. _e04ab-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
"""
raise NotImplementedError
[docs]def one_var_deriv(funct, e1, e2, a, b, maxcal, data=None):
r"""
``one_var_deriv`` searches for a minimum, in a given finite interval, of a continuous function of a single variable, using function and first derivative values.
The method (based on cubic interpolation) is intended for functions which have a continuous first derivative (although it will usually work if the derivative has occasional discontinuities).
.. _e04bb-py2-py-doc:
For full information please refer to the NAG Library document for e04bb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04bbf.html
.. _e04bb-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(xc, data=None)
You must supply this function to calculate the values of :math:`F\left(x\right)` and :math:`\frac{{dF}}{{dx}}` at any point :math:`x` in :math:`\left[a, b\right]`.
It should be tested separately before being used in conjunction with ``one_var_deriv``.
**Parameters**
**xc** : float
The point :math:`x` at which the values of :math:`F` and :math:`\frac{{dF}}{{dx}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Must be set to the value of the function :math:`F` at the current point :math:`x`.
**gc** : float
Must be set to the value of the first derivative :math:`\frac{{dF}}{{dx}}` at the current point :math:`x`.
**e1** : float
The relative accuracy to which the position of a minimum is required. (Note that, since :math:`\mathrm{e1}` is a relative tolerance, the scaling of :math:`x` is automatically taken into account.)
:math:`\mathrm{e1}` should be no smaller than :math:`2\epsilon`, and preferably not much less than :math:`\sqrt{\epsilon }`, where :math:`\epsilon` is the machine precision.
**e2** : float
The absolute accuracy to which the position of a minimum is required. :math:`\mathrm{e2}` should be no smaller than :math:`2\epsilon`.
**a** : float
The lower bound :math:`a` of the interval containing a minimum.
**b** : float
The upper bound :math:`b` of the interval containing a minimum.
**maxcal** : int
The maximum number of calls of :math:`\mathrm{funct}` to be allowed.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**e1** : float
If you set :math:`\mathrm{e1}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e1}` will be reset to the default value :math:`\sqrt{\epsilon }` before starting the minimization process.
**e2** : float
If you set :math:`\mathrm{e2}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e2}` will be reset to the default value :math:`\sqrt{\epsilon }`.
**a** : float
An improved lower bound on the position of the minimum.
**b** : float
An improved upper bound on the position of the minimum.
**maxcal** : int
The total number of times that :math:`\mathrm{funct}` was actually called.
**x** : float
The estimated position of the minimum.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float
The value of the first derivative at the final point in :math:`\mathrm{x}`.
.. _e04bb-py2-py-errors:
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
On entry, :math:`\mathrm{a}+\mathrm{e2} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{b} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{a}+\mathrm{e2} < \mathrm{b}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 2`.
(`errno` :math:`2`)
The maximum number of function calls, :math:`\langle\mathit{\boldsymbol{value}}\rangle`, have been performed. This may have happened simply because :math:`\mathrm{maxcal}` was set too small for the particular problem, or may be due to a mistake in the user-supplied function :math:`\mathrm{funct}`. If no mistake can be found in :math:`\mathrm{funct}`, restart ``one_var_deriv`` (preferably with the values of :math:`\mathrm{a}` and :math:`\mathrm{b}` given on exit from the previous call to ``one_var_deriv``).
.. _e04bb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``one_var_deriv`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right)\quad \text{ subject to }\quad a\leq x\leq b
when the first derivative :math:`\frac{{dF}}{{dx}}` can be calculated.
The function normally computes a sequence of :math:`x` values which tend in the limit to a minimum of :math:`F\left(x\right)` subject to the given bounds.
It also progressively reduces the interval :math:`\left[a, b\right]` in which the minimum is known to lie.
It uses the safeguarded cubic-interpolation method described in Gill and Murray (1973).
You must supply a :math:`\mathrm{funct}` to evaluate :math:`F\left(x\right)` and :math:`\frac{{dF}}{{dx}}`.
The arguments :math:`\mathrm{e1}` and :math:`\mathrm{e2}` together specify the accuracy
.. math::
\textit{Tol}\left(x\right) = \mathrm{e1}\times \left\lvert x\right\rvert +\mathrm{e2}
to which the position of the minimum is required.
Note that :math:`\mathrm{funct}` is never called at a point which is closer than :math:`\textit{Tol}\left(x\right)` to a previous point.
If the original interval :math:`\left[a, b\right]` contains more than one minimum, ``one_var_deriv`` will normally find one of the minima.
.. _e04bb-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
"""
raise NotImplementedError
[docs]def uncon_simplex(x, tolf, tolx, funct, maxcal, monit=None, data=None):
r"""
``uncon_simplex`` minimizes a general function :math:`F\left(\mathbf{x}\right)` of :math:`n` independent variables :math:`\mathbf{x} = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` by the Nelder and Mead simplex method (see Nelder and Mead (1965)).
Derivatives of the function need not be supplied.
.. _e04cb-py2-py-doc:
For full information please refer to the NAG Library document for e04cb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html
.. _e04cb-py2-py-parameters:
**Parameters**
**x** : float, array-like, shape :math:`\left(n\right)`
A guess at the position of the minimum. Note that the problem should be scaled so that the values of the :math:`\mathrm{x}[i-1]` are of order unity.
**tolf** : float
The error tolerable in the function values, in the following sense. If :math:`f_{\textit{i}}`, for :math:`\textit{i} = 1,2,\ldots,n+1`, are the individual function values at the vertices of the current simplex, and if :math:`f_m` is the mean of these values, then you can request that ``uncon_simplex`` should terminate if
.. math::
\sqrt{\frac{1}{{n+1}}\sum_{{i = 1}}^{{n+1}}\left(f_i-f_m\right)^2} < \mathrm{tolf}\text{.}
You may specify :math:`\mathrm{tolf} = 0` if you wish to use only the termination criterion `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn2>`__ on the spatial values: see the description of :math:`\mathrm{tolx}`.
**tolx** : float
The error tolerable in the spatial values, in the following sense. If :math:`LV` denotes the 'linearized' volume of the current simplex, and if :math:`{LV}_{\mathrm{init}}` denotes the 'linearized' volume of the initial simplex, then you can request that ``uncon_simplex`` should terminate if
.. math::
\frac{{LV}}{{{LV}_{\mathrm{init}}}} < \mathrm{tolx}\text{.}
You may specify :math:`\mathrm{tolx} = 0` if you wish to use only the termination criterion `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn1>`__ on function values: see the description of :math:`\mathrm{tolf}`.
**funct** : callable fc = funct(xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F` at a specified point.
It should be tested separately before being used in conjunction with ``uncon_simplex``.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point at which the function value is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`\mathbf{x}`.
**maxcal** : int
The maximum number of function evaluations to be allowed.
**monit** : None or callable monit(fmin, fmax, sim, ncall, serror, vratio, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` may be used to monitor the optimization process.
It is invoked once every iteration.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
**Parameters**
**fmin** : float
The smallest function value in the current simplex.
**fmax** : float
The largest function value in the current simplex.
**sim** : float, ndarray, shape :math:`\left(n+1, n\right)`
The :math:`n+1` position vectors of the current simplex.
**ncall** : int
The number of times that :math:`\mathrm{funct}` has been called so far.
**serror** : float
The current value of the standard deviation in function values used in termination test `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn1>`__.
**vratio** : float
The current value of the linearized volume ratio used in termination test `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn2>`__.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\mathbf{x}` corresponding to the function value in :math:`\mathrm{f}`.
**f** : float
The lowest function value found.
.. _e04cb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolx} = 0.0` and :math:`\mathrm{tolf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolx} = 0.0` then :math:`\mathrm{tolf}` is greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolf} = 0.0` and :math:`\mathrm{tolx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolf} = 0.0` then :math:`\mathrm{tolx}` is greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolf} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{tolx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolf}\neq 0.0` and :math:`\mathrm{tolx}\neq 0.0` then both should be greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`2`)
:math:`\mathrm{maxcal}` function evaluations have been completed without any other termination test passing. Check the coding of :math:`\mathrm{funct}` before increasing the value of :math:`\mathrm{maxcal}`.
.. _e04cb-py2-py-notes:
**Notes**
``uncon_simplex`` finds an approximation to a minimum of a function :math:`F` of :math:`n` variables.
You must supply a function to calculate the value of :math:`F` for any set of values of the variables.
The method is iterative.
A simplex of :math:`n+1` points is set up in the :math:`n`-dimensional space of the variables (for example, in :math:`2` dimensions the simplex is a triangle) under the assumption that the problem has been scaled so that the values of the independent variables at the minimum are of order unity.
The starting point you have provided is the first vertex of the simplex, the remaining :math:`n` vertices are generated by ``uncon_simplex``.
The vertex of the simplex with the largest function value is reflected in the centre of gravity of the remaining vertices and the function value at this new point is compared with the remaining function values.
Depending on the outcome of this test the new point is accepted or rejected, a further expansion move may be made, or a contraction may be carried out.
See Nelder and Mead (1965) and Parkinson and Hutchinson (1972) for more details.
When no further progress can be made the sides of the simplex are reduced in length and the method is repeated.
The method can be slow, but computational bottlenecks have been reduced following Singer and Singer (2004).
However, ``uncon_simplex`` is robust, and, therefore, very useful for functions that are subject to inaccuracies.
There are the following options for successful termination of the method: based only on the function values at the vertices of the current simplex (see `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn1>`__); based only on a volume ratio between the current simplex and the initial one (see `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04cbf.html#eqn2>`__); or based on which one of the previous two tests passes first.
The volume test may be useful if :math:`F` is discontinuous, while the function-value test should be sufficient on its own if :math:`F` is continuous.
.. _e04cb-py2-py-references:
**References**
Nelder, J A and Mead, R, 1965, `A simplex method for function minimization`, Comput. J. (7), 308--313
Parkinson, J M and Hutchinson, D, 1972, `An investigation into the efficiency of variants of the simplex method`, Numerical Methods for Nonlinear Optimization, (ed F A Lootsma), Academic Press
Singer, S and Singer, S, 2004, `Efficient implementation of the Nelder--Mead search algorithm`, Appl. Num. Anal. Comp. Math. (1(3)), 524--534
See Also
--------
:meth:`naginterfaces.library.examples.opt.uncon_simplex_ex.main`
"""
raise NotImplementedError
[docs]def uncon_conjgrd_comp(objfun, x, comm, data=None, io_manager=None):
r"""
``uncon_conjgrd_comp`` minimizes an unconstrained nonlinear function of several variables using a pre-conditioned, limited memory quasi-Newton conjugate gradient method.
First derivatives (or an 'acceptable' finite difference approximation to them) are required.
It is intended for use on large scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`uncon_conjgrd_option_file`, :meth:`uncon_conjgrd_option_string`, :meth:`nlp1_init`.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_comp`` is deprecated.
Please use :meth:`handle_solve_bounds_foas` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04dg-py2-py-doc:
For full information please refer to the NAG Library document for e04dg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dgf.html
.. _e04dg-py2-py-parameters:
**Parameters**
**objfun** : callable (objf, objgrd) = objfun(mode, x, nstate, data=None)
:math:`\mathrm{objfun}` must calculate the objective function :math:`F\left(x\right)` and possibly its gradient as well for a specified :math:`n`-element vector :math:`x`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{objf}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{objf}` and :math:`\mathrm{objgrd}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the objective function and its gradient are to be evaluated.
**nstate** : int
Will be :math:`1` on the first call of :math:`\mathrm{objfun}` by ``uncon_conjgrd_comp``, and :math:`0` for all subsequent calls. Thus, you may wish to test, :math:`\mathrm{nstate}` within :math:`\mathrm{objfun}` in order to perform certain calculations once only. For example, you may read data or initialize global variables when :math:`\mathrm{nstate} = 1`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**objf** : float
The value of the objective function at :math:`x`.
**objgrd** : float, array-like, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 2`, :math:`\mathrm{objgrd}[\textit{i}-1]` must contain the value of :math:`\frac{{\partial F}}{{\partial x_{\textit{i}}}}` evaluated at :math:`x`, for :math:`\textit{i} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**itera** : int
The total number of iterations performed.
**objf** : float
The value of the objective function at the final iterate.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`
The gradient of the objective function at the final iterate (or its finite difference approximation).
**x** : float, ndarray, shape :math:`\left(n\right)`
The final estimate of the solution.
.. _e04dg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Estimated Optimal Function Value'** : float
This value of :math:`r` specifies the user-supplied guess of the optimum objective function value :math:`F_{\mathrm{est}}`.
This value is used to calculate an initial step length :math:`\alpha_0` (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dgf.html#algdetails>`__).
If the value of :math:`r` is not specified (the default), then this has the effect of setting :math:`\alpha_0` to unity.
It should be noted that for badly scaled functions a unit step along the steepest descent direction will often compute the objective function at very large values of :math:`x`.
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.9}`
The argument defines :math:`\epsilon_r`, which is intended to be a measure of the accuracy with which the problem function :math:`F\left(x\right)` can be computed.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
The value of :math:`\epsilon_r` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`; i.e., :math:`\epsilon_r` acts as a relative precision when :math:`\left\lvert F\right\rvert` is large, and as an absolute precision when :math:`\left\lvert F\right\rvert` is small.
For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
In contrast, if :math:`F\left(x\right)` is typically of order :math:`10^{-4}` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
However when the accuracy of the computed function values is known to be significantly worse than full precision, the value of :math:`\epsilon_r` should be large enough so that no attempt will be made to distinguish between function values that differ by less than the error inherent in the calculation.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Linesearch Tolerance'** : float
Default :math:`\text{} = 0.9`
The value :math:`r` controls the accuracy with which the step :math:`\alpha` taken during each iteration approximates a minimum of the function along the search direction (the smaller the value of :math:`r`, the more accurate the linesearch).
The default value :math:`r = 0.9` requests an inaccurate search, and is appropriate for most problems.
A more accurate search may be appropriate when it is desirable to reduce the number of iterations -- for example, if the objective function is cheap to evaluate.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default for ``uncon_conjgrd_comp`` :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Maximum Step Length'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, the maximum allowable step length for the linesearch is taken as :math:`\mathrm{min}\left(\frac{1}{{\texttt{machine.real_safe}\left(\right)}}, \frac{r}{{\left\lVert p_k\right\rVert }}\right)`.
If :math:`r\leq 0`, the default value is used.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon_R^{0.8}`
The argument :math:`r` specifies the accuracy to which you wish the final iterate to approximate a solution of the problem.
Broadly speaking, :math:`r` indicates the number of correct figures desired in the objective function at the solution.
For example, if :math:`r` is :math:`10^{-6}` and termination occurs with no exception or warning is raised (see :ref:`Parameters <e04dg-py2-py-parameters>`), then the final point satisfies the termination criteria, where :math:`\tau_F` represents 'Optimality Tolerance'.
If :math:`r < \epsilon_r` or :math:`r\geq 1`, the default value is used.
If 'Optimality Tolerance' is chosen below a certain threshold, it will automatically be reset to another value.
**'Print Level'** : int
The value :math:`i` controls the amount of printout produced by ``uncon_conjgrd_comp``, as indicated below.
A detailed description of the printout is given in `Description of Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dgf.html#fcomments1>`__ (summary output at each iteration and the final solution).
.. rst-class:: nag-rules-none nag-align-left
+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+==========+==============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Description of Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dgf.html#fcomments1>`__) for each iteration (no printout of the final solution).|
+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10`|The final solution and one line of summary output for each iteration. |
+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'Start Objective Check at Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check at Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{max}\left(1, \mathrm{min}\left(n, i_2\right)\right)`, the default value is used.
If :math:`i_2\leq 0` or :math:`i_2 > n`, the default value is used.
**'Stop Objective Check at Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check at Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{max}\left(1, \mathrm{min}\left(n, i_2\right)\right)`, the default value is used.
If :math:`i_2\leq 0` or :math:`i_2 > n`, the default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify Objective Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
.. _e04dg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`8`)
Gradient at the starting point is too small.
(`errno` :math:`9`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`6`)
Current point cannot be improved upon.
(`errno` :math:`7`)
Large errors found in the derivatives.
**NagAlgorithmicMajorWarning**
(`errno` :math:`3`)
Too many iterations.
(`errno` :math:`4`)
Computed upper bound on step length is too small.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination.
.. _e04dg-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``uncon_conjgrd_comp`` is designed to solve unconstrained minimization problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad {-\infty }\leq x\leq \infty \text{,}
where :math:`x` is an :math:`n`-element vector.
You must supply an initial estimate of the solution.
For maximum reliability, it is preferable to provide all first partial derivatives.
If all of the derivatives cannot be provided, you are recommended to obtain approximate values (using finite differences) by calling :meth:`estimate_deriv` from within :math:`\mathrm{objfun}`.
The method used by ``uncon_conjgrd_comp`` is described in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dgf.html#algdetails>`__.
.. _e04dg-py2-py-references:
**References**
Gill, P E and Murray, W, 1979, `Conjugate-gradient methods for large-scale nonlinear optimization`, Technical Report SOL 79-15, Department of Operations Research, Stanford University
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
"""
raise NotImplementedError
[docs]def uncon_conjgrd_option_file(ioptns, comm, io_manager=None):
r"""
``uncon_conjgrd_option_file`` may be used to supply options to :meth:`uncon_conjgrd_comp` from an external file.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_option_file`` is deprecated.
There is no suggested replacement for this routine.
.. _e04dj-py2-py-doc:
For full information please refer to the NAG Library document for e04dj
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04djf.html
.. _e04dj-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04dj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04dj-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``uncon_conjgrd_option_file`` may be used to supply values for options to :meth:`uncon_conjgrd_comp`. ``uncon_conjgrd_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`uncon_conjgrd_comp` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`uncon_conjgrd_comp`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def uncon_conjgrd_option_string(optstr, comm, io_manager=None):
r"""
``uncon_conjgrd_option_string`` may be used to supply individual options to :meth:`uncon_conjgrd_comp`.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_option_string`` is deprecated.
There is no suggested replacement for this routine.
.. _e04dk-py2-py-doc:
For full information please refer to the NAG Library document for e04dk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04dkf.html
.. _e04dk-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04dk-py2-py-notes>` and in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04dk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04dk-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``uncon_conjgrd_option_string`` may be used to supply values for options to :meth:`uncon_conjgrd_comp`.
It is only necessary to call ``uncon_conjgrd_option_string`` for those arguments whose values are to be different from their default values.
One call to ``uncon_conjgrd_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`uncon_conjgrd_comp` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`uncon_conjgrd_comp`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_func_comp(m, lsqfun, x, lsqmon=None, iprint=1, maxcal=None, eta=None, xtol=0.0, stepmx=100000.0, data=None):
r"""
``lsq_uncon_mod_func_comp`` is a comprehensive algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
No derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04fc-py2-py-doc:
For full information please refer to the NAG Library document for e04fc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fcf.html
.. _e04fc-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` at any point :math:`x`. (However, if you do not wish to calculate the residuals at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_func_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Has a non-negative value.
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{lsqfun}` resets :math:`\mathrm{iflag}` to some negative number, ``lsq_uncon_mod_func_comp`` will terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current approximation to the Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**igrade** : int
``lsq_uncon_mod_func_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_func_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far. (However, for intermediate calls of :math:`\mathrm{lsqmon}`, :math:`\mathrm{nf}` is calculated on the assumption that the latest linear search has been successful. If this is not the case, the :math:`n` evaluations allowed for approximating the Jacobian at the new point will not in fact have been made. :math:`\mathrm{nf}` will be accurate at the final call of :math:`\mathrm{lsqmon}`.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_func_comp``.
If :math:`\mathrm{iprint} = 0`, :math:`\mathrm{lsqmon}` is just called at the final point.
If :math:`\mathrm{iprint} < 0`, :math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`400\times n`.
The limit you set on the number of times that :math:`\mathrm{lsqfun}` may be called by ``lsq_uncon_mod_func_comp``. There will be an error exit (see :ref:`Exceptions <e04fc-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_func_comp`` involves a linear minimization, i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`.
Specifies how accurately the linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`). Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_mod_func_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made each iteration. On balance it is usually more efficient to perform a low accuracy minimization.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fcf.html#accuracy>`__.)
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.) ``lsq_uncon_mod_func_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2\text{,}
where :math:`k` is the iteration number. Thus, if the problem has more than one solution, ``lsq_uncon_mod_func_comp`` is most likely to find the one nearest to the starting point. On difficult problems, a realistic choice can prevent the sequence :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`. However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The estimate of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the estimated Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the estimated Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_func_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}`).
.. _e04fc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`4`)
Failure in computing SVD of estimated Jacobian matrix.
.. _e04fc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_uncon_mod_func_comp`` is essentially identical to the function LSQNDN in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply :math:`\mathrm{lsqfun}` to calculate the values of the :math:`f_i\left(x\right)` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is an approximation to the Gauss--Newton direction; otherwise additional function evaluations are made so as to enable :math:`p^{\left(k\right)}` to be a more accurate approximation to the Newton direction.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04fc-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
See Also
--------
:meth:`naginterfaces.library.examples.opt.lsq_uncon_mod_func_comp_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfls(handle, objfun, x, nres, monit=None, data=None, io_manager=None):
r"""
``handle_solve_dfls`` is a forward communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFLS) for small to medium-scale nonlinear least squares problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04ff-py2-py-doc:
For full information please refer to the NAG Library document for e04ff
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fff.html
.. _e04ff-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfls``. It **must not** be changed between calls to the NAG optimization modelling suite.
**objfun** : callable (rx, inform) = objfun(x, nres, inform, data=None)
:math:`\mathrm{objfun}` must evaluate the value of the nonlinear residuals :math:`r_i\left(x\right)` at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`m_r`, the number of residuals in the problem, as set during the initialization of the handle by :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residuals :math:`r_i\left(x\right)` at :math:`x`.
**inform** : int
May be used to indicate that the requested objective value could not be computed. Specifically, it can be set to a negative value:
:math:`\mathrm{inform} = -1`
The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 17.
:math:`\mathrm{inform} = -2`
The solver will cleanly exit with :math:`\mathrm{errno}` = 20 and return the best available point as well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`m_r`, the number of residuals in the problem. It must be unchanged from the value set during the definition of the objective structure by :meth:`handle_set_nlnls`.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'DFO Monitor Frequency' (default value :math:`0`, :math:`\mathrm{monit}` is never called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+------------+--------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)` (sum of the squared residuals).|
+------------+--------------------------------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+------------+--------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+--------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04ff-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFLS Small Residuals Tol'** : float
Default :math:`= \epsilon^{0.75}`
This option defines the tolerance on the value of the residuals.
Namely, the solver declares convergence if
.. math::
f\left(x\right) = \sum_{{i = 1}}^{m_r}{r_i\left(x\right)}^2 < \text{‘DFLS Small Residuals Tol'}\text{.}
Constraint: :math:`\text{‘DFLS Small Residuals Tol'} > \epsilon^2`.
**'DFO Initial Interp Points'** : str
Default :math:`= \texttt{'Coordinate'}`
Determines how the initial interpolation points are chosen.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}`, the interpolation points are chosen along the coordinate directions around the initial point.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Random'}`, the initial interpolation points are chosen along random orthogonal directions around the initial point.
Set 'DFO Random Seed' to a positive value to fix the random seed and get reproducible results.
Constraint: :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}` or :math:`\texttt{'Random'}`.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function if :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Initial Points'** : int
Default :math:`= {0}`
The initial number of interpolation points in :math:`Y_0` `(9) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fff.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Initial Points'} = 0`, the number of points is chosen to be equal to the total number of interpolation points set by 'DFO Number Interp Points'.
If this parameter is chosen to be lower than the maximum set by 'DFO Number Interp Points', the solver will progressively increase the number of interpolation points until it reaches that value.
In this release, it is only possible to grow the interpolation set if 'DFO Number Interp Points' is set to the default value.
Constraint: :math:`\text{‘DFO Number Initial Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`0\leq \text{‘DFO Number Initial Points'}\leq \text{‘DFO Number Interp Points'}`.
If :math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`, 'DFO Number Interp Points' must be set to the default value.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fff.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model or build the underdetermined models when the interpolation set has not fully grown (:math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`).
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Version'** : str
Default :math:`= {\texttt{'Latest'}}`
At Mark 27, the underlying algorithm of ``handle_solve_dfls`` underwent significant changes.
This option allows you to continue using the Mark 26 version if it is set to ''26''.
By default (recommended), the latest version of the code is called.
Constraint: :math:`\text{‘DFO Version'} = \texttt{'Latest'}` or :math:`\texttt{'26'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04ff-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is greater than the maximum.
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`6`)
Initial number of interpolation points :math:`\textit{ninit} = \langle\mathit{\boldsymbol{value}}\rangle`, total number of interpolation points :math:`\textit{npts} = \langle\mathit{\boldsymbol{value}}\rangle`, number of variables :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: growing interpolation set is only supported for linear models (:math:`\textit{npts} = \textit{nvar}+1`).
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`8`)
Maximization is not possible for a nonlinear least squares problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points were not provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
A rescue procedure has been called in order to correct damage from rounding errors when computing an update to a quadratic approximation of :math:`F`, but no further progress could be made. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`20`)
User requested termination after a call to the objective function.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04ff-py2-py-notes:
**Notes**
``handle_solve_dfls`` is aimed at minimizing a sum of squares objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & \sum_{{i = 1}}^{m_r} {r_i\left(x\right)}^2 \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here the :math:`r_i\left(x\right)` are smooth nonlinear functions called residuals and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
Typically, in a calibration or data fitting context, the residuals will be defined as the difference between the data points and a nonlinear model (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optclasses>`__).
``handle_solve_dfls`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnls` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfls`` always assumes that the Jacobian of the residuals is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnls` will have no effect.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fff.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04ff-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFLS (Derivative-free Least Squares).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfls`` is the same as the one used by :meth:`handle_solve_dfls_rcomm`. ``handle_solve_dfls`` serves as a forward communication interface to the derivative-free solver for nonlinear least squares problems.
.. _e04ff-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Cartis, C and Roberts, L, 2017, `A Derivative-Free Gauss-Newton Method`
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
Zhang, H, Conn, A R and Scheinberg, K, 2010, `A Derivative-Free Algorithm for Least-Squares Minimization`, SIAM J. Optim. (20(6)), 3555--3576
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfls_rcomm(handle, irevcm, x, rx, io_manager=None):
r"""
``handle_solve_dfls_rcomm`` is a reverse communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFLS) for small to medium-scale nonlinear least squares problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04fg-py2-py-doc:
For full information please refer to the NAG Library document for e04fg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fgf.html
.. _e04fg-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfls_rcomm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**irevcm** : int
Does not need to be set on the first call of ``handle_solve_dfls_rcomm``. On subsequent calls, :math:`\mathrm{irevcm}` must be set to a positive integer if all the required function evaluations have been correctly provided in :math:`\mathrm{rx}`. Otherwise, if a problem occurred during a monitoring step or while providing objective values, it is possible to set it to a negative value:
:math:`\mathrm{irevcm} = -1`
If function evaluations were required, the solver will attempt a rescue procedure and request an alternative point. If no function were required (monitoring step), the solver will stop with :math:`\mathrm{errno}` = 20.
:math:`\mathrm{irevcm} \leq -2`
The solver will cleanly exit and return the best available point as a well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
The first column contains :math:`x_0`, the initial estimates of the variables :math:`x`.
**rx** : float, array-like, shape :math:`\left(\textit{nres}, \textit{maxeval}\right)`
Does not need to be set on the first call to ``handle_solve_dfls_rcomm``.
If :math:`\mathrm{irevcm} = 1` after the last call of ``handle_solve_dfls_rcomm``, the first :math:`\mathrm{neval}` columns must contain the residuals of the requested points.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**irevcm** : int
Indicates what action is to be performed before the next call to ``handle_solve_dfls_rcomm``.
:math:`\mathrm{irevcm} = 0`
Final exit of the solver.
:math:`\mathrm{irevcm} = 1`
:math:`\mathrm{neval}` objective evaluations are required.
:math:`\mathrm{irevcm} = 2`
Monitoring step, no evaluation is required, :math:`\mathrm{x}` and :math:`\mathrm{rx}` contain the best evaluation of the objective yet.
**neval** : int
Indicates the number of objective evaluations required for the next call of ``handle_solve_dfls_rcomm`` in :math:`\mathrm{rx}`. The coordinates of the points to evaluate are provided in the first :math:`\mathrm{neval}` columns of :math:`\mathrm{x}`.
**x** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the best computed estimate of the solution.
If :math:`\mathrm{irevcm} = 1`, the first :math:`\mathrm{neval}` columns contain the coordinates of the points to evaluate.
**rx** : float, ndarray, shape :math:`\left(\textit{nres}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the residuals of the best computed point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+------------+--------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)` (sum of the squared residuals).|
+------------+--------------------------------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+------------+--------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+--------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04fg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFLS Small Residuals Tol'** : float
Default :math:`= \epsilon^{0.75}`
This option defines the tolerance on the value of the residuals.
Namely, the solver declares convergence if
.. math::
f\left(x\right) = \sum_{{i = 1}}^{m_r}{r_i\left(x\right)}^2 < \text{‘DFLS Small Residuals Tol'}\text{.}
Constraint: :math:`\text{‘DFLS Small Residuals Tol'} > \epsilon^2`.
**'DFO Initial Interp Points'** : str
Default :math:`= \texttt{'Coordinate'}`
Determines how the initial interpolation points are chosen.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}`, the interpolation points are chosen along the coordinate directions around the initial point.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Random'}`, the initial interpolation points are chosen along random orthogonal directions around the initial point.
Set 'DFO Random Seed' to a positive value to fix the random seed and get reproducible results.
Constraint: :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}` or :math:`\texttt{'Random'}`.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, the solver will stop at the end of every :math:`i`\ th step for monitoring purposes. ``handle_solve_dfls_rcomm`` needs to be called again to continue the optimization.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Initial Points'** : int
Default :math:`= {0}`
The initial number of interpolation points in :math:`Y_0` `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fgf.html#eqn1>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Initial Points'} = 0`, the number of points is chosen to be equal to the total number of interpolation points set by 'DFO Number Interp Points'.
If this parameter is chosen to be lower than the maximum set by 'DFO Number Interp Points', the solver will progressively increase the number of interpolation points until it reaches that value.
In this release, it is only possible to grow the interpolation set if 'DFO Number Interp Points' is set to the default value.
Constraint: :math:`\text{‘DFO Number Initial Points'} \geq 0`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`0\leq \text{‘DFO Number Initial Points'}\leq \text{‘DFO Number Interp Points'}`.
If :math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`, 'DFO Number Interp Points' must be set to the default value.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fgf.html#eqn1>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model or build the underdetermined models when the interpolation set has not fully grown (:math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`).
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04fg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{maxeval} > 0`.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the first call of the function, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is greater than the maximum.
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`6`)
Initial number of interpolation points :math:`\textit{ninit} = \langle\mathit{\boldsymbol{value}}\rangle`, total number of interpolation points :math:`\textit{npts} = \langle\mathit{\boldsymbol{value}}\rangle`, number of variables :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: growing interpolation set is only supported for linear models (:math:`\textit{npts} = \textit{nvar}+1`).
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`8`)
Maximization is not possible for a nonlinear least squares problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points were not provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`20`)
User requested termination during an objective evaluation step.
.. _e04fg-py2-py-notes:
**Notes**
``handle_solve_dfls_rcomm`` uses reverse communication for function evaluations and monitoring steps.
Every time the solver requires an evaluation of the objective function, it pauses its progress, exits and waits for the function to be called again with the objective value provided in the argument :math:`\mathrm{rx}`.
``handle_solve_dfls_rcomm`` is aimed at minimizing a sum of squares objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & \sum_{{i = 1}}^{m_r} {r_i\left(x\right)}^2 \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here the :math:`r_i\left(x\right)` are smooth nonlinear functions called residuals and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
Typically, in a calibration or data fitting context, the residuals will be defined as the difference between the data points and a nonlinear model (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optclasses>`__).
``handle_solve_dfls_rcomm`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnls` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfls_rcomm`` always assumes that the Jacobian of the residuals is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnls` will have no effect.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fgf.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04fg-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFLS (Derivative-free Least Squares).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfls_rcomm`` is the same as the one used by :meth:`handle_solve_dfls`. ``handle_solve_dfls_rcomm`` serves as a reverse communication interface to the derivative-free solver for nonlinear least squares problems.
.. _e04fg-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Cartis, C and Roberts, L, 2017, `A Derivative-Free Gauss-Newton Method`
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
Zhang, H, Conn, A R and Scheinberg, K, 2010, `A Derivative-Free Algorithm for Least-Squares Minimization`, SIAM J. Optim. (20(6)), 3555--3576
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_func_easy(m, lsfun1, x, data=None):
r"""
``lsq_uncon_mod_func_easy`` is an easy-to-use algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
No derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04fy-py2-py-doc:
For full information please refer to the NAG Library document for e04fy
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04fyf.html
.. _e04fy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun1** : callable fvec = lsfun1(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_func_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04fy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`400\times n` calls to :math:`\mathrm{lsfun1}`.
(`errno` :math:`4`)
Failure in computing SVD of estimated Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04fy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_func_easy`` is essentially identical to the function LSNDN1 in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}F\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate functions :math:`f_i\left(x\right)` at any point :math:`x`.
From a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04fy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_quasi_deriv_comp(m, selct, lsqfun, eta, xtol, x, lsqmon=None, iprint=1, maxcal=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_quasi_deriv_comp`` is a comprehensive quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. deprecated:: 28.3.0.0
``lsq_uncon_quasi_deriv_comp`` is deprecated.
Please use :meth:`handle_solve_bxnl` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04gb-py2-py-doc:
For full information please refer to the NAG Library document for e04gb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gbf.html
.. _e04gb-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**selct** : int
:math:`\mathrm{selct}` enables you to specify whether the linear minimizations (i.e., minimizations of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`) are to be performed by a function which just requires the evaluation of the :math:`f_i\left(x\right)` (:math:`\mathrm{selct} = 1`), or by a function which also requires the first derivatives of the :math:`f_i\left(x\right)` (:math:`\mathrm{selct} = 2`).
It will often be possible to evaluate the first derivatives of the residuals in about the same amount of computer time that is required for the evaluation of the residuals themselves -- if this is so, then ``lsq_uncon_quasi_deriv_comp`` should be called with :math:`\mathrm{selct} = 2`.
However, if the evaluation of the derivatives takes more than about :math:`4` times as long as the evaluation of the residuals, :math:`\mathrm{selct} = 1` will usually be preferable.
If in doubt, use :math:`\mathrm{selct} = 2` as it is slightly more robust.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_quasi_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Will be set to :math:`0`, :math:`1` or :math:`2`.
:math:`\mathrm{iflag} = 0`
Indicates that only the residuals need to be evaluated
:math:`\mathrm{iflag} = 1`
Indicates that only the Jacobian matrix needs to be evaluated
:math:`\mathrm{iflag} = 2`
Indicates that both the residuals and the Jacobian matrix must be calculated.
If :math:`\mathrm{selct} = 2`, :math:`\mathrm{lsqfun}` will always be called with :math:`\mathrm{iflag}` set to :math:`2`.
**m** : int
:math:`m`, the number of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it is wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_quasi_deriv_comp``. ``lsq_uncon_quasi_deriv_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag} = 1` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[i-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag} = 0` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**eta** : float
Every iteration of ``lsq_uncon_quasi_deriv_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_quasi_deriv_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made every iteration.
On balance it is usually more efficient to perform a low accuracy minimization.
`Suggested value`:
:math:`\mathrm{eta} = 0.9` if :math:`n > 1` and :math:`\mathrm{selct} = 2`,
:math:`\mathrm{eta} = 0.5` if :math:`n > 1` and :math:`\mathrm{selct} = 1`,
:math:`\mathrm{eta} = 0.0` if :math:`n = 1`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gbf.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gbf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_quasi_deriv_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**igrade** : int
``lsq_uncon_quasi_deriv_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_quasi_deriv_comp``.
**nf** : int
The number of evaluations of the residuals. (If :math:`\mathrm{selct} = 2`, :math:`\mathrm{nf}` is also the number of evaluations of the Jacobian matrix.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_quasi_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`\mathrm{selct} = 1`: :math:`{ 75\times n }`; otherwise: :math:`{ 50\times n }`.
Enables you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_quasi_deriv_comp``. There will be an error exit (see :ref:`Exceptions <e04gb-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.)
``lsq_uncon_quasi_deriv_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``lsq_uncon_quasi_deriv_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_quasi_deriv_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., the number of calls of :math:`\mathrm{lsqfun}`). If :math:`\mathrm{selct} = 2`, :math:`\mathrm{nf}` is also the number of times that the Jacobian matrix has been evaluated.
.. _e04gb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{selct} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{selct} = 1` or :math:`2`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
**NagAlgorithmicMajorWarning**
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
.. _e04gb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_uncon_quasi_deriv_comp`` is essentially identical to the function LSQFDQ in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise the second derivatives of the :math:`f_i\left(x\right)` are taken into account using a quasi-Newton updating scheme.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04gb-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv_comp(m, lsqfun, xtol, x, lsqmon=None, iprint=1, maxcal=None, eta=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv_comp`` is a comprehensive modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04gd-py2-py-doc:
For full information please refer to the NAG Library document for e04gd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gdf.html
.. _e04gd-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`1` or :math:`2`.
:math:`\mathrm{iflag} = 1`
Indicates that only the Jacobian matrix needs to be evaluated
:math:`\mathrm{iflag} = 2`
Indicates that both the residuals and the Jacobian matrix must be calculated
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_mod_deriv_comp``. ``lsq_uncon_mod_deriv_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag} = 1` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[i-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gdf.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gdf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_mod_deriv_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem. (If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called at the initial point before the singular values have been calculated. So the elements of :math:`\mathrm{s}` are set to zero for the first call of :math:`\mathrm{lsqmon}`.)
**igrade** : int
``lsq_uncon_mod_deriv_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far with :math:`\mathrm{iflag} = 2`. (In addition to these calls monitored by :math:`\mathrm{nf}`, :math:`\mathrm{lsqfun}` is called not more than :math:`\textit{n}` times per iteration with :math:`\mathrm{iflag}` set to :math:`1`.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
Enables you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_mod_deriv_comp``. There will be an error exit (see :ref:`Exceptions <e04gd-py2-py-errors>`) after :math:`\mathrm{maxcal}` evaluations of the residuals (i.e., calls of :math:`\mathrm{lsqfun}` with :math:`\mathrm{iflag}` set to :math:`2`). It should be borne in mind that, in addition to the calls of :math:`\mathrm{lsqfun}` which are limited directly by :math:`\mathrm{maxcal}`, there will be calls of :math:`\mathrm{lsqfun}` (with :math:`\mathrm{iflag}` set to :math:`1`) to evaluate only first derivatives.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_deriv_comp`` involves a linear minimization, i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`. :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations, they will tend to increase the number of calls of :math:`\mathrm{lsqfun}` (with :math:`\mathrm{iflag}` set to :math:`2`) needed for each linear minimization.
On balance it is usually efficient to perform a low accuracy linear minimization.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.) ``lsq_uncon_mod_deriv_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2
where :math:`k` is the iteration number. Thus, if the problem has more than one solution, ``lsq_uncon_mod_deriv_comp`` is most likely to find the one nearest to the starting point. On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`. However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}` with :math:`\mathrm{iflag}` set to :math:`2`).
.. _e04gd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` evaluations of the residuals.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
.. _e04gd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv_comp`` is essentially identical to the function LSQFDN in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise finite difference estimates of the second derivatives of the :math:`f_i\left(x\right)` are taken into account.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04gd-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def handle_solve_bxnl(handle, lsqfun, lsqgrd, x, nres, lsqhes=None, lsqhprd=None, monit=None, data=None, io_manager=None, spiked_sorder='C'):
r"""
``handle_solve_bxnl`` is a bound-constrained nonlinear least squares trust region solver (BXNL) from the NAG optimization modelling suite aimed for small to medium-scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04gg-py2-py-doc:
For full information please refer to the NAG Library document for e04gg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html
.. _e04gg-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_bxnl``. It **must not** be changed between calls to the NAG optimization modelling suite.
**lsqfun** : callable (rx, inform) = lsqfun(x, nres, inform, data=None)
:math:`\mathrm{lsqfun}` must evaluate the value of the nonlinear residuals, :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right),i = 1,\ldots,n_{\mathrm{res}}`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residual vector, :math:`r\left(x\right)`, evaluated at :math:`x`.
**inform** : int
May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqgrd** : callable inform = lsqgrd(x, nres, rdx, inform, data=None)
:math:`\mathrm{lsqgrd}` evaluates the residual gradients, :math:`\nabla r_i\left(x\right)`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residual gradients, :math:`\nabla r_i\left(x\right)`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**rdx** : float, ndarray, shape :math:`\left(\textit{nnzrd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the vector containing the nonzero residual gradients evaluated at :math:`x`,
.. math::
\nabla r\left(x\right) = \left[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\ldots,\nabla r_{n_{\textit{res}}}\left(x\right)\right]\text{,}
where
.. math::
\nabla r_i\left(x\right) = \left[\frac{{\partial r_i\left(x\right)}}{{\partial x_1}},\ldots,\frac{{\partial r_i\left(x\right)}}{{\partial x_{n_{\textit{var}}}}}\right]^\mathrm{T}\text{.}
The elements must be stored in the same order as the defined sparsity pattern provided in the call to :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**lsqhes** : None or callable inform = lsqhes(x, lamda, hx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{lsqhes}` evaluates the residual Hessians, :math:`\nabla^2r_i\left(x\right)`, at a specified point :math:`x`.
By default, the option :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'NO'}` and :math:`\mathrm{lsqhes}` is never called. :math:`\mathrm{lsqhes}` may be **None**.
This function will only be called if the option :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` and if the model (see `Models <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#models>`__) requires second order information.
Under these circumstances, if you do not provide a valid :math:`\mathrm{lsqhes}` the solver will terminate with either :math:`\mathrm{errno}` = 6 or :math:`\mathrm{errno}` = 21.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**lamda** : float, ndarray, shape :math:`\left(\textit{nres}\right)`
:math:`\lambda`, the vector containing the (weighted) residuals at :math:`x`, :math:`\lambda_i = w_ir_i\left(x\right)`. See `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and `Residual Weights <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#residualweights>`__.
**hx** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{nvar}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: a dense square (symmetric) matrix containing the weighted sum of residual Hessians,
.. math::
H\left(x\right) = \sum_{{i = 1}}^{n_{\textit{res}}}\lambda_i\nabla^2r_i\left(x\right)\text{,}
where
.. math::
\nabla^2r_i\left(x\right) = \begin{pmatrix} \frac{\partial^2}{{\partial_{x_1}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_1}\partial_{x_2}}} r_i \left(x\right) &\ldots & \frac{\partial^2}{{\partial_{x_1}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \\ \frac{\partial^2}{{\partial_{x_2}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_2}\partial_{x_2}}} r_i \left(x\right) &…& \frac{\partial^2}{{\partial_{x_2}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \\⋮&⋮&⋱&⋮\\ \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_2}}} r_i \left(x\right) &…& \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \end{pmatrix}\text{,}
is also a dense square (symmetric) matrix containing the :math:`i`\ th residual Hessian evaluated at the point :math:`x`. All matrix elements must be provided: both upper and lower triangular parts.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqhprd** : None or callable (hxy, inform) = lsqhprd(x, y, hxy, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{lsqhprd}` evaluates the residual Hessians, :math:`\nabla^2r_i\left(x\right)`, at a specified point, :math:`x`, and performs matrix-vector products with a given vector, :math:`y`, returning the dense matrix :math:`\left[\nabla^2r_1\left(x\right)y,\nabla^2r_2\left(x\right)y,\ldots,\nabla^2r_{n_{\textit{res}}}\left(x\right)y\right]`.
If you do not supply this function, it may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**y** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`y`, the vector used to perform the required matrix-vector products.
**hxy** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{nres}\right)`
The elements should only be assigned and not referenced.
**inform** : int
The first call to :math:`\mathrm{lsqhprd}` will have a non-zero value and can be used to optimize your code in order to avoid recalculations of common quantities when evaluating the Hessians. For all other instances :math:`\mathrm{inform}` will have a value of zero. This notification argument may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**hxy** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{nres}\right)`
A dense matrix of size :math:`n_{\textit{var}}\times n_{\textit{res}}` containing the following matrix-vector products,
.. math::
H\left(x,y\right) = \left[\nabla^2r_1\left(x\right)y,\nabla^2r_2\left(x\right)y,\ldots,\nabla^2r_{n_{\textit{res}}}\left(x\right)y\right]\text{.}
**inform** : int
May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25. The value of :math:`\mathrm{inform}` returned on the first call is ignored.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'Bxnl Monitor Frequency' (the default value is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**spiked_sorder** : str, optional
If :math:`\mathrm{hxy}` in :math:`\mathrm{lsqhprd}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value, :math:`f\left(x\right)`. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Norm of the projected gradient at the current iterate, see **PG STEP** in `Bound Constraints <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#subprob_bcon>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.|
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Norm of the scaled projected gradient at the current iterate, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__ |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Norm of the step between the current and previous iterate. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Convergence tests result. A scalar value between :math:`0-7` indicates whether a convergence test has passed. Specifically, :math:`1` indicates small objective test passed, :math:`2` indicates small (scaled) gradient test passed, :math:`4` indicates small step test passed. In the case where two or more tests passed, they are accumulated. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Norm of the current iterate :math:`x`. If regularization is requested, then this value was used in the regularization and it might differ from :math:`\left\lVert x\right\rVert` if :math:`x` has fixed or disabled elements. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6`--:math:`99`|Reserved for future use. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of calls to the objective function :math:`\mathrm{lsqfun}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of calls to the objective gradient function :math:`\mathrm{lsqgrd}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`3` |Total number of calls to the objective Hessian function :math:`\mathrm{lsqhes}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`4` |Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of calls to the objective function :math:`\mathrm{lsqfun}` required by linesearch steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`6` |Number of calls to the objective gradient function :math:`\mathrm{lsqgrd}` required by linesearch steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`7` |Number of calls to the objective function :math:`\mathrm{lsqfun}` required by projected gradient steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`8` |Number of calls to the objective gradient function :math:`\mathrm{lsqgrd}` required by projected gradient steps.|
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`9` |Number of inner iterations performed, see option :math:`\text{‘Bxnl Model'} = \texttt{'TENSOR-NEWTON'}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`10` |Number of linesearch iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`11` |Number of projected gradient iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`12` |Total number of calls to the objective auxiliary Hessian function :math:`\mathrm{lsqhprd}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`13`--:math:`99`|Reserved for future use. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
.. _e04gg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Bxnl Basereg Pow'** : float
Default :math:`= 2.0`
This argument defines the regularization power :math:`p` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and for the tensor Newton subproblem (when :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
Some values are restricted depending on the type of regularization specified, see 'Bxnl Basereg Type' for more details.
Constraint: :math:`\text{‘Bxnl Basereg Pow'} > 0`.
**'Bxnl Basereg Term'** : float
Default :math:`= 0.01`
This argument defines the regularization term :math:`\sigma` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and for the tensor Newton subproblem (when :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
Constraint: :math:`\text{‘Bxnl Basereg Term'} > 0`.
**'Bxnl Basereg Type'** : str
Default :math:`= \texttt{'NONE'}`
This argument specifies the method used to incorporate the regularizer into `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and optionally into the tensor Newton subproblem (when :math:`\text{‘Bxnl Model'} = \texttt{'Tensor-Newton'}` and :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
The option :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-NVAR-DOF'}` reformulates the original problem by expanding it with :math:`n_{\textit{var}}` degrees of freedom that is subsequently solved.
For the case :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-1-DOF'}` the residual vector is extended with a new term of the form :math:`\frac{\sigma }{p}\left\lVert x\right\rVert_2^p`; for this method a value of :math:`p = 3` is recommended.
If :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-NVAR-DOF'}` then the regularization power term :math:`p` must be :math:`2.0`, that is :math:`\text{‘Bxnl Basereg Pow'} = \texttt{'2.0'}`.
For further details see `Subproblems <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#subprob>`__.
Constraint: :math:`\text{‘Bxnl Basereg Type'} = \texttt{'NONE'}`, :math:`\texttt{'EXPAND-NVAR-DOF'}` or :math:`\texttt{'EXPAND-1-DOF'}`.
**'Bxnl Save Covariance Matrix'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates to the solver to store the covariance matrix into the handle.
If :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'YES'}` then the lower triangle part of the covariance matrix is stored in packed column order (see `the F07 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f07/f07intro.html#recomm_32>`__) into the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'COVARIANCE MATRIX'}` with :math:`{\textit{lrarr}} = \left(n_{\textit{var}}\times \left(n_{\textit{var}}+1\right)\right)/2`.
In the special case where :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'VARIANCE'}`, only the diagonal elements of the covariance matrix are stored in the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'VARIANCE'}` with :math:`{\textit{lrarr}} = n_{\textit{var}}`.
Similarly, if :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'HESSIAN'}` then the lower triangle part of the matrix :math:`H\left(x\right) = \nabla r\left(x\right)\left. \nabla r\left(x\right)\right.^\mathrm{T} = \left. J\left(x\right)\right.^\mathrm{T}J\left(x\right)` is stored in packed column order into the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'HESSIAN MATRIX'}` with :math:`{\textit{lrarr}} = \left(n_{\textit{var}}\times \left(n_{\textit{var}}+1\right)\right)/2`.
**Limitations:** If the number of enabled residuals is not greater than the number of enabled variables, or the pseudo-inverse of :math:`H\left(x\right)` could not be calculated, then the covariance matrix (variance vector) is not stored in the handle and will not be available.
For more information on how the covariance matrix is estimated, see :meth:`lsq_uncon_covariance`.
Constraint: :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'NO'}`, :math:`\texttt{'YES'}`, :math:`\texttt{'VARIANCE'}` or :math:`\texttt{'HESSIAN'}`.
**'Bxnl Stop Abs Tol Fun'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the error test, specifically, it sets the value of :math:`\epsilon_{\textit{abs}}^f` of equation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit1>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Abs Tol Fun' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Abs Tol Fun'} > 0`.
**'Bxnl Stop Abs Tol Grd'** : float
Default :math:`\text{} = 2.2\epsilon^{\frac{1}{3}}`
This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of :math:`\epsilon_{\textit{abs}}^g` of equation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Abs Tol Grd' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Abs Tol Grd'} > 0`.
**'Bxnl Stop Rel Tol Fun'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the error test, specifically, it sets the value of :math:`\epsilon_{\textit{rel}}^f` of equation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit1>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Rel Tol Fun' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Rel Tol Fun'} > 0`.
**'Bxnl Stop Rel Tol Grd'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of :math:`\epsilon_{\textit{rel}}^g` of equation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Rel Tol Grd' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Rel Tol Grd'} > 0`.
**'Bxnl Stop Step Tol'** : float
Default :math:`\text{} = 2\epsilon`
Specifies the stopping tolerance for the step length test, specifically, it sets the value for :math:`\epsilon_{\textit{step}}` of equation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stopcrit3>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Step Tol' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Under certain circumstances, e.g., when in doubt of the quality of the first - or second-order derivatives, in the event of the solver exiting with a successful step length test, it is recommended to verify that either the error or the gradient norm is acceptably small.
Constraint: :math:`\text{‘Bxnl Stop Step Tol'} > 0`.
**'Bxnl Reg Order'** : str
Default :math:`\text{} = \texttt{'AUTO'}`
This argument specifies the order of the regularization :math:`p` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:reg>`__ used when :math:`\text{‘Bxnl Glob Method'} = \texttt{'Reg'}`.
Some values for :math:`p` are restricted depending on the method chosen in 'Bxnl Nlls Method', see `Regularization <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#subprob_reg>`__ for more details.
Constraint: :math:`\text{‘Bxnl Reg Order'} = \texttt{'AUTO'}`, :math:`\texttt{'QUADRATIC'}` or :math:`\texttt{'CUBIC'}`.
**'Bxnl Glob Method'** : str
Default :math:`= \texttt{'TR'}`
This argument specifies the globalization method used to estimate the next step :math:`s_k`.
It also determines the class of subproblem to solve.
The trust region subproblem finds the step by minimizing the specified model withing a given radius.
On the other hand, when :math:`\text{‘Bxnl Glob Method'} = REG`, the problem is reformulated by adding an aditional regularization term and minimized in order to find the next step :math:`s_k`.
See `Subproblems <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#subprob>`__ for more details.
Constraint: :math:`\text{‘Bxnl Glob Method'} = \texttt{'TR'}` or :math:`\texttt{'REG'}`.
**'Bxnl Nlls Method'** : str
Default :math:`= \texttt{'GALAHAD'}`
This argument defines the method used to estimate the next step :math:`s_k` in :math:`x_{{k+1}} = x_k+s_k`.
It only applies to :math:`\text{‘Bxnl Model'} = \texttt{'GAUSS-NEWTON'}`, :math:`\texttt{'QUASI-NEWTON'}` or :math:`\texttt{'HYBRID'}`.
When the globalization technique chosen is trust region (:math:`\text{‘Bxnl Glob Method'} = TR`) the methods for 'Bxnl Nlls Method' available are Powell's dogleg method, a generalized eigenvalue method (AINT) Adachi `et al.` (2015), a variant of Moré--Sorensen's method, and GALAHAD's DTRS method.
Otherwise, when the globalization method chosen is via regularization (:math:`\text{‘Bxnl Glob Method'} = REG`) the methods available are comprised by a linear system solver and GALAHAD's DRQS method.
See `Subproblems <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#subprob>`__ for more details.
Constraint: :math:`\text{‘Bxnl Nlls Method'} = \texttt{'POWELL-DOGLEG'}`, :math:`\texttt{'AINT'}`, :math:`\texttt{'MORE-SORENSEN'}`, :math:`\texttt{'LINEAR SOLVER'}` or :math:`\texttt{'GALAHAD'}`.
**'Bxnl Model'** : str
Default :math:`= \texttt{'HYBRID'}`
This argument specifies which model is used to approximate the objective function and estimate the next point that reduces the error.
This is one of the most important options and should be chosen according to the problem characteristics.
The models are briefly described in `Models <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#models>`__.
Constraint: :math:`\text{‘Bxnl Model'} = \texttt{'GAUSS-NEWTON'}`, :math:`\texttt{'QUASI-NEWTON'}`, :math:`\texttt{'HYBRID'}` or :math:`\texttt{'TENSOR-NEWTON'}`.
**'Bxnl Tn Method'** : str
Default :math:`= \texttt{'MIN-1-VAR'}`
This argument specifies how to solve the subproblem and find the next step :math:`s_k` for the tensor Newton model, :math:`\text{‘Bxnl Model'} = \texttt{'TENSOR-NEWTON'}`.
The subproblems are solved using a range of regularization schemes.
See `Tensor Newton subproblem <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#tensor_newton>`__.
Constraint: :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`, :math:`\texttt{'MIN-1-VAR'}`, :math:`\texttt{'MIN-NVAR'}`, :math:`\texttt{'ADD-1-VAR'}` or :math:`\texttt{'ADD-NVAR'}`.
**'Bxnl Use Second Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates whether the weighted sum of residual Hessians are available through the call-back :math:`\mathrm{lsqhes}`.
If :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'NO'}` and the specified model in 'Bxnl Model' requires user-suppied second derivatives, then the solver will terminate with :math:`\mathrm{errno}` = 6.
Constraint: :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Bxnl Use Weights'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates whether to use a weighted nonlinear least square model.
If :math:`\text{‘Bxnl Use Weights'} = \texttt{'YES'}` then the weights :math:`w_i > 0,i = 1,\ldots,n_{\textit{res}}` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#eq:lsq11>`__ must be supplied by you via :meth:`handle_set_get_real`.
If weights are to be used, then all :math:`n_{\textit{res}}` elements must be provided, see `Residual Weights <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#residualweights>`__.
If the solver is expecting to use weights but they are not provided or have non-positive values, then the solver will terminate with :math:`\mathrm{errno}` = 11.
Constraint: :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Bxnl Iteration Limit'** : int
Default :math:`= 1000`
This argument specifies the maximum amount of major iterations the solver is alloted.
If this limit is reached, then the solver will terminate with :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘Bxnl Iteration Limit'} \geq 1`.
**'Bxnl Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘Bxnl Monitor Frequency'} > 0`, the user-supplied function :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘Bxnl Monitor Frequency'} \geq 0`.
**'Bxnl Print Header'** : int
Default :math:`= 30`
This argument defines, in number of iterations, the frequency with which to print the iteration log header.
Constraint: :math:`\text{‘Bxnl Print Header'} \geq 1`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as for 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timing.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If at the end of an iteration this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04gg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`3`)
Unsupported option combinations.
(`errno` :math:`3`)
Unsupported model and method chosen.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
There are no decision variables. :math:`\textit{nvar}` must be greater than zero.
(`errno` :math:`6`)
Exact second derivatives needed for tensor model.
(`errno` :math:`11`)
Data for residual weights not found or is invalid.
(`errno` :math:`21`)
The current starting point is unusable.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`18`)
Numerical difficulties encountered and solver was terminated.
(`errno` :math:`19`)
Iteration limit reached while solving a subproblem.
(`errno` :math:`19`)
Line Search failed.
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user-supplied function and recovery failed.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04gg-py2-py-notes:
**Notes**
``handle_solve_bxnl`` computes a solution :math:`x` to the nonlinear least squares problem
.. math::
\begin{array}{ll} \textit{minimize}_{{x \in ℝ^{n_{\textit{var}}}}} & f\left(x\right) = \frac{1}{2} \sum_{{i = 1}}^{n_{\textit{res}}} {\left[w_ir_i\left(x\right)\right]}^2 + \frac{\sigma }{p} \left\lVert x\right\rVert_2^p \\\text{subject to}& l_x \leq x \leq u_x \text{,} \end{array}
where :math:`r_i\left(x\right),i = 1,\ldots,n_{\textit{res}}`, are smooth nonlinear functions called residuals, :math:`w_i,i = 1,\ldots,n_{\textit{res}}` are weights (by default they are all defined to :math:`1`, see `Residual Weights <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#residualweights>`__ on how to change them), and the rightmost element represents the regularization term with argument :math:`\sigma \geq 0` and power :math:`p > 0` (by default the regularization term is not used, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#algdetails>`__ on how to enable it).
The constraint elements :math:`l_x` and :math:`u_x` are :math:`n_{\textit{var}}`-dimensional vectors defining the bounds on the variables.
Typically in a calibration or data fitting context, the residuals will be defined as the difference between the observed values :math:`y_i` at :math:`t_i` and the values provided by a nonlinear model :math:`\phi \left(t; x\right)`, i.e., :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right)`.
If these residuals (errors) follow a Gaussian distribution, then the values of the optimal parameter vector :math:`x^*` are the maximum likelihood estimates.
For a description of the various algorithms implemented for solving this problem see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#algdetails>`__.
It is also recommended that you read `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optclasses>`__.
``handle_solve_bxnl`` serves as a solver for problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
A nonlinear least square residual objective can be added by calling :meth:`handle_set_nlnls` and, optionally, (simple) box constraints can be defined by calling :meth:`handle_set_simplebounds`.
It should be noted that ``handle_solve_bxnl`` internally works with a dense representation of the residual Jacobian even if a sparse structure was defined in the call to :meth:`handle_set_nlnls`.
Once the problem is fully described, the handle may be passed to the solver ``handle_solve_bxnl``.
When the handle is not needed anymore, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
For more information refer to the NAG optimization modelling suite in `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__.
The algorithm is based on the trust region framework and its behaviour can be modified by various options (see :ref:`Other Parameters <e04gg-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
Several options might have significant impact on the performance of the solver.
Even though the defaults were chosen to suit the majority of anticipated problems, it is recommended that you experiment with the option settings to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ggf.html#algdetails>`__ and :ref:`Other Parameters <e04gg-py2-py-other_params>` for further details.
.. _e04gg-py2-py-references:
**References**
Adachi, S, Iwata, S, Nakatsukasa, Y, and Takeda, A, 2015, `Solving the trust region subproblem by a generalized eigenvalue problem`, Technical report, METR 2015-14., Mathematical Engineering, The University of Tokyo, https://www.keisu.t.u-tokyo.ac.jp/data/2015/METR15-14.pdf
Conn, A R, Gould, N I M and Toint, Ph L, 2000, `Trust Region Methods`, SIAM, Philadephia
Gould, N I M, Orban, D, and Toint, Ph L, 2003, `GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization`, ACM Transactions on Mathematical Software (TOMS) (29(4)), 353--372
Gould, N I M, Rees, T, and Scott, J A, 2017, `A higher order method for solving nonlinear least-squares problems`, Technical report, RAL-P-1027-010, RAL Library. STFC Rutherford Appleton Laboratory, http://www.numerical.rl.ac.uk/people/rees/pdf/RAL-P-2017-010.pdf
Kanzow, C, Yamashita, N, and Fukushima, M, 2004, `Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints`, Journal of Computational and Applied Mathematics (174), 375--397
Lanczos, C, 1956, `Applied Analysis`, 272--280, Prentice Hall, Englewood Cliffs, NJ, USA
Nielsen, H B, 1999, `Damping parameter in Marquadt’s Method`, Technical report TR IMM-REP-1999-05., Department of Mathematical Modelling, Technical University of Denmark, http://www2.imm.dtu.dk/documents/ftp/tr99/tr05_99.pdf
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, confun=None, congrd=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_nldf`` is a solver from the NAG optimization modelling suite for general nonlinear data-fitting problems with constraints.
Various loss and regularization functions are supported.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04gn-py2-py-doc:
For full information please refer to the NAG Library document for e04gn
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html
.. _e04gn-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_nldf``. It **must not** be changed between calls to the NAG optimization modelling suite.
**lsqfun** : callable (rx, inform) = lsqfun(x, nres, inform, data=None)
:math:`\mathrm{lsqfun}` must evaluate the value of the nonlinear residuals, :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right),i = 1,\ldots,n_{\mathrm{res}}`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residual vector, :math:`r\left(x\right)`, evaluated at :math:`x`.
**inform** : int
May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqgrd** : callable inform = lsqgrd(x, nres, rdx, inform, data=None)
:math:`\mathrm{lsqgrd}` evaluates the residual gradients, :math:`\nabla r_i\left(x\right)`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residual gradients, :math:`\nabla r_i\left(x\right)`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**rdx** : float, ndarray, shape :math:`\left(\textit{nnzrd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the vector containing the nonzero residual gradients evaluated at :math:`x`,
.. math::
\nabla r\left(x\right) = \left[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\ldots,\nabla r_{n_{\textit{res}}}\left(x\right)\right]\text{,}
where
.. math::
\nabla r_i\left(x\right) = \left[\frac{{\partial r_i\left(x\right)}}{{\partial x_1}},\ldots,\frac{{\partial r_i\left(x\right)}}{{\partial x_{n_{\textit{var}}}}}\right]^\mathrm{T}\text{.}
The elements must be stored in the same order as the defined sparsity pattern provided in the call to :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**confun** : None or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the values of the :math:`m_g`-element vector :math:`g_i\left(x\right)` of nonlinear constraint functions at a specified value of the :math:`n_{\textit{var}}`-element vector of :math:`x` variables.
If there are no nonlinear constraints then :math:`\mathrm{confun}` will never be called by ``handle_solve_nldf`` and it may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the constraint functions are to be evaluated.
**ncnln** : int
:math:`m_g`, the number of nonlinear constraints, as specified in an earlier call to :meth:`handle_set_nlnconstr`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**gx** : float, array-like, shape :math:`\left(\mathrm{ncnln}\right)`
The :math:`m_g` values of the nonlinear constraint functions at :math:`x`.
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{confun}`. Specifically, if the value is negative, then the value of :math:`\mathrm{gx}` will be discarded and the solver will either attempt to find a different trial point or terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, the solver will proceed normally.
**congrd** : None or callable inform = congrd(x, gdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{congrd}` must calculate the nonzero values of the sparse Jacobian of the nonlinear constraint functions, :math:`\frac{{\partial g_i}}{{\partial x}}`, at a specified value of the :math:`n_{\textit{var}}`-element vector of :math:`x` variables.
If there are no nonlinear constraints, :math:`\mathrm{congrd}` will never be called by ``handle_solve_nldf`` and :math:`\mathrm{congrd}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Jacobian of the constraint functions is to be evaluated.
**gdx** : float, ndarray, shape :math:`\left(\textit{nnzgd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by :math:`\textit{irowgd}` and :math:`\textit{icolgd}` in an earlier call to :meth:`handle_set_nlnconstr`. :math:`\mathrm{gdx}[\textit{i}-1]` will be the gradient :math:`\frac{{\partial g_j}}{{\partial x_k}}`, where :math:`j = {\textit{irowgd}}[\textit{i}-1]` and :math:`k = {\textit{icolgd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{congrd}`. Specifically, if the value is negative the solution of the current problem will terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'NLDF Monitor Frequency' (if the value is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are:
.. rst-class:: nag-rules-none nag-align-left
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value, :math:`f\left(x\right)` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:df>`__. |
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Loss function value, :math:`\sum_{{i = 1}}^{n_{\textit{res}}}\chi \left(r_i\left(x\right)\right)` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:df>`__.|
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Regularization term value, :math:`\sum_{{i = 1}}^{n_{\textit{var}}}\psi \left(x_i\right)` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:df>`__. |
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Solution optimality measure. |
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration:
.. rst-class:: nag-rules-none nag-align-left
+------------+-----------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations performed. |
+------------+-----------------------------------------------------------------------------------------------+
|:math:`1` |Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.|
+------------+-----------------------------------------------------------------------------------------------+
|:math:`2-99`|Reserved for future use. |
+------------+-----------------------------------------------------------------------------------------------+
.. _e04gn-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'NLDF Iteration Limit'** : int
Default :math:`= 10000000`
The maximum number of iterations to be performed by ``handle_solve_nldf``.
If this limit is reached, then the solver will terminate with :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘NLDF Iteration Limit'} \geq 1`.
**'NLDF Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘NLDF Monitor Frequency'} > 0`, the user-supplied function :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘NLDF Monitor Frequency'} \geq 0`.
**'NLDF Stop Tolerance'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This argument sets the value of :math:`\epsilon_{\textit{tol}}` which specifies the tolerance for the optimality measure.
When both loss function and regularization are differentiable, and with only simple bound constraints, the optimality measures are defined by `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit1>`__ and 'NLDF Stop Tolerance' is passed to the solver :meth:`handle_solve_bounds_foas` as 'FOAS Stop Tolerance'.
When any of the loss function or regularization is non-differentiable, or there presents linear, quadratic or general nonlinear constraint, the optimality measure is defined by `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#eqn5>`__ and 'NLDF Stop Tolerance' is passed to the solver :meth:`handle_solve_ipopt` as 'Stop Tolerance 1'.
Constraint: :math:`0\leq \epsilon_{\textit{tol}} < 1`.
**'NLDF Loss Function Type'** : str
Default :math:`=`'L2'
This argument sets the loss function type used in the objective function.
Constraint: :math:`\text{‘NLDF Loss Function Type'} = \texttt{'HUBER'}`, :math:`\texttt{'L2'}`, :math:`\texttt{'CAUCHY'}`, :math:`\texttt{'ATAN'}`, :math:`\texttt{'SMOOTHL1'}`, :math:`\texttt{'LINF'}`, :math:`\texttt{'L1'}` or :math:`\texttt{'QUANTILE'}`.
**'Reg Term Type'** : str
Default :math:`=`'OFF'
This argument sets the regularization function type used in the objective function.
**Note:** if there is no residual in the model, regularization will be turned off.
Constraint: :math:`\text{‘Reg Term Type'} = \texttt{'OFF'}`, :math:`\texttt{'L2'}`, :math:`\texttt{'L1'}` or :math:`\texttt{'ELASTIC NET'}`.
**'NLDF Huber Function Width'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{\textit{hub}}` defined in the Huber loss function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_huber>`__.
Constraint: :math:`\text{‘NLDF Huber Function Width'} > 0`.
**'NLDF Cauchy Function Sharpness'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{\textit{cau}}` defined in the Cauchy loss function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_cauchy>`__.
Constraint: :math:`\text{‘NLDF Cauchy Function Sharpness'} > 0`.
**'NLDF SmoothL1 Function Width'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{{\textit{sl}1}}` defined in the SmoothL1 loss function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_smoothl1>`__.
Constraint: :math:`\text{‘NLDF SmoothL1 Function Width'} > 0`.
**'NLDF Quantile Parameter'** : float
Default :math:`\text{} = 0.5`
Sets the parameter :math:`d_{\textit{qnt}}` defined in the Quantile loss function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_quantile>`__.
Constraint: :math:`0 < \text{‘NLDF Quantile Parameter'} < 1`.
**'Reg Coefficient'** : float
Default :math:`\text{} = 1.0`
Sets the regularization coefficient :math:`\rho` in the definition of the objective function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:df>`__. **Note:** if set to :math:`0`, regularization will be turned off.
Constraint: :math:`\text{‘Reg Coefficient'}\geq 0`.
**'NLDF Elastic Net Alpha'** : float
Default :math:`\text{} = 0.5`
Sets the regularization coefficient :math:`\alpha` in the definition of the elastic net regularization function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:enreg>`__.
**Note:** if set to :math:`0`, the regularization function reduces to the :math:`l_2`-norm.
In this case, only the regularization coefficient 'Reg Coefficient' is used, and the value of 'NLDF Elastic Net Beta' is ignored.
Constraint: :math:`\text{‘NLDF Elastic Net Alpha'}\geq 0`.
**'NLDF Elastic Net Beta'** : float
Default :math:`\text{} = 0.5`
Sets the regularization coefficient :math:`\beta` in the definition of the elastic net regularization function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#eq:enreg>`__.
**Note:** if set to :math:`0`, the regularization function reduces to the :math:`l_1`-norm.
In this case, only the regularization coefficient 'Reg Coefficient' is used, and the value of 'NLDF Elastic Net Alpha' is ignored.
Constraint: :math:`\text{‘NLDF Elastic Net Beta'}\geq 0`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as for 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the initialization of the options, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}` or :math:`\texttt{'X'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timing.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If at the end of an iteration this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04gn-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`-199`)
``handle_solve_nldf`` is not available in this implementation.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nres}` must match the current number of residuals defined in the :math:`\mathrm{handle}`.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{confun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{congrd}` function.
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`28`)
The solver terminated with not enough degrees of freedom.
(`errno` :math:`51`)
The solver detected an infeasible problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Problem was solved to an acceptable level; full accuracy was not achieved.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user function.
(`errno` :math:`28`)
The solver terminated after failure during line search.
(`errno` :math:`28`)
The solver terminated after an error in the step computation.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04gn-py2-py-notes:
**Notes**
``handle_solve_nldf`` solves a data-fitting problem of the form
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^{n_{\textit{var}}}}} & f\left(x\right) = \sum_{{i = 1}}^{n_{\textit{res}}} \chi \left(r_i\left(x\right)\right) + \rho \sum_{{i = 1}}^{n_{\textit{var}}} \psi \left(x_i\right) \\ \text{subject to} & l_g \leq g\left(x\right) \leq u_g \text{,} \\& \frac{1}{2} x^\mathrm{T} Q_ix + p_i^\mathrm{T}x + s_i \leq 0 \text{, }\quad 1 \leq i \leq m_Q \text{,} \\& l_B \leq Bx \leq u_B \text{,} \\& l_x \leq x \leq u_x \text{,} \end{array}
where
:math:`n_{\textit{var}}` is the number of decision variables,
:math:`m_g` is the number of the nonlinear constraints and :math:`g\left(x\right)`, :math:`l_g` and :math:`u_g` are :math:`m_g`-dimensional vectors,
:math:`m_Q` is the number of quadratic constraints,
:math:`m_B` is the number of the linear constraints and :math:`B` is a :math:`m_B\times n_{\textit{var}}` matrix, :math:`l_B` and :math:`u_B` are :math:`m_B`-dimensional vectors,
there are :math:`n_{\textit{var}}` box constraints and :math:`l_x` and :math:`u_x` are :math:`n_{\textit{var}}`-dimensional vectors.
Here,
:math:`x` is an :math:`n_{\textit{var}}`-dimensional vector representing the model parameters,
:math:`\chi` is the loss function,
:math:`\psi` is the regularization function,
:math:`\rho` is the regularization coefficient,
:math:`n_{\textit{res}}` is the number of residuals and :math:`r_i` is the :math:`i`\ th residual, which is defined as
.. math::
r_i\left(x\right) = y_i-\phi \left(t_i; x\right)\text{, }\quad i = 1,\ldots,n_{\textit{res}}
where :math:`\phi \left(t_i; x\right)` is the predicted value of the :math:`i`\ th data point, given :math:`x`.
For the :math:`i`\ th data point, :math:`y_i` and :math:`t_i` are the observed values of the independent and dependant variables respectively.
The available loss and regularization function types are summarized in Table [label omitted], where :math:`d` is the function parameter and :math:`I\left(L\right)` denotes an indicator function taking the value :math:`1` if the logical expression :math:`L` is true and :math:`0` otherwise.
Loss function and regularization types can be specified by options 'NLDF Loss Function Type' and 'Reg Term Type', respectively.
For example, set :math:`\text{‘NLDF Loss Function Type'} = \texttt{'LINF'}` and :math:`\text{‘Reg Term Type'} = \texttt{'L2'}` to use :math:`l_{\infty }`-norm loss function with :math:`l_2`-norm (Ridge) regularization.
See `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#algdetails>`__ for more details on the loss functions.
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Loss function |:math:`\chi \left(r_i\right)` |'NLDF Loss Function Type'|
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_2`-norm |:math:`r_i^2` |'L2' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_1`-norm |:math:`\left\lvert r_i\right\rvert` |'L1' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_{\infty }`-norm |:math:`\mathrm{max}_{{1\leq j\leq n_{\textit{res}}}}\left\lvert r_j\right\rvert /n_{\textit{res}}` |'LINF' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Huber (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_huber>`__) |:math:`\left\{\begin{array}{ll} 0.5 * r_i^2 & \text{if } \left\lvert r_i\right\rvert < d \\ d * \left(\left\lvert r_i\right\rvert -0.5*d\right) &\text{otherwise}\end{array}\right.`|'HUBER' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Cauchy (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_cauchy>`__) |:math:`\mathrm{ln}\left(1+\left(r_i/d\right)^2\right)` |'CAUCHY' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Atan |:math:`\arctan\left(r_i^2\right)` |'ATAN' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|SmoothL1 (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_smoothl1>`__)|:math:`\left\{\begin{array}{ll} 0.5 * r_i^2 / d & \text{if } \left\lvert r_i\right\rvert < d \\ \left\lvert r_i\right\rvert - 0.5 * d &\text{otherwise}\end{array}\right.` |'SMOOTHL1' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Quantile (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gnf.html#func_quantile>`__)|:math:`r_i*\left(d-I_{\left(r_i < 0\right)}\right)` |'QUANTILE' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Regularization |:math:`\psi \left(x_i\right)` |'Reg Term Type' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Lasso (:math:`l_1`-norm) |:math:`\left\lvert x_i\right\rvert` |'L1' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Ridge (:math:`l_2`-norm) |:math:`x_i^2` |'L2' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Elastic net |:math:`\alpha \left\lvert x_i\right\rvert +\beta x_i^2` |'ELASTIC NET' |
+-------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
``handle_solve_nldf`` serves as a solver for problems stored as a :math:`\mathrm{handle}`.
The :math:`\mathrm{handle}` points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), :meth:`handle_set_nlnls` can be used to add a model and define its residual sparsity structure. :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac` may be used to set or modify quadratic constraints.
Linear constraints :math:`l_B`, :math:`B`, :math:`u_B` are handled by :meth:`handle_set_linconstr`.
Variable box bounds :math:`l_x` and :math:`u_x` can be specified with :meth:`handle_set_simplebounds`, and :meth:`handle_set_nlnconstr` can set or modify nonlinear constraints.
Once the problem is fully described, the :math:`\mathrm{handle}` may be passed to the solver ``handle_solve_nldf``.
When the :math:`\mathrm{handle}` is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
Nonlinear Programming (NLP) solvers :meth:`handle_solve_bounds_foas` and :meth:`handle_solve_ipopt` are used as solver engines by ``handle_solve_nldf``, which defines the selected loss function and regularization, then transforms the problem into standard form that the NLP solvers allow.
For best performance, when the objective function :math:`f\left(x\right)` is differentiable and without any constraint other than simple bound constraints, :meth:`handle_solve_bounds_foas` is used.
For non-differentiable objective functions or cases where constraints other than simple variable bounds are present, :meth:`handle_solve_ipopt` is used.
See `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#algdetails>`__ and :meth:`handle_solve_ipopt` for more details on algorithmic details.
The algorithm behaviour can be modified by various options (see :ref:`Other Parameters <e04gn-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the :math:`\mathrm{handle}` by e.g., :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
Option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_nldf_ex.main`
"""
raise NotImplementedError
[docs]def lsq_uncon_quasi_deriv_easy(m, lsfun2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_quasi_deriv_easy`` is an easy-to-use quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. deprecated:: 28.3.0.0
``lsq_uncon_quasi_deriv_easy`` is deprecated.
Please use :meth:`handle_solve_bxnl` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04gy-py2-py-doc:
For full information please refer to the NAG Library document for e04gy
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gyf.html
.. _e04gy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_quasi_deriv_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the first derivatives calculated by :math:`\mathrm{lsfun2}` at the starting point and so is more likely to detect an error in your function if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04gy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04gy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_quasi_deriv_easy`` is similar to the function LSFDQ2 in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left({F\left(x\right)}\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.) You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`.
Before attempting to minimize the sum of squares, the algorithm checks the function for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04gy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv_easy(m, lsfun2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv_easy`` is an easy-to-use modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04gz-py2-py-doc:
For full information please refer to the NAG Library document for e04gz
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04gzf.html
.. _e04gz-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv_easy``.
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[i-1]` must be set to the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must be set to the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the first derivatives calculated by :math:`\mathrm{lsfun2}` at the starting point and so is more likely to detect any error in your functions if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04gz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04gz-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv_easy`` is similar to the function LSFDN2 in the NPL Algorithms NAG Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`.
Before attempting to minimize the sum of squares, the algorithm checks the function for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04gz-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def check_deriv(funct, x, data=None):
r"""
``check_deriv`` checks that a function for evaluating an objective function and its first derivatives produces derivative values which are consistent with the function values calculated.
.. _e04hc-py2-py-doc:
For full information please refer to the NAG Library document for e04hc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hcf.html
.. _e04hc-py2-py-parameters:
**Parameters**
**funct** : callable (iflag, fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function and its first derivatives at a given point. (The minimization functions mentioned in :ref:`Notes <e04hc-py2-py-notes>` gives you the option of resetting arguments of :math:`\mathrm{funct}` to cause the minimization process to terminate immediately. ``check_deriv`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
Will be set to :math:`2`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which :math:`F` and its derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you reset :math:`\mathrm{iflag}` to a negative number in :math:`\mathrm{funct}` and return control to ``check_deriv``, ``check_deriv`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fc** : float
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{fc}` must be set to the value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must be set to the coordinates of a suitable point at which to check the derivatives calculated by :math:`\mathrm{funct}`. 'Obvious' settings, such as :math:`0.0` or :math:`1.0`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. Similarly, it is preferable that no two elements of :math:`\mathrm{x}` should be the same.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**f** : float
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{f}` contains the value of the objective function :math:`F\left(x\right)` at the point given by you in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{g}[\textit{j}-1]` contains the value of the derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{funct}`, for :math:`\textit{j} = 1,2,\ldots,n`.
.. _e04hc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
(`errno` :math:`2`)
Large errors were found in the derivatives of :math:`F` computed by :math:`\mathrm{funct}`.
.. _e04hc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
Functions for minimizing a function of several variables may require you to supply a function to evaluate the objective function :math:`F\left(x_1, x_2, \ldots, x_n\right)` and its first derivatives. ``check_deriv`` is designed to check the derivatives calculated by such functions.
As well as the function to be checked ( :math:`\mathrm{funct}`), you must supply a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the check will be made.
Note that ``check_deriv`` checks functions of the form required for :meth:`bounds_mod_deriv_comp` and :meth:`bounds_mod_deriv2_comp`.
``check_deriv`` first calls :math:`\mathrm{funct}` to evaluate :math:`F` and its first derivatives :math:`g_{\textit{j}} = \frac{{\partial F}}{{\partial x_{\textit{j}}}}`, for :math:`\textit{j} = 1,2,\ldots,n` at :math:`x`.
The components of the user-supplied derivatives along two orthogonal directions (defined by unit vectors :math:`p_1` and :math:`p_2`, say) are then calculated; these will be :math:`g^\mathrm{T}p_1` and :math:`g^\mathrm{T}p_2` respectively.
The same components are also estimated by finite differences, giving quantities
.. math::
v_k = \frac{{F\left(x+hp_k\right)-F\left(x\right)}}{h}\text{, }\quad k = 1,2
where :math:`h` is a small positive scalar.
If the relative difference between :math:`v_1` and :math:`g^\mathrm{T}p_1` or between :math:`v_2` and :math:`g^\mathrm{T}p_2` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def check_deriv2(funct, h, x, lh, data=None):
r"""
``check_deriv2`` checks that a function for calculating second derivatives of an objective function is consistent with a function for calculating the corresponding first derivatives.
.. _e04hd-py2-py-doc:
For full information please refer to the NAG Library document for e04hd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hdf.html
.. _e04hd-py2-py-parameters:
**Parameters**
**funct** : callable (iflag, fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function and its first derivatives at a given point. (:meth:`bounds_mod_deriv2_comp` gives you the option of resetting arguments of :math:`\mathrm{funct}` to cause the minimization process to terminate immediately. ``check_deriv2`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
To :math:`\mathrm{funct}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and first derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you set :math:`\mathrm{iflag}` to some negative number in :math:`\mathrm{funct}` and return control to ``check_deriv2``, ``check_deriv2`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fc** : float
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{fc}` must be set to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**h** : callable (iflag, fhesl, fhesd) = h(iflag, xc, lh, fhesd, data=None)
:math:`\mathrm{h}` must evaluate the second derivatives of the function at a given point. (As with :math:`\mathrm{funct}`, an argument can be set to cause immediate termination.)
**Parameters**
**iflag** : int
Is set to a non-negative number.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the second derivatives of :math:`F\left(x\right)` are required.
**lh** : int
The length of the array :math:`\mathrm{fhesl}`.
**fhesd** : float, ndarray, shape :math:`\left(n\right)`
Contains the value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`. Functions written to take advantage of a similar feature of :meth:`bounds_mod_deriv2_comp` can be tested as they stand by ``check_deriv2``.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{h}` resets :math:`\mathrm{iflag}` to a negative number, ``check_deriv2`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fhesl** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
Unless :math:`\mathrm{iflag}` is reset, :math:`\mathrm{h}` must place the strict lower triangle of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesl}`, stored by rows, i.e., :math:`\mathrm{fhesl}[\left(\textit{i}-1\right)\left(\textit{i}-2\right)/2+\textit{j}-1]` must be set to the value of :math:`\frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,\textit{i}-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**fhesd** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{iflag}` is reset, :math:`\mathrm{h}` must place the diagonal elements of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesd}`, i.e., :math:`\mathrm{fhesd}[\textit{j}-1]` must be set to the value of :math:`\frac{{\partial^2F}}{{\partial x_{\textit{j}}^2}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must contain the coordinates of a suitable point at which to check the derivatives calculated by :math:`\mathrm{funct}`. 'Obvious' settings, such as :math:`0.0` or :math:`1.0`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. Similarly, it is advisable that no two elements of :math:`\mathrm{x}` should be the same.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**g** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{g}[\textit{j}-1]` contains the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{funct}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
Unless you set :math:`\mathrm{iflag}` negative in :math:`\mathrm{h}`, :math:`\mathrm{hesl}` contains the strict lower triangle of the second derivative matrix of :math:`F`, as evaluated by :math:`\mathrm{h}` at the point given in :math:`\mathrm{x}`, stored by rows.
**hesd** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in :math:`\mathrm{h}`, :math:`\mathrm{hesd}` contains the diagonal elements of the second derivative matrix of :math:`F`, as evaluated by :math:`\mathrm{h}` at the point given in :math:`\mathrm{x}`.
.. _e04hd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \mathrm{max}\left(1, {n\times \frac{{n-1}}{2}}\right)`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
(`errno` :math:`2`)
Large errors were found in the derivatives of :math:`F` computed by :math:`\mathrm{funct}`.
.. _e04hd-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
Functions for minimizing a function :math:`F\left(x_1, x_2, \ldots, x_n\right)` of the variables :math:`x_1,x_2,\ldots,x_n` may require you to provide a function to evaluate the second derivatives of :math:`F`. ``check_deriv2`` is designed to check the second derivatives calculated by such functions.
As well as the function to be checked (:math:`\mathrm{h}`), you must supply a function (:math:`\mathrm{funct}`) to evaluate the first derivatives, and a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the checks will be made.
Note that ``check_deriv2`` checks functions of the form required for :meth:`bounds_mod_deriv2_comp`.
``check_deriv2`` first calls functions :math:`\mathrm{funct}` and :math:`\mathrm{h}` to evaluate the first and second derivatives of :math:`F` at :math:`x`.
The user-supplied Hessian matrix (:math:`H`, say) is projected onto two orthogonal vectors :math:`y` and :math:`z` to give the scalars :math:`y^\mathrm{T}Hy` and :math:`z^\mathrm{T}Hz` respectively.
The same projections of the Hessian matrix are also estimated by finite differences, giving
.. math::
\begin{array}{ll}p = \left(y^\mathrm{T}g\left(x+hy\right)-y^\mathrm{T}g\left(x\right)\right)/h&\text{and}\\&q = \left(z^\mathrm{T}g\left(x+hz\right)-z^\mathrm{T}g\left(x\right)\right)/h\end{array}
respectively, where :math:`g\left(\right)` denotes the vector of first derivatives at the point in brackets and :math:`h` is a small positive scalar.
If the relative difference between :math:`p` and :math:`y^\mathrm{T}Hy` or between :math:`q` and :math:`z^\mathrm{T}Hz` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv2_comp(m, lsqfun, lsqhes, xtol, x, lsqmon=None, iprint=1, maxcal=None, eta=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv2_comp`` is a comprehensive modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First and second derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04he-py2-py-doc:
For full information please refer to the NAG Library document for e04he
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hef.html
.. _e04he-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_mod_deriv2_comp``. ``lsq_uncon_mod_deriv2_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**lsqhes** : callable (iflag, b) = lsqhes(iflag, fvec, xc, lb, data=None)
:math:`\mathrm{lsqhes}` must calculate the elements of the symmetric matrix
.. math::
B\left(x\right) = \sum_{{i = 1}}^mf_i\left(x\right)G_i\left(x\right)\text{,}
at any point :math:`x`, where :math:`G_i\left(x\right)` is the Hessian matrix of :math:`f_i\left(x\right)`. (As with :math:`\mathrm{lsqfun}`, there is the option of causing ``lsq_uncon_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Is set to a non-negative number.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The value of the residual :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`, so that the values of the :math:`f_{\textit{i}}` can be used in the calculation of the elements of :math:`\mathrm{b}`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the elements of :math:`\mathrm{b}` are to be evaluated.
**lb** : int
The length of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{lsqhes}` resets :math:`\mathrm{iflag}` to some negative number, ``lsq_uncon_mod_deriv2_comp`` will terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**b** : float, array-like, shape :math:`\left(\mathrm{lb}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{b}` must contain the lower triangle of the matrix :math:`\mathrm{b}[x-1]`, evaluated at the point :math:`x`, stored by rows. (The upper triangle is not required because the matrix is symmetric.) More precisely, :math:`\mathrm{b}[\textit{j}\left(\textit{j}-1\right)/2+\textit{k}-1]` must contain :math:`\sum_{{\textit{i} = 1}}^mf_{\textit{i}}\frac{{\partial^2f_{\textit{i}}}}{{\partial x_{\textit{j}}\partial x_{\textit{k}}}}` evaluated at the point :math:`x`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hef.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hef.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_mod_deriv2_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem. (If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called at the initial point before the singular values have been calculated, so the elements of :math:`\mathrm{s}` are set to zero for the first call of :math:`\mathrm{lsqmon}`.)
**igrade** : int
``lsq_uncon_mod_deriv2_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv2_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far. Thus :math:`\mathrm{nf}` gives the number of evaluations of the residuals and the Jacobian matrix.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
Specifies the frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_deriv2_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
This argument is present so as to enable you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_mod_deriv2_comp``. There will be an error exit (see :ref:`Exceptions <e04he-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_deriv2_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`). :math:`\mathrm{eta}` must lie in the range :math:`0.0\leq \mathrm{eta} < 1.0`, and specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_mod_deriv2_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made each iteration.
On balance it is usually more efficient to perform a low accuracy minimization.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.)
``lsq_uncon_mod_deriv2_comp`` will ensure that, for each iteration
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2\text{,}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``lsq_uncon_mod_deriv2_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv2_comp``.
**nf** : int
The number of times that the residuals and Jacobian matrix have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}`).
.. _e04he-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}` or :math:`\mathrm{lsqhes}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
.. _e04he-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv2_comp`` is essentially identical to the function LSQSDN in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply functions to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives and second derivatives at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise the second derivatives of the :math:`f_i\left(x\right)` are taken into account.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04he-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv2_easy(m, lsfun2, lshes2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv2_easy`` is an easy-to-use modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First and second derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04hy-py2-py-doc:
For full information please refer to the NAG Library document for e04hy
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04hyf.html
.. _e04hy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv2_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[i-1]` must be set to the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must be set to the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**lshes2** : callable b = lshes2(fvec, xc, lb, data=None)
You must supply this function to calculate the elements of the symmetric matrix
.. math::
B\left(x\right) = \sum_{{i = 1}}^mf_i\left(x\right)G_i\left(x\right)\text{,}
at any point :math:`x`, where :math:`G_i\left(x\right)` is the Hessian matrix of :math:`f_i\left(x\right)`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv2_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The value of the residual :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`, so that the values of the :math:`f_{\textit{i}}` can be used in the calculation of the elements of :math:`\mathrm{b}`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the elements of :math:`\mathrm{b}` are to be evaluated.
**lb** : int
The length of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**b** : float, array-like, shape :math:`\left(\mathrm{lb}\right)`
Must contain the lower triangle of the matrix :math:`B\left(x\right)`, evaluated at the point :math:`x`, stored by rows. (The upper triangle is not required because the matrix is symmetric.) More precisely, :math:`\mathrm{b}[\textit{j}\left(\textit{j}-1\right)/2+\textit{k}-1]` must contain :math:`\sum_{{\textit{i} = 1}}^mf_{\textit{i}}\frac{{\partial^2f_{\textit{i}}}}{{\partial x_{\textit{j}}\partial x_{\textit{k}}}}` evaluated at the point :math:`x`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks :math:`\mathrm{lsfun2}` and :math:`\mathrm{lshes2}` at the starting point and so is more likely to detect any error in your functions if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04hy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
(`errno` :math:`10`)
It is very likely that you have made an error in setting up the array :math:`\mathrm{b}` in :math:`\mathrm{lshes2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04hy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv2_easy`` is similar to the function LSSDN2 in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`, and a function to evaluate the elements of the second derivative term of the Hessian matrix of :math:`F\left(x\right)`.
Before attempting to minimize the sum of squares, the algorithm checks the functions for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04hy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def bounds_bobyqa_func(objfun, npt, x, bl, bu, rhobeg, rhoend, maxcal, monfun=None, data=None):
r"""
``bounds_bobyqa_func`` is an easy-to-use algorithm that uses methods of quadratic approximation to find a minimum of an objective function :math:`F` over :math:`\mathbf{x} \in R^n`, subject to fixed lower and upper bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`.
Derivatives of :math:`F` are not required.
The function is intended for functions that are continuous and that have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
Efficiency is maintained for large :math:`n`.
.. deprecated:: 27.0.0.0
``bounds_bobyqa_func`` is deprecated.
Please use :meth:`handle_solve_dfno` and :meth:`handle_solve_dfno_rcomm` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04jc-py2-py-doc:
For full information please refer to the NAG Library document for e04jc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jcf.html
.. _e04jc-py2-py-parameters:
**Parameters**
**objfun** : callable f = objfun(x, data=None)
:math:`\mathrm{objfun}` must evaluate the objective function :math:`F` at a specified vector :math:`\mathbf{x}`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`\mathbf{x}`, the vector at which the objective function is to be evaluated.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**f** : float
Must be set to the value of the objective function at :math:`\mathbf{x}`.
**npt** : int
:math:`m`, the number of interpolation conditions imposed on the quadratic approximation at each iteration.
`Suggested value`: :math:`\mathrm{npt} = 2\times n_r+1`, where :math:`n_r` denotes the number of non-fixed variables.
**x** : float, array-like, shape :math:`\left(n\right)`
An estimate of the position of the minimum. If any component is out-of-bounds it is replaced internally by the bound it violates.
**bl** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bl}` must contain the fixed vector of lower bounds, :math:`\ell`.
**bu** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bu}` must contain the fixed vector of upper bounds, :math:`\mathbf{u}`.
**rhobeg** : float
An initial lower bound on the value of the trust region radius.
`Suggested value`: :math:`\mathrm{rhobeg}` should be about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`\mathrm{x}` of length :math:`\mathrm{rhobeg}` along each coordinate direction.
**rhoend** : float
A final lower bound on the value of the trust region radius.
`Suggested value`: :math:`\mathrm{rhoend}` should indicate the absolute accuracy that is required in the final values of the variables.
**maxcal** : int
The maximum permitted number of calls to :math:`\mathrm{objfun}`.
**monfun** : None or callable monfun(nf, x, f, rho, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monfun}` may be used to monitor the optimization process.
It is invoked every time a new trust region radius is chosen.
If no monitoring is required, :math:`\mathrm{monfun}` may be **None**.
**Parameters**
**nf** : int
The cumulative number of calls made to :math:`\mathrm{objfun}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The current best point.
**f** : float
The value of :math:`\mathrm{objfun}` at :math:`\mathrm{x}`.
**rho** : float
A lower bound on the current trust region radius.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}` is the position of the minimum.
**f** : float
The function value at the lowest point found (:math:`\mathrm{x}`).
**nf** : int
Unless :math:`\mathrm{errno}` = 1 or ``NagMemoryError`` on exit, the total number of calls made to :math:`\mathrm{objfun}`.
.. _e04jc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and :math:`\mathrm{npt} = \langle\mathit{\boldsymbol{value}}\rangle` on entry.
Constraint: :math:`n_r+2\leq \mathrm{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
(`errno` :math:`1`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds.
Constraint: :math:`n_r\geq 2`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{bl}[i-1]\neq \mathrm{bu}[i-1]` in coordinate :math:`i`, :math:`\mathrm{bu}[i-1]-\mathrm{bl}[i-1]\geq 2\times \mathrm{rhobeg}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhobeg} > 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhoend} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhoend}\geq \textit{macheps}`, where :math:`\textit{macheps} = \texttt{machine.precision}\left(\right)`, the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{rhoend} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhoend}\leq \mathrm{rhobeg}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`2`)
The function evaluations limit was reached: :math:`\mathrm{objfun}` has been called :math:`\mathrm{maxcal}` times.
(`errno` :math:`3`)
The predicted reduction in a trust region step was non-positive. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`4`)
A rescue procedure has been called in order to correct damage from rounding errors when computing an update to a quadratic approximation of :math:`F`, but no further progess could be made. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
**NagCallbackTerminateWarning**
(`errno` :math:`5`)
User-supplied objective function requested termination.
(`errno` :math:`5`)
User-supplied monitoring function requested termination.
.. _e04jc-py2-py-notes:
**Notes**
``bounds_bobyqa_func`` is applicable to problems of the form:
.. math::
\textit{minimize}_{{\mathbf{x} \in R^n}}F\left(\mathbf{x}\right)\quad \text{ subject to }\quad \ell \leq \mathbf{x}\leq \mathbf{u}\quad \text{ and }\quad \ell \leq \mathbf{u}\text{,}
where :math:`F` is a nonlinear scalar function whose derivatives may be unavailable, and where the bound vectors are elements of :math:`R^n`.
Relational operators between vectors are interpreted elementwise.
Fixing variables (that is, setting :math:`\ell_i = u_i` for some :math:`i`) is allowed in ``bounds_bobyqa_func``.
You must supply a function to calculate the value of :math:`F` at any given point :math:`\mathbf{x}`.
The method used by ``bounds_bobyqa_func`` is based on BOBYQA, the method of Bound Optimization BY Quadratic Approximation described in Powell (2009).
In particular, each iteration of ``bounds_bobyqa_func`` generates a quadratic approximation :math:`Q` to :math:`F` that agrees with :math:`F` at :math:`m` automatically chosen interpolation points.
The value of :math:`m` is a constant prescribed by you.
Updates to the independent variables mostly occur from approximate solutions to trust region subproblems, using the current quadratic model.
.. _e04jc-py2-py-references:
**References**
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
"""
raise NotImplementedError
[docs]def handle_solve_dfno(handle, x, objfun=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_dfno`` is a forward communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFNO) for small to medium-scale nonlinear problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04jd-py2-py-doc:
For full information please refer to the NAG Library document for e04jd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jdf.html
.. _e04jd-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfno``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` calculates the value of the objective function :math:`f\left(x\right)` at a specified point :math:`x`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objfun}` will never be called by ``handle_solve_dfno`` and may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
May be used to indicate that the requested objective value could not be computed. Specifically, it can be set to a negative value:
:math:`\mathrm{inform} = -1`
The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 17.
:math:`\mathrm{inform} = -2`
The solver will cleanly exit with :math:`\mathrm{errno}` = 20 and return the best available point as well as the solve statistics.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the 'DFO Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in the main argument :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+----------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+---------------------+----------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region.|
+---------------------+----------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+---------------------+----------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+---------------------+----------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+----------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04jd-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFNO Detect Unbounded'** : str
Default :math:`= {\texttt{'YES'}}`
The solver can try to detect whether the problem is unbounded.
This option allows you to turn on or off the unboundedness detection heuristic.
Constraint: :math:`\text{‘DFNO Detect Unbounded'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'DFNO Objective Limit'** : float
Default :math:`\text{} = {-\infty }`
This option sets an additional convergence criterion.
The solver will stop if it finds a point for which the function value is lower than this parameter.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function if :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jdf.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model.
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04jd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is different from the total set by 'DFO Number Interp Points'.
Growing the interpolation set is not supported for this solver.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points could not be provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
The solver failed at the model building phase and the maximum number of restarts was reached. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`20`)
User requested termination after a call to the objective function.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
(`errno` :math:`54`)
The problem seems to be unbounded.
The unboundedness detection heuristic can be turned off with the option 'DFNO Detect Unbounded'.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04jd-py2-py-notes:
**Notes**
``handle_solve_dfno`` is aimed at minimizing a nonlinear objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & {f\left(x\right)} \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here :math:`f` is a smooth nonlinear function and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
``handle_solve_dfno`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnobj` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfno`` always assumes that the gradient of the objective is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnobj` will have no effect.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jdf.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04jd-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFNO (Derivative-free Nonlinear Optimization).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfno`` is the same as the one used by :meth:`handle_solve_dfno_rcomm`. ``handle_solve_dfno`` serves as a forward communication interface to the derivative-free solver for nonlinear objective functions.
.. _e04jd-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfno_rcomm(handle, irevcm, x, f, io_manager=None):
r"""
``handle_solve_dfno_rcomm`` is a reverse communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFNO) for small to medium-scale nonlinear problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04je-py2-py-doc:
For full information please refer to the NAG Library document for e04je
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jef.html
.. _e04je-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfno_rcomm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**irevcm** : int
Does not need to be set on the first call of ``handle_solve_dfno_rcomm``. On subsequent calls, :math:`\mathrm{irevcm}` must be set to a positive integer if all the required function evaluations have been correctly provided in :math:`\mathrm{f}`. Otherwise, if a problem occurred during a monitoring step or while providing objective values, it is possible to set it to a negative value:
:math:`\mathrm{irevcm} = -1`
If function evaluations were required, the solver will attempt a rescue procedure and request an alternative point. If no function were required (monitoring step), the solver will stop with :math:`\mathrm{errno}` = 20.
:math:`\mathrm{irevcm} \leq -2`
The solver will cleanly exit and return the best available point as a well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
The first column contains :math:`x_0`, the initial estimates of the variables :math:`x`.
**f** : float, array-like, shape :math:`\left(\textit{maxeval}\right)`
Does not need to be set on the first call to ``handle_solve_dfno_rcomm``.
If :math:`\mathrm{irevcm} = 1` after the last call of ``handle_solve_dfno_rcomm``, the first :math:`\mathrm{neval}` elements must contain the function values of the requested points.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**irevcm** : int
Indicates what action is to be performed before the next call to ``handle_solve_dfno_rcomm``.
:math:`\mathrm{irevcm} = 0`
Final exit of the solver.
:math:`\mathrm{irevcm} = 1`
:math:`\mathrm{neval}` objective evaluations are required.
:math:`\mathrm{irevcm} = 2`
Monitoring step, no evaluation is required, :math:`\mathrm{x}` and :math:`\mathrm{f}` contain the best evaluation of the objective yet.
**neval** : int
Indicates the number of objective evaluations required for the next call of ``handle_solve_dfno_rcomm`` in :math:`\mathrm{f}`. The coordinates of the points to evaluate are provided in the first :math:`\mathrm{neval}` columns of :math:`\mathrm{x}`.
**x** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the best computed estimate of the solution.
If :math:`\mathrm{irevcm} = 1`, the first :math:`\mathrm{neval}` columns contain the coordinates of the points to evaluate.
**f** : float, ndarray, shape :math:`\left(\textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first element contains the function value of the best computed point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+----------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+---------------------+----------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region.|
+---------------------+----------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+---------------------+----------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+---------------------+----------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+----------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04je-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFNO Detect Unbounded'** : str
Default :math:`= {\texttt{'YES'}}`
The solver can try to detect whether the problem is unbounded.
This option allows you to turn on or off the unboundedness detection heuristic.
Constraint: :math:`\text{‘DFNO Detect Unbounded'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'DFNO Objective Limit'** : float
Default :math:`\text{} = {-\infty }`
This option sets an additional convergence criterion.
The solver will stop if it finds a point for which the function value is lower than this parameter.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, the solver will stop at the end of every :math:`i`\ th step for monitoring purposes. ``handle_solve_dfno_rcomm`` needs to be called again to continue the optimization.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jef.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+2\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model.
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04je-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{maxeval} > 0`.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the first call of the function, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is different from the total set by 'DFO Number Interp Points'.
Growing the interpolation set is not supported for this solver.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points could not be provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
The solver failed at the model building phase and the maximum number of restarts was reached. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
(`errno` :math:`54`)
The problem seems to be unbounded.
The unboundedness detection heuristic can be turned off with the option 'DFNO Detect Unbounded'.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`20`)
User requested termination during an objective evaluation step.
.. _e04je-py2-py-notes:
**Notes**
``handle_solve_dfno_rcomm`` uses reverse communication for function evaluations and monitoring steps.
Every time the solver requires an evaluation of the objective function, it pauses its progress, exits and waits for the function to be called again with the objective value provided in the argument :math:`\mathrm{f}`.
``handle_solve_dfno_rcomm`` is aimed at minimizing a nonlinear objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & {f\left(x\right)} \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here :math:`f` is a smooth nonlinear function and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
``handle_solve_dfno_rcomm`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnobj` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfno_rcomm`` always assumes that the gradient of the objective is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnobj` will have no effect.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jef.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04je-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFNO (Derivative-free Nonlinear Optimization).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfno_rcomm`` is the same as the one used by :meth:`handle_solve_dfno`. ``handle_solve_dfno_rcomm`` serves as a reverse communication interface to the derivative-free solver for nonlinear objective functions.
.. _e04je-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
"""
raise NotImplementedError
[docs]def bounds_quasi_func_easy(ibound, funct1, bl, bu, x, liw=None, lw=None, data=None):
r"""
``bounds_quasi_func_easy`` is an easy-to-use quasi-Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds of the independent variables :math:`x_1,x_2,\ldots,x_n`, using function values only.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04jy-py2-py-doc:
For full information please refer to the NAG Library document for e04jy
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04jyf.html
.. _e04jy-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used.
It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct1** : callable fc = funct1(xc, data=None)
You must supply :math:`\mathrm{funct1}` to calculate the value of the function :math:`F\left(x\right)` at any point :math:`x`.
It should be tested separately before being used with ``bounds_quasi_func_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function value is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_quasi_func_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_quasi_func_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to an estimate of the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**liw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`n+2`.
The dimension of the array :math:`\mathrm{iw}`.
**lw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`\max\left({n\times \left(n-1\right) /2+12\times n},13\right)`.
The dimension of the array :math:`\mathrm{w}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_quasi_func_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_quasi_func_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**iw** : int, ndarray, shape :math:`\left(\mathrm{liw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, the first :math:`\textit{n}` elements of :math:`\mathrm{iw}` contain information about which variables are currently on their bounds and which are free. Specifically, if :math:`x_i` is:
- fixed on its upper bound, :math:`\mathrm{iw}[i-1]` is :math:`-1`;
- fixed on its lower bound, :math:`\mathrm{iw}[i-1]` is :math:`-2`;
- effectively a constant (i.e., :math:`l_j = u_j`), :math:`\mathrm{iw}[i-1]` is :math:`-3`;
- free, :math:`\mathrm{iw}[i-1]` gives its position in the sequence of free variables.
In addition, :math:`\mathrm{iw}[n]` contains the number of free variables (i.e., :math:`n_z`).
The rest of the array is used as workspace.
**w** : float, ndarray, shape :math:`\left(\mathrm{lw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, :math:`\mathrm{w}[i-1]` contains a finite difference approximation to the :math:`\textit{i}`\ th element of the projected gradient vector :math:`g_z`, for :math:`\textit{i} = 1,2,\ldots,n`. In addition, :math:`\mathrm{w}[n]` contains an estimate of the condition number of the projected Hessian matrix (i.e., :math:`k`). The rest of the array is used as workspace.
.. _e04jy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{liw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{liw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, either :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`, or :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`2`)
There have been :math:`400\times n` calls to :math:`\mathrm{funct1}`.
(`errno` :math:`4`)
An overflow occurred during computation.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct1}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
One of the forward differences was negative.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been met but a lower point could not be found and the algorithm has failed.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04jy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_quasi_func_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}F\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when derivatives of :math:`F\left(x\right)` are unavailable.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the value of :math:`F\left(x\right)` at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the gradient and the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
An attempt is made to verify that the final point is a minimum.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The projected gradient vector :math:`g_z`, whose elements are finite difference approximations to the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
A unit lower triangular matrix :math:`L` and a diagonal matrix :math:`D` (both of dimension :math:`n_z`), such that :math:`LDL^\mathrm{T}` is a positive definite approximation of the matrix of second derivatives with respect to the free variables (i.e., the projected Hessian) are also held.
The equations
.. math::
LDL^\mathrm{T}p_z = -g_z
are solved to give a search direction :math:`p_z`, which is expanded to an :math:`n`-vector :math:`p` by an insertion of appropriate zero elements.
Then :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; :math:`x` is replaced by :math:`x+\alpha p`, and the matrices :math:`L` and :math:`D` are updated so as to be consistent with the change produced in the estimated gradient by the step :math:`\alpha p`.
If any variable actually reaches a bound during the search along :math:`p`, it is fixed and :math:`n_z` is reduced for the next iteration.
Most iterations calculate :math:`g_z` using forward differences, but central differences are used when they seem necessary.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace provided that this is practicable.
When it is not, or when the stronger convergence criteria are already satisfied, then, if one or more Lagrange multiplier estimates are close to zero, a slight perturbation is made in the values of the corresponding variables in turn until a lower function value is obtained.
The normal algorithm is then resumed from the perturbed point.
If a saddle point is suspected, a local search is carried out with a view to moving away from the saddle point.
A local search is also performed when a point is found which is thought to be a constrained minimum.
.. _e04jy-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv_comp(funct, monit, eta, ibound, bl, bu, x, lh, iprint=1, maxcal=None, xtol=0.0, delta=0.0, stepmx=100000.0, data=None, io_manager=None):
r"""
``bounds_mod_deriv_comp`` is a comprehensive modified Newton algorithm for finding:
- an unconstrained minimum of a function of several variables;
- a minimum of a function of several variables subject to fixed upper and/or lower bounds on the variables.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04kd-py2-py-doc:
For full information please refer to the NAG Library document for e04kd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kdf.html
.. _e04kd-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at a specified point. (However, if you do not wish to calculate :math:`F` or its first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``bounds_mod_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Will have been set to :math:`1` or :math:`2`. The value :math:`1` indicates that only the first derivatives of :math:`F` need be supplied, and the value :math:`2` indicates that both :math:`F` itself and its first derivatives must be calculated.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the :math:`\frac{{\partial F}}{{\partial x_j}}`, or :math:`F` and the :math:`\frac{{\partial F}}{{\partial x_j}}`, are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Unless :math:`\mathrm{iflag} = 1` on entry, :math:`\mathrm{funct}` must set :math:`\mathrm{fc}` to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{funct}` must set :math:`\mathrm{gc}[\textit{j}-1]` to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**monit** : callable monit(xc, fc, gc, istate, gpjnrm, cond, posdef, niter, nf, data=None)
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{monit}` which is suitable for monitoring the minimization process. :math:`\mathrm{monit}` must not change the values of any of its arguments.
If :math:`\mathrm{iprint} < 0`, a :math:`\mathrm{monit}` with the correct argument list must still be supplied, although it will not be called.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fc** : float
The value of :math:`F\left(x\right)` at the current point :math:`x`.
**gc** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently fixed on their bounds and which are free.
If :math:`\mathrm{istate}[j-1]` is negative, :math:`x_j` is currently:
- fixed on its upper bound if :math:`\mathrm{istate}[j-1] = -1`
- fixed on its lower bound if :math:`\mathrm{istate}[j-1] = -2`
- effectively a constant (i.e., :math:`l_j = u_j`) if :math:`\mathrm{istate}[j-1] = -3`
If :math:`\mathrm{istate}[j-1]` is positive, its value gives the position of :math:`x_j` in the sequence of free variables.
**gpjnrm** : float
The Euclidean norm of the current projected gradient vector :math:`g_z`.
**cond** : float
The ratio of the largest to the smallest elements of the diagonal factor :math:`D` of the approximated projected Hessian matrix. This quantity is usually a good estimate of the condition number of the projected Hessian matrix. (If no variables are currently free, :math:`\mathrm{cond}` is set to zero.)
**posdef** : bool
Specifies :math:`\mathbf{True}` or :math:`\mathbf{False}` according to whether or not the approximation to the second derivative matrix for the current subspace, :math:`H`, is positive definite.
**niter** : int
The number of iterations (as outlined in :ref:`Notes <e04kd-py2-py-notes>`) which have been performed by ``bounds_mod_deriv_comp`` so far.
**nf** : int
The number of evaluations of :math:`F\left(x\right)` so far, i.e., the number of calls of :math:`\mathrm{funct}` with :math:`\mathrm{iflag}` set to :math:`2`. Each such call of :math:`\mathrm{funct}` also calculates the first derivatives of :math:`F`. (In addition to these calls monitored by :math:`\mathrm{nf}`, :math:`\mathrm{funct}` is called with :math:`\mathrm{iflag}` set to :math:`1` not more than :math:`\textit{n}` times per iteration.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**eta** : float
Every iteration of ``bounds_mod_deriv_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x+\alpha p\right)` with respect to :math:`\alpha`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations (and hence the number of calls of :math:`\mathrm{funct}` to estimate the second derivatives), they will tend to increase the number of calls of :math:`\mathrm{funct}` needed for each linear minimization.
On balance, it is usually more efficient to perform a low accuracy linear minimization when :math:`n` is small and a high accuracy minimization when :math:`n` is large.
`Suggested value`:
:math:`\mathrm{eta} = 0.5` if :math:`1 < n < 10`
:math:`\mathrm{eta} = 0.1` if :math:`10\leq n\leq 20`
:math:`\mathrm{eta} = 0.01` if :math:`n > 20`
**If** :math:`n = 1` **, eta should be set to** :math:`0.0` (also when the problem is effectively one-dimensional even though :math:`n > 1`; i.e., if for all except one of the variables the lower and upper bounds are equal).
**ibound** : int
Indicates whether the problem is unconstrained or bounded. If there are bounds on the variables, :math:`\mathrm{ibound}` can be used to indicate whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If the variables are bounded and you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If the problem is unconstrained.
:math:`\mathrm{ibound} = 2`
If the variables are bounded, but all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If all the variables are bounded, and :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
:math:`\mathrm{ibound} = 4`
If the problem is unconstrained. (The :math:`\mathrm{ibound} = 4` option is provided for consistency with other functions. In ``bounds_mod_deriv_comp`` it produces the same effect as :math:`\mathrm{ibound} = 1\text{.}`)
**bl** : float, array-like, shape :math:`\left(n\right)`
The fixed lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to a large negative number, e.g., :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv_comp`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bl}` will be initialized by ``bounds_mod_deriv_comp``.
**bu** : float, array-like, shape :math:`\left(n\right)`
The fixed upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any variable, the corresponding :math:`\mathrm{bu}[j-1]` should be set to a large positive number, e.g., :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv_comp`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bu}` will be initialized by ``bounds_mod_deriv_comp``.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**iprint** : int, optional
The frequency with which :math:`\mathrm{monit}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{monit}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``bounds_mod_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{monit}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{monit}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
The maximum permitted number of evaluations of :math:`F\left(x\right)`, i.e., the maximum permitted number of calls of :math:`\mathrm{funct}` with :math:`\mathrm{iflag}` set to :math:`2`. It should be borne in mind that, in addition to the calls of :math:`\mathrm{funct}` which are limited directly by :math:`\mathrm{maxcal}`, there will be calls of :math:`\mathrm{funct}` (with :math:`\mathrm{iflag}` set to :math:`1`) to evaluate only first derivatives.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that :math:`\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)` where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus, and if :math:`\mathrm{xtol}` is set to :math:`10^{-5}`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kdf.html#accuracy>`__.)
If the problem is scaled as described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kdf.html#fcomments2>`__ and :math:`\epsilon` is the machine precision, then :math:`\sqrt{\epsilon }` is probably the smallest reasonable choice for :math:`\mathrm{xtol}`.
This is because, normally, to machine accuracy, :math:`F\left(x+\sqrt{\epsilon }e_j\right) = F\left(x\right)`, for any :math:`j` where :math:`e_j` is the :math:`j`\ th column of the identity matrix.
If you set :math:`\mathrm{xtol}` to :math:`0.0` (or any positive value less than :math:`\epsilon`), ``bounds_mod_deriv_comp`` will use :math:`10.0\times \sqrt{\epsilon }` instead of :math:`\mathrm{xtol}`.
**delta** : float, optional
The differencing interval to be used for approximating the second derivatives of :math:`F\left(x\right)`. Thus, for the finite difference approximations, the first derivatives of :math:`F\left(x\right)` are evaluated at points which are :math:`\mathrm{delta}` apart. If :math:`\epsilon` is the machine precision, :math:`\sqrt{\epsilon }` will usually be a suitable setting for :math:`\mathrm{delta}`. If you set :math:`\mathrm{delta}` to :math:`0.0` (or to any positive value less than :math:`\epsilon`), ``bounds_mod_deriv_comp`` will automatically use :math:`\sqrt{\epsilon }` as the differencing interval.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency a slight overestimate is preferable.)
``bounds_mod_deriv_comp`` will ensure that, for each iteration,
.. math::
\sqrt{\sum_{{j = 1}}^n\left[x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right]^2}\leq \mathrm{stepmx}\text{,}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``bounds_mod_deriv_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can also help to avoid possible overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bl}[0] = \mathrm{bl}[1] = \cdots = \mathrm{bl}[n-1] = 0.0`.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bu}[0] = \mathrm{bu}[1] = \cdots = \mathrm{bu}[n-1] = 10^6`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04kd-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv_comp`` (see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kdf.html#accuracy>`__).
**hesd** : float, ndarray, shape :math:`\left(n\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04kd-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv_comp`` (see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kdf.html#accuracy>`__).
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently on their bounds and which are free. If :math:`\mathrm{istate}[j-1]` is:
- equal to :math:`-1`, :math:`x_j` is fixed on its upper bound;
- equal to :math:`-2`, :math:`x_j` is fixed on its lower bound;
- equal to :math:`-3`, :math:`x_j` is effectively a constant (i.e., :math:`l_j = u_j`);
- positive, :math:`\mathrm{istate}[j-1]` gives the position of :math:`x_j` in the sequence of free variables.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The first derivative vector corresponding to the final point given in :math:`\mathrm{x}`. The components of :math:`\mathrm{g}` corresponding to free variables should normally be close to zero.
.. _e04kd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 4`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{delta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{delta}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal}` function evaluations.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
No further progress can be made.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
.. _e04kd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv_comp`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n\text{.}
Special provision is made for unconstrained minimization (i.e., problems which actually have no bounds on the :math:`x_j`), problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
It is possible to specify that a particular :math:`x_j` should be held constant.
You must supply a starting point, and a :math:`\mathrm{funct}` to calculate the value of :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from their bounds.
The vector :math:`g_z`, whose elements are the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
The matrix of second derivatives with respect to the free variables, :math:`H`, is estimated by finite differences. (Note that :math:`g_z` and :math:`H` are both of dimension :math:`n_z`.) The equations
.. math::
\left(H+E\right)p_z = -g_z
are solved to give a search direction :math:`p_z`. (The matrix :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`p_z` is then expanded to an :math:`n`-vector :math:`p` by the insertion of appropriate zero elements, :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; and :math:`x` is replaced by :math:`x+\alpha p`. (If a saddle point is found, a special search is carried out so as to move away from the saddle point.) If any variable actually reaches a bound, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace until the stronger convergence criteria are satisfied.
If at this point there are no negative or near-zero Lagrange multiplier estimates, the process is terminated.
If you specify that the problem is unconstrained, ``bounds_mod_deriv_comp`` sets the :math:`l_j` to :math:`{-10^6}` and the :math:`u_j` to :math:`10^6`.
Thus, provided that the problem has been sensibly scaled, no bounds will be encountered during the minimization process and ``bounds_mod_deriv_comp`` will act as an unconstrained minimization algorithm.
.. _e04kd-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
Gill, P E and Murray, W, 1974, `Newton-type methods for unconstrained and linearly constrained optimization`, Math. Programming (7), 311--350
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def handle_solve_bounds_foas(handle, x, objfun=None, objgrd=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_bounds_foas`` is a solver from the NAG optimization modelling suite for bound-constrained large-scale Nonlinear Programming (NLP) problems.
It is a first-order active-set method (FOAS) that has low memory requirements and thus is suitable for very large-scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04kf-py2-py-doc:
For full information please refer to the NAG Library document for e04kf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html
.. _e04kf-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_bounds_foas``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` must calculate the value of the nonlinear objective function :math:`f\left(x\right)` at a specified point :math:`x`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objfun}` will never be called by ``handle_solve_bounds_foas`` and may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
In some cases, it is known beforehand that the evaluations of the objective function and its gradient are required at the same point :math:`x`, in such cases, :math:`\mathrm{inform} = 1`.
This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient; the objective function is always evaluated before the objective gradient.
This notification parameter may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
May be used to indicate that the function cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. Returning NaN or :math:`\pm \infty` in :math:`\mathrm{fx}` has the same effect. The algorithm will try to recover if the objective cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 25.
**objgrd** : None or callable inform = objgrd(x, fdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objgrd}` must calculate the values of the nonlinear objective function gradient :math:`\frac{{\partial f}}{{\partial x}}` at a specified point :math:`x`.
Every call to :math:`\mathrm{objgrd}` is preceded by a call to :math:`\mathrm{objfun}` at the same point, if this is known in advance, both functions will be notified via :math:`\mathrm{inform} = 1`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objgrd}` will never be called by ``handle_solve_bounds_foas`` and :math:`\mathrm{objgrd}` may be **None**.
If the option :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'YES'}`, then after returning from :math:`\mathrm{objgrd}` the gradient vector is checked for missing entries which you have not supplied.
Missing entries are estimated using the finite difference method, see option 'FOAS Estimate Derivatives' description for more details.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function gradient is to be evaluated.
**fdx** : float, ndarray, shape :math:`\left(\textit{nnzfd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the values of the nonzero elements in the sparse gradient vector of the objective function, in the order specified by :math:`\textit{idxfd}` in a previous call to :meth:`handle_set_nlnobj`. :math:`\mathrm{fdx}[\textit{i}-1]` will store the gradient element :math:`\frac{{\partial f}}{{\partial x_j}}`, where :math:`j = {\textit{idxfd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
If :math:`\mathrm{inform} = 1`, then the previous call to :math:`\mathrm{objfun}` was also made at the same point :math:`x` with :math:`\mathrm{inform} = 1`.
This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient.
This notification parameter may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to inform that the gradient cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. Returning NaN or :math:`\pm \infty` in any element of :math:`\mathrm{fdx}` has the same effect. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 25.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization and optionally to terminate the solver early if necessary.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the 'FOAS Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
:math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of decision variables at the current iteration.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`\mathrm{f}\left(x\right)`. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Norm of inactive gradient, the objective gradient over the current search space. If the problem is unconstrained, then elements :math:`1`--:math:`3` coincide. See `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__ for details. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Norm of projected direction, used in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#projdir>`__, see `Constrained Subproblem (NPG) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#constrainedsubproblem>`__. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Norm of objective gradient. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Last step size (:math:`\alpha_k`) used in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#npgstep>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#iterstep>`__, see `Constrained Subproblem (NPG) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#constrainedsubproblem>`__ and `Unconstrained Subproblem (CG) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#unconstrainedsubproblem>`__.|
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Progress score, a positive value that measures the progress of the solver. A low score close to zero indicates poor progress, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#moncrit1>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6`--:math:`99`|Reserved for future use. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of function evaluations performed by NPG, see `Constrained Subproblem (NPG) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#constrainedsubproblem>`__. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Number of gradient evaluations performed by NPG. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Number of function evaluations performed by CG, see `Unconstrained Subproblem (CG) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#unconstrainedsubproblem>`__.|
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Number of gradient evaluations performed by CG. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Number of function evaluations performed by LCG. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of gradient evaluations performed by LCG. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Number of function evaluations used by the finite difference method to estimate missing components of the gradient. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Number of iterations. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent in the solver (including user-supplied function calls). |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent in user-supplied objective function. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Total time spent in user-supplied objective gradient. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11`--:math:`99`|Reserved for future use. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. _e04kf-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'FOAS Estimate Derivatives'** : str
Default :math:`\text{} = \texttt{'NO'}`
This option indicates whether to check for and estimate missing entries of the user-supplied gradient vector.
Since the associated cost of estimating missing elements can be high, this option should only be used if strictly necessary.
In general terms, if the gradient is not provided (in its entirety or for arbitrary points) potential degradation in the progress of the solver is to be expected.
Depending on the complexity of the objective, the function may not achieve the desired optimality accuracy or even terminate with no possible further progress error :math:`\mathrm{errno}` = 24, it is advisable to increase the values of 'FOAS Stop Tolerance' and 'FOAS Rel Stop Tolerance' when using this option.
Missing elements from the gradient vector are estimated by finite-differences using the perturbation interval specified by the option 'FOAS Finite Diff Interval'.
If :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'NO'}`, the entries are not checked and all derivative elements need to be provided.
Constraint: :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'FOAS Finite Diff Interval'** : float
Default :math:`= \sqrt{\epsilon }`
Specifies the relative perturbation size used to estimate a derivative using the forward (or backward) finite-difference method.
Setting the value too small or too big may lead ``handle_solve_bounds_foas`` to terminated with :math:`\mathrm{errno}` = 24 or 25.
Constraint: :math:`10^{-12}\leq \text{‘FOAS Finite Diff Interval'}\leq 10^{-1}`.
**'FOAS Iteration Limit'** : int
Default :math:`= 10^7`
This argument sets the maximum number of iterations to be performed by ``handle_solve_bounds_foas``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘FOAS Iteration Limit'} \geq 1`.
**'FOAS Memory'** : int
Default :math:`= 11`
This argument specifies the maximum number of memory vectors to use in the LCG solver.
Constraint: :math:`0\leq \text{‘FOAS Memory'}\leq 100`.
**'FOAS Monitor Frequency'** : int
Default :math:`= 0`
This argument specifies the frequency on which to call the monitor function :math:`\mathrm{monit}`.
If zero, the monitor function will not be called.
Constraint: :math:`\text{‘FOAS Monitor Frequency'} \geq 0`.
**'FOAS Print Frequency'** : int
Default :math:`= \texttt{'1'}`
This argument specifies the frequency with which to print information regarding each iteration to 'Print File' and/or 'Monitoring File'.
By default, it will print information of every iteration.
Constraint: :math:`\text{‘FOAS Print Frequency'} \geq 1`.
**'FOAS Progress Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{3}{4}}`
Specifies the tolerance for :math:`\epsilon_{\mathrm{prog}}` (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#moncrit1>`__) for which the function characterises a poor rate of progress given that it deems to be far from a solution.
If this behaviour is persistent, then the function asserts that no substantial further progress can be achieved and the process is terminated with :math:`\mathrm{errno}` = 24.
Setting a high tolerance can lead to misinterpret reasonable progress for unsatisfactory progress or even issue a premature stop, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#moncrit1>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0 < \text{‘FOAS Progress Tolerance'} < 1`.
**'FOAS Rel Stop Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{3}{4}}`
This argument sets the value of :math:`\epsilon_{\textit{rel}}` which specifies the relative tolerance for the convergence measures in the stopping criteria, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0\leq \epsilon_{\textit{rel}} < 1`.
**'FOAS Restart Factor'** : float
Default :math:`= 6.0`
This factor specifies the frequency :math:`\textit{nvar}\times \text{‘FOAS Restart Factor'}` with which the CG/LCG directions are replaced by the steepest descent direction (:math:`d_k = -g_k`).
Setting the value too small can potentially slow the convergence speed.
Constraint: :math:`\text{‘FOAS Restart Factor'} \geq 0`.
**'FOAS Slow Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{8}}`
Specifies the tolerance for :math:`\epsilon_{\mathrm{slow}}` (see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#moncrit2>`__) for which the function characterises a slow rate of convergence.
If this behaviour is deemed permanent, then the function asserts that no substantial improvement can be achieved and the process is terminated with :math:`\mathrm{errno}` = 50.
Setting a large tolerance can lead to incorrectly identifying a suboptimal solution, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#moncrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘FOAS Slow Tolerance'} > 0`.
**'FOAS Stop Tolerance'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This argument sets the value of :math:`\epsilon_{\textit{tol}}` which specifies the tolerance for the convergence measures in the stopping criteria, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stopcrit2>`__ in `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0\leq \epsilon_{\textit{tol}} < 1`.
**'FOAS Tolerance Norm'** : str
Default :math:`= \texttt{'INFINITY'}`
This argument specifies the norm used to measure some stopping metrics, such as optimality tolerances (see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#stoppingcriteria>`__).
It is possible to choose between 2-norm and ∞-norm.
Solving problems using ∞-norm generally has lower computational costs than those based on 2-norm.
Constraint: :math:`\text{‘FOAS Tolerance Norm'} = \texttt{'INFINITY'}` or :math:`\texttt{'TWO'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options;
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the option's initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but each iteration line is longer and includes step length and progress measure |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the final values of the solution vector are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
This argument specifies a limit in seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23 error message.
Constraint: :math:`\text{‘Time Limit'} > 0`.
**'Verify Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument specifies whether the function should perform numerical checks on the consistency of the user-supplied gradient function.
If any discrepancies are found, :math:`\mathrm{errno}` = 26 is returned.
It is recommended that such checks are enabled when first developing the formulation of the problem, however, the verification process results in a significant increase of the number of the function evaluations and thus it shouldn't be used in production code.
Constraint: :math:`\text{‘Verify Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
.. _e04kf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objgrd}` function.
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`26`)
User-provided gradient is likely to be incorrect.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Problem was solved to an acceptable level; full accuracy was not achieved.
(`errno` :math:`54`)
The problem seems to be unbounded and the algorithm was stopped.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user-supplied function and recovery failed.
(`errno` :math:`25`)
Invalid number detected in user-supplied gradient and recovery failed.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04kf-py2-py-notes:
**Notes**
``handle_solve_bounds_foas`` solves large-scale bound-constrained nonlinear optimization problems of the form
.. math::
\textit{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }\quad \ell_x\leq x\leq u_x\text{,}
where :math:`n` is the number of decision variables, :math:`\ell_x`, :math:`u_x`, and :math:`x` are :math:`n`-dimensional vectors, and the nonlinear objective function :math:`f\left(x\right)` is assumed to be sufficiently smooth.
The solver is a first-order method (i.e., uses only first derivatives) that has very low memory requirements and, therefore, is suitable for very large bound-constrained problems.
It is based on an active-set method coupled to a nonmonotone projected gradient algorithm (NPG), nonlinear conjugate gradient method (CG) and its limited-memory variant (LCG).
The active-set method is based on alternating between both solvers, the NPG step handles the box constraints and identifies a suitable search space while the CG step explores it for a solution.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#algdetails>`__.
Under standard assumptions on the problem (smoothness of the first derivative of the objective) the algorithm converges to a local solution or to a critical point.
``handle_solve_bounds_foas`` serves as a solver for problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_nlnobj` or :meth:`handle_set_simplebounds` may be called to formulate the objective and to define (simple) box constraints for the problem.
Once the problem is fully described, the handle may be passed to the solver ``handle_solve_bounds_foas``.
When the handle is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The algorithm behaviour can be modified by various options (see :ref:`Other Parameters <e04kf-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
Option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
The option 'Task' may be used to switch the problem to maximization, while 'FOAS Estimate Derivatives' can be used to complete missing elements from the gradient. Option 'Verify Derivatives' may help verify the correctness of the gradient vector before starting to solve a problem.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended that you experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kff.html#algdetails>`__ and :ref:`Other Parameters <e04kf-py2-py-other_params>` for further details.
.. _e04kf-py2-py-references:
**References**
Dai, Y-H and Kou, C-X, 2013, `A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search`, SIAM J. Optim. (23(1)), 296--320
Gill, P E and Leonard, M W, 2003, `Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization`, SIAM J. Optim. (14(2)), 380--401
Hager, W W and Zhang, H, 2005, `A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search`, SIAM J. Optim. (16(1)), 170--192
Hager, W W and Zhang, H, 2006, `Algorithm 851: CG DESCENT, a Conjugate Gradient Method with Guaranteed Descent`, ACM Trans. Math. Software (32(1)), 113--137
Hager, W W and Zhang, H, 2006, `A New Active Set Algorithm for Box Constrained Optimization`, SIAM J. Optim. (17(2)), 525--557
Hager, W W and Zhang, H, 2013, `The Limited Memory Conjugate Gradient Method`, SIAM J. Optim. (23(4)), 2150--2168
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
"""
raise NotImplementedError
[docs]def bounds_quasi_deriv_easy(ibound, funct2, bl, bu, x, liw=None, lw=None, data=None):
r"""
``bounds_quasi_deriv_easy`` is an easy-to-use quasi-Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`, when first derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04ky-py2-py-doc:
For full information please refer to the NAG Library document for e04ky
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kyf.html
.. _e04ky-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply :math:`\mathrm{funct2}` to calculate the values of the function :math:`F\left(x\right)` and its first derivative :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_quasi_deriv_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_quasi_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_quasi_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**liw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`n+2`.
The dimension of the array :math:`\mathrm{iw}`.
**lw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`\max\left({10\times n+n\times \left(n-1\right) /2},11\right)`.
The dimension of the array :math:`\mathrm{w}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_quasi_deriv_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_quasi_deriv_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
**iw** : int, ndarray, shape :math:`\left(\mathrm{liw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, the first :math:`\textit{n}` elements of :math:`\mathrm{iw}` contain information about which variables are currently on their bounds and which are free. Specifically, if :math:`x_i` is:
- fixed on its upper bound, :math:`\mathrm{iw}[i-1]` is :math:`-1`;
- fixed on its lower bound, :math:`\mathrm{iw}[i-1]` is :math:`-2`;
- effectively a constant (i.e., :math:`l_j = u_j`), :math:`\mathrm{iw}[i-1]` is :math:`-3`;
- free, :math:`\mathrm{iw}[i-1]` gives its position in the sequence of free variables.
In addition, :math:`\mathrm{iw}[n]` contains the number of free variables (i.e., :math:`n_z`).
The rest of the array is used as workspace.
**w** : float, ndarray, shape :math:`\left(\mathrm{lw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, :math:`\mathrm{w}[\textit{i}-1]` contains the :math:`\textit{i}`\ th element of the projected gradient vector :math:`g_z`, for :math:`\textit{i} = 1,2,\ldots,n`. In addition, :math:`\mathrm{w}[n]` contains an estimate of the condition number of the projected Hessian matrix (i.e., :math:`k`). The rest of the array is used as workspace.
.. _e04ky-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{liw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{liw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`100\times n` function evaluations.
(`errno` :math:`4`)
An overflow occurred during computation.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct2}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04ky-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_quasi_deriv_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first derivatives are available.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
An attempt is made to verify that the final point is a minimum.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The projected gradient vector :math:`g_z`, whose elements are the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
A unit lower triangular matrix :math:`L` and a diagonal matrix :math:`D` (both of dimension :math:`n_z`), such that :math:`LDL^\mathrm{T}` is a positive definite approximation of the matrix of second derivatives with respect to the free variables (i.e., the projected Hessian) are also held.
The equations
.. math::
LDL^\mathrm{T}p_z = -g_z
are solved to give a search direction :math:`p_z`, which is expanded to an :math:`n`-vector :math:`p` by an insertion of appropriate zero elements.
Then :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; :math:`x` is replaced by :math:`x+\alpha p`, and the matrices :math:`L` and :math:`D` are updated so as to be consistent with the change produced in the gradient by the step :math:`\alpha p`.
If any variable actually reaches a bound during the search along :math:`p`, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace provided that this is practicable.
When it is not, or when the stronger convergence criteria are already satisfied, then, if one or more Lagrange multiplier estimates are close to zero, a slight perturbation is made in the values of the corresponding variables in turn until a lower function value is obtained.
The normal algorithm is then resumed from the perturbed point.
If a saddle point is suspected, a local search is carried out with a view to moving away from the saddle point.
A local search is also performed when a point is found which is thought to be a constrained minimum.
.. _e04ky-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv_easy(ibound, funct2, bl, bu, x, data=None):
r"""
``bounds_mod_deriv_easy`` is an easy-to-use modified Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`, when first derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04kz-py2-py-doc:
For full information please refer to the NAG Library document for e04kz
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04kzf.html
.. _e04kz-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply this function to calculate the values of the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv_easy`` (see submodule ``opt``).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`,
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
.. _e04kz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` function evaluations.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct2}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04kz-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first derivatives are known.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the gradient of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
.. _e04kz-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv2_comp(funct, h, monit, ibound, bl, bu, x, lh, iprint=1, maxcal=None, eta=None, xtol=0.0, stepmx=100000.0, data=None, io_manager=None):
r"""
``bounds_mod_deriv2_comp`` is a comprehensive modified Newton algorithm for finding:
an unconstrained minimum of a function of several variables
a minimum of a function of several variables subject to fixed upper and/or lower bounds on the variables.
First and second derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04lb-py2-py-doc:
For full information please refer to the NAG Library document for e04lb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lbf.html
.. _e04lb-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which :math:`F` and the :math:`\frac{{\partial F}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
:math:`\mathrm{funct}` must set :math:`\mathrm{fc}` to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{funct}` must set :math:`\mathrm{gc}[j-1]` to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**h** : callable (fhesl, fhesd) = h(xc, lh, fhesd, data=None)
:math:`\mathrm{h}` must calculate the second derivatives of :math:`F` at any point :math:`x`. (As with :math:`\mathrm{funct}`, there is the option of causing ``bounds_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the second derivatives of :math:`F` are required.
**lh** : int
The length of the array :math:`\mathrm{fhesl}`.
**fhesd** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
These values may be useful in the evaluation of the second derivatives.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fhesl** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
:math:`\mathrm{h}` must place the strict lower triangle of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesl}`, stored by rows, i.e., set :math:`\mathrm{fhesl}[ \left(\textit{i}-1\right) \left(\textit{i}-2\right) / 2 +\textit{j} -1] = \left. \frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}\right|_{\mathrm{xc}}`, for :math:`\textit{j} = 1,2,\ldots,i-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**fhesd** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{h}` must place the diagonal elements of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesd}`, i.e., set :math:`\mathrm{fhesd}[j-1] = \left. \frac{{\partial^2F}}{{\partial x_j^2}}\right|_{\mathrm{xc}}`, :math:`j = 1,2,\ldots,n`.
**monit** : callable monit(xc, fc, gc, istate, gpjnrm, cond, posdef, niter, nf, data=None)
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{monit}` which is suitable for monitoring the minimization process. :math:`\mathrm{monit}` must not change the values of any of its arguments.
If :math:`\mathrm{iprint} < 0`, a :math:`\mathrm{monit}` with the correct argument list should still be supplied, although it will not be called.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fc** : float
The value of :math:`F\left(x\right)` at the current point :math:`x`.
**gc** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently fixed on their bounds and which are free.
If :math:`\mathrm{istate}[j-1]` is negative, :math:`x_j` is currently:
- fixed on its upper bound if :math:`\mathrm{istate}[j-1] = -1`;
- fixed on its lower bound if :math:`\mathrm{istate}[j-1] = -2`;
- effectively a constant (i.e., :math:`l_j = u_j`) if :math:`\mathrm{istate}[j-1] = -3`.
If :math:`\mathrm{istate}` is positive, its value gives the position of :math:`x_j` in the sequence of free variables.
**gpjnrm** : float
The Euclidean norm of the projected gradient vector :math:`g_z`.
**cond** : float
The ratio of the largest to the smallest elements of the diagonal factor :math:`D` of the projected Hessian matrix (see specification of :math:`\mathrm{h}`). This quantity is usually a good estimate of the condition number of the projected Hessian matrix. (If no variables are currently free, :math:`\mathrm{cond}` is set to zero.)
**posdef** : bool
Is set :math:`\mathbf{True}` or :math:`\mathbf{False}` according to whether the second derivative matrix for the current subspace, :math:`H`, is positive definite or not.
**niter** : int
The number of iterations (as outlined in :ref:`Notes <e04lb-py2-py-notes>`) which have been performed by ``bounds_mod_deriv2_comp`` so far.
**nf** : int
The number of times that :math:`\mathrm{funct}` has been called so far. Thus :math:`\mathrm{nf}` is the number of function and gradient evaluations made so far.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**ibound** : int
Specifies whether the problem is unconstrained or bounded. If there are bounds on the variables, :math:`\mathrm{ibound}` can be used to indicate whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If the variables are bounded and you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If the problem is unconstrained.
:math:`\mathrm{ibound} = 2`
If the variables are bounded, but all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If all the variables are bounded, and :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
:math:`\mathrm{ibound} = 4`
If the problem is unconstrained. (The :math:`\mathrm{ibound} = 4` option is provided purely for consistency with other functions. In ``bounds_mod_deriv2_comp`` it produces the same effect as :math:`\mathrm{ibound} = 1`.)
**bl** : float, array-like, shape :math:`\left(n\right)`
The fixed lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to a large negative number, e.g., :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv2_comp`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bl}` will be initialized by ``bounds_mod_deriv2_comp``.
**bu** : float, array-like, shape :math:`\left(n\right)`
The fixed upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any variable, the corresponding :math:`\mathrm{bu}[j-1]` should be set to a large positive number, e.g., :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv2_comp`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bu}` will then be initialized by ``bounds_mod_deriv2_comp``.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**iprint** : int, optional
The frequency with which :math:`\mathrm{monit}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{monit}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``bounds_mod_deriv2_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{monit}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{monit}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
The maximum permitted number of evaluations of :math:`F\left(x\right)`, i.e., the maximum permitted number of calls of :math:`\mathrm{funct}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.9 }`.
Every iteration of ``bounds_mod_deriv2_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x+\alpha p\right)` with respect to :math:`\alpha`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations of ``bounds_mod_deriv2_comp``, this usually results in an increase in the number of function and gradient evaluations required for each iteration.
On balance, it is usually more efficient to perform a low accuracy linear minimization.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that :math:`\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)`, where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus, and if :math:`\mathrm{xtol}` is set to :math:`10^{-5}`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lbf.html#accuracy>`__.)
If the problem is scaled roughly as described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lbf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then :math:`\sqrt{\epsilon }` is probably the smallest reasonable choice for :math:`\mathrm{xtol}`. (This is because, normally, to machine accuracy, :math:`F\left({x+\sqrt{\epsilon }}, e_j\right) = F\left(x\right)` where :math:`e_j` is any column of the identity matrix.)
If you set :math:`\mathrm{xtol}` to :math:`0.0` (or any positive value less than :math:`\epsilon`), ``bounds_mod_deriv2_comp`` will use :math:`10.0\times \sqrt{\epsilon }` instead of :math:`\mathrm{xtol}`.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency a slight overestimate is preferable.)
``bounds_mod_deriv2_comp`` will ensure that, for each iteration,
.. math::
\sqrt{\sum_{{j = 1}}^n\left[x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right]^2}\leq \mathrm{stepmx}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``bounds_mod_deriv2_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can also help to avoid possible overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv2_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bl}[0] = \mathrm{bl}[1] = \cdots = \mathrm{bl}[n-1] = 0.0`.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv2_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bu}[0] = \mathrm{bu}[1] = \cdots = \mathrm{bu}[n-1] = 10^6`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04lb-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv2_comp`` (see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lbf.html#accuracy>`__).
**hesd** : float, ndarray, shape :math:`\left(n\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04lb-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the result produced by ``bounds_mod_deriv2_comp`` (see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lbf.html#accuracy>`__).
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently on their bounds and which are free. If :math:`\mathrm{istate}[j-1]` is:
- equal to :math:`-1`, :math:`x_j` is fixed on its upper bound;
- equal to :math:`-2`, :math:`x_j` is fixed on its lower bound;
- equal to :math:`-3`, :math:`x_j` is effectively a constant (i.e., :math:`l_j = u_j`);
- positive, :math:`\mathrm{istate}[j-1]` gives the position of :math:`x_j` in the sequence of free variables.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The first derivative vector corresponding to the final point given in :math:`\mathrm{x}`. The components of :math:`\mathrm{g}` corresponding to free variables should normally be close to zero.
.. _e04lb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 4`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal}` function evaluations.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
No further progress can be made.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}` or :math:`\mathrm{h}`.
.. _e04lb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``bounds_mod_deriv2_comp`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}F\left(x_1, x_2, \ldots, x_n\right)\text{subject to }l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n\text{.}
Special provision is made for unconstrained minimization (i.e., problems which actually have no bounds on the :math:`x_j`), problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
It is possible to specify that a particular :math:`x_j` should be held constant.
You must supply a starting point, a :math:`\mathrm{funct}` to calculate the value of :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`, and a :math:`\mathrm{h}` to calculate the second derivatives :math:`\frac{{\partial^2F}}{{\partial x_i\partial x_j}}`.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The vector of first derivatives of :math:`F\left(x\right)` with respect to the free variables, :math:`g_z`, and the matrix of second derivatives with respect to the free variables, :math:`H`, are obtained. (These both have dimension :math:`n_z`.)
The equations
.. math::
\left(H+E\right)p_z = -g_z
are solved to give a search direction :math:`p_z`. (The matrix :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`p_z` is then expanded to an :math:`n`-vector :math:`p` by the insertion of appropriate zero elements; :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`, and :math:`x` is replaced by :math:`x+\alpha p`. (If a saddle point is found, a special search is carried out so as to move away from the saddle point.)
If any variable actually reaches a bound, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise, minimization continues in the current subspace until the stronger criteria are satisfied.
If at this point there are no negative or near-zero Lagrange multiplier estimates, the process is terminated.
If you specify that the problem is unconstrained, ``bounds_mod_deriv2_comp`` sets the :math:`l_j` to :math:`{-10^6}` and the :math:`u_j` to :math:`10^6`.
Thus, provided that the problem has been sensibly scaled, no bounds will be encountered during the minimization process and ``bounds_mod_deriv2_comp`` will act as an unconstrained minimization algorithm.
.. _e04lb-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
Gill, P E and Murray, W, 1974, `Newton-type methods for unconstrained and linearly constrained optimization`, Math. Programming (7), 311--350
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv2_easy(ibound, funct2, hess2, bl, bu, x, data=None):
r"""
``bounds_mod_deriv2_easy`` is an easy-to-use modified-Newton algorithm for finding a minimum of a function, :math:`F\left(x_1, x_2, \ldots, x_n\right)` subject to fixed upper and lower bounds on the independent variables, :math:`x_1,x_2,\ldots,x_n` when first and second derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04ly-py2-py-doc:
For full information please refer to the NAG Library document for e04ly
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04lyf.html
.. _e04ly-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply this function to calculate the values of the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv2_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and its derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**hess2** : callable (heslc, hesdc) = hess2(xc, lh, data=None)
You must supply this function to evaluate the elements :math:`H_{{ij}} = \frac{{\partial^2F}}{{\partial x_i\partial x_j}}` of the matrix of second derivatives of :math:`F\left(x\right)` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv2_easy`` (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the derivatives are required.
**lh** : int
The length of the array :math:`\mathrm{heslc}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**heslc** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
:math:`\mathrm{hess2}` must place the strict lower triangle of the second derivative matrix :math:`H` in :math:`\mathrm{heslc}`, stored by rows, i.e., set :math:`\mathrm{heslc}[\left(\textit{i}-1\right)\left(\textit{i}-2\right)/2+\textit{j}-1] = \frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}`, for :math:`\textit{j} = 1,2,\ldots,\textit{i}-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**hesdc** : float, array-like, shape :math:`\left(n\right)`
Must contain the diagonal elements of the second derivative matrix, i.e., set :math:`\mathrm{hesdc}[\textit{j}-1] = \frac{{\partial^2F}}{{\partial x_{\textit{j}}^2}}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, :math:`\mathrm{bl}[\textit{j}-1]` must be set to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv2_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, :math:`\mathrm{bu}[\textit{j}-1]` must be set to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any :math:`x_j` the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv2_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient and the Hessian matrix at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv2_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv2_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
.. _e04ly-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` function evaluations.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in your supplied functions, your problem has no finite solution, or the problem needs rescaling.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
(`errno` :math:`11`)
It is very likely that you have made an error forming the 2nd derivatives.
.. _e04ly-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv2_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first and second derivatives of :math:`F\left(x\right)` are available.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x` and a function to calculate the second derivatives.
From a starting point you supplied there is generated, on the basis of estimates of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
.. _e04ly-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def lp_solve(bl, bu, x, comm, a=None, cvec=None, istate=None, io_manager=None):
r"""
``lp_solve`` solves general linear programming problems.
It is not intended for large sparse problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`lp_option_file`, :meth:`lp_option_string`, :meth:`nlp1_init`.
.. _e04mf-py2-py-doc:
For full information please refer to the NAG Library document for e04mf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html
.. _e04mf-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**a** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; if :math:`\textit{nclin}=0`: :math:`1`; otherwise: :math:`0`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` must contain the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,m_L`.
**cvec** : None or float, array-like, shape :math:`\left(n\right)`, optional
The coefficients of the objective function when the problem is of type LP.
If the problem is of type FP, :math:`\mathrm{cvec}` is not referenced.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{istate}` specifies the desired status of the constraints at the start of the feasibility phase.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, and the next :math:`m_L` elements refer to the general linear constraints (if any).
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+========================================================================================================================================================+
|0 |The corresponding constraint should `not` be in the initial working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraint should be in the initial working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraint should be in the initial working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The constraint should be in the initial working set as an equality. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be reset to zero by the function.
If ``lp_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\textit{nclin}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The status of the constraints in the working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================+
|:math:`-2` |The constraint violates its lower bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |The constraint violates its upper bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |This corresponds to optimality being declared with :math:`\mathrm{x}[j-1]` being temporarily fixed at its current value. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{errno}` = 1 on exit.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**x** : float, ndarray, shape :math:`\left(n\right)`
The point at which ``lp_solve`` terminated. If the function exits successfully or :math:`\mathrm{errno}` = 1 or 4, :math:`\mathrm{x}` contains an estimate of the solution.
**itera** : int
The total number of iterations performed.
**obj** : float
The value of the objective function at :math:`x` if :math:`x` is feasible, or the sum of infeasibiliites at :math:`x` otherwise. If the problem is of type FP and :math:`x` is feasible, :math:`\mathrm{obj}` is set to zero.
**ax** : None or float, ndarray, shape :math:`\left(\max\left(1,\textit{nclin}\right)\right)`
The final values of the linear constraints :math:`Ax`.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The values of the Lagrange multipliers for each constraint with respect to the current working set. The first :math:`n` elements contain the multipliers for the bound constraints on the variables, and the next :math:`m_L` elements contain the multipliers for the general linear constraints (if any). If :math:`\mathrm{istate}[j-1] = 0` (i.e., constraint :math:`j` is not in the working set), :math:`\mathrm{clamda}[j-1]` is zero. If :math:`x` is optimal, :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1`, non-positive if :math:`\mathrm{istate}[j-1] = 2` and zero if :math:`\mathrm{istate}[j-1] = 4`.
.. _e04mf-py2-py-other_params:
**Other Parameters**
**'Check Frequency'** : int
Default :math:`\text{} = 50`
Every :math:`i`\ th iteration, a numerical test is made to see if the current solution :math:`x` satisfies the constraints in the working set.
If the largest residual of the constraints in the working set is judged to be too large, the current working set is refactorized and the variables are recomputed to satisfy the constraints more accurately.
If :math:`i\leq 0`, the default value is used.
**'Cold Start'** : valueless
Default
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lp_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lp_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lp_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lp_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lp_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lp_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``lp_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Expand Frequency'** : int
Default :math:`\text{} = 5`
This option is part of an anti-cycling procedure designed to guarantee progress even on highly degenerate problems.
The strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of the option 'Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``lp_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
At certain stages the following 'resetting procedure' is used to remove constraint infeasibilities.
First, all variables whose upper or lower bounds are in the working set are moved exactly onto their bounds.
A count is kept of the number of nontrivial adjustments made.
If the count is positive, iterative refinement is used to give variables that satisfy the working set to (essentially) machine precision.
Finally, the working feasibility tolerance is reinitialized to :math:`0.5\delta`.
If a problem requires more than :math:`i` iterations, the resetting procedure is invoked and a new cycle of :math:`i` iterations is started with :math:`i` incremented by :math:`10`. (The decision to resume the feasibility phase or optimality phase is based on comparing any constraint infeasibilities with :math:`\delta`.)
The resetting procedure is also invoked when ``lp_solve`` reaches an apparently optimal, infeasible or unbounded solution, unless this situation has already occurred twice.
If any nontrivial adjustments are made, iterations are continued.
If :math:`i\leq 0`, the default value is used.
If :math:`i\geq 9999999`, no anti-cycling procedure is invoked.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
If :math:`r\geq \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point.
For example, if the variables and the coefficients in the general constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-6}`.
If :math:`0\leq r < \epsilon`, the default value is used.
``lp_solve`` attempts to find a feasible solution before optimizing the objective function.
If the sum of infeasibilities cannot be reduced to zero, the option 'Minimum Sum of Infeasibilities' can be used to find the minimum value of the sum.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Note that a 'feasible solution' is a solution that satisfies the current constraints to within the tolerance :math:`r`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the problem is of type LP.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Minimum Sum of Infeasibilities'** : str
Default :math:`= \mathrm{NO}`
If no feasible point exists for the constraints, this option is used to control whether or not ``lp_solve`` will calculate a point that minimizes the constraint violations.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'NO'}`, ``lp_solve`` will terminate as soon as it is evident that no feasible point exists for the constraints.
The final point will generally not be the point at which the sum of infeasibilities is minimized.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'YES'}`, ``lp_solve`` will continue until the sum of infeasibilities is minimized.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Print Level'} \geq 5`, monitoring information produced by ``lp_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.8}`
If :math:`r\geq \epsilon`, :math:`r` defines the tolerance used to determine if the bounds and general constraints have the right 'sign' for the solution to be judged to be optimal.
If :math:`0\leq r < \epsilon`, the default value is used.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``lp_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#monitoring>`__ (monitoring information at each iteration).
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=======================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#fc-printedoutput>`__) for each iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#monitoring>`__) for each iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each iteration, the Lagrange multipliers, the variables :math:`x`, the constraint values :math:`Ax` and the constraint status (see :math:`\mathrm{istate}`). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each iteration, the diagonal elements of the upper triangular matrix :math:`T` associated with the :math:`TQ` factorization `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#eqn3>`__ (see `Definition of Search Direction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#ad-search>`__) of the working set.|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Problem Type'** : str
Default :math:`=` LP
This option specifies the type of objective function to be minimized during the optimality phase.
The following is the optional keyword and the dimensions of the array that must be specified in order to define the objective function:
.. rst-class:: nag-rules-none
+--+------------------------------------------------------------------------------+
|LP|length-:math:`\textit{n}` :math:`\mathrm{cvec}` required. |
+--+------------------------------------------------------------------------------+
|FP|the objective function is omitted and :math:`\mathrm{cvec}` is not referenced.|
+--+------------------------------------------------------------------------------+
For problems of type FP, the objective function is omitted and :math:`\mathrm{cvec}` is not referenced.
The following keywords are also acceptable.
.. rst-class:: nag-rules-none nag-align-left
+---------+------+
|:math:`a`|Option|
+=========+======+
|Linear |LP |
+---------+------+
|Feasible |FP |
+---------+------+
.. _e04mf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`6`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Cold Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
'Problem Type' not recognized. Problem abandoned.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Weak :math:`\langle\mathit{\boldsymbol{value}}\rangle` solution.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` solution is unbounded.
(`errno` :math:`3`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
Cannot satisfy the working-set constraints to the accuracy requested.
.. _e04mf-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lp_solve`` is designed to solve Linear Programming (LP) problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}c^\mathrm{T}x\text{, subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{,}
where :math:`c` is an :math:`n`-element vector and :math:`A` is an :math:`m_L\times n` matrix.
This is the default type of problem, referred to as type LP.
The option 'Problem Type' may be used to specify an alternative problem type FP, in which the objective function is omitted and the function attempts to find a feasible point for the set of constraints.
The constraints involving :math:`A` are called the `general` constraints.
Note that upper and lower bounds are specified for all the variables and for all the general constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
You must supply an initial estimate of the solution.
The method used by ``lp_solve`` is described in detail in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mff.html#algdetails>`__.
.. _e04mf-py2-py-references:
**References**
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E and Murray, W, 1978, `Numerically stable methods for quadratic programming`, Math. Programming (14), 349--372
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1991, `Inertia-controlling methods for general quadratic programming`, SIAM Rev. (33), 1--36
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
"""
raise NotImplementedError
[docs]def lp_option_file(ioptns, comm, io_manager=None):
r"""
``lp_option_file`` may be used to supply options to :meth:`lp_solve` from an external file.
.. _e04mg-py2-py-doc:
For full information please refer to the NAG Library document for e04mg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mgf.html
.. _e04mg-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04mg-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lp_option_file`` may be used to supply values for options to :meth:`lp_solve`. ``lp_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lp_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lp_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lp_option_string(optstr, comm, io_manager=None):
r"""
``lp_option_string`` may be used to supply individual options to :meth:`lp_solve`.
.. _e04mh-py2-py-doc:
For full information please refer to the NAG Library document for e04mh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mhf.html
.. _e04mh-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04mh-py2-py-notes>` and in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04mh-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lp_option_string`` may be used to supply values for options to :meth:`lp_solve`.
It is only necessary to call ``lp_option_string`` for those arguments whose values are to be different from their default values.
One call to ``lp_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lp_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lp_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def handle_solve_lp_simplex(handle, x, u=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_lp_simplex`` is a solver from the NAG optimization modelling suite for large-scale Linear Programming (LP) problems.
It is a simplex method optimization solver based on the HiGHS software package.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04mk-py2-py-doc:
For full information please refer to the NAG Library document for e04mk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mkf.html
.. _e04mk-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_lp_simplex``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the bound constraints and linear constraints. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
Optionally provides the initial estimates of Lagrange multipliers. If there are no initial estimates available, then set to zero.
**monit** : None or callable monit(handle, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is reserved for future releases of the NAG Library which will allow you to monitor the progress of the optimization.
It will never be called in the current implementation and :math:`\mathrm{monit}` may be **None**.
**Parameters**
**handle** : Handle
The handle to the problem as provided on entry to ``handle_solve_lp_simplex``. It may be used to query the model during the solve, and extract the current approximation of the solution by :meth:`handle_set_get_real`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`, however, elements :math:`2`, :math:`3`, :math:`5`, :math:`9`, :math:`10`, :math:`11` and :math:`15` refer to the quantities in the last iteration rather than accumulated over all iterations through the whole algorithm run.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final values of the variables :math:`u`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators of the algorithm as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-----------------------------------------------+
|:math:`0` |Value of the primal objective. |
+---------------------+-----------------------------------------------+
|:math:`1` |The maximum violation of a bound on a variable.|
+---------------------+-----------------------------------------------+
|:math:`2` |The sum of violations of bounds by variables. |
+---------------------+-----------------------------------------------+
|:math:`3` |The maximum dual feasibility violation. |
+---------------------+-----------------------------------------------+
|:math:`4` |The sum of dual feasibility violations. |
+---------------------+-----------------------------------------------+
|:math:`5`--:math:`99`|Reserved for future use. |
+---------------------+-----------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics as given in the table below.
.. rst-class:: nag-rules-none nag-align-left
+---------------------+---------------------------------------------+
|:math:`0` |Total number of simplex iterations performed.|
+---------------------+---------------------------------------------+
|:math:`1` |Total time spent in the solver. |
+---------------------+---------------------------------------------+
|:math:`2`--:math:`99`|Reserved for future use. |
+---------------------+---------------------------------------------+
.. _e04mk-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'Simplex Presolve'** : str
Default :math:`= \texttt{'YES'}`
This argument allows you to turn the presolving of the problem off completely.
If the presolver is turned off, the solver will try to handle the original problem you have given.
In such a case, the presence of linear dependencies in the constraint matrix can cause numerical instabilities to occur.
In normal circumstances, it is recommended to use the presolve which is the default.
Constraint: :math:`\text{‘Simplex Presolve'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Simplex Start Type'** : str
Default :math:`\text{} = \texttt{'COLD'}`
Defines whether to perform a `cold` or `warm` start.
If warm start data is not provided or is considered to have an unexpected size or content, then the solver will revert to perform a cold start on the problem.
See `Warm Starting <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mkf.html#warmstart>`__ on how to correctly warm start a problem.
Constraint: :math:`\text{‘Simplex Start Type'} = \texttt{'COLD'}` or :math:`\texttt{'WARM'}`.
**'Simplex Strategy'** : str
Default :math:`= \texttt{'DUAL SERIAL'}`
This argument controls the strategy employed by the simplex algorithm implemetation.
By default the dual simplex solver runs in serial.
Unless a Linear Programming (LP) problem has significantly more variables than constraints, the parallel dual simplex solver is unlikely to be worth using.
If a parallel strategy is chosen, ``handle_solve_lp_simplex`` will use half the available threads on the machine and automatically choose maximum level of concurrency.
.. rst-class:: nag-rules-none nag-align-left
+-------------+-------------------------------------------------------------------+
|:math:`a` |Meaning |
+=============+===================================================================+
|'AUTO' |The solver chooses the strategy automatically |
+-------------+-------------------------------------------------------------------+
|'DUAL SERIAL'|Dual simplex method running in serial |
+-------------+-------------------------------------------------------------------+
|'DUAL PAMI' |Dual simplex method with Parallelization Across Multiple Iterations|
+-------------+-------------------------------------------------------------------+
|'DUAL SIP' |Dual simplex method with Single Iteration Parallelism |
+-------------+-------------------------------------------------------------------+
|'PRIMAL' |Primal simplex method running in serial |
+-------------+-------------------------------------------------------------------+
Constraint: :math:`\text{‘Simplex Strategy'} = \texttt{'AUTO'}`, :math:`\texttt{'DUAL SERIAL'}`, :math:`\texttt{'DUAL PAMI'}`, :math:`\texttt{'DUAL SIP'}` or :math:`\texttt{'PRIMAL'}`.
**'Simplex Random Seed'** : int
Default :math:`= 0`
Initial seed used for random permutation and factor accuracy assessment.
Constraint: :math:`\text{‘Simplex Random Seed'}\geq 0`.
**'Simplex Iteration Limit'** : int
Default :math:`= \textit{Imax}`
The maximum number of iterations to be performed by ``handle_solve_lp_simplex``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘Simplex Iteration Limit'} \geq 0`.
**'Simplex Small Matrix Value'** : float
Default :math:`\text{} = 10^{-9}`
Lower limit on the absolute value of the linear constraint coefficients in the matrix :math:`A` defined in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mkf.html#eqn1>`__.
Values smaller than this will be treated as zero.
Constraint: :math:`\text{‘Simplex Small Matrix Value'}\geq 10^{-12}`.
**'Simplex Primal Feasibility Tol'** : float
Default :math:`= 10^{-7}`
The maximum acceptable absolute violation in each primal constraint (bound and linear constraint) at a 'feasible' point; i.e., a primal constraint is considered satisfied if its violation does not exceed 'Simplex Primal Feasibility Tol' :math:`r`.
For example, a variable :math:`x_j` is considered to be feasible with respect to the bound constraint :math:`l_j\leq x_j\leq u_j` only if :math:`\left(l_j-r\right)\leq x_j\leq \left(u_j+r\right)`.
Constraint: :math:`\text{‘Simplex Primal Feasibility Tol'} > 10^{-10}`.
**'Simplex Dual Feasibility Tol'** : float
Default :math:`= 10^{-7}`
Similar to 'Simplex Primal Feasibility Tol', this argument defines the maximum acceptable absolute violation in each dual constraint (bound and linear constraint) at a 'feasible' point; i.e., a dual constraint is considered satisfied if its violation does not exceed 'Simplex Dual Feasibility Tol' :math:`r`.
Constraint: :math:`\text{‘Simplex Dual Feasibility Tol'} > 10^{-10}`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options if set by 'Print Options';
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+-------------------------------+
|:math:`i` |Output |
+==========================================+===============================+
|:math:`0` |No output from the solver |
+------------------------------------------+-------------------------------+
|:math:`1` |The Header and Summary |
+------------------------------------------+-------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log|
+------------------------------------------+-------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
This argument specifies a limit in seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23 error message.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04mk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is either :math:`0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is :math:`0` for no constraints.
(`errno` :math:`51`)
The problem was found to be primal infeasible.
(`errno` :math:`52`)
The problem was found to be dual infeasible.
(`errno` :math:`53`)
The problem seems to be primal or dual infeasible, the algorithm was stopped.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
.. _e04mk-py2-py-notes:
**Notes**
``handle_solve_lp_simplex`` solves a large-scale linear optimization problem in the following form:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_A\leq Ax\leq u_A&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
where :math:`n` is the number of decision variables and :math:`m` is the number of linear constraints.
Here :math:`c`, :math:`x`, :math:`l_x`, :math:`u_x` are :math:`n`-dimensional vectors, :math:`A` is an :math:`m\times n` sparse matrix and :math:`l_A`, :math:`u_A` are :math:`m`-dimensional vectors.
``handle_solve_lp_simplex`` solves linear programming problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_simplebounds` or :meth:`handle_set_linconstr` may be called to formulate the objective function, bounds of the variables, and the block of linear constraints, respectively.
Once the problem is fully set, the handle may be passed to the solver.
When the handle is not needed anymore, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver method can be modified by various options (see :ref:`Other Parameters <e04mk-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various options.
The option 'Task' may be used to switch the problem to maximization or to ignore the objective function and find only a feasible point.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended to experiment in order to find the most suitable set of options for a particular problem, see :ref:`Other Parameters <e04mk-py2-py-other_params>` for further details.
``handle_solve_lp_simplex`` is a complement to the interior point method solver :meth:`handle_solve_lp_ipm`.
It is recommended to try both solvers to determine which best suits your application.
**Structure of the Lagrangian Multipliers**
The algorithm works internally with estimates of both the decision variables, denoted by :math:`x`, and the Lagrangian multipliers (dual variables), denoted by :math:`u`.
The multipliers :math:`u` are stored in the array :math:`\mathrm{u}` and conform to the structure of the constraints.
If the simple bounds have been defined (:meth:`handle_set_simplebounds` was successfully called), the first :math:`2n` elements of :math:`\mathrm{u}` belong to the corresponding Lagrangian multipliers, interleaving a multiplier for the lower and the upper bound for each :math:`x_i`.
If any of the bounds were set to infinity, the corresponding Lagrangian multipliers are set to :math:`0` and may be ignored.
Similarly, the following :math:`2m` elements of :math:`\mathrm{u}` belong to multipliers for the linear constraints (if :meth:`handle_set_linconstr` has been successfully called).
The organization is the same, i.e., the multipliers for each constraint for the lower and upper bounds are alternated and zeros are used for any missing (infinite bound) constraints.
Some solvers merge multipliers for both lower and upper inequality into one element whose sign determines the inequality.
Negative multipliers are associated with the upper bounds and positive with the lower bounds.
An equivalent result can be achieved with this storage scheme by subtracting the upper bound multiplier from the lower one.
This is also consistent with equality constraints.
.. _e04mk-py2-py-references:
**References**
Huangfu, Q, and Hall, J.A. J., 2018, `Parallelizing the dual revised simplex method`, Mathematical Programming Computation (10(1)), 119--142
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_lp_ipm(handle, x=None, u=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_lp_ipm`` is a solver from the NAG optimization modelling suite for large-scale Linear Programming (LP) problems based on an interior point method (IPM).
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04mt-py2-py-doc:
For full information please refer to the NAG Library document for e04mt
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html
.. _e04mt-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_lp_ipm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : None or float, array-like, shape :math:`\left(\textit{nvar}\right)`, optional
The input of :math:`\mathrm{x}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the bound constraints and linear constraints. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
The input of :math:`\mathrm{u}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**monit** : None or callable monit(handle, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the option 'LPIPM Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**handle** : Handle
The handle to the problem as provided on entry to ``handle_solve_lp_ipm``. It may be used to query the model during the solve, and extract the current approximation of the solution by :meth:`handle_set_get_real`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`, however, elements :math:`2`, :math:`3`, :math:`5`, :math:`9`, :math:`10`, :math:`11` and :math:`15` refer to the quantities in the last iteration rather than accumulated over all iterations through the whole algorithm run.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final values of the variables :math:`u`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators of the algorithm (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails>`__ for details) as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Value of the primal objective. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Value of the dual objective. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Flag indicating the system formulation used by the solver, :math:`0`: augmented system, :math:`1`: normal equation. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Factorization type, :math:`3`: Cholesky, :math:`4`: Bunch--Parlett. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4`--:math:`13` |Primal-Dual specific information (will be :math:`0` if the Self-Dual algorithm is chosen). |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`4` |Relative dual feasibility (optimality), see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#pdopt>`__. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`5` |Relative primal feasibility, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#pdfeas>`__. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`6` |Relative duality gap (complementarity), see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#pdcompl>`__. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`7` |Average complementarity error :math:`\mu` (see `The Infeasible-interior-point Primal-Dual Algorithm <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_pd>`__).||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`8` |Centring parameter :math:`\sigma` (see `The Infeasible-interior-point Primal-Dual Algorithm <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_pd>`__). ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`9` |Primal step length. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`10` |Dual step length. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`11`--:math:`13`|Reserved for future use. ||
| |+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14`--:math:`23`|Self-Dual specific information (will be :math:`0` if the Primal-Dual algorithm is chosen). |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`14` |Relative primal infeasibility, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#sdfeas>`__.| |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`15` |Relative dual infeasibility, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#sdopt>`__. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`16` |Relative duality gap, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#sdgap>`__. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`17` |Accuracy, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#sdacc>`__. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`18` |:math:`\tau`, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#hlf>`__. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`19` |:math:`\kappa`, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#hlf>`__. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`20` |Step length. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`21`--:math:`23`|Reserved for future use. | |
| |+----------------------+---------------------------------------------------------------------------------------------------------------------------------+ |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`24`--:math:`99`|Reserved for future use. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics as given in the table below. Note that time statistics are provided only if 'Stats Time' is set (the default is 'NO'), the measured time is returned in seconds.
.. rst-class:: nag-rules-none nag-align-left
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of centrality correction steps performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of iterative refinements performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Value of the perturbation added to the diagonal in the normal equation formulation or on the zero block in the augmented system formulation.|
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Total number of factorizations performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Total time spent in the solver. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Time spent in the presolve phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Time spent in the last iteration. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent factorizing the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent backsolving the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Total time spent in the multiple centrality correctors phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Time spent in the initialization phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Number of nonzeros in the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13` |Number of nonzeros in the system matrix factor. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Maximum error of the backsolve. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15` |Number of columns in :math:`A` considered dense by the solver. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`16` |Maximum number of centrality corrector steps. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`17`--:math:`99`|Reserved for future use. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
.. _e04mt-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'LP Presolve'** : str
Default :math:`= \texttt{'FULL'}`
This argument allows you to reduce the level of presolving of the problem or turn it off completely.
If the presolver is turned off, the solver will try to handle the problem you have given.
In such a case, the presence of fixed variables or linear dependencies in the constraint matrix can cause numerical instabilities to occur.
In normal circumstances, it is recommended to use the full presolve which is the default.
Constraint: :math:`\text{‘LP Presolve'} = \texttt{'FULL'}`, :math:`\texttt{'BASIC'}` or :math:`\texttt{'NO'}`.
**'LPIPM Algorithm'** : str
Default :math:`= \texttt{'PRIMAL-DUAL'}`
As described in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails>`__, ``handle_solve_lp_ipm`` implements the infeasible Primal-Dual algorithm, see `The Infeasible-interior-point Primal-Dual Algorithm <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_pd>`__, and the homogeneous Self-Dual algorithm, see `Homogeneous Self-Dual Algorithm <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_sd>`__.
This argument controls which one to use.
Constraint: :math:`\text{‘LPIPM Algorithm'} = \texttt{'PRIMAL-DUAL'}`, :math:`\texttt{'PD'}`, :math:`\texttt{'SELF-DUAL'}` or :math:`\texttt{'SD'}`.
**'LPIPM Centrality Correctors'** : int
Default :math:`\text{} = 6`
This argument controls the number of centrality correctors (see `Weighted Multiple Centrality Correctors <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_mcc>`__) used at each iteration.
Each corrector step attempts to improve the current iterate for the price of additional solve(s) of the factorized system matrix in order to reduce the total number of iterations.
Therefore, it trades the additional solves of the system with the number of factorizations.
The more expensive the factorization is with respect to the solve, the more corrector steps should be allowed.
If :math:`i > 0`, the maximum number of corrector steps will be computed by timing heuristics (the ratio between the times of the factorization and the solve in the first iteration) but will not be greater than :math:`i`.
The number computed by the heuristic can be recovered after the solve or during a monitoring step in :math:`\mathrm{stats}`.
This might cause non-repeatable results.
If :math:`i < 0`, the maximum number of corrector steps will be set to :math:`\left\lvert i\right\rvert`.
If it is set to :math:`0`, no additional centrality correctors will be used and the algorithm reverts to Mehrotra's predictor-corrector.
**'LPIPM Iteration Limit'** : int
Default :math:`\text{} = 100`
The maximum number of iterations to be performed by ``handle_solve_lp_ipm``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘LPIPM Iteration Limit'} \geq 1`.
**'LPIPM Max Iterative Refinement'** : int
Default :math:`\text{} = 5`
This argument controls the maximum number of iterative refinement iterations (see `Solving the KKT System <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_system>`__) used at each main iteration when :math:`\text{‘LPIPM System Formulation'} = \texttt{'Normal Equations'}`.
When solving the Normal Equations linear system for numerically challenging problems, mixed-precision iterative refinement may be used until the roundoff errors are reduced to an acceptable level or until the number of refinements reached its maximum value set by this argument.
Constraint: :math:`\text{‘LPIPM Max Iterative Refinement'} \geq 0`.
**'LPIPM Scaling'** : str
Default :math:`= \texttt{'ARITHMETIC'}`
This argument controls the type of scaling to be applied on the constraint matrix :math:`A` before solving the problem.
More precisely, the scaling procedure will try to find diagonal matrices :math:`D_1` and :math:`D_2` such that the values in :math:`D_1AD_2` are of a similar order of magnitude.
The solver is less likely to run into numerical difficulties when the constraint matrix is well scaled.
Constraint: :math:`\text{‘LPIPM Scaling'} = \texttt{'ARITHMETIC'}`, :math:`\texttt{'GEOMETRIC'}` or :math:`\texttt{'NONE'}`.
**'LPIPM Monitor Frequency'** : int
Default :math:`\text{} = 0`
This argument defines the frequency of how often function :math:`\mathrm{monit}` is called.
If :math:`i > 0`, the solver calls :math:`\mathrm{monit}` at the end of every :math:`i`\ th iteration.
If it is set to :math:`0`, the function is not called at all.
Constraint: :math:`\text{‘LPIPM Monitor Frequency'} \geq 0`.
**'LPIPM Stop Tolerance'** : float
Default :math:`= \sqrt{\epsilon }`
This argument sets the value :math:`\epsilon_1` which is the tolerance for the convergence measures in the stopping criteria, see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘LPIPM Stop Tolerance'} > \epsilon`.
**'LPIPM Stop Tolerance 2'** : float
Default :math:`= \epsilon^{0.6}`
This argument sets the additional tolerance :math:`\epsilon_2` used in the stopping criteria for the Self-Dual algorithm, see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘LPIPM Stop Tolerance 2'} > \epsilon`.
**'LPIPM System Formulation'** : str
Default :math:`= \texttt{'AUTO'}`
As described in `Solving the KKT System <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_system>`__, ``handle_solve_lp_ipm`` can internally work either with the normal equations formulation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#normaleq>`__ or with the augmented system `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#augmented>`__.
A brief discussion of advantages and disadvantages is presented in `Solving the KKT System <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails_system>`__.
Option 'AUTO' leaves the decision to the solver based on the structure of the constraints and it is the recommended option.
This will typically lead to the normal equations formulation unless there are many dense columns or the system is significantly cheaper to factorize as the augmented system.
Note that in some cases even if :math:`\text{‘LPIPM System Formulation'} = \texttt{'NORMAL EQUATIONS'}` the solver might switch the formulation through the computation to the augmented system due to numerical instabilities or computational cost.
Constraint: :math:`\text{‘LPIPM System Formulation'} = \texttt{'AUTO'}`, :math:`\texttt{'AUGMENTED SYSTEM'}`, :math:`\texttt{'AS'}`, :math:`\texttt{'NORMAL EQUATIONS'}` or :math:`\texttt{'NE'}`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options if set by 'Print Options';
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the primal and dual objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but each iteration line is longer, including step lengths and errors |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
.. _e04mt-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is either :math:`0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is :math:`0` for no constraints.
(`errno` :math:`51`)
The problem was found to be primal infeasible.
(`errno` :math:`52`)
The problem was found to be dual infeasible.
(`errno` :math:`53`)
The problem seems to be primal or dual infeasible, the algorithm was stopped.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Suboptimal solution.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations exceeded.
(`errno` :math:`24`)
No progress, stopping early.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04mt-py2-py-notes:
**Notes**
``handle_solve_lp_ipm`` solves a large-scale linear optimization problem in the following form:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_A\leq Ax\leq u_A&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
where :math:`n` is the number of decision variables and :math:`m` is the number of linear constraints.
Here :math:`c`, :math:`x`, :math:`l_x`, :math:`u_x` are :math:`n`-dimensional vectors, :math:`A` is an :math:`m\times n` sparse matrix and :math:`l_A`, :math:`u_A` are :math:`m`-dimensional vectors.
``handle_solve_lp_ipm`` implements two algorithmic variants of the interior point method for solving linear optimization problems: the infeasible Primal-Dual interior point method and homogeneous Self-Dual interior point method.
In general, the Self-Dual algorithm has a slightly higher price per iteration, however, it is able to declare infeasibility or unboundness of the problem, whereas the Primal-Dual relies, in this case, on heuristics.
For a detailed description of both algorithms see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails>`__.
The algorithm is chosen by the 'LPIPM Algorithm', the default is Primal-Dual.
``handle_solve_lp_ipm`` solves linear programming problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_simplebounds` or :meth:`handle_set_linconstr` may be called to formulate the objective function, bounds of the variables, and the block of linear constraints, respectively.
Once the problem is fully set, the handle may be passed to the solver.
When the handle is not needed anymore, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver method can be modified by various options (see :ref:`Other Parameters <e04mt-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various options.
The option 'Task' may be used to switch the problem to maximization or to ignore the objective function and find only a feasible point.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended to experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mtf.html#algdetails>`__ and :ref:`Other Parameters <e04mt-py2-py-other_params>` for further details.
``handle_solve_lp_ipm`` is a complement to the simplex solver :meth:`handle_solve_lp_simplex`.
It is recommended to try both solvers to determine which best suits your application.
**Structure of the Lagrangian Multipliers**
The algorithm works internally with estimates of both the decision variables, denoted by :math:`x`, and the Lagrangian multipliers (dual variables), denoted by :math:`u`.
The multipliers :math:`u` are stored in the array :math:`\mathrm{u}` and conform to the structure of the constraints.
If the simple bounds have been defined (:meth:`handle_set_simplebounds` was successfully called), the first :math:`2n` elements of :math:`\mathrm{u}` belong to the corresponding Lagrangian multipliers, interleaving a multiplier for the lower and the upper bound for each :math:`x_i`.
If any of the bounds were set to infinity, the corresponding Lagrangian multipliers are set to :math:`0` and may be ignored.
Similarly, the following :math:`2m` elements of :math:`\mathrm{u}` belong to multipliers for the linear constraints (if :meth:`handle_set_linconstr` has been successfully called).
The organization is the same, i.e., the multipliers for each constraint for the lower and upper bounds are alternated and zeros are used for any missing (infinite bound) constraints.
Some solvers merge multipliers for both lower and upper inequality into one element whose sign determines the inequality.
Negative multipliers are associated with the upper bounds and positive with the lower bounds.
An equivalent result can be achieved with this storage scheme by subtracting the upper bound multiplier from the lower one.
This is also consistent with equality constraints.
.. _e04mt-py2-py-references:
**References**
Andersen, E D, Gondzio, J, Mészáros, C and Xu, X, 1996, `Implementation of interior point methods for large scale linear programming`, HEC/Université de Genève
Colombo, M and Gondzio, J, 2008, `Further development of multiple centrality correctors for interior point methods`, Computational Optimization and Algorithms (41(3)), 277--305
Goldfard, D and Scheinberg, K, 2004, `A product-form Cholesky factorization method for handling dense columns in interior point methods for linear programming`, Mathematical Programming (99(1)), 1--34
Gondzio, J, 1996, `Multiple centrality corrections in a primal-dual method for linear programming`, Computational Optimization and Algorithms (6(2)), 137--156
Gondzio, J, 2012, `Interior point methods 25 years later`, European Journal of Operations Research (218(3)), 587--601
Hogg, J D and Scott, J A, 2011, `HSL MA97: a bit-compatible multifrontal code for sparse symmetric systems`, RAL Technical Report. RAL-TR-2011-024
HSL, `a collection of Fortran codes for large-scale scientific computation`, http://www.hsl.rl.ac.uk/
Karypis, G and Kumar, V, 1998, `A fast and high quality multilevel scheme for partitioning irregular graphs`, SIAM J. Sci. Comput. (20(1)), 359--392
Mészáros, C, 1996, `The efficient implementation of interior point methods for linear programming and their applications`, PhD Thesis, Eötvös Loránd University of Science, Budapest
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
Wright, S W, 1997, `Primal-dual interior point methods`, SIAM, Philadelphia
Xu, X, Hung, P-F and Ye, Y, 1996, `A simplified homogeneous and self-dual linear programming algorithm and its implementation`, Annals of Operations Research (62(1)), 151--171
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
"""
raise NotImplementedError
[docs]def miqp_mps_write(outfile, n, m, ncolh, iobj, bl, bu, pnames, crname, minmax, idxc=None, c=None, a=None, irowa=None, iccola=None, h=None, irowh=None, iccolh=None, intvar=None, io_manager=None):
r"""
``miqp_mps_write`` writes data for sparse linear programming, mixed integer linear programming, quadratic programming or mixed integer quadratic programming problems to a file in MPS format.
.. _e04mw-py2-py-doc:
For full information please refer to the NAG Library document for e04mw
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mwf.html
.. _e04mw-py2-py-parameters:
**Parameters**
**outfile** : int
The ID of the file to store the problem data as associated by a call to the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`.
**n** : int
:math:`n`, the number of variables in the problem.
**m** : int
:math:`m`, the number of constraints in the problem. This is the number of rows in the linear constraint matrix :math:`A`, including the free row (if any; see :math:`\mathrm{iobj}`).
**ncolh** : int
The number of leading nonzero columns of the Hessian matrix :math:`H`.
If :math:`\mathrm{ncolh} = 0`, the quadratic term :math:`H` of the objective function is considered zero (e.g., LP problems), and arrays :math:`\mathrm{h}`, :math:`\mathrm{irowh}` and :math:`\mathrm{iccolh}` will not be referenced.
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the linear terms of the objective function. In this case :math:`\textit{nnzc}` is set to :math:`0`.
If :math:`\mathrm{iobj} = 0`, there is no free row in :math:`A`, and the linear terms might be supplied in array :math:`\mathrm{c}`.
**bl** : float, array-like, shape :math:`\left(\mathrm{n}+\mathrm{m}\right)`
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` contains the lower bounds :math:`l` and the upper bounds :math:`u`, respectively.
The first :math:`\mathrm{n}` elements refer to the bounds for the variables :math:`x` and the rest to the bounds for the linear constraints (including the objective row :math:`\mathrm{iobj}` if present).
To specify a nonexistent lower bound (i.e., :math:`l_j = -\mathrm{inf}`), set :math:`\mathrm{bl}[j-1]\leq -10^{20}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq 10^{20}`.
**bu** : float, array-like, shape :math:`\left(\mathrm{n}+\mathrm{m}\right)`
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` contains the lower bounds :math:`l` and the upper bounds :math:`u`, respectively.
The first :math:`\mathrm{n}` elements refer to the bounds for the variables :math:`x` and the rest to the bounds for the linear constraints (including the objective row :math:`\mathrm{iobj}` if present).
To specify a nonexistent lower bound (i.e., :math:`l_j = -\mathrm{inf}`), set :math:`\mathrm{bl}[j-1]\leq -10^{20}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq 10^{20}`.
**pnames** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
The names can be composed only from 'printable' characters (ASCII codes between :math:`32` and :math:`126`).
If any of the names are blank, the default name is used.
:math:`\mathrm{pnames}[0]`
Contains the name of the problem.
:math:`\mathrm{pnames}[1]`
Contains the name of the objective row if the objective is provided in :math:`\mathrm{c}` instead of :math:`\mathrm{iobj}` and all names :math:`\mathrm{crname}` are given. The name must be nonempty and unique. In all other cases :math:`\mathrm{pnames}[1]` is not used.
:math:`\mathrm{pnames}[2]`
Contains the name of the RHS set.
:math:`\mathrm{pnames}[3]`
Contains the name of the RANGE.
:math:`\mathrm{pnames}[4]`
Contains the name of the BOUNDS.
**crname** : str, length 8, array-like, shape :math:`\left(\textit{nname}\right)`
The names of all the variables and constraints in the problem in that order.
The names can be composed only from 'printable' characters and must be unique.
**minmax** : int
:math:`\mathrm{minmax}` defines the direction of optimization problem.
:math:`\mathrm{minmax} = -1`
Minimization.
:math:`\mathrm{minmax} = 1`
Maximization.
**idxc** : None or int, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of sparse vector :math:`c`. :math:`\mathrm{idxc}[\textit{i}-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`.
The elements are stored in ascending order.
**c** : None or float, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of sparse vector :math:`c`. :math:`\mathrm{idxc}[\textit{i}-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`.
The elements are stored in ascending order.
**a** : None or float, array-like, shape :math:`\left(\textit{nnza}\right)`, optional
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**irowa** : None or int, array-like, shape :math:`\left(\textit{nnza}\right)`, optional
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**iccola** : None or int, array-like, shape :math:`\left(:\right)`, optional
Note: the required length for this argument is determined as follows: if :math:`\textit{nnza} > 0`: :math:`{ \mathrm{n}+1 }`; otherwise: :math:`0`.
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**h** : None or float, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{h}` is not referenced.
**irowh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{irowh}` is not referenced.
**iccolh** : None or int, array-like, shape :math:`\left(:\right)`, optional
Note: the required length for this argument is determined as follows: if :math:`\mathrm{ncolh} > 0`: :math:`{ \mathrm{ncolh}+1 }`; otherwise: :math:`0`.
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{iccolh}` is not referenced.
**intvar** : None or int, array-like, shape :math:`\left(\textit{lintvar}\right)`, optional
:math:`\mathrm{intvar}` contains the indices :math:`k` of variables :math:`x_k` which are defined as integers. Duplicate indices are not allowed.
If :math:`\textit{lintvar} = 0`, :math:`\mathrm{intvar}` is not referenced.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mw-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{outfile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{outfile}\geq 0`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{n}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nnzc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzc}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nnza} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnza}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{lintvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lintvar}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nname} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nname} = 0` or :math:`\mathrm{n}+\mathrm{m}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ncolh}\leq \mathrm{n}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncolh} > 0`, :math:`\textit{nnzh} > 0`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncolh} = 0`, :math:`\textit{nnzh} = 0`.
(`errno` :math:`5`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxc}[\textit{j}-1]\leq \mathrm{n}`.
(`errno` :math:`5`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idxc}[j] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idxc}[j-1] < \mathrm{idxc}[j]`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{minmax} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{minmax} = -1` or :math:`1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{iobj}\leq \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: at most one of :math:`\mathrm{iobj}` or :math:`\textit{nnzc}` may be nonzero.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` are incorrect.
Constraint: :math:`\mathrm{bl}[\textit{j}-1]\leq \mathrm{bu}[\textit{j}-1]`.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1]` is incorrect.
Constraint: :math:`\mathrm{bl}[\textit{j}-1] < {1e+20}`.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[\textit{j}-1]` is incorrect.
Constraint: :math:`\mathrm{bu}[\textit{j}-1] > {-1e+20}`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, if :math:`\mathrm{iobj} > 0` the bounds must be infinite.
Constraints: :math:`\mathrm{bl}[\textit{j}-1]\leq {-1e+20}`, :math:`\mathrm{bu}[\textit{j}-1]\geq 1e+20`.
(`errno` :math:`8`)
On entry, :math:`\textit{j} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, the integer variable :math:`\textit{j}` requires at least one bound finite.
Constraint: at least one of the following conditions must be met for integer variable j: :math:`\mathrm{bl}[\textit{j}-1] > {-1e+20}`, :math:`\mathrm{bu}[\textit{j}-1] < 1e+20`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{crname}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` is incorrect.
Constraint: the names in :math:`\mathrm{crname}` must consist only of printable characters.
(`errno` :math:`9`)
On entry, :math:`\mathrm{pnames}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` is incorrect.
Constraint: the names in :math:`\mathrm{pnames}` must consist only of printable characters.
(`errno` :math:`9`)
On entry, :math:`\mathrm{crname}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` has been already used.
Constraint: the names in :math:`\mathrm{crname}` must be unique.
(`errno` :math:`9`)
The name specified in :math:`\mathrm{pnames}[1]` is empty or has been already used among row names.
Constraint: the names in :math:`\mathrm{pnames}[1]` must be unique and nonempty if :math:`\mathrm{crname}` is provided and :math:`\textit{nnzc} > 0`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{intvar}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{lintvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{intvar}[\textit{j}-1]\leq \textit{lintvar}`.
(`errno` :math:`10`)
On entry, :math:`\mathrm{intvar}[\langle\mathit{\boldsymbol{value}}\rangle] = \mathrm{intvar}[\langle\mathit{\boldsymbol{value}}\rangle] =` :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: all entries in :math:`\mathrm{intvar}` must be unique.
(`errno` :math:`11`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowa}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowa}[\textit{i}-1]\leq \mathrm{m}`.
(`errno` :math:`11`)
On entry, more than one element of :math:`\mathrm{a}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{a}` must have a unique row and column index.
(`errno` :math:`12`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iccola}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{iccola}[j] = \langle\mathit{\boldsymbol{value}}\rangle`, the values of :math:`\mathrm{iccola}` must be nondecreasing.
Constraint: :math:`\mathrm{iccola}[\textit{j}-1]\leq \mathrm{iccola}[j]`.
(`errno` :math:`12`)
On entry, :math:`\mathrm{iccola}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccola}[0] = 1`.
(`errno` :math:`12`)
On entry, :math:`\mathrm{iccola}[\mathrm{n}] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnza} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccola}[\mathrm{n}] = \textit{nnza}+1`.
(`errno` :math:`13`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle`
Constraint: :math:`j\leq \mathrm{irowh}[\textit{i}-1]\leq \mathrm{ncolh}` (within the lower triangle).
(`errno` :math:`13`)
On entry, more than one element of :math:`\mathrm{h}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{h}` must have a unique row and column index.
(`errno` :math:`14`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iccolh}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{iccolh}[j] = \langle\mathit{\boldsymbol{value}}\rangle`, the values of :math:`\mathrm{iccolh}` must be nondecreasing.
Constraint: :math:`\mathrm{iccolh}[\textit{j}-1]\leq \mathrm{iccolh}[j]`.
(`errno` :math:`14`)
On entry, :math:`\mathrm{iccolh}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccolh}[0] = 1`.
(`errno` :math:`14`)
On entry, :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \textit{nnzh}+1`.
(`errno` :math:`15`)
An error occurred when writing to file.
.. _e04mw-py2-py-notes:
**Notes**
``miqp_mps_write`` writes data for Linear Programming (LP) or Quadratic Programming (QP) problems (or their mixed integer variants) from an optimization problem to a MPS output file, see `MPS Input Format for miqp_mps_read <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#mpsinput>`__ for the format description.
The problem is expected in the form
.. math::
\mathrm{minimize}_x\left({c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx}\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
Where :math:`n` is the number of variables, :math:`m` is the number of general linear constraints, :math:`A` is the linear constraint matrix with dimension :math:`m\times n`, the vectors :math:`l` and :math:`u` are the lower and upper bounds, respectively. :math:`H` is the Hessian matrix with dimension :math:`n\times n`, however, only leading :math:`\mathrm{ncolh}` columns might contain nonzero elements and the rest is assumed to be zero.
Note that the linear term of the objective function :math:`c` might be supplied either as :math:`\mathrm{c}` or via :math:`\mathrm{iobj}`.
If :math:`\mathrm{c}` is supplied then :math:`\mathrm{idxc}` contains the indices of the nonzero elements of sparse vector :math:`c`, whereas if :math:`\mathrm{iobj}` is supplied (:math:`\mathrm{iobj} > 0`), row :math:`\mathrm{iobj}` of matrix :math:`A` is a free row storing the nonzero elements of :math:`c`.
Note: that this function uses fixed MPS format, see IBM (1971).
.. _e04mw-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def miqp_mps_read(infile, maxn, maxm, maxnnz, maxncolh, maxnnzh, maxlintvar, pnames, mpslst=0, io_manager=None):
r"""
``miqp_mps_read`` reads data for sparse linear programming, mixed integer linear programming, quadratic programming or mixed integer quadratic programming problems from an external file which is in standard or compatible MPS input format.
.. _e04mx-py2-py-doc:
For full information please refer to the NAG Library document for e04mx
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html
.. _e04mx-py2-py-parameters:
**Parameters**
**infile** : int
The ID of the MPSX data file to be read as returned by a call to the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`.
**maxn** : int
An upper limit for the number of variables in the problem.
If :math:`\mathrm{maxn} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxm** : int
An upper limit for the number of general linear constraints (including the objective row) in the problem.
If :math:`\mathrm{maxm} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxnnz** : int
An upper limit for the number of nonzeros (including the objective row) in the problem.
If :math:`\mathrm{maxnnz} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxncolh** : int
An upper limit for the dimension of the matrix :math:`H`.
If :math:`\mathrm{maxncolh} < 0`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxnnzh** : int
An upper limit for the number of nonzeros of the matrix :math:`H`.
If :math:`\mathrm{maxnnzh} < 0`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxlintvar** : int
If :math:`\mathrm{maxlintvar}\geq 0`, an upper limit for the number of integer variables.
If :math:`\mathrm{maxlintvar} < 0`, ``miqp_mps_read`` will treat all integer variables in the file as continuous variables.
**pnames** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
:math:`\mathrm{pnames}[0]`
Must either contain the name of the problem or be blank.
:math:`\mathrm{pnames}[1]`
Must either be blank or contain the name of the objective row (in which case it overrides OBJNAME section and the default choice of the first objective free row).
:math:`\mathrm{pnames}[2]`
Must either contain the name of the RHS set to be used or be blank (in which case the first RHS set is used).
:math:`\mathrm{pnames}[3]`
Must either contain the name of the RANGE set to be used or be blank (in which case the first RANGE set (if any) is used).
:math:`\mathrm{pnames}[4]`
Must either contain the name of the BOUNDS set to be used or be blank (in which case the first BOUNDS set (if any) is used).
**mpslst** : int, optional
If :math:`\mathrm{mpslst} \neq 0`, summary messages are sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`) as ``miqp_mps_read`` reads through the data file. This can be useful for debugging the file. If :math:`\mathrm{mpslst} = 0`, then no summary is produced.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**n** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of variables of the problem. Otherwise, :math:`n`, the actual number of variables in the problem.
**m** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of general linear constraints in the problem (including the objective row). Otherwise, :math:`m`, the actual number of general linear constraints of the problem.
**nnz** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of nonzeros in the problem (including the objective row). Otherwise, the actual number of nonzeros in the problem (including the objective row).
**ncolh** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the value of :math:`\textit{ncolh}` required by :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve`. In this context :math:`\mathrm{ncolh}` is the number of leading nonzero columns of the Hessian matrix :math:`H`. Otherwise, the actual dimension of the matrix :math:`H`.
**nnzh** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of nonzeros of the matrix :math:`H`. Otherwise, the actual number of nonzeros of the matrix :math:`H`.
**lintvar** : int
If on entry :math:`\mathrm{maxlintvar} < 0`, all integer variables are treated as continuous and :math:`\mathrm{lintvar} = -1`.
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of integer variables of the problem.
Otherwise, the actual number of integer variables of the problem.
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the vector :math:`c`.
If :math:`\mathrm{iobj} = 0`, the coefficients of :math:`c` are assumed to be zero.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__) :math:`\mathrm{iobj}` is not referenced.
**a** : float, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{a}` is not referenced.
**irowa** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The row indices of the nonzero elements stored in :math:`\mathrm{a}`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{irowa}` is not referenced.
**iccola** : int, ndarray, shape :math:`\left(:\right)`
A set of pointers to the beginning of each column of :math:`A`. More precisely, :math:`\mathrm{iccola}[\textit{i}-1]` contains the index in :math:`\mathrm{a}` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{n}`. Note that :math:`\mathrm{iccola}[0] = 1` and :math:`\mathrm{iccola}[\mathrm{n}] = \mathrm{nnz}+1`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{iccola}` is not referenced.
**bl** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bl}` contains the vector :math:`l` (the lower bounds) for all the variables and constraints
**bu** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bu}` contains the vector :math:`u` (the upper bounds) for all the variables and constraints
**pnames** : str, length 8, ndarray, shape :math:`\left(5\right)`
A set of names associated with the problem as defined in the MPSX data file as follows:
:math:`\mathrm{pnames}[0]`
Contains the name of the problem (or blank if none).
:math:`\mathrm{pnames}[1]`
Contains the name of the objective row (or blank if none).
:math:`\mathrm{pnames}[2]`
Contains the name of the RHS set (or blank if none).
:math:`\mathrm{pnames}[3]`
Contains the name of the RANGE set (or blank if none).
:math:`\mathrm{pnames}[4]`
Contains the name of the BOUNDS set (or blank if none).
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{pnames}` is not referenced.
**nname** : int
:math:`n+m`, the total number of variables and constraints in the problem (including the objective row).
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{nname}` is not set.
**crname** : str, length 8, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
The MPS names of all the variables and constraints in the problem in the following order. The first :math:`\mathrm{n}` elements contain the MPS names for the variables and the next :math:`\mathrm{m}` elements contain the MPS names for the objective row and general linear constraints (if any). Note that the MPS name for the objective row is stored in :math:`\mathrm{crname}[\mathrm{n}+\mathrm{iobj}-1]`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{crname}` is not referenced.
**h** : float, ndarray, shape :math:`\left(\mathrm{maxnnzh}\right)`
The :math:`\mathrm{nnzh}` nonzero elements of :math:`H`, arranged by increasing column index.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{h}` is not referenced.
**irowh** : int, ndarray, shape :math:`\left(\mathrm{maxnnzh}\right)`
The :math:`\mathrm{nnzh}` row indices of the elements stored in :math:`H`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{irowh}` is not referenced.
**iccolh** : int, ndarray, shape :math:`\left(:\right)`
A set of pointers to the beginning of each column of :math:`H`. More precisely, :math:`\mathrm{iccolh}[\textit{i}-1]` contains the index in :math:`H` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{ncolh}`. Note that :math:`\mathrm{iccolh}[0] = 1` and :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \mathrm{nnzh}+1`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{iccolh}` is not referenced.
**minmax** : int
:math:`\mathrm{minmax}` defines the direction of the optimization as read from the MPS file. By default the function assumes the objective function should be minimized and will return :math:`\mathrm{minmax} = -1`. If the function discovers in the OBJSENSE section that the objective function should be maximized it will return :math:`\mathrm{minmax} = 1`. If the function discovers that there is neither the linear objective term :math:`c` (the objective row) nor the Hessian matrix :math:`H`, the problem is considered as a feasible point problem and :math:`\mathrm{minmax} = 0` is returned.
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{minmax}` is not set.
**intvar** : int, ndarray, shape :math:`\left(\mathrm{maxlintvar}\right)`
If :math:`\mathrm{maxlintvar} > 0` on entry, :math:`\mathrm{intvar}` contains pointers to the columns that are defined as integer variables. More precisely, :math:`\mathrm{intvar}[\textit{i}-1] = k`, where :math:`k` is the index of a column that is defined as an integer variable, for :math:`\textit{i} = 1,2,\ldots,\mathrm{lintvar}`.
If :math:`\mathrm{maxlintvar}\leq 0` on entry, or ``miqp_mps_read`` was run in query mode (see `Query Mode <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#querymode>`__), or it returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{intvar}` is not set.
.. _e04mx-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`3`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
OBJNAME indicator line found after ROWS indicator line.
(`errno` :math:`4`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
COLUMNS indicator line found before ROWS indicator line.
(`errno` :math:`5`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RHS indicator line found before COLUMNS indicator line.
(`errno` :math:`6`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RANGES indicator line found before RHS indicator line.
(`errno` :math:`7`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
BOUNDS indicator line found before COLUMNS indicator line.
(`errno` :math:`8`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before BOUNDS indicator line.
(`errno` :math:`9`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before COLUMNS indicator line.
(`errno` :math:`10`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown indicator line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`12`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Indicator line ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been found more than once in the MPS file.
(`errno` :math:`13`)
End of file found before ENDATA indicator line.
(`errno` :math:`14`)
No indicator line found in file. It may be an empty file.
(`errno` :math:`15`)
At least one mandatory section not found in MPS file.
(`errno` :math:`16`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: An illegal line was detected in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
This is neither a comment nor a valid data line.
(`errno` :math:`17`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown inequality key :math:`\langle\mathit{\boldsymbol{value}}\rangle` in ROWS section.
Expected '``N``', '``G``', '``L``' or '``E``'.
(`errno` :math:`18`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Empty ROWS section.
Neither the objective row nor the constraints were defined.
(`errno` :math:`19`)
The supplied name, in :math:`\mathrm{pnames}[1]` or in OBJNAME, of the objective row was not found among the free rows in the ROWS section.
(`errno` :math:`20`)
The supplied name, in :math:`\mathrm{pnames}[4]`, of the BOUNDS set to be used was not found in the BOUNDS section.
(`errno` :math:`21`)
The supplied name, in :math:`\mathrm{pnames}[2]`, of the RHS set to be used was not found in the RHS section.
(`errno` :math:`22`)
The supplied name, in :math:`\mathrm{pnames}[3]`, of the RANGES set to be used was not found in the RANGES section.
(`errno` :math:`23`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal row name.
Row names must consist of printable characters only.
(`errno` :math:`24`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal column name.
Column names must consist of printable characters only.
(`errno` :math:`25`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` has been defined more than once in the ROWS section.
(`errno` :math:`26`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been defined more than once in the COLUMNS section. Column definitions must be continuous. (See `COLUMNS Section <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#columnssection>`__).
(`errno` :math:`27`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` marker within ``'INTORG'`` to ``'INTEND'`` range.
(`errno` :math:`28`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTEND'`` marker without previous marker being ``'INTORG'``.
(`errno` :math:`29`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` but not ``'INTEND'`` before the end of the COLUMNS section.
(`errno` :math:`30`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal marker type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Should be either ``'INTORG'`` or ``'INTEND'``.
(`errno` :math:`31`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All row names must be specified in the ROWS section.
(`errno` :math:`32`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All column names must be specified in the COLUMNS section.
(`errno` :math:`33`)
Inconsistent bounds for row ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`33`)
Inconsistent bounds for column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`33`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown bound type :math:`\langle\mathit{\boldsymbol{value}}\rangle` in BOUNDS section.
(`errno` :math:`34`)
More than one nonzero of :math:`A` has row name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' and column name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' in the COLUMNS section.
(`errno` :math:`35`)
Field :math:`\langle\mathit{\boldsymbol{value}}\rangle` did not contain a number (see :ref:`Notes <e04mx-py2-py-notes>`).
(`errno` :math:`36`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{infile}\geq 0`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Warning: MPS file not strictly fixed format, although the problem was read anyway. The data may have been read incorrectly. You should set :math:`\mathrm{mpslst} = 1` and repeat the call to ``miqp_mps_read`` for more details.
(`errno` :math:`2`)
At least one of :math:`\mathrm{maxm}`, :math:`\mathrm{maxn}`, :math:`\mathrm{maxnnz}`, :math:`\mathrm{maxnnzh}` or :math:`\mathrm{maxncolh}` is too small. You must provide :math:`\mathrm{maxm}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxn}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnz}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnzh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxncolh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
At least one of :math:`\mathrm{maxm}`, :math:`\mathrm{maxn}`, :math:`\mathrm{maxnnz}`, :math:`\mathrm{maxnnzh}`, :math:`\mathrm{maxncolh}` or :math:`\mathrm{maxlintvar}` is too small. You must provide :math:`\mathrm{maxm}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxn}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnz}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnzh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxncolh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxlintvar}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04mx-py2-py-notes:
**Notes**
``miqp_mps_read`` reads data for Linear Programming (LP) or Quadratic Programming (QP) problems (or their mixed integer variants) from an external file which is prepared in standard or compatible MPS (see IBM (1971)) input format.
It then initializes :math:`n` (the number of variables), :math:`m` (the number of general linear constraints), the :math:`m\times n` matrix :math:`A`, the vectors :math:`l`, :math:`u`, :math:`c` (stored in row :math:`\mathrm{iobj}` of :math:`A`) and the :math:`n\times n` Hessian matrix :math:`H` for use with :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve`. These functions are designed to solve problems of the form
.. math::
\mathrm{minimize}_x\left({c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx}\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
**MPS input format**
The input file of data may only contain two types of lines:
(1) Indicator lines (specifying the type of data which is to follow).
(#) Data lines (specifying the actual data).
A `section` is a combination of an indicator line and its corresponding data line(s).
Any characters beyond column 80 are ignored.
Indicator lines must not contain leading blank characters (in other words they must begin in column 1).
The following displays the order in which the indicator lines must appear in the file:
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------+----------+
|NAME |user-supplied name|(optional)|
+--------+------------------+----------+
|OBJSENSE| |(optional)|
+--------+------------------+----------+
| |data line | |
+--------+------------------+----------+
|OBJNAME | |(optional)|
+--------+------------------+----------+
| |data line | |
+--------+------------------+----------+
|ROWS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|COLUMNS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|RHS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|RANGES | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|BOUNDS | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|QUADOBJ | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|ENDATA | | |
+--------+------------------+----------+
A data line follows a fixed format, being made up of fields as defined below.
The contents of the fields may have different significance depending upon the section of data in which they appear.
.. rst-class:: nag-rules-none
+--------+-----------+------------+-------------+-------------+-------------+-------------+
| |Field 1 |Field 2 |Field 3 |Field 4 |Field 5 |Field 6 |
+========+===========+============+=============+=============+=============+=============+
|Columns |:math:`2-3`|:math:`5-12`|:math:`15-22`|:math:`25-36`|:math:`40-47`|:math:`50-61`|
+--------+-----------+------------+-------------+-------------+-------------+-------------+
|Contents|Code |Name |Name |Value |Name |Value |
+--------+-----------+------------+-------------+-------------+-------------+-------------+
Each name and code must consist of 'printable' characters only; names and codes supplied must match the case used in the following descriptions.
Values are read using a field width of :math:`12`.
This allows values to be entered in several equivalent forms.
For example, :math:`1.2345678`, :math:`1.2345678e+0`, :math:`123.45678e-2` and :math:`12345678e-07` all represent the same number.
It is safest to include an explicit decimal point.
Lines with an asterisk (:math:`*`) in column :math:`1` will be considered comment lines and will be ignored by the function.
Columns outside the six fields must be blank, except for columns 72--80, whose contents are ignored by the function.
A non-blank character outside the predefined six fields and columns 72--80 is considered to be a major error (:math:`\mathrm{errno}` = 16; see :ref:`Exceptions <e04mx-py2-py-errors>`), unless it is part of a comment.
`NAME Section (optional)`
The NAME section is the only section where the data must be on the same line as the indicator.
The 'user-supplied name' must be in field :math:`3` but may be blank.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-------------------+
|Field |Required|Description |
+=========+========+===================+
|:math:`3`|No |Name of the problem|
+---------+--------+-------------------+
`OBJSENSE Section (optional)`
The data line in this section can be used to specify the sense of the objective function.
If this section is present it must contain only one data line.
If the section is missing or empty, minimization is assumed.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-------------------------------+
|Field |Required|Description |
+=========+========+===============================+
|:math:`2`|No |Sense of the objective function|
+---------+--------+-------------------------------+
Field 2 may contain either ``MIN``, ``MAX``, ``MINIMIZE`` or ``MAXIMIZE``.
`OBJNAME Section (optional)`
The data line in this section can be used to specify the name of a free row (see `ROWS Section <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#rowssection>`__) that should be used as the objective function.
If this section is present it must contain only one data line.
If the section is missing or is empty, the first free row will be chosen instead.
Alternatively, OBJNAME can be overridden by setting nonempty :math:`\mathrm{pnames}[1]` (see :ref:`Parameters <e04mx-py2-py-parameters>`).
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------------------------------+
|Field |Required|Description |
+=========+========+=============================================+
|:math:`2`|No |Row name to be used as the objective function|
+---------+--------+---------------------------------------------+
Field 2 must contain a valid row name.
`ROWS Section`
The data lines in this section specify unique row (constraint) names and their inequality types (i.e., unconstrained, :math:`=`, :math:`\geq` or :math:`\leq`).
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+--------------+
|Field |Required|Description |
+=========+========+==============+
|:math:`1`|Yes |Inequality key|
+---------+--------+--------------+
|:math:`2`|Yes |Row name |
+---------+--------+--------------+
The inequality key specifies each row's type.
It must be ``E``, ``G``, ``L`` or ``N`` and can be in either column :math:`2` or :math:`3`.
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------+---------------+--------------+
|Inequality Key|Description |:math:`l` |:math:`u` |
+==============+========================+===============+==============+
|``N`` |Free row |:math:`-\infty`|:math:`\infty`|
+--------------+------------------------+---------------+--------------+
|``G`` |Greater than or equal to|finite |:math:`\infty`|
+--------------+------------------------+---------------+--------------+
|``L`` |Less than or equal to |:math:`-\infty`|finite |
+--------------+------------------------+---------------+--------------+
|``E`` |Equal to |finite |:math:`l` |
+--------------+------------------------+---------------+--------------+
Row type ``N`` stands for 'Not binding'.
It can be used to define the objective row.
The objective row is a free row that specifies the vector :math:`c` in the linear objective term :math:`c^\mathrm{T}x`.
If there is more than one free row, the first free row is chosen, unless another free row name is specified by OBJNAME (see `OBJNAME Section (optional) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#objnamesection>`__) or :math:`\mathrm{pnames}[1]` (see :ref:`Parameters <e04mx-py2-py-parameters>`).
Note that :math:`c` is assumed to be zero if either the chosen row does not appear in the COLUMNS section (i.e., has no nonzero elements) or there are no free rows defined in the ROWS section.
`COLUMNS Section`
Data lines in this section specify the names to be assigned to the variables (columns) in the general linear constraint matrix :math:`A`, and define, in terms of column vectors, the actual values of the corresponding matrix elements.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |Column name|
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
Each data line in the COLUMNS section defines the nonzero elements of :math:`A` or :math:`c`.
Any elements of :math:`A` or :math:`c` that are undefined are assumed to be zero.
Nonzero elements of :math:`A` must be grouped by column, that is to say that all of the nonzero elements in the jth column of :math:`A` must be specified before those in the :math:`\textit{j}+1`\ th column, for :math:`\textit{j} = 1,2,\ldots,n-1`.
Rows may appear in any order within the column.
`Integer Markers`
For backward compatibility ``miqp_mps_read`` allows you to define the integer variables within the COLUMNS section using integer markers, although this is not recommended as markers can be treated differently by different MPS readers; you should instead define any integer variables in the BOUNDS section (see below).
Each marker line must have the following format:
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|No |Marker ID |
+---------+--------+-----------+
|:math:`3`|Yes |Marker tag |
+---------+--------+-----------+
|:math:`5`|Yes |Marker type|
+---------+--------+-----------+
The marker tag must be ``'MARKER'``.
The marker type must be ``'INTORG'`` to start reading integer variables and ``'INTEND'`` to finish reading integer variables.
This implies that a row cannot be named ``'MARKER'``, ``'INTORG'`` or ``'INTEND'``.
Please note that both marker tag and marker type comprise of :math:`8` characters as a ``'`` is the mandatory first and last character in the string.
You may wish to have several integer marker sections within the COLUMNS section, in which case each marker section must begin with an ``'INTORG'`` marker and end with an ``'INTEND'`` marker and there should not be another marker between them.
Field 2 is ignored by ``miqp_mps_read``.
When an integer variable is declared it will keep its default bounds unless they are changed in the BOUNDS section.
This may vary between different MPS readers.
`RHS Section`
This section specifies the right-hand side values (if any) of the general linear constraint matrix :math:`A`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |RHS name |
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
The MPS file may contain several RHS sets distinguished by RHS name.
If an RHS name is defined in :math:`\mathrm{pnames}[2]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then ``miqp_mps_read`` will read in only that RHS vector, otherwise the first RHS set will be used.
Only the nonzero RHS elements need to be specified.
Note that if an RHS is given to the objective function it will be ignored by ``miqp_mps_read``.
An RHS given to the objective function is dealt with differently by different MPS readers, therefore, it is safer to not define an RHS of the objective function in your MPS file.
Note that this section may be empty, in which case the RHS vector is assumed to be zero.
`RANGES Section (optional)`
Ranges are used to modify the interpretation of constraints defined in the ROWS section (see `ROWS Section <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mxf.html#rowssection>`__) to the form :math:`l\leq Ax\leq u`, where both :math:`l` and :math:`u` are finite.
The range of the constraint is :math:`r = u-l`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |Range name |
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
The range of each constraint implies an upper and lower bound dependent on the inequality key of each constraint, on the RHS :math:`b` of the constraint (as defined in the RHS section), and on the range :math:`r`.
.. rst-class:: nag-rules-none nag-align-left
+--------------+-----------------+-----------------------------------+-----------------------------------+
|Inequality Key|Sign of :math:`r`|:math:`l` |:math:`u` |
+==============+=================+===================================+===================================+
|``E`` |:math:`+` |:math:`b` |:math:`b+r` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``E`` |:math:`-` |:math:`b+r` |:math:`b` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``G`` |:math:`+/-` |:math:`b` |:math:`b+\left\lvert r\right\rvert`|
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``L`` |:math:`+/-` |:math:`b-\left\lvert r\right\rvert`|:math:`b` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``N`` |:math:`+/-` |:math:`-\infty` |:math:`+\infty` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
If a range name is defined in :math:`\mathrm{pnames}[3]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then the function will read in only the range set of that name, otherwise the first set will be used.
`BOUNDS Section (optional)`
These lines specify limits on the values of the variables (the quantities :math:`l` and :math:`u` in :math:`l\leq x\leq u`).
If a variable is not specified in the bound set then it is automatically assumed to lie between :math:`0` and :math:`{+\infty }`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------+
|Field |Required|Description |
+=========+========+=====================+
|:math:`1`|Yes |Bound type identifier|
+---------+--------+---------------------+
|:math:`2`|Yes |Bound name |
+---------+--------+---------------------+
|:math:`3`|Yes |Column name |
+---------+--------+---------------------+
|:math:`4`|Yes/No |Value |
+---------+--------+---------------------+
Note: field 4 is required only if the bound type identifier is one of ``UP``, ``LO``, ``FX``, ``UI`` or ``LI`` in which case it gives the value :math:`k` below. If the bound type identifier is ``FR``, ``MI``, ``PL`` or ``BV``, field 4 is ignored and it is recommended to leave it blank.
The table below describes the acceptable bound type identifiers and how each determines the variables' bounds.
.. rst-class:: nag-rules-none nag-align-left
+---------------------+---------------+--------------+-----------------+
|Bound Type Identifier|:math:`l` |:math:`u` |Integer Variable?|
+=====================+===============+==============+=================+
|``UP`` |unchanged |:math:`k` |No |
+---------------------+---------------+--------------+-----------------+
|``LO`` |:math:`k` |unchanged |No |
+---------------------+---------------+--------------+-----------------+
|``FX`` |:math:`k` |:math:`k` |No |
+---------------------+---------------+--------------+-----------------+
|``FR`` |:math:`-\infty`|:math:`\infty`|No |
+---------------------+---------------+--------------+-----------------+
|``MI`` |:math:`-\infty`|unchanged |No |
+---------------------+---------------+--------------+-----------------+
|``PL`` |unchanged |:math:`\infty`|No |
+---------------------+---------------+--------------+-----------------+
|``BV`` |:math:`0` |:math:`1` |Yes |
+---------------------+---------------+--------------+-----------------+
|``UI`` |unchanged |:math:`k` |Yes |
+---------------------+---------------+--------------+-----------------+
|``LI`` |:math:`k` |unchanged |Yes |
+---------------------+---------------+--------------+-----------------+
If a bound name is defined in :math:`\mathrm{pnames}[4]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then the function will read in only the bound set of that name, otherwise the first set will be used.
`QUADOBJ Section (optional)`
The QUADOBJ section defines nonzero elements of the upper or lower triangle of the Hessian matrix :math:`H`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------------+
|Field |Required|Description |
+=========+========+===========================+
|:math:`2`|Yes |Column name (HColumn Index)|
+---------+--------+---------------------------+
|:math:`3`|Yes |Column name (HRow Index) |
+---------+--------+---------------------------+
|:math:`4`|Yes |Value |
+---------+--------+---------------------------+
|:math:`5`|No |Column name (HRow Index) |
+---------+--------+---------------------------+
|:math:`6`|No |Value |
+---------+--------+---------------------------+
Each data line in the QUADOBJ section defines one (or optionally two) nonzero elements :math:`H_{{ij}}` of the matrix :math:`H`.
Each element :math:`H_{{ij}}` is given as a triplet of row index :math:`i`, column index :math:`j` and a value.
The column names (as defined in the COLUMNS section) are used to link the names of the variables and the indices :math:`i` and :math:`j`.
More precisely, the matrix :math:`H` on output will have a nonzero element
.. math::
H_{{ij}} = \text{Value}
where index :math:`j` belongs to HColumn Index and index :math:`i` to one of the HRow Indices such that
:math:`\mathrm{crname}[j-1] = \text{Column name (HColumn Index)}` and
:math:`\mathrm{crname}[i-1] = \text{Column name (HRow Index)}`.
It is only necessary to define either the upper or lower triangle of the :math:`H` matrix; either will suffice.
Any elements that have been defined in the upper triangle of the matrix will be moved to the lower triangle of the matrix, then any repeated nonzeros will be summed.
Note: it is much more efficient for :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve` to have the :math:`H` matrix defined by the first :math:`\mathrm{ncolh}` column names. If the nonzeros of :math:`H` are defined by any columns that are not in the first :math:`\mathrm{ncolh}` of :math:`\mathrm{n}` then ``miqp_mps_read`` will rearrange the matrices :math:`A` and :math:`H` so that they are.
**Query Mode**
``miqp_mps_read`` offers a 'query mode' to quickly give upper estimates on the sizes of user arrays.
In this mode any expensive checks of the data and of the file format are skipped, providing a prompt count of the number of variables, constraints and matrix nonzeros.
This might be useful in the common case where the size of the problem is not known in advance.
You may activate query mode by setting any of the following: :math:`\mathrm{maxn} < 1`, :math:`\mathrm{maxm} < 1`, :math:`\mathrm{maxnnz} < 1`, :math:`\mathrm{maxncolh} < 0` or :math:`\mathrm{maxnnzh} < 0`.
If no major formatting error is detected in the data file, no exception or warning is raised is returned and the upper estimates are given as stated in Table [label omitted].
Alternatively, the function switches to query mode while the file is being read if it is discovered that the provided space is insufficient (that is, if :math:`\mathrm{n} > \mathrm{maxn}`, :math:`\mathrm{m} > \mathrm{maxm}`, :math:`\mathrm{nnz} > \mathrm{maxnnz}`, :math:`\mathrm{ncolh} > \mathrm{maxncolh}`, :math:`\mathrm{nnzh} > \mathrm{maxnnzh}` or :math:`\mathrm{lintvar} > \mathrm{maxlintvar}`).
In this case :math:`\mathrm{errno}` = 2 is returned.
.. rst-class:: nag-rules-none
+------------------------+---------------------------+
|Argument Name |Upper Estimate for |
+========================+===========================+
|:math:`\mathrm{n}` |:math:`\mathrm{maxn}` |
+------------------------+---------------------------+
|:math:`\mathrm{m}` |:math:`\mathrm{maxm}` |
+------------------------+---------------------------+
|:math:`\mathrm{nnz}` |:math:`\mathrm{maxnnz}` |
+------------------------+---------------------------+
|:math:`\mathrm{ncolh}` |:math:`\mathrm{maxncolh}` |
+------------------------+---------------------------+
|:math:`\mathrm{nnzh}` |:math:`\mathrm{maxnnzh}` |
+------------------------+---------------------------+
|:math:`\mathrm{lintvar}`|:math:`\mathrm{maxlintvar}`|
+------------------------+---------------------------+
.. _e04mx-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def qpconvex1_sparse_mps(infile, maxn, maxm, maxnnz, xbldef, xbudef, mpslst, names, io_manager=None):
r"""
``qpconvex1_sparse_mps`` reads data for a sparse linear programming or quadratic programming problem from an external file which is in standard or compatible MPSX input format.
.. deprecated:: 26.2.0.0
``qpconvex1_sparse_mps`` is deprecated.
Please use :meth:`miqp_mps_read` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04mz-py2-py-doc:
For full information please refer to the NAG Library document for e04mz
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04mzf.html
.. _e04mz-py2-py-parameters:
**Parameters**
**infile** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) associated with the MPSX data file.
**maxn** : int
An upper limit for the number of variables in the problem.
**maxm** : int
An upper limit for the number of constraints (including the objective row) in the problem.
**maxnnz** : int
An upper limit for the number of nonzeros (including the objective row) in the problem.
**xbldef** : float
The default lower bound to be used for the variables in the problem when none is specified in the BOUNDS section of the MPSX data file. For a standard LP or QP problem :math:`\mathrm{xbldef}` would normally be set to zero.
**xbudef** : float
The default upper bound to be used for the variables in the problem when none is specified in the BOUNDS section of the MPSX data file. For a standard LP or QP problem :math:`\mathrm{xbudef}` would normally be set to 'infinity' (i.e., :math:`\mathrm{xbudef}\geq 10^{20}`).
**mpslst** : bool
If :math:`\mathrm{mpslst} = \mathbf{True}`, a listing of the input data is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`). This can be useful for debugging the MPSX data file. If :math:`\mathrm{mpslst} = \mathbf{False}`, no listing is produced.
**names** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
:math:`\mathrm{names}[0]`
Must contain either the name of the problem or be blank.
:math:`\mathrm{names}[1]`
Must contain either the name of the objective row or be blank (in which case the first objective free row is used).
:math:`\mathrm{names}[2]`
Must contain either the name of the RHS set to be used or be blank (in which case the first RHS set is used).
:math:`\mathrm{names}[3]`
Must contain either the name of the RANGE set to be used or be blank (in which case the first RANGE set (if any) is used).
:math:`\mathrm{names}[4]`
Must contain either the name of the BOUNDS set to be used or be blank (in which case the first BOUNDS set (if any) is used).
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**n** : int
:math:`n`, the actual number of variables in the problem.
**m** : int
:math:`m`, the actual number of general linear constraints in the problem (including the objective row).
**nnz** : int
The actual number of nonzeros in the problem (including the objective row).
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the vector :math:`c`.
If :math:`\mathrm{iobj} = 0`, the coefficients of :math:`c` are assumed to be zero.
If :math:`\mathrm{iobj} = -1`, no such row was found and the function terminates with :math:`\mathrm{errno}` = 4 or 5 (see :ref:`Exceptions <e04mz-py2-py-errors>`).
**ncolh** : int
:math:`\mathrm{ncolh} = 0`. For QP problems, :math:`\mathrm{ncolh}` is the number of leading nonzero columns of the Hessian matrix :math:`H` and must, therefore, be set :math:`\text{} > 0` before calling :meth:`qpconvex1_sparse_solve`.
**a** : float, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index.
**ha** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The row indices of the nonzero elements stored in :math:`\mathrm{a}`.
**ka** : int, ndarray, shape :math:`\left(\mathrm{maxn}+1\right)`
A set of pointers to the beginning of each column of :math:`A`. More precisely, :math:`\mathrm{ka}[\textit{i}-1]` contains the index in :math:`\mathrm{a}` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{n}`. Note that :math:`\mathrm{ka}[0] = 1` and :math:`\mathrm{ka}[\mathrm{n}] = \mathrm{nnz}+1`.
**bl** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bl}` contains the vector :math:`l` (the lower bounds) for all the variables and constraints
**bu** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bu}` contains the vector :math:`u` (the upper bounds) for all the variables and constraints
**start** : str, length 1
:math:`\mathrm{start} = \texttt{'C'}` and an internal Crash procedure will be used by :meth:`qpconvex1_sparse_solve` to choose an initial basis.
**names** : str, length 8, ndarray, shape :math:`\left(5\right)`
A set of names associated with the problem as defined in the MPSX data file as follows:
:math:`\mathrm{names}[0]`
Contains the name of the problem (or blank if none).
:math:`\mathrm{names}[1]`
Contains the name of the objective row (or blank if none).
:math:`\mathrm{names}[2]`
Contains the name of the RHS set (or blank if none).
:math:`\mathrm{names}[3]`
Contains the name of the RANGE set (or blank if none).
:math:`\mathrm{names}[4]`
Contains the name of the BOUNDS set (or blank if none).
**nname** : int
:math:`n+m`, the total number of variables and constraints in the problem.
**crname** : str, length 8, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
The MPSX names of all the variables and constraints in the problem in the following order. The first :math:`\mathrm{n}` elements contain the MPSX names for the variables and the next :math:`\mathrm{m}` elements contain the MPSX names for the objective row and general linear constraints (if any). Note that the MPSX name for the objective row is stored in :math:`\mathrm{crname}[\mathrm{n}+\mathrm{iobj}-1]`.
**xs** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
A set of initial values for the variables and constraints in the problem. More precisely, :math:`\mathrm{xs}[\textit{j}-1] = \mathrm{min}\left(\mathrm{max}\left(0.0, {\mathrm{bl}[\textit{j}-1]}\right), \mathrm{bu}[\textit{j}-1]\right)`, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nname}`.
**istate** : int, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
A set of initial states for the variables and constraints in the problem. More precisely, :math:`\mathrm{istate}[\textit{j}-1] = 1` if :math:`\mathrm{xs}[\textit{j}-1] = \mathrm{bu}[\textit{j}-1]` and :math:`0` otherwise, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nname}`.
.. _e04mz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
Too many rows. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
Too many columns. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`3`)
Too many nonzeros. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
No objective row found.
(`errno` :math:`5`)
Objective row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the ROWS section.
(`errno` :math:`6`)
No rows specified.
(`errno` :math:`7`)
Illegal constraint type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`8`)
Row name with leading blank or non-alphanumeric character: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
Column name with leading blank or non-alphanumeric character: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`10`)
Illegal bound type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`11`)
Column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the COLUMNS section.
(`errno` :math:`12`)
The last line must be the ENDATA indicator line.
(`errno` :math:`13`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not a comment nor a valid line.
(`errno` :math:`14`)
Row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the ROWS section.
(`errno` :math:`15`)
No columns specified.
(`errno` :math:`16`)
BOUNDS name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`16`)
RANGES name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`16`)
RHS name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`17`)
On entry, :math:`\mathrm{xbldef} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xbudef} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xbldef}\leq \mathrm{xbudef}`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxnnz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxnnz}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxm} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxm}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxn} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxn}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{infile}\leq 2147483647`.
.. _e04mz-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``qpconvex1_sparse_mps`` reads Linear Programming (LP) or Quadratic Programming (QP) problem data from an external file which is prepared in standard or compatible MPSX (see IBM (1971)) input format and then initializes :math:`n` (the number of variables), :math:`m` (the number of general linear constraints), the :math:`m\times n` matrix :math:`A`, and the vectors :math:`l`, :math:`u` and :math:`c` (stored in row :math:`\mathrm{iobj}` of :math:`A`) for use with :meth:`qpconvex1_sparse_solve`, which is designed to solve problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
For LP problems, :math:`H = 0`.
For QP problems, you must set :math:`\mathrm{ncolh} > 0` (see :ref:`Parameters <e04mz-py2-py-parameters>`) and provide a function to :meth:`qpconvex1_sparse_solve` to compute :math:`Hx` for any given vector :math:`x`.
The option 'Maximize' may be used to specify an alternative problem in which the objective function is maximized (see :ref:`Other Parameters for qpconvex1_sparse_solve <e04nk-py2-py-other_params>`).
**MPSX input format**
The input file of data may only contain two types of lines:
(1) Indicator lines (specifying the type of data which is to follow).
(#) Data lines (specifying the actual data).
The input file must not contain any blank lines.
Any characters beyond column 80 are ignored.
Indicator lines must not contain leading blank characters (in other words they must begin in column 1).
The following displays the order in which the indicator lines must appear in the file:
.. rst-class:: nag-rules-none nag-align-left
+-------+------------+------------------+
|NAME | |user-supplied name|
+-------+------------+------------------+
|ROWS | | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|COLUMNS| | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|RHS | | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|RANGES | |(optional) |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|BOUNDS | |(optional) |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|ENDATA | | |
+-------+------------+------------------+
The 'user-supplied name' specifies a name for the problem and must occupy columns :math:`15-22`.
The name can either be blank or up to a maximum of :math:`8` characters.
A data line follows the same fixed format made up of fields defined below.
The contents of the fields may have different significance depending upon the section of data in which they appear.
.. rst-class:: nag-rules-none
+--------+-----------+------------+-------------+-------------+-------------+-------------+
| |Field 1 |Field 2 |Field 3 |Field 4 |Field 5 |Field 6 |
+========+===========+============+=============+=============+=============+=============+
|Columns |:math:`2-3`|:math:`5-12`|:math:`15-22`|:math:`25-36`|:math:`40-47`|:math:`50-61`|
+--------+-----------+------------+-------------+-------------+-------------+-------------+
|Contents|Code |Name |Name |Value |Name |Value |
+--------+-----------+------------+-------------+-------------+-------------+-------------+
The names and codes consist of 'alphanumeric' characters (i.e., a--z, A--Z, :math:`0`--:math:`9`, :math:`+`, :math:`-`, :math:`*`, blank (), :, $ or full stop (.) only) and the names must not contain leading blank characters.
Values are read using Fortran format :math:`e12.0`.
This allows values to be entered in several equivalent forms.
For example, :math:`1.2345678`, :math:`1.2345678e+0`, :math:`123.45678e-2` and :math:`12345678e-07` all represent the same number.
It is safest to include an explicit decimal point.
Note that in order to ensure numeric values are interpreted as intended, they should be `right-justified` in the :math:`12`-character field, with no trailing blanks.
This is because in some situations trailing blanks may be interpreted as zeros and this can dramatically affect the interpretation of the value.
This is relevant if the value contains an exponent, or if it contains neither an exponent nor an explicit decimal point.
For example, the fields
::
%%%%1.23e-2%
%%%%%%%123%%
may be interpreted as :math:`1.23e-20` and :math:`12300` respectively (where ``%`` denotes a blank).
The actual behaviour is system-dependent.
Comment lines are allowed in the data file.
These must have an asterisk (``*``) in column 1 and any characters in columns 2--80.
In any data line, a dollar sign (``$``) as the first character in Field 3 or 5 indicates that the information from that point through column 80 consists of comments.
Columns outside the six fields must be blank, except for columns 72--80, whose contents are ignored by the function.
These columns may be used to enter a sequence number.
A non-blank character outside the predefined six fields and columns 72--80 is considered to be a major error (:math:`\mathrm{errno}` = 13; see :ref:`Exceptions <e04mz-py2-py-errors>`), unless it is part of a comment.
**ROWS Data Lines**
These lines specify row (constraint) names and their inequality types (i.e., :math:`=`, :math:`\geq` or :math:`\leq`).
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------------------------------------------------------------------------+
|Field 1:|defines the constraint type. It may be in column 2 or column 3. |
+--------+------------------------------------------------------------------------------------+
| |+-----+----------------------------------------------------------------------------+|
| ||``N``|free row, that is no constraint. It may be used to define the objective row.||
| |+-----+----------------------------------------------------------------------------+|
| ||``G``|greater than or equal to (i.e., :math:`\geq`). ||
| |+-----+----------------------------------------------------------------------------+|
| ||``L``|less than or equal to (i.e., :math:`\leq`). ||
| |+-----+----------------------------------------------------------------------------+|
| ||``E``|exactly equal to (i.e., :math:`=`). ||
| |+-----+----------------------------------------------------------------------------+|
+--------+------------------------------------------------------------------------------------+
|Field 2:|defines the row name. |
+--------+------------------------------------------------------------------------------------+
Row type ``N`` stands for 'Not binding', also known as 'Free'.
It can be used to define the objective row.
The objective row is a free row that specifies the vector :math:`c` in the linear objective term :math:`c^\mathrm{T}x`.
It is taken to be the first free row, unless some other free row name is specified by the :math:`\mathrm{names}` array (see :ref:`Parameters <e04mz-py2-py-parameters>`).
Note that :math:`c` is assumed to be zero if (for example) the line
::
%N%%DUMMYROW
(where ``%`` denotes a blank) appears in the ROWS section of the MPSX data file, and the row name ``DUMMYROW`` is omitted from the COLUMNS section.
**COLUMNS Data Lines**
These lines specify the names to be assigned to the variables (columns) in the general linear constraint matrix :math:`A`, and define, in terms of column vectors, the actual values of the corresponding matrix elements.
.. rst-class:: nag-rules-none nag-align-left
+--------+--------------------------------------------------------------------------------------------+
|Field 1:|blank (ignored). |
+--------+--------------------------------------------------------------------------------------------+
|Field 2:|gives the name of the column associated with the elements specified in the following fields.|
+--------+--------------------------------------------------------------------------------------------+
|Field 3:|contains the name of a row. |
+--------+--------------------------------------------------------------------------------------------+
|Field 4:|used in conjunction with Field 3 contains the value of the matrix element. |
+--------+--------------------------------------------------------------------------------------------+
|Field 5:|is optional (may be used like Field 3). |
+--------+--------------------------------------------------------------------------------------------+
|Field 6:|is optional (may be used like Field 4). |
+--------+--------------------------------------------------------------------------------------------+
Note that only the nonzero elements of :math:`A` and :math:`c` need to be specified in the COLUMNS section, as any zero elements of :math:`A` are removed and any unspecified elements of :math:`c` are assumed to be zero.
In addition, any nonzero elements in the :math:`\textit{j}`\ th column of :math:`A` must be grouped together before those in the :math:`\left(\textit{j}+1\right)`\ th column, for :math:`\textit{j} = 1,2,\ldots,n-1`.
Nonzero elements within a column may however appear in any order.
**RHS Data Lines**
This section specifies the right-hand side values of the general linear constraint matrix :math:`A` (if any).
The lines specify the name to be given to the right-hand side (RHS) vector along with the numerical values of the elements of the vector, which may appear in any order.
The data lines have exactly the same format as the COLUMNS data lines, except that the column name is replaced by the RHS name.
Only the nonzero elements need be specified.
Note that this section may be empty, in which case the RHS vector is assumed to be zero.
**RANGES Data Lines (optional)**
Ranges are used for constraints of the form :math:`l\leq Ax\leq u`, where both :math:`l` and :math:`u` are finite.
The range of the constraint is :math:`r = u-l`.
Either :math:`l` or :math:`u` must be specified in the RHS section and :math:`r` must be defined in this section.
The data lines have exactly the same format as the COLUMNS data lines, except that the column name is replaced by the RANGES name.
**BOUNDS Data Lines (optional)**
These lines specify limits on the values of the variables (:math:`l` and :math:`u` in :math:`l\leq x\leq u`).
If the variable is not specified in the bound set then it is automatically assumed to lie between default lower and upper bounds (usually :math:`0` and :math:`{+\infty }`).
Like an RHS column which is given a name, the set of variables in one bound set is also given a name.
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 1:|+--+---------------------------------------------------------------------+ |
| ||LO|lower bound | |
| |+--+---------------------------------------------------------------------+ |
| ||UP|upper bound | |
| |+--+---------------------------------------------------------------------+ |
| ||FX|fixed variable | |
| |+--+---------------------------------------------------------------------+ |
| ||FR|free variable (:math:`{-\infty }` to :math:`{+\infty }`) | |
| |+--+---------------------------------------------------------------------+ |
| ||MI|lower bound is :math:`{-\infty }` | |
| |+--+---------------------------------------------------------------------+ |
| ||PL|upper bound is :math:`{+\infty }`. This is the default variable type.| |
| |+--+---------------------------------------------------------------------+ |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 2:|identifies a name for the bound set. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 3:|identifies the column name of the variable belonging to this set. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 4:|identifies the value of the bound; this has a numerical value only in association with LO, UP, FX in Field 1, otherwise it is blank.|
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 5:|is blank and ignored. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 6:|is blank and ignored. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
Note that if RANGES and BOUNDS sections are both present, the RANGES section must appear first.
.. _e04mz-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def lsq_lincon_solve(bl, bu, x, comm, c=None, cvec=None, istate=None, kx=None, a=None, b=None, io_manager=None):
r"""
``lsq_lincon_solve`` solves linearly constrained linear least squares problems and convex quadratic programming problems.
It is not intended for large sparse problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`lsq_lincon_option_file`, :meth:`lsq_lincon_option_string`, :meth:`nlp1_init`.
.. _e04nc-py2-py-doc:
For full information please refer to the NAG Library document for e04nc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html
.. _e04nc-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
Note: that it may be best to avoid the choice :math:`\mathrm{x} = 0.0`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**c** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{c}` must contain the coefficients of the :math:`\textit{i}`\ th general constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nclin}`.
**cvec** : None or float, array-like, shape :math:`\left(n\right)`, optional
The coefficients of the explicit linear term of the objective function.
If the problem is of type FP, QP1, QP3, LS1 (the default) or LS3, :math:`\mathrm{cvec}` is not referenced.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{istate}` specifies the desired status of the constraints at the start of the feasibility phase.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, and the next :math:`n_L` elements refer to the general linear constraints (if any).
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+========================================================================================================================================================+
|0 |The constraint should `not` be in the initial working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraint should be in the initial working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraint should be in the initial working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The constraint should be in the initial working set as an equality. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be reset to zero by the function.
If ``lsq_lincon_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\textit{nclin}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**kx** : None or int, array-like, shape :math:`\left(n\right)`, optional
Need not be initialized for problems of type FP, LP, QP1, QP2, LS1 (the default) or LS2.
For problems QP3, QP4, LS3 or LS4, :math:`\mathrm{kx}` must specify the order of the columns of the matrix :math:`A` with respect to the ordering of :math:`\mathrm{x}`.
Thus if column :math:`j` of :math:`A` is the column associated with the variable :math:`x_i` then :math:`\mathrm{kx}[j-1] = i`.
**a** : None or float, array-like, shape :math:`\left(m, n\right)`, optional
The matrix :math:`A`.
**b** : None or float, array-like, shape :math:`\left(m\right)`, optional
The :math:`m` elements of the vector of observations.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The status of the constraints in the working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+====================================================================================================================================================================+
|:math:`-2` |The constraint violates its lower bound by more than the feasibility tolerance. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |The constraint violates its upper bound by more than the feasibility tolerance. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |The constraint is included in the working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |This corresponds to optimality being declared with :math:`\mathrm{x}[j-1]` being temporarily fixed at its current value. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**kx** : int, ndarray, shape :math:`\left(n\right)`
Defines the order of the columns of :math:`\mathrm{a}` with respect to the ordering of :math:`\mathrm{x}`, as described above.
**x** : float, ndarray, shape :math:`\left(n\right)`
The point at which ``lsq_lincon_solve`` terminated. If the function exits successfully or :math:`\mathrm{errno}` = 1 or 4, :math:`\mathrm{x}` contains an estimate of the solution.
**a** : float, ndarray, shape :math:`\left(m, n\right)`
The matrix :math:`A`.
**b** : None or float, ndarray, shape :math:`\left(m\right)`
The transformed residual vector of `(0) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#eqn10>`__ (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#ad-mainiteration>`__).
If the problem is of type FP, LP, QP1, QP2, QP3 or QP4, :math:`\mathrm{b}` is not referenced.
**itera** : int
The total number of iterations performed.
**obj** : float
The value of the objective function at :math:`x` if :math:`x` is feasible, or the sum of infeasibiliites at :math:`x` otherwise. If the problem is of type FP and :math:`x` is feasible, :math:`\mathrm{obj}` is set to zero.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The values of the Lagrange multipliers for each constraint with respect to the current working set. The first :math:`n` elements contain the multipliers for the bound constraints on the variables, and the next :math:`n_L` elements contain the multipliers for the general linear constraints (if any). If :math:`\mathrm{istate}[j-1] = 0` (i.e., constraint :math:`j` is not in the working set), :math:`\mathrm{clamda}[j-1]` is zero. If :math:`x` is optimal, :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1`, non-positive if :math:`\mathrm{istate}[j-1] = 2` and zero if :math:`\mathrm{istate}[j-1] = 4`.
.. _e04nc-py2-py-other_params:
**Other Parameters**
**'Cold Start'** : valueless
Default
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lsq_lincon_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lsq_lincon_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_lincon_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lsq_lincon_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lsq_lincon_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_lincon_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``lsq_lincon_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`c_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert c_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Feasibility Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
The scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. Option 'Optimality Phase Iteration Limit' is equivalent to option 'Iteration Limit'.
Setting :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Optimality Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
The scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. Option 'Optimality Phase Iteration Limit' is equivalent to option 'Iteration Limit'.
Setting :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
If :math:`r > \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point.
For example, if the variables and the coefficients in the general constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-6}`.
If :math:`0\leq r < \epsilon`, the default value is used.
Note that a 'feasible solution' is a solution that satisfies the current constraints to within the tolerance :math:`r`.
**'Hessian'** : str
Default :math:`\text{} = \mathrm{NO}`
This option controls the contents of the upper triangular matrix :math:`R` (see the description of :math:`\mathrm{a}` in :ref:`Parameters <e04nc-py2-py-parameters>`). ``lsq_lincon_solve`` works exclusively with the transformed and reordered matrix :math:`H_Q` `(8) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#eqn8>`__, and hence extra computation is required to form the Hessian itself.
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{a}` contains the Cholesky factor of the matrix :math:`H_Q` with columns ordered as indicated by :math:`\mathrm{kx}` (see :ref:`Parameters <e04nc-py2-py-parameters>`).
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, :math:`\mathrm{a}` contains the Cholesky factor of the matrix :math:`H`, with columns ordered as indicated by :math:`\mathrm{kx}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the Hessian is singular and the objective contains an explicit linear term.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Print Level'} \geq 5`, monitoring information produced by ``lsq_lincon_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} < 5`, no monitoring information is produced.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``lsq_lincon_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#monitoring>`__ (monitoring information at each iteration).
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=======================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=====================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#monitoring>`__) for each iteration (no printout of the final solution). |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each iteration, the Lagrange multipliers, the variables :math:`x`, the constraint values :math:`Cx` and the constraint status. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(4) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#eqn4>`__ (see `Definition of Search Direction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#ad-search>`__) of the working set, and the diagonal elements of the upper triangular matrix :math:`R`.|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Problem Type'** : str
Default :math:`=` LS1
This option specifies the type of objective function to be minimized during the optimality phase.
The following are the nine optional keywords and the dimensions of the arrays that must be specified in order to define the objective function:
.. rst-class:: nag-rules-none nag-align-left
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LP |:math:`\mathrm{a}` and :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP1|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` symmetric, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP2|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` symmetric, :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP3|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP4|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS1|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS2|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS3|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS4|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, length-:math:`\textit{n}` :math:`\mathrm{cvec}`.|
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
For problems of type FP, the objective function is omitted and :math:`\mathrm{a}`, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` are not referenced.
The following keywords are also acceptable.
.. rst-class:: nag-rules-none nag-align-left
+---------+------+
|:math:`a`|Option|
+=========+======+
|Least |LS1 |
+---------+------+
|Quadratic|QP2 |
+---------+------+
|Linear |LP |
+---------+------+
In addition, the keywords LS and LSQ are equivalent to the default option LS1, and the keyword QP is equivalent to the option QP2.
If :math:`A = 0`, i.e., the objective function is purely linear, the efficiency of ``lsq_lincon_solve`` may be increased by specifying :math:`a` as LP.
**'Rank Tolerance'** : float
Default :math:`\text{} = 100\epsilon` or :math:`10\sqrt{\epsilon }` (see below)
Note that this option does not apply to problems of type FP or LP.
The default value of :math:`r` depends on the problem type.
If :math:`A` occurs as a least squares matrix, as it does in problem types QP1, LS1 and LS3, then the default value of :math:`r` is :math:`100\epsilon`.
In all other cases, :math:`A` is treated as the 'square root' of the Hessian matrix :math:`H` and :math:`r` has the default value :math:`10\sqrt{\epsilon }`.
This argument enables you to control the estimate of the triangular factor :math:`R_1` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#ad-mainiteration>`__).
If :math:`\rho_i` denotes the function :math:`\rho_i = \mathrm{max}\left\{\left\lvert R_{11}\right\rvert,\left\lvert R_{22}\right\rvert,\ldots,\left\lvert R_{{ii}}\right\rvert \right\}`, the rank of :math:`R` is defined to be smallest index `i` such that :math:`\left\lvert R_{{i+1,i+1}}\right\rvert \leq r\left\lvert \rho_{{i+1}}\right\rvert`.
If :math:`r\leq 0`, the default value is used.
.. _e04nc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`5`)
Too many iterations without changing :math:`x`.
(`errno` :math:`6`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`m = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`m > 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{kx}` has not been supplied as a valid permutation.
(`errno` :math:`6`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
The problem to be solved is of type QP1 or QP2, but the Hessian matrix supplied in :math:`\mathrm{a}` is not positive semidefinite.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Weak :math:`\langle\mathit{\boldsymbol{value}}\rangle` solution.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` solution is unbounded.
(`errno` :math:`3`)
Cannot satisfy the linear constraints.
.. _e04nc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_lincon_solve`` is designed to solve a class of quadratic programming problems of the following general form:
.. math::
\mathrm{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Cx\end{array}\right\}\leq u
where :math:`\mathrm{c}` is an :math:`n_L\times n` matrix and the objective function :math:`F\left(x\right)` may be specified in a variety of ways depending upon the particular problem to be solved.
The available forms for :math:`F\left(x\right)` are listed in Table [label omitted], in which the prefixes FP, LP, QP and LS stand for 'feasible point', 'linear programming', 'quadratic programming' and 'least squares' respectively, :math:`c` is an :math:`n`-element vector, :math:`b` is an :math:`m` element vector and :math:`\left\lVert z\right\rVert` denotes the Euclidean length of :math:`z`.
+------------+---------------------------------------------------------------+-------------------------------------------------+
|Problem type|:math:`F\left(x\right)` |Matrix :math:`A` |
+============+===============================================================+=================================================+
|FP |None |Not applicable |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LP |:math:`c^\mathrm{T}x` |Not applicable |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP1 |:math:`\frac{1}{2}x^\mathrm{T}Ax` |:math:`n\times n` symmetric positive semidefinite|
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP2 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Ax` |:math:`n\times n` symmetric positive semidefinite|
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP3 |:math:`\frac{1}{2}x^\mathrm{T}A^\mathrm{T}Ax` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP4 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}A^\mathrm{T}Ax` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS1 |:math:`\frac{1}{2}\left\lVert b-Ax\right\rVert^2` |:math:`m\times n` |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS2 |:math:`c^\mathrm{T}x+\frac{1}{2}\left\lVert b-Ax\right\rVert^2`|:math:`m\times n` |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS3 |:math:`\frac{1}{2}\left\lVert b-Ax\right\rVert^2` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS4 |:math:`c^\mathrm{T}x+\frac{1}{2}\left\lVert b-Ax\right\rVert^2`|:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
In the standard LS problem :math:`F\left(x\right)` will usually have the form LS1, and in the standard convex QP problem :math:`F\left(x\right)` will usually have the form QP2.
The default problem type is LS1 and other objective functions are selected by using the option 'Problem Type'.
When :math:`A` is upper trapezoidal it will usually be the case that :math:`m = n`, so that :math:`A` is upper triangular, but full generality has been allowed for in the specification of the problem.
The upper trapezoidal form is intended for cases where a previous factorization, such as a :math:`QR` factorization, has been performed.
The constraints involving :math:`\mathrm{c}` are called the `general` constraints.
Note that upper and lower bounds are specified for all the variables and for all the general constraints.
An equality constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
The defining feature of a quadratic function :math:`F\left(x\right)` is that the second-derivative matrix :math:`H` (the `Hessian matrix`) is constant.
For the LP case :math:`H = 0`; for QP1 and QP2, :math:`H = A`; for QP3 and QP4, :math:`H = A^\mathrm{T}A` and for LS1 (the default), LS2, LS3 and LS4, :math:`H = A^\mathrm{T}A`.
Problems of type QP3 and QP4 for which :math:`A` is not in upper trapezoidal form should be solved as types LS1 and LS2 respectively, with :math:`b = 0`.
For problems of type LS, we refer to :math:`A` as the `least squares` matrix, or the `matrix of observations` and to :math:`b` as the `vector of observations.`
You must supply an initial estimate of the solution.
If :math:`H` is nonsingular then ``lsq_lincon_solve`` will obtain the unique (global) minimum.
If :math:`H` is singular then the solution may still be a global minimum if all active constraints have nonzero Lagrange multipliers.
Otherwise the solution obtained will be either a weak minimum (i.e., with a unique optimal objective value, but an infinite set of optimal :math:`x`), or else the objective function is unbounded below in the feasible region.
The last case can only occur when :math:`F\left(x\right)` contains an explicit linear term (as in problems LP, QP2, QP4, LS2 and LS4).
The method used by ``lsq_lincon_solve`` is described in detail in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#algdetails>`__.
.. _e04nc-py2-py-references:
**References**
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Stoer, J, 1971, `On the numerical solution of constrained least squares problems`, SIAM J. Numer. Anal. (8), 382--411
"""
raise NotImplementedError
[docs]def lsq_lincon_option_file(ioptns, comm, io_manager=None):
r"""
``lsq_lincon_option_file`` may be used to supply options to :meth:`lsq_lincon_solve` from an external file.
.. _e04nd-py2-py-doc:
For full information please refer to the NAG Library document for e04nd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ndf.html
.. _e04nd-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04nd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_lincon_option_file`` may be used to supply values for options to :meth:`lsq_lincon_solve`. ``lsq_lincon_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lsq_lincon_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lsq_lincon_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lsq_lincon_solve <e04nc-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_lincon_option_string(optstr, comm, io_manager=None):
r"""
``lsq_lincon_option_string`` may be used to supply individual options to :meth:`lsq_lincon_solve`.
.. _e04ne-py2-py-doc:
For full information please refer to the NAG Library document for e04ne
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nef.html
.. _e04ne-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04ne-py2-py-notes>` and in :ref:`Other Parameters for lsq_lincon_solve <e04nc-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ne-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ne-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_lincon_option_string`` may be used to supply values for options to :meth:`lsq_lincon_solve`.
It is only necessary to call ``lsq_lincon_option_string`` for those arguments whose values are to be different from their default values.
One call to ``lsq_lincon_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lsq_lincon_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lsq_lincon_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lsq_lincon_solve <e04nc-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qp_dense_solve(bl, bu, h, x, comm, a=None, cvec=None, qphess=None, istate=None, data=None, io_manager=None):
r"""
``qp_dense_solve`` solves general quadratic programming problems.
It is not intended for large sparse problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`qp_dense_option_file`, :meth:`qp_dense_option_string`, :meth:`nlp1_init`.
.. _e04nf-py2-py-doc:
For full information please refer to the NAG Library document for e04nf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html
.. _e04nf-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**h** : float, array-like, shape :math:`\left(:, :\right)`
May be used to store the quadratic term :math:`H` of the QP objective function if desired. In some cases, you need not use :math:`\mathrm{h}` to store :math:`H` explicitly (see the specification of function :math:`\mathrm{qphess}`). The elements of :math:`\mathrm{h}` are referenced only by function :math:`\mathrm{qphess}`. The number of rows of :math:`H` is denoted by :math:`m`, whose default value is :math:`n`. (The option 'Hessian Rows' may be used to specify a value of :math:`m < n`.)
If the default version of :math:`\mathrm{qphess}` is used and the problem is of type QP1 or QP2 (the default), the first :math:`m` rows and columns of :math:`\mathrm{h}` must contain the leading :math:`m\times m` rows and columns of the symmetric Hessian matrix :math:`H`.
Only the diagonal and upper triangular elements of the leading :math:`m` rows and columns of :math:`\mathrm{h}` are referenced.
The remaining elements need not be assigned.
If the default version of :math:`\mathrm{qphess}` is used and the problem is of type QP3 or QP4, the first :math:`m` rows of :math:`\mathrm{h}` must contain an :math:`m\times n` upper trapezoidal factor of the symmetric Hessian matrix :math:`H^\mathrm{T}H`.
The factor need not be of full rank, i.e., some of the diagonal elements may be zero.
However, as a general rule, the larger the dimension of the leading nonsingular sub-matrix of :math:`\mathrm{h}`, the fewer iterations will be required.
Elements outside the upper trapezoidal part of the first :math:`m` rows of :math:`\mathrm{h}` need not be assigned.
If a non-default version of :math:`\mathrm{qphess}` is supplied, then in some cases it may be desirable to use a one-dimensional array to transmit data to :math:`\mathrm{qphess}`. :math:`\mathrm{h}` is then declared as an :math:`\textit{ldh}` by :math:`1` array, where :math:`\textit{ldh}\geq n\times \left(n+1\right)/2`.
In other situations, it may be desirable to compute :math:`Hx` or :math:`H^\mathrm{T}Hx` without accessing :math:`\mathrm{h}` -- for example, if :math:`H` or :math:`H^\mathrm{T}H` is sparse or has special structure.
The arguments :math:`\mathrm{h}` and :math:`\textit{ldh}` may then refer to any convenient array.
If the problem is of type FP or LP, :math:`\mathrm{h}` is not referenced.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**a** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; if :math:`\textit{nclin}=0`: :math:`1`; otherwise: :math:`0`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` must contain the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,m_L`.
**cvec** : None or float, array-like, shape :math:`\left(n\right)`, optional
The coefficients of the explicit linear term of the objective function when the problem is of type LP, QP2 (the default) and QP4.
If the problem is of type FP, QP1, or QP3, :math:`\mathrm{cvec}` is not referenced.
**qphess** : None or callable hx = qphess(jthcol, h, x, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
In general, you need not provide a version of :math:`\mathrm{qphess}`.
However, the algorithm of ``qp_dense_solve`` requires only the product of :math:`H` or :math:`H^\mathrm{T}H` and a vector :math:`x`; and in some cases you may obtain increased efficiency by providing a version of :math:`\mathrm{qphess}` that avoids the need to define the elements of the matrices :math:`H` or :math:`H^\mathrm{T}H` explicitly.
:math:`\mathrm{qphess}` is not referenced if the problem is of type FP or LP, in which case :math:`\mathrm{qphess}` may be **None**.
**Parameters**
**jthcol** : int
Specifies whether or not the vector :math:`x` is a column of the identity matrix.
:math:`\mathrm{jthcol} = j > 0`
The vector :math:`x` is the :math:`j`\ th column of the identity matrix, and hence :math:`Hx` or :math:`H^\mathrm{T}Hx` is the :math:`j`\ th column of :math:`H` or :math:`H^\mathrm{T}H`, respectively. This may in some cases require very little computation and :math:`\mathrm{qphess}` may be coded to take advantage of this. However special code is not necessary because :math:`x` is always stored explicitly in the array :math:`\mathrm{x}`.
:math:`\mathrm{jthcol} = 0`
:math:`x` has no special form.
**h** : float, ndarray, shape :math:`\left(:, :\right)`
See: :math:`\mathrm{h}`
**x** : float, ndarray, shape :math:`\left(n\right)`
The vector :math:`x`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**hx** : float, array-like, shape :math:`\left(n\right)`
The product :math:`Hx` if the problem is of type QP1 or QP2 (the default), or the product :math:`H^\mathrm{T}Hx` if the problem is of type QP3 or QP4.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{istate}` specifies the desired status of the constraints at the start of the feasibility phase.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, and the next :math:`m_L` elements refer to the general linear constraints (if any).
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+========================================================================================================================================================+
|0 |The corresponding constraint should `not` be in the initial working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraint should be in the initial working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraint should be in the initial working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The constraint should be in the initial working set as an equality. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be reset to zero by the function.
If ``qp_dense_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\textit{nclin}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The status of the constraints in the working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================+
|:math:`-2` |The constraint violates its lower bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |The constraint violates its upper bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |This corresponds to optimality being declared with :math:`\mathrm{x}[j-1]` being temporarily fixed at its current value. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{errno}` = 1 on exit.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**x** : float, ndarray, shape :math:`\left(n\right)`
The point at which ``qp_dense_solve`` terminated. If the function exits successfully or :math:`\mathrm{errno}` = 1 or 4, :math:`\mathrm{x}` contains an estimate of the solution.
**itera** : int
The total number of iterations performed.
**obj** : float
The value of the objective function at :math:`x` if :math:`x` is feasible, or the sum of infeasibilities at :math:`x` otherwise. If the problem is of type FP and :math:`x` is feasible, :math:`\mathrm{obj}` is set to zero.
**ax** : None or float, ndarray, shape :math:`\left(\max\left(1,\textit{nclin}\right)\right)`
The final values of the linear constraints :math:`Ax`.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The values of the Lagrange multipliers for each constraint with respect to the current working set. The first :math:`n` elements contain the multipliers for the bound constraints on the variables, and the next :math:`m_L` elements contain the multipliers for the general linear constraints (if any). If :math:`\mathrm{istate}[j-1] = 0` (i.e., constraint :math:`j` is not in the working set), :math:`\mathrm{clamda}[j-1]` is zero. If :math:`x` is optimal, :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1`, non-positive if :math:`\mathrm{istate}[j-1] = 2` and zero if :math:`\mathrm{istate}[j-1] = 4`.
.. _e04nf-py2-py-other_params:
**Other Parameters**
**'Check Frequency'** : float
Default :math:`\text{} = 50`
Every :math:`i`\ th iteration, a numerical test is made to see if the current solution :math:`x` satisfies the constraints in the working set.
If the largest residual of the constraints in the working set is judged to be too large, the current working set is refactorized and the variables are recomputed to satisfy the constraints more accurately.
If :math:`i\leq 0`, the default value is used.
**'Cold Start'** : valueless
Default
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``qp_dense_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``qp_dense_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``qp_dense_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``qp_dense_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``qp_dense_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``qp_dense_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``qp_dense_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Expand Frequency'** : int
Default :math:`\text{} = 5`
This option is part of an anti-cycling procedure designed to guarantee progress even on highly degenerate problems.
The strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of the option 'Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``qp_dense_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
At certain stages the following 'resetting procedure' is used to remove constraint infeasibilities.
First, all variables whose upper or lower bounds are in the working set are moved exactly onto their bounds.
A count is kept of the number of nontrivial adjustments made.
If the count is positive, iterative refinement is used to give variables that satisfy the working set to (essentially) machine precision.
Finally, the working feasibility tolerance is reinitialized to :math:`0.5\delta`.
If a problem requires more than :math:`i` iterations, the resetting procedure is invoked and a new cycle of :math:`i` iterations is started with :math:`i` incremented by :math:`10`. (The decision to resume the feasibility phase or optimality phase is based on comparing any constraint infeasibilities with :math:`\delta`.)
The resetting procedure is also invoked when ``qp_dense_solve`` reaches an apparently optimal, infeasible or unbounded solution, unless this situation has already occurred twice.
If any nontrivial adjustments are made, iterations are continued.
If :math:`i\leq 0`, the default value is used.
If :math:`i\geq 9999999`, no anti-cycling procedure is invoked.
**'Feasibility Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
For problems of type FP, the scalar :math:`i_1` specifies the maximum number of iterations allowed before temination.
Setting :math:`i_1 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
For problems of type LP, the maximum number of iterations allowed before temination is taken as :math:`\mathrm{max}\left(i_1, i_2\right)`.
Setting :math:`i_1 = 0`, :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
For problems of type QP, the scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. 'Optimality Phase Iteration Limit' is equivalent to 'Iteration Limit'.
Setting :math:`i_1 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Optimality Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
For problems of type FP, the scalar :math:`i_1` specifies the maximum number of iterations allowed before temination.
Setting :math:`i_1 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
For problems of type LP, the maximum number of iterations allowed before temination is taken as :math:`\mathrm{max}\left(i_1, i_2\right)`.
Setting :math:`i_1 = 0`, :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
For problems of type QP, the scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. 'Optimality Phase Iteration Limit' is equivalent to 'Iteration Limit'.
Setting :math:`i_1 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
If :math:`r\geq \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point.
For example, if the variables and the coefficients in the general constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-6}`.
If :math:`0\leq r < \epsilon`, the default value is used.
``qp_dense_solve`` attempts to find a feasible solution before optimizing the objective function.
If the sum of infeasibilities cannot be reduced to zero, the option 'Minimum Sum of Infeasibilities' can be used to find the minimum value of the sum.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Note that a 'feasible solution' is a solution that satisfies the current constraints to within the tolerance :math:`r`.
**'Hessian Rows'** : int
Default :math:`\text{} = n`
Note that this option does not apply to problems of type FP or LP.
This specifies :math:`m`, the number of rows of the Hessian matrix :math:`H`.
The default value of :math:`m` is :math:`n`, the number of variables of the problem.
If the problem is of type QP, then :math:`m` will usually be :math:`n`, the number of variables.
However, a value of :math:`m` less than :math:`n` is appropriate for QP3 or QP4 if :math:`H` is an upper trapezoidal matrix with :math:`m` rows.
Similarly, :math:`m` may be used to define the dimension of a leading block of nonzeros in the Hessian matrices of QP1 or QP2.
In this case the last :math:`n-m` rows and columns of :math:`H` are assumed to be zero.
In the QP case :math:`m` should not be greater than :math:`n`; if it is, the last :math:`m-n` rows of :math:`H` are ignored.
If :math:`i < 0` or :math:`i > n`, the default value is used.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the Hessian is not positive definite.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, then the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Maximum Degrees of Freedom'** : int
Default :math:`\text{} = n`
Note that this option does not apply to problems of type FP or LP.
This places a limit on the storage allocated for the triangular factor :math:`R` of the reduced Hessian :math:`H_R`.
Ideally, :math:`i` should be set slightly larger than the value of :math:`n_R` expected at the solution.
It need not be larger than :math:`m_n+1`, where :math:`m_n` is the number of variables that appear nonlinearly in the quadratic objective function.
For many problems it can be much smaller than :math:`m_n`.
For quadratic problems, a minimizer may lie on any number of constraints, so that :math:`n_R` may vary between :math:`1` and :math:`n`.
The default value of :math:`i` is, therefore, the number of variables :math:`n`.
If 'Hessian Rows' :math:`m` is specified, the default value of :math:`i` is the same number, :math:`m`.
**'Minimum Sum of Infeasibilities'** : str
Default :math:`= \mathrm{NO}`
If no feasible point exists for the constraints, then this option is used to control whether or not ``qp_dense_solve`` will calculate a point that minimizes the constraint violations.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'NO'}`, ``qp_dense_solve`` will terminate as soon as it is evident that no feasible point exists for the constraints.
The final point will generally not be the point at which the sum of infeasibilities is minimized.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'YES'}`, ``qp_dense_solve`` will continue until the sum of infeasibilities is minimized.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i > 6` and :math:`\text{‘Print Level'} \geq 5`, monitoring information produced by ``qp_dense_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.5}`
If :math:`r\geq \epsilon`, :math:`r` defines the tolerance used to determine if the bounds and general constraints have the right 'sign' for the solution to be judged to be optimal.
If :math:`0\leq r < \epsilon`, the default value is used.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``qp_dense_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#monitoring>`__ (monitoring information at each iteration).
If :math:`i < 0`, the default value is used.
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=======================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#fc-printedoutput>`__) for each iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=====================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#monitoring>`__) for each iteration (no printout of the final solution). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each iteration: the Lagrange multipliers, the variables :math:`x`, the constraint values :math:`Ax` and the constraint status (see :math:`\mathrm{istate}`). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each iteration: the diagonal elements of the upper triangular matrix :math:`T` associated with the :math:`TQ` factorization `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#eqn3>`__ (see `Definition of Search Direction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#ad-search>`__) of the working set and the diagonal elements of the upper triangular matrix :math:`R`.|
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output is suppressed.
**'Problem Type'** : str
Default :math:`=` QP2
This option specifies the type of objective function to be minimized during the optimality phase.
The following are the five optional keywords and the dimensions of the arrays that must be specified in order to define the objective function:
.. rst-class:: nag-rules-none
+---+--------------------------------------------------------------------------------------------------------------+
|LP |:math:`\mathrm{h}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}` required; |
+---+--------------------------------------------------------------------------------------------------------------+
|QP1|rank-:math:`2` :math:`\mathrm{h}` symmetric, :math:`\mathrm{cvec}` not referenced; |
+---+--------------------------------------------------------------------------------------------------------------+
|QP2|rank-:math:`2` :math:`\mathrm{h}` symmetric, length-:math:`\textit{n}` :math:`\mathrm{cvec}` required; |
+---+--------------------------------------------------------------------------------------------------------------+
|QP3|rank-:math:`2` :math:`\mathrm{h}` upper trapezoidal, :math:`\mathrm{cvec}` not referenced; |
+---+--------------------------------------------------------------------------------------------------------------+
|QP4|rank-:math:`2` :math:`\mathrm{h}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{cvec}` required.|
+---+--------------------------------------------------------------------------------------------------------------+
|FP |the objective function is omitted and neither :math:`\mathrm{h}` nor :math:`\mathrm{cvec}` are referenced. |
+---+--------------------------------------------------------------------------------------------------------------+
For problems of type FP the objective function is omitted and neither :math:`\mathrm{h}` nor :math:`\mathrm{cvec}` are referenced.
The following keywords are also acceptable.
.. rst-class:: nag-rules-none nag-align-left
+---------+------+
|:math:`a`|Option|
+=========+======+
|Quadratic|QP2 |
+---------+------+
|Linear |LP |
+---------+------+
|Feasible |FP |
+---------+------+
In addition, the keyword QP is equivalent to the default option QP2.
If :math:`H = 0` (i.e., the objective function is purely linear), the efficiency of ``qp_dense_solve`` may be increased by specifying :math:`a` as LP.
**'Rank Tolerance'** : float
Default :math:`\text{} = 100\epsilon`
Note that this option does not apply to problems of type FP or LP.
This option enables you to control the condition number of the triangular factor :math:`R` (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#algdetails>`__).
If :math:`\rho_i` denotes the function :math:`\rho_i = \mathrm{max}\left\{\left\lvert R_{11}\right\rvert,\left\lvert R_{22}\right\rvert,\ldots,\left\lvert R_{{ii}}\right\rvert \right\}`, the dimension of :math:`R` is defined to be smallest index :math:`i` such that :math:`\left\lvert R_{{i+1,i+1}}\right\rvert \leq \sqrt{r}\left\lvert \rho_{{i+1}}\right\rvert`.
If :math:`r\leq 0`, the default value is used.
.. _e04nf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`5`)
Reduced Hessian exceeds assigned dimension.
(`errno` :math:`6`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Cold Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
'Problem Type' not recognized. Problem abandoned.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Iterations terminated at a dead point.
(`errno` :math:`1`)
Optimal solution is not unique.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` solution is unbounded.
(`errno` :math:`3`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
Cannot satisfy the working set constraints to the accuracy requested.
.. _e04nf-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``qp_dense_solve`` is designed to solve a class of quadratic programming problems that are assumed to be stated in the following general form:
.. math::
\mathrm{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }\quad l\leq \begin{pmatrix}x\\Ax\end{pmatrix}\leq u\text{,}
where :math:`A` is an :math:`m_L\times n` matrix and :math:`f\left(x\right)` may be specified in a variety of ways depending upon the particular problem to be solved.
The available forms for :math:`f\left(x\right)` are listed in Table [label omitted], in which the prefixes FP, LP and QP stand for 'feasible point', 'linear programming' and 'quadratic programming' respectively and :math:`c` is an :math:`n`-element vector.
+------------+-----------------------------------------------------------+-----------------------------------+
|Problem type|:math:`f\left(x\right)` |Matrix :math:`H` |
+============+===========================================================+===================================+
|FP |Not applicable |Not applicable |
+------------+-----------------------------------------------------------+-----------------------------------+
|LP |:math:`c^\mathrm{T}x` |Not applicable |
+------------+-----------------------------------------------------------+-----------------------------------+
|QP1 |:math:`\frac{1}{2}x^\mathrm{T}Hx` |symmetric |
+------------+-----------------------------------------------------------+-----------------------------------+
|QP2 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx` |symmetric |
+------------+-----------------------------------------------------------+-----------------------------------+
|QP3 |:math:`\frac{1}{2}x^\mathrm{T}H^\mathrm{T}Hx` |:math:`m\times n` upper trapezoidal|
+------------+-----------------------------------------------------------+-----------------------------------+
|QP4 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}H^\mathrm{T}Hx`|:math:`m\times n` upper trapezoidal|
+------------+-----------------------------------------------------------+-----------------------------------+
There is no restriction on :math:`H` or :math:`H^\mathrm{T}H` apart from symmetry.
If the quadratic function is convex, a global minimum is found; otherwise, a local minimum is found.
The default problem type is QP2 and other objective functions are selected by using the option 'Problem Type'.
For problems of type FP, the objective function is omitted and the function attempts to find a feasible point for the set of constraints.
The constraints involving :math:`A` are called the `general` constraints.
Note that upper and lower bounds are specified for all the variables and for all the general constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
The defining feature of a quadratic function :math:`f\left(x\right)` is that the second-derivative matrix :math:`\nabla^2f\left(x\right)` (the `Hessian matrix`) is constant.
For QP1 and QP2 (the default), :math:`\nabla^2f\left(x\right) = H`; for QP3 and QP4, :math:`\nabla^2f\left(x\right) = H^\mathrm{T}H`; and for the LP case, :math:`\nabla^2f\left(x\right) = 0`.
If :math:`H` is positive semidefinite, it is usually more efficient to use :meth:`lsq_lincon_solve`.
If :math:`H` is defined as the zero matrix, ``qp_dense_solve`` will still attempt to solve the resulting linear programming problem; however, this can be accomplished more efficiently by setting the option the problem is of type 'LP', or by using :meth:`lp_solve` instead.
You must supply an initial estimate of the solution.
In the QP case, you may supply :math:`H` either `explicitly` as an :math:`m\times n` matrix, or `implicitly` in a function that computes the product :math:`Hx` or :math:`H^\mathrm{T}Hx` for any given vector :math:`x`.
In general, a successful run of ``qp_dense_solve`` will indicate one of three situations:
(i) a minimizer has been found;
(#) the algorithm has terminated at a so-called `dead-point`; or
(#) the problem has no bounded solution.
If a minimizer is found, and :math:`\nabla^2f\left(x\right)` is positive definite or positive semidefinite, ``qp_dense_solve`` will obtain a global minimizer; otherwise, the solution will be a `local` minimizer (which may or may not be a global minimizer).
A dead-point is a point at which the necessary conditions for optimality are satisfied but the sufficient conditions are not.
At such a point, a feasible direction of decrease may or may not exist, so that the point is not necessarily a local solution of the problem.
Verification of optimality in such instances requires further information, and is in general an NP-hard problem (see Pardalos and Schnitger (1988)).
Termination at a dead-point can occur only if :math:`\nabla^2f\left(x\right)` is not positive definite.
If :math:`\nabla^2f\left(x\right)` is positive semidefinite, the dead-point will be a `weak minimizer` (i.e., with a unique optimal objective value, but an infinite set of optimal :math:`x`).
The method used by ``qp_dense_solve`` (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nff.html#algdetails>`__) is most efficient when many constraints or bounds are active at the solution.
.. _e04nf-py2-py-references:
**References**
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E and Murray, W, 1978, `Numerically stable methods for quadratic programming`, Math. Programming (14), 349--372
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1991, `Inertia-controlling methods for general quadratic programming`, SIAM Rev. (33), 1--36
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Pardalos, P M and Schnitger, G, 1988, `Checking local optimality in constrained quadratic programming is NP-hard`, Operations Research Letters (7), 33--35
"""
raise NotImplementedError
[docs]def qp_dense_option_file(ioptns, comm, io_manager=None):
r"""
``qp_dense_option_file`` may be used to supply options to :meth:`qp_dense_solve` from an external file.
.. _e04ng-py2-py-doc:
For full information please refer to the NAG Library document for e04ng
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ngf.html
.. _e04ng-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ng-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04ng-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``qp_dense_option_file`` may be used to supply values for options to :meth:`qp_dense_solve`. ``qp_dense_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`qp_dense_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qp_dense_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qp_dense_solve <e04nf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qp_dense_option_string(optstr, comm, io_manager=None):
r"""
``qp_dense_option_string`` may be used to supply individual options to :meth:`qp_dense_solve`.
.. _e04nh-py2-py-doc:
For full information please refer to the NAG Library document for e04nh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nhf.html
.. _e04nh-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04nh-py2-py-notes>` and in :ref:`Other Parameters for qp_dense_solve <e04nf-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nh-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``qp_dense_option_string`` may be used to supply values for options to :meth:`qp_dense_solve`.
It is only necessary to call ``qp_dense_option_string`` for those arguments whose values are to be different from their default values.
One call to ``qp_dense_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`qp_dense_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qp_dense_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qp_dense_solve <e04nf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex1_sparse_solve(m, iobj, ncolh, a, ha, ka, bl, bu, start, names, crname, ns, xs, istate, leniz, lenz, comm, qphx=None, data=None, io_manager=None):
r"""
``qpconvex1_sparse_solve`` solves sparse linear programming or convex quadratic programming problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`qpconvex1_sparse_option_file`, :meth:`qpconvex1_sparse_option_string`, :meth:`nlp1_init`.
.. _e04nk-py2-py-doc:
For full information please refer to the NAG Library document for e04nk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html
.. _e04nk-py2-py-parameters:
**Parameters**
**m** : int
:math:`m`, the number of general linear constraints (or slacks). This is the number of rows in :math:`A`, including the free row (if any; see :math:`\mathrm{iobj}`).
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero elements of the vector :math:`c` appearing in the linear objective term :math:`c^\mathrm{T}x`.
If :math:`\mathrm{iobj} = 0`, there is no free row, i.e., the problem is either an FP problem (in which case :math:`\mathrm{iobj}` must be set to zero), or a QP problem with :math:`c = 0`.
**ncolh** : int
:math:`n_H`, the number of leading nonzero columns of the Hessian matrix :math:`H`. For FP and LP problems, :math:`\mathrm{ncolh}` must be set to zero.
**a** : float, array-like, shape :math:`\left(\textit{nnz}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index. Note that elements with the same row and column indices are not allowed.
**ha** : int, array-like, shape :math:`\left(\textit{nnz}\right)`
:math:`\mathrm{ha}[\textit{i}-1]` must contain the row index of the nonzero element stored in :math:`\mathrm{a}[\textit{i}-1]`, for :math:`\textit{i} = 1,2,\ldots,\textit{nnz}`. Note that the row indices for a column may be supplied in any order.
**ka** : int, array-like, shape :math:`\left(n+1\right)`
:math:`\mathrm{ka}[\textit{j}-1]` must contain the index in :math:`\mathrm{a}` of the start of the :math:`\textit{j}`\ th column, for :math:`\textit{j} = 1,2,\ldots,n`. :math:`\mathrm{ka}[n]` must be set to :math:`\textit{nnz}+1`. To specify the :math:`j`\ th column as empty, set :math:`\mathrm{ka}[j-1] = \mathrm{ka}[j]`. As a consequence :math:`\mathrm{ka}[0]` is always :math:`1`.
**bl** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`l`, the lower bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bl}` must contain the bounds on the variables :math:`x`, and the next :math:`\mathrm{m}` elements the bounds for the general linear constraints :math:`Ax` (or slacks :math:`s`) and the free row (if any). To specify a nonexistent lower bound (i.e., :math:`l_j = {-\infty }`), set :math:`\mathrm{bl}[j-1]\leq -\textit{bigbnd}`, where :math:`\textit{bigbnd}` is the value of the option 'Infinite Bound Size'. To specify the :math:`j`\ th constraint as an `equality`, set :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1] = \beta`, say, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. Note that the lower bound corresponding to the free row must be set to :math:`{-\infty }` and stored in :math:`\mathrm{bl}[n+\mathrm{iobj}-1]`.
**bu** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`u`, the upper bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bu}` must contain the bounds on the variables :math:`x`, and the next :math:`\mathrm{m}` elements the bounds for the general linear constraints :math:`Ax` (or slacks :math:`s`) and the free row (if any). To specify a nonexistent upper bound (i.e., :math:`u_j = {+\infty }`), set :math:`\mathrm{bu}[j-1]\geq \textit{bigbnd}`. Note that the upper bound corresponding to the free row must be set to :math:`{+\infty }` and stored in :math:`\mathrm{bu}[n+\mathrm{iobj}-1]`.
**start** : str, length 1
Indicates how a starting basis is to be obtained.
:math:`\mathrm{start} = \texttt{'C'}`
An internal Crash procedure will be used to choose an initial basis matrix :math:`B`.
:math:`\mathrm{start} = \texttt{'W'}`
A basis is already defined in :math:`\mathrm{istate}` (probably from a previous call).
**names** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the so-called MPSX form of the problem, as follows:
:math:`\mathrm{names}[0]`
Must contain the name for the problem (or be blank).
:math:`\mathrm{names}[1]`
Must contain the name for the free row (or be blank).
:math:`\mathrm{names}[2]`
Must contain the name for the constraint right-hand side (or be blank).
:math:`\mathrm{names}[3]`
Must contain the name for the ranges (or be blank).
:math:`\mathrm{names}[4]`
Must contain the name for the bounds (or be blank).
(These names are used in the monitoring file output; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#monitoring>`__.)
**crname** : str, length 8, array-like, shape :math:`\left(\textit{nname}\right)`
The optional column and row names, respectively.
If :math:`\textit{nname} = 1`, :math:`\mathrm{crname}` is not referenced and the printed output will use default names for the columns and rows.
If :math:`\textit{nname} = n+\mathrm{m}`, the first :math:`\textit{n}` elements must contain the names for the columns and the next :math:`\mathrm{m}` elements must contain the names for the rows.
Note that the name for the free row (if any) must be stored in :math:`\mathrm{crname}[n+\mathrm{iobj}-1]`.
**ns** : int
:math:`n_S`, the number of superbasics. For QP problems, :math:`\mathrm{ns}` need not be specified if :math:`\mathrm{start} = \texttt{'C'}`, but must retain its value from a previous call when :math:`\mathrm{start} = \texttt{'W'}`. For FP and LP problems, :math:`\mathrm{ns}` need not be initialized.
**xs** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
The initial values of the variables and slacks :math:`\left(x, s\right)`. (See the description for :math:`\mathrm{istate}`.)
**istate** : int, array-like, shape :math:`\left(n+\mathrm{m}\right)`
If :math:`\mathrm{start} = \texttt{'C'}`, the first :math:`\textit{n}` elements of :math:`\mathrm{istate}` and :math:`\mathrm{xs}` must specify the initial states and values, respectively, of the variables :math:`x`. (The slacks :math:`s` need not be initialized.) An internal Crash procedure is then used to select an initial basis matrix :math:`B`. The initial basis matrix will be triangular (neglecting certain small elements in each column). It is chosen from various rows and columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}`. Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|State of :math:`\mathrm{xs}[j-1]` during Crash procedure |
+============================+=====================================================================+
|:math:`0` or :math:`1` |Eligible for the basis |
+----------------------------+---------------------------------------------------------------------+
|:math:`2` |Ignored |
+----------------------------+---------------------------------------------------------------------+
|:math:`3` |Eligible for the basis (given preference over :math:`0` or :math:`1`)|
+----------------------------+---------------------------------------------------------------------+
|:math:`4` or :math:`5` |Ignored |
+----------------------------+---------------------------------------------------------------------+
If nothing special is known about the problem, or there is no wish to provide special information, you may set :math:`\mathrm{istate}[\textit{j}-1] = 0` and :math:`\mathrm{xs}[\textit{j}-1] = 0.0`, for :math:`\textit{j} = 1,2,\ldots,n`.
All variables will then be eligible for the initial basis.
Less trivially, to say that the :math:`\textit{j}`\ th variable will probably be equal to one of its bounds, set :math:`\mathrm{istate}[j-1] = 4` and :math:`\mathrm{xs}[j-1] = \mathrm{bl}[j-1]` or :math:`\mathrm{istate}[j-1] = 5` and :math:`\mathrm{xs}[j-1] = \mathrm{bu}[j-1]` as appropriate.
Following the Crash procedure, variables for which :math:`\mathrm{istate}[j-1] = 2` are made superbasic.
Other variables not selected for the basis are then made nonbasic at the value :math:`\mathrm{xs}[j-1]` if :math:`\mathrm{bl}[j-1]\leq \mathrm{xs}[j-1]\leq \mathrm{bu}[j-1]`, or at the value :math:`\mathrm{bl}[j-1]` or :math:`\mathrm{bu}[j-1]` closest to :math:`\mathrm{xs}[j-1]`.
If :math:`\mathrm{start} = \texttt{'W'}`, :math:`\mathrm{istate}` and :math:`\mathrm{xs}` must specify the initial states and values, respectively, of the variables and slacks :math:`\left(x, s\right)`.
If ``qpconvex1_sparse_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\mathrm{m}`, :math:`\mathrm{istate}` already contains satisfactory information.
**leniz** : int
The dimension of the internal workspace array :math:`\textit{iz}`.
**lenz** : int
The dimension of the internal workspace array :math:`\textit{z}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**qphx** : None or callable hx = qphx(nstate, x, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
For QP problems, you must supply a version of :math:`\mathrm{qphx}` to compute the matrix product :math:`Hx`.
If :math:`H` has zero rows and columns, it is most efficient to order the variables :math:`x = \begin{pmatrix}y&z\end{pmatrix}^\mathrm{T}` so that
.. math::
Hx = \begin{pmatrix}H_1&0\\0&0\end{pmatrix}\begin{pmatrix}y\\z\end{pmatrix} = \begin{pmatrix}H_1y\\0\end{pmatrix}\text{,}
where the nonlinear variables :math:`y` appear first as shown.
For FP and LP problems, :math:`\mathrm{qphx}` will never be called by ``qpconvex1_sparse_solve`` and hence :math:`\mathrm{qphx}` may be **None**.
**Parameters**
**nstate** : int
If :math:`\mathrm{nstate} = 1`, ``qpconvex1_sparse_solve`` is calling :math:`\mathrm{qphx}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
If :math:`\mathrm{nstate}\geq 2`, ``qpconvex1_sparse_solve`` is calling :math:`\mathrm{qphx}` for the last time.
This argument setting allows you to perform some additional computation on the final solution.
In general, the last call to :math:`\mathrm{qphx}` is made with :math:`\mathrm{nstate} = 2+\textit{errno}` (see :ref:`Exceptions <e04nk-py2-py-errors>`).
Otherwise, :math:`\mathrm{nstate} = 0`.
**x** : float, ndarray, shape :math:`\left(\textit{ncolh}\right)`
The first :math:`\mathrm{ncolh}` elements of the vector :math:`x`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**hx** : float, array-like, shape :math:`\left(\textit{ncolh}\right)`
The product :math:`Hx`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**ns** : int
The final number of superbasics. This will be zero for FP and LP problems.
**xs** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final values of the variables and slacks :math:`\left(x, s\right)`.
**istate** : int, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final states of the variables and slacks :math:`\left(x, s\right)`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|State of variable :math:`j`|Normal value of :math:`\mathrm{xs}[j-1]` |
+============================+===========================+=============================================================+
|:math:`0` |Nonbasic |:math:`\mathrm{bl}[j-1]` |
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`1` |Nonbasic |:math:`\mathrm{bu}[j-1]` |
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`2` |Superbasic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`3` |Basic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+----------------------------+---------------------------+-------------------------------------------------------------+
If :math:`\mathrm{ninf} = 0`, basic and superbasic variables may be outside their bounds by as much as the value of the option 'Feasibility Tolerance'.
Note that unless the :math:`\text{‘Scale Option'} = 0` is specified, the option 'Feasibility Tolerance' applies to the variables of the scaled problem.
In this case, the variables of the original problem may be as much as :math:`0.1` outside their bounds, but this is unlikely unless the problem is very badly scaled.
Very occasionally some nonbasic variables may be outside their bounds by as much as the option 'Feasibility Tolerance', and there may be some nonbasic variables for which :math:`\mathrm{xs}[j-1]` lies strictly between its bounds.
If :math:`\mathrm{ninf} > 0`, some basic and superbasic variables may be outside their bounds by an arbitrary amount (bounded by :math:`\mathrm{sinf}` if :math:`\text{‘Scale Option'} = 0`).
**miniz** : int
The minimum value of :math:`\mathrm{leniz}` required to start solving the problem. If :math:`\mathrm{errno}` = 12, ``qpconvex1_sparse_solve`` may be called again with :math:`\mathrm{leniz}` suitably larger than :math:`\mathrm{miniz}`. (The bigger the better, since it is not certain how much workspace the basis factors need.)
**minz** : int
The minimum value of :math:`\mathrm{lenz}` required to start solving the problem. If :math:`\mathrm{errno}` = 13, ``qpconvex1_sparse_solve`` may be called again with :math:`\mathrm{lenz}` suitably larger than :math:`\mathrm{minz}`. (The bigger the better, since it is not certain how much workspace the basis factors need.)
**ninf** : int
The number of infeasibilities. This will be zero if the function exits successfully or :math:`\mathrm{errno}` = 1.
**sinf** : float
The sum of infeasibilities. This will be zero if :math:`\mathrm{ninf} = 0`. (Note that ``qpconvex1_sparse_solve`` does `not` attempt to compute the minimum value of :math:`\mathrm{sinf}` if :math:`\mathrm{errno}` = 3.)
**obj** : float
The value of the objective function.
If :math:`\mathrm{ninf} = 0`, :math:`\mathrm{obj}` includes the quadratic objective term :math:`\frac{1}{2}x^\mathrm{T}Hx` (if any).
If :math:`\mathrm{ninf} > 0`, :math:`\mathrm{obj}` is just the linear objective term :math:`c^\mathrm{T}x` (if any).
For FP problems, :math:`\mathrm{obj}` is set to zero.
**clamda** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
A set of Lagrange multipliers for the bounds on the variables and the general constraints. More precisely, the first :math:`\textit{n}` elements contain the multipliers (`reduced costs`) for the bounds on the variables, and the next :math:`\mathrm{m}` elements contain the multipliers (`shadow prices`) for the general linear constraints.
.. _e04nk-py2-py-other_params:
**Other Parameters**
**'Check Frequency'** : int
Default :math:`\text{} = 60`
Every :math:`i`\ th iteration after the most recent basis factorization, a numerical test is made to see if the current solution :math:`\left(x, s\right)` satisfies the linear constraints :math:`Ax-s = 0`.
If the largest element of the residual vector :math:`r = Ax-s` is judged to be too large, the current basis is refactorized and the basic variables recomputed to satisfy the constraints more accurately.
If :math:`i < 0`, the default value is used.
If :math:`i = 0`, the value :math:`i = 99999999` is used and effectively no checks are made.
**'Crash Option'** : int
Default :math:`\text{} = 2`
Note that this option does not apply when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04nk-py2-py-parameters>`).
If :math:`\mathrm{start} = \texttt{'C'}`, an internal Crash procedure is used to select an initial basis from various rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The value of :math:`i` determines which rows and columns are initially eligible for the basis, and how many times the Crash procedure is called.
If :math:`i = 0`, the all-slack basis :math:`B = -I` is chosen.
If :math:`i = 1`, the Crash procedure is called once (looking for a triangular basis in all rows and columns of the linear constraint matrix :math:`A`).
If :math:`i = 2`, the Crash procedure is called twice (looking at any `equality` constraints first followed by any `inequality` constraints).
If :math:`i < 0` or :math:`i > 2`, the default value is used.
If :math:`i = 1` or :math:`2`, certain slacks on inequality rows are selected for the basis first. (If :math:`i = 2`, numerical values are used to exclude slacks that are close to a bound.) The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.1`
This value allows the Crash procedure to ignore certain 'small' nonzero elements in the constraint matrix :math:`A` while searching for a triangular basis.
For each column of :math:`A`, if :math:`a_{\mathrm{max}}` is the largest element in the column, other nonzeros in that column are ignored if they are less than (or equal to) :math:`a_{\mathrm{max}}\times r`.
When :math:`r > 0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis with more column variables and fewer (arbitrary) slacks.
A feasible solution may be reached earlier for some problems.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Expand Frequency'** : int
Default :math:`\text{} = 10000`
This option is part of an anti-cycling procedure (see `Miscellaneous <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#algdetails4>`__) designed to allow progress even on highly degenerate problems.
For LP problems, the strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of the option 'Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``qpconvex1_sparse_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
For QP problems, the same procedure is used for iterations in which there is only one superbasic variable. (Cycling can only occur when the current solution is at a vertex of the feasible region.) Thus, zero steps are allowed if there is more than one superbasic variable, but otherwise positive steps are enforced.
Increasing the value of :math:`i` helps reduce the number of slightly infeasible nonbasic basic variables (most of which are eliminated during the resetting procedure).
However, it also diminishes the freedom to choose a large pivot element (see option 'Pivot Tolerance').
If :math:`i < 0`, the default value is used.
If :math:`i = 0`, the value :math:`i = 99999999` is used and effectively no anti-cycling procedure is invoked.
**'Factorization Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i > 0`, at most :math:`i` basis changes will occur between factorizations of the basis matrix.
For LP problems, the basis factors are usually updated at every iteration.
For QP problems, fewer basis updates will occur as the solution is approached.
The number of iterations between basis factorizations will, therefore, increase.
During these iterations a test is made regularly according to the value of option 'Check Frequency' to ensure that the linear constraints :math:`Ax-s = 0` are satisfied.
If necessary, the basis will be refactorized before the limit of :math:`i` updates is reached.
If :math:`i\leq 0`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left(10^{-6}, {\sqrt{\epsilon }}\right)`
If :math:`r\geq \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point (including slack variables).
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about five decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-5}`.
If :math:`r < \epsilon`, the default value is used.
``qpconvex1_sparse_solve`` attempts to find a feasible solution before optimizing the objective function.
If the sum of infeasibilities cannot be reduced to zero, the problem is assumed to be `infeasible`.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Note that the function does `not` attempt to find the minimum value of Sinf.
If the constraints and variables have been scaled (see 'Scale Option'), then feasibility is defined in terms of the scaled problem (since it is more likely to be meaningful).
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r\leq 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the Hessian is not positive definite.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed to start solving the problem will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed to start solving the problem will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed to start solving the problem will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'LU Factor Tolerance'** : float
Default :math:`\text{} = 100.0`
The values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
For large and relatively dense problems, setting :math:`r_1` and :math:`r_2` to :math:`25` (say) may give a marked improvement in sparsity without impairing stability to a serious degree.
Note that for band matrices it may be necessary to set :math:`r_1` in the range :math:`1\leq r_1 < 2` in order to achieve stability.
If :math:`r_1 < 1` or :math:`r_2 < 1`, the default value is used.
**'LU Update Tolerance'** : float
Default :math:`\text{} = 10.0`
The values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
For large and relatively dense problems, setting :math:`r_1` and :math:`r_2` to :math:`25` (say) may give a marked improvement in sparsity without impairing stability to a serious degree.
Note that for band matrices it may be necessary to set :math:`r_1` in the range :math:`1\leq r_1 < 2` in order to achieve stability.
If :math:`r_1 < 1` or :math:`r_2 < 1`, the default value is used.
**'LU Singularity Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.67}`
If :math:`r > 0`, :math:`r` defines the singularity tolerance used to guard against ill-conditioned basis matrices.
Whenever the basis is refactorized, the diagonal elements of :math:`U` are tested as follows.
If :math:`\left\lvert u_{{jj}}\right\rvert \leq r` or :math:`\left\lvert u_{{jj}}\right\rvert < r\times \mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable.
If :math:`r\leq 0`, the default value is used.
**'Minimize'** : valueless
Default
This option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#ad-mainiteration>`__) will be reversed.
**'Maximize'** : valueless
This option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#ad-mainiteration>`__) will be reversed.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Print Level'} > 0` (see 'Print Level'), monitoring information produced by ``qpconvex1_sparse_solve`` is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} = 0`, the default value is used and hence no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left(10^{-6}, {\sqrt{\epsilon }}\right)`
If :math:`r\geq \epsilon`, :math:`r` is used to judge the size of the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j`.
By definition, the reduced gradients for basic variables are always zero.
Optimality is declared if the reduced gradients for any nonbasic variables at their lower or upper bounds satisfy :math:`{-r}\times \mathrm{max}\left(1, {\left\lVert \pi \right\rVert }\right)\leq d_j\leq r\times \mathrm{max}\left(1, {\left\lVert \pi \right\rVert }\right)`, and if :math:`\left\lvert d_j\right\rvert \leq r\times \mathrm{max}\left(1, {\left\lVert \pi \right\rVert }\right)` for any superbasic variables.
If :math:`r < \epsilon`, the default value is used.
**'Partial Price'** : int
Default :math:`\text{} = 10`
Note that this option does not apply to QP problems.
This option is recommended for large FP or LP problems that have significantly more variables than constraints (i.e., :math:`n≫m`).
It reduces the work required for each pricing operation (i.e., when a nonbasic variable is selected to enter the basis).
If :math:`i = 1`, all columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}` are searched.
If :math:`i > 1`, :math:`A` and :math:`{-I}` are partitioned to give :math:`i` roughly equal segments :math:`A_{\textit{j}},K_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,p` (modulo :math:`p`).
If the previous pricing search was successful on :math:`A_{{j-1}},K_{{j-1}}`, the next search begins on the segments :math:`A_j,K_j`.
If a reduced gradient is found that is larger than some dynamic tolerance, the variable with the largest such reduced gradient (of appropriate sign) is selected to enter the basis.
If nothing is found, the search continues on the next segments :math:`A_{{j+1}},K_{{j+1}}`, and so on.
If :math:`i\leq 0`, the default value is used.
**'Pivot Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.67}`
If :math:`r > 0`, :math:`r` is used to prevent columns entering the basis if they would cause the basis to become almost singular.
If :math:`r\leq 0`, the default value is used.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``qpconvex1_sparse_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#monitoring>`__ (monitoring information at each iteration).
Note that the summary output will not exceed :math:`80` characters per line and that the monitoring information will not exceed :math:`120` characters per line.
If :math:`i < 0`, the default value is used.
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==================================================================================+
|:math:`0` |No output. |
+----------------------+----------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+----------------------------------------------------------------------------------+
|:math:`5` |One line of summary output for each iteration (no printout of the final solution).|
+----------------------+----------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+----------------------------------------------------------------------------------+
The following printout is sent to the logical unit number defined by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One long line of output for each iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one long line of output for each iteration. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|The final solution, one long line of output for each iteration, matrix statistics (initial status of rows and columns, number of elements, density, biggest and smallest elements, etc.), details of the scale factors resulting from the scaling procedure (if :math:`\text{‘Scale Option'} = 1` or :math:`2` (see the description of the option 'Scale Option'), basis factorization statistics and details of the initial basis resulting from the Crash procedure (if :math:`\mathrm{start} = \texttt{'C'}`; see :ref:`Parameters <e04nk-py2-py-parameters>`).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} > 0` and the unit number defined by option 'Monitoring File' is the advisory unit number, then the summary output is suppressed.
**'Rank Tolerance'** : float
Default :math:`\text{} = 100\epsilon`
See above.
**'Scale Option'** : int
Default :math:`\text{} = 2`
This option enables you to scale the variables and constraints using an iterative procedure due to Fourer (1982), which attempts to compute row scales :math:`r_i` and column scales :math:`c_j` such that the scaled matrix coefficients :math:`\bar{a}_{{ij}} = a_{{ij}}\times \left(c_j/r_i\right)` are as close as possible to unity.
This may improve the overall efficiency on some problems. (The lower and upper bounds on the variables and slacks for the scaled problem are redefined as :math:`\bar{l}_j = l_j/c_j` and :math:`\bar{u}_j = u_j/c_j` respectively, where :math:`c_j\equiv r_{{j-n}}` if :math:`j > n`.)
If :math:`i = 0`, no scaling is performed.
If :math:`i = 1`, all rows and columns of the constraint matrix :math:`A` are scaled.
If :math:`i = 2`, an additional scaling is performed that may be helpful when the solution :math:`x` is large; it takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.
If :math:`i < 0` or :math:`i > 2`, the default value is used.
**'Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
Note that this option does not apply when :math:`\text{‘Scale Option'} = 0`.
If :math:`0 < r < 1`, :math:`r` is used to control the number of scaling passes to be made through the constraint matrix :math:`A`.
At least :math:`3` (and at most :math:`10`) passes will be made.
More precisely, let :math:`a_p` denote the largest column ratio (i.e., :math:`\frac{{\text{‘biggest'}\text{ element}}}{{\text{‘smallest'}\text{ element}}}` in some sense) after the :math:`p`\ th scaling pass through :math:`A`.
The scaling procedure is terminated if :math:`a_p\geq a_{{p-1}}\times r` for some :math:`p\geq 3`.
Thus, increasing the value of :math:`r` from :math:`0.9` to :math:`0.99` (say) will probably increase the number of passes through :math:`A`.
If :math:`r\leq 0` or :math:`r\geq 1`, the default value is used.
**'Superbasics Limit'** : int
Default :math:`\text{} = \mathrm{min}\left({n_H+1}, n\right)`
Note that this option does not apply to FP or LP problems.
The value of :math:`i` specifies 'how nonlinear' you expect the QP problem to be.
If :math:`i\leq 0`, the default value is used.
.. _e04nk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`5`)
Reduced Hessian matrix :math:`Z^\mathrm{T}\times H\times Z` exceeds its assigned dimension.
(`errno` :math:`6`)
Hessian matrix :math:`H` appears to be indefinite.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` then :math:`\mathrm{istate}[i]\leq 3` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` then :math:`\mathrm{istate}[i]\leq 5` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle` then :math:`\mathrm{istate}[i]\geq 0` for all :math:`i`.
(`errno` :math:`7`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
On entry, duplicate element found in row :math:`\langle\mathit{\boldsymbol{value}}\rangle`, column :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{lenz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lenz}\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{leniz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{leniz}\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ka}[j] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ka}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ka}[j+1]-\mathrm{ka}[j]\leq \mathrm{m}` for all :math:`j`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ka}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[j]\geq 1` for all :math:`j`.
(`errno` :math:`7`)
On entry, :math:`n+1 = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ka}[n] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnz}+1 = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[n] = \textit{nnz}+1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ka}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[0] = 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ha}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{ha}[j]\leq \mathrm{m}` for all :math:`j`.
(`errno` :math:`7`)
On entry, :math:`\textit{nname} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nname} = 1` or :math:`\textit{nname} = n+\mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[n+\mathrm{iobj}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bu}[n+\mathrm{iobj}-1]\geq \textit{bigbnd}`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[n+\mathrm{iobj}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[n+\mathrm{iobj}-1]\leq {-\textit{bigbnd}}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{iobj}\leq \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ncolh}\leq n`.
(`errno` :math:`7`)
On entry, :math:`\textit{nnz} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \textit{nnz}\leq n\times \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq 1`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{start} = \texttt{'C'}` or :math:`\texttt{'W'}`.
(`errno` :math:`8`)
Cannot satisfy the general constraints.
(`errno` :math:`9`)
Not enough integer workspace for the basis factors.
(`errno` :math:`10`)
Not enough real workspace for the basis factors.
(`errno` :math:`11`)
The basis is singular after :math:`15` factorization attempts.
(`errno` :math:`21`)
Error in basis package. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`32`)
System error. Wrong number of basic variables. Please contact `NAG <https://www.nag.com>`__.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Weak :math:`\langle\mathit{\boldsymbol{value}}\rangle` solution found.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` problem is unbounded (or badly scaled).
(`errno` :math:`3`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` problem is infeasible.
(`errno` :math:`12`)
Not enough integer workspace to start solving the problem.
(`errno` :math:`13`)
Not enough real workspace to start solving the problem.
.. _e04nk-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``qpconvex1_sparse_solve`` is designed to solve a class of quadratic programming problems that are assumed to be stated in the following general form:
.. math::
\mathrm{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{,}
where :math:`x` is a set of variables, :math:`A` is an :math:`m\times n` matrix and the objective function :math:`f\left(x\right)` may be specified in a variety of ways depending upon the particular problem to be solved.
The option 'Maximize' may be used to specify an alternative problem in which :math:`f\left(x\right)` is maximized.
The possible forms for :math:`f\left(x\right)` are listed in Table [label omitted], in which the prefixes FP, LP and QP stand for 'feasible point', 'linear programming' and 'quadratic programming' respectively, :math:`c` is an :math:`n`-element vector and :math:`H` is the :math:`n\times n` second-derivative matrix :math:`\nabla^2f\left(x\right)` (the `Hessian matrix`).
+------------+-----------------------------------------------+-------------------------------+
|Problem type|Objective function :math:`f\left(x\right)` |Hessian matrix :math:`H` |
+============+===============================================+===============================+
|FP |Not applicable |Not applicable |
+------------+-----------------------------------------------+-------------------------------+
|LP |:math:`c^\mathrm{T}x` |Not applicable |
+------------+-----------------------------------------------+-------------------------------+
|QP |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx`|Symmetric positive semidefinite|
+------------+-----------------------------------------------+-------------------------------+
For LP and QP problems, the unique global minimum value of :math:`f\left(x\right)` is found.
For FP problems, :math:`f\left(x\right)` is omitted and the function attempts to find a feasible point for the set of constraints.
For QP problems, you must also provide a function that computes :math:`Hx` for any given vector :math:`x`. (:math:`H` need not be stored explicitly.) If :math:`H` is the zero matrix, the function will still solve the resulting LP problem; however, this can be accomplished more efficiently by setting :math:`\mathrm{ncolh} = 0` (see :ref:`Parameters <e04nk-py2-py-parameters>`).
The defining feature of a `convex` QP problem is that the matrix :math:`H` must be `positive semidefinite`, i.e., it must satisfy :math:`x^\mathrm{T}Hx\geq 0` for all :math:`x`.
Otherwise, :math:`f\left(x\right)` is said to be `nonconvex` and it may be more appropriate to call :meth:`handle_solve_ssqp` instead.
``qpconvex1_sparse_solve`` is intended to solve large-scale linear and quadratic programming problems in which the constraint matrix :math:`A` is `sparse` (i.e., when the number of zero elements is sufficiently large that it is worthwhile using algorithms which avoid computations and storage involving zero elements).
The function also takes advantage of sparsity in :math:`c`. (Sparsity in :math:`H` can be exploited in the function that computes :math:`Hx`.) For problems in which :math:`A` can be treated as a `dense` matrix, it is usually more efficient to use :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve`.
The upper and lower bounds on the :math:`m` elements of :math:`Ax` are said to define the `general constraints` of the problem.
Internally, ``qpconvex1_sparse_solve`` converts the general constraints to equalities by introducing a set of `slack variables` :math:`s`, where :math:`s = \left(s_1,s_2,\ldots,s_m\right)^\mathrm{T}`.
For example, the linear constraint :math:`5\leq 2x_1+3x_2\leq +\infty` is replaced by :math:`2x_1+3x_2-s_1 = 0`, together with the bounded slack :math:`5\leq s_1\leq {+\infty }`.
The problem defined by `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#eqn1>`__ can, therefore, be re-written in the following equivalent form:
.. math::
\mathrm{minimize}_{{x \in R^n,s \in R^m}}f\left(x\right)\quad \text{ subject to }\quad Ax-s = 0\text{, }\quad l\leq \left\{\begin{array}{c}x\\s\end{array}\right\}\leq u\text{.}
Since the slack variables :math:`s` are subject to the same upper and lower bounds as the elements of :math:`Ax`, the bounds on :math:`Ax` and :math:`x` can simply be thought of as bounds on the combined vector :math:`\left(x, s\right)`. (In order to indicate their special role in QP problems, the original variables :math:`x` are sometimes known as 'column variables', and the slack variables :math:`s` are known as 'row variables'.)
Each LP or QP problem is solved using an `active-set` method.
This is an iterative procedure with two phases: a `feasibility phase`, in which the sum of infeasibilities is minimized to find a feasible point; and an `optimality phase`, in which :math:`f\left(x\right)` is minimized by constructing a sequence of iterations that lies within the feasible region.
A constraint is said to be `active` or `binding` at :math:`x` if the associated element of either :math:`x` or :math:`Ax` is equal to one of its upper or lower bounds.
Since an active constraint in :math:`Ax` has its associated slack variable at a bound, the status of both simple and general upper and lower bounds can be conveniently described in terms of the status of the variables :math:`\left(x, s\right)`.
A variable is said to be `nonbasic` if it is temporarily fixed at its upper or lower bound.
It follows that regarding a general constraint as being `active` is equivalent to thinking of its associated slack as being `nonbasic`.
At each iteration of an active-set method, the constraints :math:`Ax-s = 0` are (conceptually) partitioned into the form
.. math::
Bx_B+Sx_S+Nx_N = 0\text{,}
where :math:`x_N` consists of the nonbasic elements of :math:`\left(x, s\right)` and the `basis matrix` :math:`B` is square and nonsingular.
The elements of :math:`x_B` and :math:`x_S` are called the `basic` and `superbasic` variables respectively; with :math:`x_N` they are a permutation of the elements of :math:`x` and :math:`s`.
At a QP solution, the basic and superbasic variables will lie somewhere between their upper or lower bounds, while the nonbasic variables will be equal to one of their bounds.
At each iteration, :math:`x_S` is regarded as a set of independent variables that are free to move in any desired direction, namely one that will improve the value of the objective function (or sum of infeasibilities).
The basic variables are then adjusted in order to ensure that :math:`\left(x, s\right)` continues to satisfy :math:`Ax-s = 0`.
The number of superbasic variables (:math:`n_S` say), therefore, indicates the number of degrees of freedom remaining after the constraints have been satisfied.
In broad terms, :math:`n_S` is a measure of `how nonlinear` the problem is.
In particular, :math:`n_S` will always be zero for FP and LP problems.
If it appears that no improvement can be made with the current definition of :math:`B`, :math:`S` and :math:`N`, a nonbasic variable is selected to be added to :math:`S`, and the process is repeated with the value of :math:`n_S` increased by one.
At all stages, if a basic or superbasic variable encounters one of its bounds, the variable is made nonbasic and the value of :math:`n_S` is decreased by one.
Associated with each of the :math:`m` equality constraints :math:`Ax-s = 0` is a `dual variable` :math:`\pi_i`.
Similarly, each variable in :math:`\left(x, s\right)` has an associated `reduced gradient` :math:`d_j` (also known as a `reduced cost`).
The reduced gradients for the variables :math:`x` are the quantities :math:`g-A^\mathrm{T}\pi`, where :math:`g` is the gradient of the QP objective function; and the reduced gradients for the slack variables :math:`s` are the dual variables :math:`\pi`.
The QP subproblem is optimal if :math:`d_j\geq 0` for all nonbasic variables at their lower bounds, :math:`d_j\leq 0` for all nonbasic variables at their upper bounds and :math:`d_j = 0` for all superbasic variables.
In practice, an `approximate` QP solution is found by slightly relaxing these conditions on :math:`d_j` (see the description of the option 'Optimality Tolerance').
The process of computing and comparing reduced gradients is known as `pricing` (a term first introduced in the context of the simplex method for linear programming).
To 'price' a nonbasic variable :math:`x_j` means that the reduced gradient :math:`d_j` associated with the relevant active upper or lower bound on :math:`x_j` is computed via the formula :math:`d_j = g_j-a^\mathrm{T}\pi`, where :math:`a_j` is the :math:`j`\ th column of :math:`\begin{pmatrix}A&-I\end{pmatrix}`. (The variable selected by such a process and the corresponding value of :math:`d_j` (i.e., its reduced gradient) are the quantities +S and dj in the monitoring file output; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#monitoring>`__.) If :math:`A` has significantly more columns than rows (i.e., :math:`n≫m`), pricing can be computationally expensive.
In this case, a strategy known as `partial pricing` can be used to compute and compare only a subset of the :math:`d_j`'s.
``qpconvex1_sparse_solve`` is based on SQOPT, which is part of the SNOPT package described in Gill `et al.` (2002), which in turn utilizes functions from the MINOS package (see Murtagh and Saunders (1995)).
It uses stable numerical methods throughout and includes a reliable basis package (for maintaining sparse :math:`LU` factors of the basis matrix :math:`B`), a practical anti-degeneracy procedure, efficient handling of linear constraints and bounds on the variables (by an active-set strategy), as well as automatic scaling of the constraints.
Further details can be found in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nkf.html#algdetails>`__.
.. _e04nk-py2-py-references:
**References**
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E and Murray, W, 1978, `Numerically stable methods for quadratic programming`, Math. Programming (14), 349--372
Gill, P E, Murray, W and Saunders, M A, 2002, `SNOPT: An SQP Algorithm for Large-scale Constrained Optimization` (12), 979--1006, SIAM J. Optim.
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1987, `Maintaining` :math:`LU` `factors of a general sparse matrix`, Linear Algebra and its Applics. (88/89), 239--270
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1991, `Inertia-controlling methods for general quadratic programming`, SIAM Rev. (33), 1--36
Hall, J A J and McKinnon, K I M, 1996, `The simplest examples where the simplex method cycles and conditions where EXPAND fails to prevent cycling`, Report MS, 96--100, Department of Mathematics and Statistics, University of Edinburgh
Murtagh, B A and Saunders, M A, 1995, `MINOS 5.4 users' guide`, Report SOL 83-20R, Department of Operations Research, Stanford University
"""
raise NotImplementedError
[docs]def qpconvex1_sparse_option_file(ioptns, comm, io_manager=None):
r"""
``qpconvex1_sparse_option_file`` may be used to supply options to :meth:`qpconvex1_sparse_solve` from an external file.
.. _e04nl-py2-py-doc:
For full information please refer to the NAG Library document for e04nl
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nlf.html
.. _e04nl-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nl-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04nl-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``qpconvex1_sparse_option_file`` may be used to supply values for options to :meth:`qpconvex1_sparse_solve`. ``qpconvex1_sparse_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`qpconvex1_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex1_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex1_sparse_solve <e04nk-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex1_sparse_option_string(optstr, comm, io_manager=None):
r"""
``qpconvex1_sparse_option_string`` may be used to supply individual options to :meth:`qpconvex1_sparse_solve`.
.. _e04nm-py2-py-doc:
For full information please refer to the NAG Library document for e04nm
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nmf.html
.. _e04nm-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04nm-py2-py-notes>` and in :ref:`Other Parameters for qpconvex1_sparse_solve <e04nk-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nm-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nm-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``qpconvex1_sparse_option_string`` may be used to supply values for options to :meth:`qpconvex1_sparse_solve`.
It is only necessary to call ``qpconvex1_sparse_option_string`` for those arguments whose values are to be different from their default values.
One call to ``qpconvex1_sparse_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`qpconvex1_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex1_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex1_sparse_solve <e04nk-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_init(io_manager=None):
r"""
``qpconvex2_sparse_init`` is used to initialize the function :meth:`qpconvex2_sparse_solve`.
.. _e04np-py2-py-doc:
For full information please refer to the NAG Library document for e04np
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04npf.html
.. _e04np-py2-py-parameters:
**Parameters**
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**comm** : dict, communication object
Communication structure.
.. _e04np-py2-py-notes:
**Notes**
``qpconvex2_sparse_init`` initializes the arrays :math:`\mathrm{comm}`\ ['cw'], :math:`\mathrm{comm}`\ ['iw'] and :math:`\mathrm{comm}`\ ['rw'] for the function :meth:`qpconvex2_sparse_solve`.
See Also
--------
:meth:`naginterfaces.library.examples.opt.qpconvex2_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_solve(start, m, ncolh, iobj, objadd, prob, acol, inda, loca, bl, bu, names, helast, hs, x, ns, comm, qphx=None, c=None, data=None, io_manager=None):
r"""
``qpconvex2_sparse_solve`` solves sparse linear programming or convex quadratic programming problems.
The initialization function :meth:`qpconvex2_sparse_init` **must** have been called before calling ``qpconvex2_sparse_solve``.
Note: this function uses optional algorithmic parameters, see also: :meth:`qpconvex2_sparse_option_file`, :meth:`qpconvex2_sparse_option_string`, :meth:`qpconvex2_sparse_option_integer_set`, :meth:`qpconvex2_sparse_option_double_set`, :meth:`qpconvex2_sparse_init`, :meth:`qpconvex2_sparse_option_integer_get`, :meth:`qpconvex2_sparse_option_double_get`.
.. _e04nq-py2-py-doc:
For full information please refer to the NAG Library document for e04nq
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html
.. _e04nq-py2-py-parameters:
**Parameters**
**start** : str, length 1
Indicates how a starting basis (and certain other items) will be obtained.
:math:`\mathrm{start} = \texttt{'C'}`
Requests that an internal Crash procedure be used to choose an initial basis, unless a Basis file is provided via options 'Old Basis File', 'Insert File' or 'Load File'.
:math:`\mathrm{start} = \texttt{'B'}`
Is the same as :math:`\mathrm{start} = \texttt{'C'}` but is more meaningful when a Basis file is given.
:math:`\mathrm{start} = \texttt{'W'}`
Means that a basis is already defined in :math:`\mathrm{hs}` and a start point is already defined in :math:`\mathrm{x}` (probably from an earlier call).
**m** : int
:math:`m`, the number of general linear constraints (or slacks). This is the number of rows in the linear constraint matrix :math:`A`, including the free row (if any; see :math:`\mathrm{iobj}`). Note that :math:`A` must have at least one row. If your problem has no constraints, or only upper or lower bounds on the variables, then you must include a dummy row with sufficiently wide upper and lower bounds (see also :math:`\mathrm{acol}`, :math:`\mathrm{inda}` and :math:`\mathrm{loca}`).
**ncolh** : int
:math:`n_H`, the number of leading nonzero columns of the Hessian matrix :math:`H`. For FP and LP problems, :math:`\mathrm{ncolh}` must be set to zero.
The first :math:`\mathrm{ncolh}` elements of :math:`x` belong to variables corresponding to the nonzero block of the QP Hessian.
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero elements of the vector :math:`c` appearing in the linear objective term :math:`c^\mathrm{T}x`.
If :math:`\mathrm{iobj} = 0`, there is no free row, and the linear objective vector should be supplied in array :math:`\mathrm{c}`.
**objadd** : float
The constant :math:`q`, to be added to the objective for printing purposes. Typically :math:`\mathrm{objadd} = 0.0e0`.
**prob** : str, length 8
The name for the problem. It is used in the printed solution and in some functions that output Basis files. A blank name may be used.
**acol** : float, array-like, shape :math:`\left(\textit{ne}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index. Note that all elements must be assigned a value in the calling program.
**inda** : int, array-like, shape :math:`\left(\textit{ne}\right)`
:math:`\mathrm{inda}[\textit{i}-1]` must contain the row index of the nonzero element stored in :math:`\mathrm{acol}[\textit{i}-1]`, for :math:`\textit{i} = 1,2,\ldots,\textit{ne}`. Thus a pair of values :math:`\left(\mathrm{acol}[i-1], \mathrm{inda}[i-1]\right)` contains a matrix element and its corresponding row index.
Note that the row indices for a column may be supplied in any order.
**loca** : int, array-like, shape :math:`\left(n+1\right)`
:math:`\mathrm{loca}[\textit{j}-1]` must contain the index in :math:`\mathrm{acol}` and :math:`\mathrm{inda}` of the start of the :math:`\textit{j}`\ th column, for :math:`\textit{j} = 1,2,\ldots,n`. Thus for :math:`j = 1:n`, the entries of column :math:`j` are held in :math:`\mathrm{acol}[k-1:l]` and their corresponding row indices are in :math:`\mathrm{inda}[k-1:l]`, where :math:`k = \mathrm{loca}[j-1]` and :math:`l = \mathrm{loca}[j]-1`. To specify the :math:`j`\ th column as empty, set :math:`\mathrm{loca}[j-1] = \mathrm{loca}[j]`. Note that the first and last elements of :math:`\mathrm{loca}` must be :math:`\mathrm{loca}[0] = 1` and :math:`\mathrm{loca}[n] = \textit{ne}+1`. If your problem has no constraints, or just bounds on the variables, you may include a dummy 'free' row with a single (zero) element by setting :math:`\textit{ne} = 1`, :math:`\mathrm{acol}[0] = 0.0`, :math:`\mathrm{inda}[0] = 1`, :math:`\mathrm{loca}[0] = 1`, and :math:`\mathrm{loca}[j-1] = 2`, for :math:`j = 2:n+1`. This row is made 'free' by setting its bounds to be :math:`\mathrm{bl}[n] = -\textit{bigbnd}` and :math:`\mathrm{bu}[n] = \textit{bigbnd}`, where :math:`\textit{bigbnd}` is the value of the option 'Infinite Bound Size'.
**bl** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`l`, the lower bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bl}` must contain the bounds on the variables :math:`x`, and the next :math:`\mathrm{m}` elements the bounds for the general linear constraints :math:`Ax` (which, equivalently, are the bounds for the slacks, :math:`s`) and the free row (if any). To fix the :math:`j`\ th variable, set :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1] = \beta`, say, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. To specify a nonexistent lower bound (i.e., :math:`l_j = {-\infty }`), set :math:`\mathrm{bl}[j-1]\leq -\textit{bigbnd}`. Here, :math:`\textit{bigbnd}` is the value of the option 'Infinite Bound Size'. To specify the :math:`j`\ th constraint as an `equality`, set :math:`\mathrm{bl}[n+j-1] = \mathrm{bu}[n+j-1] = \beta`, say, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. Note that the lower bound corresponding to the free row must be set to :math:`{-\infty }` and stored in :math:`\mathrm{bl}[n+\mathrm{iobj}-1]`.
**bu** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`u`, the upper bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bu}` must contain the bounds on the variables :math:`x`, and the next :math:`\mathrm{m}` elements the bounds for the general linear constraints :math:`Ax` (which, equivalently, are the bounds for the slacks, :math:`s`) and the free row (if any). To specify a nonexistent upper bound (i.e., :math:`u_j = {+\infty }`), set :math:`\mathrm{bu}[j-1]\geq \textit{bigbnd}`. Note that the upper bound corresponding to the free row must be set to :math:`{+\infty }` and stored in :math:`\mathrm{bu}[n+\mathrm{iobj}-1]`.
**names** : str, length 8, array-like, shape :math:`\left(\textit{nname}\right)`
The optional column and row names, respectively.
If :math:`\textit{nname} = 1`, :math:`\mathrm{names}` is not referenced and the printed output will use default names for the columns and rows.
If :math:`\textit{nname} = n+\mathrm{m}`, the first :math:`\textit{n}` elements must contain the names for the columns and the next :math:`\mathrm{m}` elements must contain the names for the rows.
Note that the name for the free row (if any) must be stored in :math:`\mathrm{names}[n+\mathrm{iobj}-1]`.
**helast** : int, array-like, shape :math:`\left(n+\mathrm{m}\right)`
Defines which variables are to be treated as being elastic in elastic mode. The allowed values of :math:`\mathrm{helast}` are:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------+
|:math:`\mathrm{helast}[j-1]`|Status in elastic mode |
+============================+==============================================================+
|:math:`0` |Variable :math:`j` is non-elastic and cannot be infeasible |
+----------------------------+--------------------------------------------------------------+
|:math:`1` |Variable :math:`j` can violate its lower bound |
+----------------------------+--------------------------------------------------------------+
|:math:`2` |Variable :math:`j` can violate its upper bound |
+----------------------------+--------------------------------------------------------------+
|:math:`3` |Variable :math:`j` can violate either its lower or upper bound|
+----------------------------+--------------------------------------------------------------+
:math:`\mathrm{helast}` need not be assigned if option :math:`\text{‘Elastic Mode'} = 0`.
**hs** : int, array-like, shape :math:`\left(n+\mathrm{m}\right)`
If :math:`\mathrm{start} = \texttt{'C'}` or :math:`\texttt{'B'}`, and a Basis file of some sort is to be input (see the description of the options 'Old Basis File', 'Insert File' or 'Load File'), then :math:`\mathrm{hs}` and :math:`\mathrm{x}` need not be set at all.
If :math:`\mathrm{start} = \texttt{'C'}` or :math:`\texttt{'B'}` and there is no Basis file, the first :math:`\textit{n}` elements of :math:`\mathrm{hs}` and :math:`\mathrm{x}` must specify the initial states and values, respectively, of the variables :math:`x`. (The slacks :math:`s` need not be initialized.) An internal Crash procedure is then used to select an initial basis matrix :math:`B`.
The initial basis matrix will be triangular (neglecting certain small elements in each column).
It is chosen from various rows and columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
Possible values for :math:`\mathrm{hs}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+------------------------+---------------------------------------------------------------------+
|:math:`\mathrm{hs}[j-1]`|State of :math:`\mathrm{x}[j-1]` during Crash procedure |
+========================+=====================================================================+
|:math:`0` or :math:`1` |Eligible for the basis |
+------------------------+---------------------------------------------------------------------+
|:math:`2` |Ignored |
+------------------------+---------------------------------------------------------------------+
|:math:`3` |Eligible for the basis (given preference over :math:`0` or :math:`1`)|
+------------------------+---------------------------------------------------------------------+
|:math:`4` or :math:`5` |Ignored |
+------------------------+---------------------------------------------------------------------+
If nothing special is known about the problem, or there is no wish to provide special information, you may set :math:`\mathrm{hs}[\textit{j}-1] = 0` and :math:`\mathrm{x}[\textit{j}-1] = 0.0`, for :math:`\textit{j} = 1,2,\ldots,n`.
All variables will then be eligible for the initial basis.
Less trivially, to say that the :math:`j`\ th variable will probably be equal to one of its bounds, set :math:`\mathrm{hs}[j-1] = 4` and :math:`\mathrm{x}[j-1] = \mathrm{bl}[j-1]` or :math:`\mathrm{hs}[j-1] = 5` and :math:`\mathrm{x}[j-1] = \mathrm{bu}[j-1]` as appropriate.
Following the Crash procedure, variables for which :math:`\mathrm{hs}[j-1] = 2` are made superbasic.
Other variables not selected for the basis are then made nonbasic at the value :math:`\mathrm{x}[j-1]` if :math:`\mathrm{bl}[j-1]\leq \mathrm{x}[j-1]\leq \mathrm{bu}[j-1]`, or at the value :math:`\mathrm{bl}[j-1]` or :math:`\mathrm{bu}[j-1]` closest to :math:`\mathrm{x}[j-1]`.
If :math:`\mathrm{start} = \texttt{'W'}`, :math:`\mathrm{hs}` and :math:`\mathrm{x}` must specify the initial states and values, respectively, of the variables and slacks :math:`\left(x, s\right)`.
If ``qpconvex2_sparse_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\mathrm{m}`, :math:`\mathrm{hs}` already contains satisfactory information.
**x** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
The initial values of the variables :math:`x`, and, if :math:`\mathrm{start} = \texttt{'W'}`, the slacks :math:`s`, i.e., :math:`\left(x, s\right)`. (See the description for argument :math:`\mathrm{hs}`.)
**ns** : int
:math:`n_S`, the number of superbasics. For QP problems, :math:`\mathrm{ns}` need not be specified if :math:`\mathrm{start} = \texttt{'C'}`, but must retain its value from a previous call when :math:`\mathrm{start} = \texttt{'W'}`. For FP and LP problems, :math:`\mathrm{ns}` need not be initialized.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**qphx** : None or callable hx = qphx(x, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
For QP problems, you must supply a version of :math:`\mathrm{qphx}` to compute the matrix product :math:`Hx` for a given vector :math:`x`.
If :math:`H` has rows and columns of zeros, it is most efficient to order :math:`x` so that the nonlinear variables appear first.
For example, if :math:`x = \left(y, z\right)^\mathrm{T}` and only :math:`y` enters the objective quadratically, then
.. math::
Hx = \begin{pmatrix}H_1&0\\0&0\end{pmatrix}\begin{pmatrix}y\\z\end{pmatrix} = \begin{pmatrix}H_1y\\0\end{pmatrix}\text{.}
In this case, :math:`\mathrm{ncolh}` should be the dimension of :math:`y`, and :math:`\mathrm{qphx}` should compute :math:`H_1y`.
For FP and LP problems, :math:`\mathrm{qphx}` will never be called by ``qpconvex2_sparse_solve`` and hence :math:`\mathrm{qphx}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{ncolh}\right)`
The first :math:`\mathrm{ncolh}` elements of the vector :math:`x`.
**nstate** : int
Allows you to save computation time if certain data must be read or calculated only once. To preserve this data for a subsequent calculation place it in one of :math:`\mathrm{cuser}`, :math:`\mathrm{ruser}` or :math:`\mathrm{iuser}`.
:math:`\mathrm{nstate} = 1`
``qpconvex2_sparse_solve`` is calling :math:`\mathrm{qphx}` for the first time.
:math:`\mathrm{nstate} = 0`
There is nothing special about the current call of :math:`\mathrm{qphx}`.
:math:`\mathrm{nstate}\geq 2`
``qpconvex2_sparse_solve`` is calling :math:`\mathrm{qphx}` for the last time. This argument setting allows you to perform some additional computation on the final solution.
:math:`\mathrm{nstate} = 2`
The current :math:`x` is optimal.
:math:`\mathrm{nstate} = 3`
The problem appears to be infeasible.
:math:`\mathrm{nstate} = 4`
The problem appears to be unbounded.
:math:`\mathrm{nstate} = 5`
The iterations limit was reached.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**hx** : float, array-like, shape :math:`\left(\textit{ncolh}\right)`
The product :math:`Hx`. If :math:`\mathrm{ncolh}` is less than the input argument :math:`\textit{n}`, :math:`Hx` is really the product :math:`H_1y` in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#eqnqphx>`__.
**c** : None or float, array-like, shape :math:`\left(\textit{lenc}\right)`, optional
Contains the explicit objective vector :math:`c` (if any). If the problem is of type FP, or if :math:`\textit{lenc} = 0`, :math:`\mathrm{c}` is not referenced. (In that case, :math:`\mathrm{c}` may be dimensioned `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#eqn1>`__, or it could be any convenient array.)
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**hs** : int, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final states of the variables and slacks :math:`\left(x, s\right)`. The significance of each possible value of :math:`\mathrm{hs}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+------------------------+---------------------------+-------------------------------------------------------------+
|:math:`\mathrm{hs}[j-1]`|State of variable :math:`j`|Normal value of :math:`\mathrm{x}[j-1]` |
+========================+===========================+=============================================================+
|:math:`0` |Nonbasic |:math:`\mathrm{bl}[j-1]` |
+------------------------+---------------------------+-------------------------------------------------------------+
|:math:`1` |Nonbasic |:math:`\mathrm{bu}[j-1]` |
+------------------------+---------------------------+-------------------------------------------------------------+
|:math:`2` |Superbasic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+------------------------+---------------------------+-------------------------------------------------------------+
|:math:`3` |Basic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+------------------------+---------------------------+-------------------------------------------------------------+
If :math:`\mathrm{ninf} = 0`, basic and superbasic variables may be outside their bounds by as much as the value of the option 'Feasibility Tolerance'.
Note that unless the option :math:`\text{‘Scale Option'} = 0` is specified, the option 'Feasibility Tolerance' applies to the variables of the scaled problem.
In this case, the variables of the original problem may be as much as :math:`0.1` outside their bounds, but this is unlikely unless the problem is very badly scaled.
Very occasionally some nonbasic variables may be outside their bounds by as much as the option 'Feasibility Tolerance', and there may be some nonbasic variables for which :math:`\mathrm{x}[j-1]` lies strictly between its bounds.
If :math:`\mathrm{ninf} > 0`, some basic and superbasic variables may be outside their bounds by an arbitrary amount (bounded by :math:`\mathrm{sinf}` if :math:`\text{‘Scale Option'} = 0`).
**x** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final values of the variables and slacks :math:`\left(x, s\right)`.
**pi** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
Contains the dual variables :math:`\pi` (a set of Lagrange multipliers (shadow prices) for the general constraints).
**rc** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
Contains the reduced costs, :math:`g-\begin{pmatrix}A&-I\end{pmatrix}^\mathrm{T}\pi`. The vector :math:`g` is the gradient of the objective if :math:`\mathrm{x}` is feasible; otherwise, it is the gradient of the Phase 1 objective. In the former case, :math:`g\left(i\right) = 0`, for :math:`i = n+1:\mathrm{m}`, hence :math:`\mathrm{rc}[n+1:\mathrm{m}-1] = \pi`.
**ns** : int
The final number of superbasics. This will be zero for FP and LP problems.
**ninf** : int
The number of infeasibilities.
**sinf** : float
The sum of the scaled infeasibilities. This will be zero if :math:`\mathrm{ninf} = 0`, and is most meaningful when :math:`\text{‘Scale Option'} = 0`.
**obj** : float
The value of the objective function.
If :math:`\mathrm{ninf} = 0`, :math:`\mathrm{obj}` includes the quadratic objective term :math:`\frac{1}{2}x^\mathrm{T}Hx` (if any).
If :math:`\mathrm{ninf} > 0`, :math:`\mathrm{obj}` is just the linear objective term :math:`c^\mathrm{T}x` (if any).
For FP problems, :math:`\mathrm{obj}` is set to zero.
Note that :math:`\mathrm{obj}` does not include contributions from the constant term :math:`\mathrm{objadd}` or the objective row, if any.
.. _e04nq-py2-py-other_params:
**Other Parameters**
**'Check Frequency'** : int
Default :math:`\text{} = 60`
Every :math:`i`\ th iteration after the most recent basis factorization, a numerical test is made to see if the current solution :math:`\left(x, s\right)` satisfies the linear constraints :math:`Ax-s = 0`.
If the largest element of the residual vector :math:`r = Ax-s` is judged to be too large, the current basis is refactorized and the basic variables recomputed to satisfy the constraints more accurately.
If :math:`i\leq 0`, the value :math:`i = 99999999` is used and effectively no checks are made.
:math:`\text{‘Check Frequency'} = 1` is useful for debugging purposes, but otherwise this option should not be needed.
**'Crash Option'** : int
Default :math:`\text{} = 3`
Note that these options do not apply when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04nq-py2-py-parameters>`).
If :math:`\mathrm{start} = \texttt{'C'}`, an internal Crash procedure is used to select an initial basis from various rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The value of :math:`i` determines which rows and columns of :math:`A` are initially eligible for the basis, and how many times the Crash procedure is called.
Columns of :math:`-I` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+===================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of the matrix :math:`A`.|
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called once, looking for a triangular basis in rows. |
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called twice, treating linear equalities and linear inequalities separately. |
+---------+-------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound.) The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' allows the Crash procedure to ignore certain 'small' nonzero elements in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros :math:`a_{{ij}}` in the column are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` should be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.1`
Note that these options do not apply when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04nq-py2-py-parameters>`).
If :math:`\mathrm{start} = \texttt{'C'}`, an internal Crash procedure is used to select an initial basis from various rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The value of :math:`i` determines which rows and columns of :math:`A` are initially eligible for the basis, and how many times the Crash procedure is called.
Columns of :math:`-I` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+===================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of the matrix :math:`A`.|
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called once, looking for a triangular basis in rows. |
+---------+-------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called twice, treating linear equalities and linear inequalities separately. |
+---------+-------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound.) The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' allows the Crash procedure to ignore certain 'small' nonzero elements in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros :math:`a_{{ij}}` in the column are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` should be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Dump File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i`.
If :math:`i_2 > 0`, the 'Load File' containing basis information will be read. The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Load File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i`.
If :math:`i_2 > 0`, the 'Load File' containing basis information will be read. The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Elastic Mode'** : int
Default :math:`\text{} = 1`
This argument determines if (and when) elastic mode is to be started.
Three elastic modes are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|Elastic mode is never invoked. ``qpconvex2_sparse_solve`` will terminate as soon as infeasibility is detected. There may be other points with significantly smaller sums of infeasibilities. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|Elastic mode is invoked only if the constraints are found to be infeasible (the default). If the constraints are infeasible, continue in elastic mode with the composite objective determined by the values of the options 'Elastic Objective' and 'Elastic Weight'. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The iterations start and remain in elastic mode. This option allows you to minimize the composite objective function directly without first performing Phase 1 iterations. The success of this option will depend critically on your choice of 'Elastic Weight'. If 'Elastic Weight' is sufficiently large and the constraints are feasible, the minimizer of the composite objective and the solution of the original problem are identical. However, if the 'Elastic Weight' is not sufficiently large, the minimizer of the composite function may be infeasible, even if a feasible point exists.|
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'Elastic Objective'** : int
Default :math:`\text{} = 1`
This determines the form of the composite objective :math:`f\left(x\right)+\gamma \sum_j\left(v_j+w_j\right)` in Phase 2 (:math:`\gamma`).
Three types of composite objectives are available.
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+====================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|Include only the true objective :math:`f\left(x\right)` in the composite objective. This option sets :math:`\gamma = 0` in the composite objective and allows ``qpconvex2_sparse_solve`` to ignore the elastic bounds and find a solution that minimizes :math:`f\left(x\right)` subject to the non-elastic constraints. This option is useful if there are some 'soft' constraints that you would like to ignore if the constraints are infeasible.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|Use a composite objective defined with :math:`\gamma` determined by the value of 'Elastic Weight'. This value is intended to be used in conjunction with :math:`\text{‘Elastic Mode'} = 2`. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|Include only the elastic variables in the composite objective. The elastics are weighted by :math:`\gamma = 1`. This choice minimizes the violations of the elastic variables at the expense of possibly increasing the true objective. This option can be used to find a point that minimizes the sum of the violations of a subset of constraints specified by the input array :math:`\mathrm{helast}`. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'Elastic Weight'** : float
Default :math:`\text{} = 1.0`
This defines the value of :math:`\gamma` in the composite objective in Phase 2 (:math:`\gamma`).
At each iteration of elastic mode, the composite objective is defined to be
.. math::
{\mathrm{minimize}\text{ }\sigma }\text{ }f\left(x\right)+\gamma \text{ (sum of infeasibilities);}
where :math:`\sigma = 1` for 'Minimize', :math:`\sigma = -1` for 'Maximize', and :math:`f\left(x\right)` is the quadratic objective.
Note that the effect of :math:`\gamma` is `not` disabled once a feasible point is obtained.
**'Expand Frequency'** : int
Default :math:`\text{} = 10000`
This option is part of an anti-cycling procedure (see `Miscellaneous <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#algdetails4>`__) designed to allow progress even on highly degenerate problems.
The strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of the option 'Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``qpconvex2_sparse_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
Increasing the value of :math:`i` helps reduce the number of slightly infeasible nonbasic variables (most of which are eliminated during the resetting procedure).
However, it also diminishes the freedom to choose a large pivot element (see the description of the option 'Pivot Tolerance').
If :math:`i\leq 0`, the value :math:`i = 99999999` is used and effectively no anti-cycling procedure is invoked.
**'Factorization Frequency'** : int
Default :math:`\text{} = 100\left(\mathrm{LP}\right)` or :math:`50\left(\mathrm{QP}\right)`
If :math:`i > 0`, at most :math:`i` basis changes will occur between factorizations of the basis matrix.
For LP problems, the basis factors are usually updated at every iteration.
Higher values of :math:`i` may be more efficient on problems that are extremely sparse and well scaled.
For QP problems, fewer basis updates will occur as the solution is approached.
The number of iterations between basis factorizations will, therefore, increase.
During these iterations a test is made regularly according to the value of option 'Check Frequency' to ensure that the linear constraints :math:`Ax-s = 0` are satisfied.
Occasionally, the basis will be refactorized before the limit of :math:`i` updates is reached.
If :math:`i\leq 0`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left\{10^{-6}, \sqrt{\epsilon }\right\}`
A `feasible problem` is one in which all variables satisfy their upper and lower bounds to within the absolute tolerance :math:`r`. (This includes slack variables.
Hence, the general constraints are also satisfied to within :math:`r`.)
``qpconvex2_sparse_solve`` attempts to find a feasible solution before optimizing the objective function.
If the sum of infeasibilities cannot be reduced to zero, the problem is assumed to be `infeasible`.
Let sInf be the corresponding sum of infeasibilities.
If sInf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Note that if sInf is not small and you have not asked ``qpconvex2_sparse_solve`` to minimize the violations of the elastic variables (i.e., you have not specified :math:`\text{‘Elastic Objective'} = 2`), there may be other points that have a `significantly smaller sum of infeasibilities`. ``qpconvex2_sparse_solve`` will not attempt to find the solution that minimizes the sum unless :math:`\text{‘Elastic Objective'} = 2`.
If the constraints and variables have been scaled (see the description of the option 'Scale Option'), then feasibility is defined in terms of the scaled problem (since it is more likely to be meaningful).
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r\geq 0`, :math:`r` defines the 'infinite' bound :math:`\textit{infbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{infbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{infbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Iterations Limit'** : int
Default :math:`\text{} = \mathrm{max}\left\{10000, {10\mathrm{max}\left\{m, n\right\}}\right\}`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Print Level'} > 0` means that: the workspace needed to start solving the problem will be computed and printed; and feasibility and optimality will be checked.
No iterations will be performed.
If :math:`i < 0`, the default value is used.
**'LU Density Tolerance'** : float
Default :math:`\text{} = 0.6`
The density tolerance :math:`r_1` is used during :math:`LU` factorization of the basis matrix.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated.
The remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
If :math:`r_2 > 0`, :math:`r_2` defines the singularity tolerance used to guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows.
If :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable.
If :math:`r_2\leq 0`, the default value is used.
**'LU Singularity Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
The density tolerance :math:`r_1` is used during :math:`LU` factorization of the basis matrix.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated.
The remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
If :math:`r_2 > 0`, :math:`r_2` defines the singularity tolerance used to guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows.
If :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable.
If :math:`r_2\leq 0`, the default value is used.
**'LU Factor Tolerance'** : float
Default :math:`\text{} = 100.0`
The values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Update Tolerance'** : float
Default :math:`\text{} = 10.0`
The values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Partial Pivoting'** : valueless
Default
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshold partial pivoting.
The options 'LU Complete Pivoting' and 'LU Rook Pivoting' are more expensive but more stable and better at revealing rank, as long as the 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
**'LU Complete Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshold partial pivoting.
The options 'LU Complete Pivoting' and 'LU Rook Pivoting' are more expensive but more stable and better at revealing rank, as long as the 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
**'LU Rook Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshold partial pivoting.
The options 'LU Complete Pivoting' and 'LU Rook Pivoting' are more expensive but more stable and better at revealing rank, as long as the 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
**'Minimize'** : valueless
Default
This option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#ad-mainiteration>`__) will be reversed.
The option 'Feasible Point' means 'ignore the objective function, while finding a feasible point for the linear constraints'.
It can be used to check that the constraints are feasible without altering the call to ``qpconvex2_sparse_solve``.
**'Maximize'** : valueless
This option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#ad-mainiteration>`__) will be reversed.
The option 'Feasible Point' means 'ignore the objective function, while finding a feasible point for the linear constraints'.
It can be used to check that the constraints are feasible without altering the call to ``qpconvex2_sparse_solve``.
**'Feasible Point'** : valueless
This option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` (see `Main Iteration <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#ad-mainiteration>`__) will be reversed.
The option 'Feasible Point' means 'ignore the objective function, while finding a feasible point for the linear constraints'.
It can be used to check that the constraints are feasible without altering the call to ``qpconvex2_sparse_solve``.
**'New Basis File'** : int
Default :math:`\text{} = 0`
Options 'New Basis File' and 'Backup Basis File' are sometimes referred to as `basis maps`.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on file :math:`i_1` every :math:`i_3`\ th iteration, where :math:`i_3` is the 'Save Frequency'.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``qpconvex2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The current basis will then be saved every :math:`100` iterations, first on 'New Basis File' and then immediately on 'Backup Basis File'.
If the run is interrupted at iteration :math:`2000` during the save on 'New Basis File', there will still be a usable basis on 'Backup Basis File' (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis on 'New Basis File' will correspond to iteration :math:`2050`, but the last basis saved on 'Backup Basis File' will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Backup Basis File'** : int
Default :math:`\text{} = 0`
Options 'New Basis File' and 'Backup Basis File' are sometimes referred to as `basis maps`.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on file :math:`i_1` every :math:`i_3`\ th iteration, where :math:`i_3` is the 'Save Frequency'.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``qpconvex2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The current basis will then be saved every :math:`100` iterations, first on 'New Basis File' and then immediately on 'Backup Basis File'.
If the run is interrupted at iteration :math:`2000` during the save on 'New Basis File', there will still be a usable basis on 'Backup Basis File' (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis on 'New Basis File' will correspond to iteration :math:`2050`, but the last basis saved on 'Backup Basis File' will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Save Frequency'** : int
Default :math:`\text{} = 100`
Options 'New Basis File' and 'Backup Basis File' are sometimes referred to as `basis maps`.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on file :math:`i_1` every :math:`i_3`\ th iteration, where :math:`i_3` is the 'Save Frequency'.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``qpconvex2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The current basis will then be saved every :math:`100` iterations, first on 'New Basis File' and then immediately on 'Backup Basis File'.
If the run is interrupted at iteration :math:`2000` during the save on 'New Basis File', there will still be a usable basis on 'Backup Basis File' (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis on 'New Basis File' will correspond to iteration :math:`2050`, but the last basis saved on 'Backup Basis File' will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Nolist'** : valueless
Default
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Old Basis File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the basis maps information will be obtained from this file. The file will usually have been output previously as a 'New Basis File' or 'Backup Basis File'.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
The file will not be acceptable if the number of rows or columns in the problem has been altered.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left\{10^{-6}, \sqrt{\epsilon }\right\}`
This is used to judge the size of the reduced gradients :math:`d_j = g_j-a_j^\mathrm{T}\pi`, where :math:`g_j` is the :math:`j`\ th component of the gradient, :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`, and :math:`\pi` is the set of dual variables.
By construction, the reduced gradients for basic variables are always zero.
The problem will be declared optimal if the reduced gradients for nonbasic variables at their lower or upper bounds satisfy
.. math::
d_j/\left\lVert \pi \right\rVert \geq {-r}\quad \text{ or }\quad d_j/\left\lVert \pi \right\rVert \leq r
respectively, and if :math:`\left\lvert d_j\right\rvert /\left\lVert \pi \right\rVert \leq r` for superbasic variables.
In the above tests, :math:`\left\lVert \pi \right\rVert` is a measure of the size of the dual variables.
It is included to make the tests independent of a scale factor on the objective function.
The quantity :math:`\left\lVert \pi \right\rVert` actually used is defined by
.. math::
\left\lVert \pi \right\rVert = \mathrm{max}\left({\sigma /\sqrt{m}}, 1\right)\text{, where }\sigma = \sum_{{i = 1}}^m\left\lvert \pi_i\right\rvert \text{,}
so that only large scale factors are allowed for.
If the objective is scaled down to be very `small`, the optimality test reduces to comparing :math:`d_j` against :math:`0.01r`.
**'Partial Price'** : int
Default :math:`\text{} = 10\left(\mathrm{LP}\right)` or :math:`1\left(\mathrm{QP}\right)`
This option is recommended for large FP or LP problems that have significantly more variables than constraints (i.e., :math:`n≫m`).
It reduces the work required for each pricing operation (i.e., when a nonbasic variable is selected to enter the basis).
If :math:`i = 1`, all columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}` are searched.
If :math:`i > 1`, :math:`A` and :math:`I` are partitioned to give :math:`i` roughly equal segments :math:`A_{\textit{j}},I_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,i` (modulo :math:`i`).
If the previous pricing search was successful on :math:`A_{{j-1}},I_{{j-1}}`, the next search begins on the segments :math:`A_j` and :math:`I_j`.
If a reduced gradient is found that is larger than some dynamic tolerance, the variable with the largest such reduced gradient (of appropriate sign) is selected to enter the basis.
If nothing is found, the search continues on the next segments :math:`A_{{j+1}},I_{{j+1}}`, and so on.
If :math:`i\leq 0`, the default value is used.
**'Pivot Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
Broadly speaking, the pivot tolerance is used to prevent columns entering the basis if they would cause the basis to become almost singular.
When :math:`x` changes to :math:`x+\alpha p` for some search direction :math:`p`, a 'ratio test' determines which component of :math:`x` reaches an upper or lower bound first.
The corresponding element of :math:`p` is called the pivot element.
Elements of :math:`p` are ignored (and, therefore, cannot be pivot elements) if they are smaller than the pivot tolerance :math:`r`.
It is common for two or more variables to reach a bound at essentially the same time.
In such cases, the option 'Feasibility Tolerance' (say :math:`t`) provides some freedom to maximize the pivot element and thereby improve numerical stability.
Excessively small values of :math:`t` should, therefore, not be specified.
To a lesser extent, the option 'Expand Frequency' (say :math:`f`) also provides some freedom to maximize the pivot element.
Excessively `large` values of :math:`f` should, therefore, not be specified.
**'Print File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the following information is output to :math:`i` during the solution of each problem:
- a listing of the options;
- some statistics about the problem;
- the amount of storage available for the :math:`LU` factorization of the basis matrix;
- notes about the initial basis resulting from a Crash procedure or a Basis file;
- the iteration log;
- basis factorization statistics;
- the exit :math:`\textit{errno}` condition and some statistics about the solution obtained;
- the printed solution, if requested.
The last four items are described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#fcomments>`__ and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#fcomments>`__.
Further brief output may be directed to the 'Summary File'.
**'Print Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i > 0`, one line of the iteration log will be printed every :math:`i`\ th iteration.
A value such as :math:`i = 10` is suggested for those interested only in the final solution.
If :math:`i\leq 0`, the value of :math:`i = 99999999` is used and effectively no checks are made.
**'Print Level'** : int
Default :math:`\text{} = 1`
This controls the amount of printing produced by ``qpconvex2_sparse_solve`` as follows.
.. rst-class:: nag-rules-none nag-align-left
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+===============+===================================================================================================================================================================================================================================================================================================================+
|0 |No output except error messages. If you want to suppress all output, set :math:`\text{‘Print File'} = 0`. |
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`= 1` |The set of selected options, problem statistics, summary of the scaling procedure, information about the initial basis resulting from a Crash or a Basis file, a single line of output at each iteration (controlled by the option 'Print Frequency'), and the exit condition with a summary of the final solution.|
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 10`|Basis factorization statistics. |
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'Punch File'** : int
Default :math:`= 0`
These files provide compatibility with commercial mathematical programming systems.
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to file :math:`i_1`. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0`, the 'Insert File' containing basis information will be read.
The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'Insert File'** : int
Default :math:`= 0`
These files provide compatibility with commercial mathematical programming systems.
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to file :math:`i_1`. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0`, the 'Insert File' containing basis information will be read.
The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'QPSolver Cholesky'** : valueless
Default
Specifies the active-set algorithm used to solve the quadratic program in Phase 2 (:math:`\gamma`). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of Phase 2 (:math:`\gamma`).
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``qpconvex2_sparse_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'QPSolver CG'** : valueless
Specifies the active-set algorithm used to solve the quadratic program in Phase 2 (:math:`\gamma`). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of Phase 2 (:math:`\gamma`).
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``qpconvex2_sparse_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'QPSolver QN'** : valueless
Specifies the active-set algorithm used to solve the quadratic program in Phase 2 (:math:`\gamma`). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of Phase 2 (:math:`\gamma`).
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``qpconvex2_sparse_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'Reduced Hessian Dimension'** : int
Default :math:`= 1\left(\mathrm{LP}\right)\text{ or }\mathrm{min}\left(2000, {n_H+1}, n\right)\left(\mathrm{QP}\right)`
This specifies that an :math:`i\times i` triangular matrix :math:`R` (to define the reduced Hessian according to :math:`R^\mathrm{T}R = Z^\mathrm{T}HZ`). is to be available for use by the Cholesky QP solver.
**'Scale Option'** : int
Default :math:`\text{} = 2`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Print'** : valueless
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Solution Yes'** : valueless
Default
This option determines if the final obtained solution is to be output to the 'Print File'.
Note that the 'Solution File' option operates independently.
**'Solution No'** : valueless
This option determines if the final obtained solution is to be output to the 'Print File'.
Note that the 'Solution File' option operates independently.
**'Solution File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the final solution will be output to file :math:`i` (whether optimal or not).
To see more significant digits in the printed solution, it will sometimes be useful to make :math:`i` refer to the system 'Print File'.
**'Summary File'** : int
Default :math:`\text{} = 0`
If :math:`i_1 > 0`, the 'Summary File' is output to file :math:`i_1`, including a line of the iteration log every :math:`i_2`\ th iteration. In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#monitoring>`__.
If :math:`i_2\leq 0`, the value of :math:`i_2 = 99999999` is used and effectively no checks are made.
**'Summary Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i_1 > 0`, the 'Summary File' is output to file :math:`i_1`, including a line of the iteration log every :math:`i_2`\ th iteration. In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#monitoring>`__.
If :math:`i_2\leq 0`, the value of :math:`i_2 = 99999999` is used and effectively no checks are made.
**'Superbasics Limit'** : int
Default :math:`\text{} = 1\left(\mathrm{LP}\right)\text{ or }\mathrm{min}\left\{{n_H+1}, n\right\}\left(\mathrm{QP}\right)`
This places a limit on the storage allocated for superbasic variables.
Ideally, :math:`i` should be set slightly larger than the 'number of degrees of freedom' expected at an optimal solution.
For linear programs, an optimum is normally a basic solution with no degrees of freedom. (The number of variables lying strictly between their bounds is no more than :math:`m`, the number of general constraints.) The default value of :math:`i` is, therefore, :math:`1`.
For quadratic problems, the number of degrees of freedom is often called the 'number of independent variables'.
Normally, :math:`i` need not be greater than :math:`n_H+1`, where :math:`n_H` is the number of leading nonzero columns of :math:`H`.
For many problems, :math:`i` may be considerably smaller than :math:`n_H`.
This will save storage if :math:`n_H` is very large.
**'Suppress Parameters'** : valueless
Normally ``qpconvex2_sparse_solve`` prints the options file as it is being read, and then prints a complete list of the available keywords and their final values.
The option 'Suppress Parameters' tells ``qpconvex2_sparse_solve`` not to print the full list.
**'System Information No'** : valueless
Default
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#monitoring>`__.
**'System Information Yes'** : valueless
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#monitoring>`__.
**'Timing Level'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, some timing information will be output to the Print file, if :math:`\text{‘Print File'} > 0`.
**'Unbounded Step Size'** : float
Default :math:`\text{} = \textit{infbnd}`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the Hessian is not positive definite.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
See 'Infinite Bound Size' for the definition of :math:`\textit{infbnd}`.
.. _e04nq-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{start} = \texttt{'B'}`, :math:`\texttt{'C'}` or :math:`\texttt{'W'}`.
(`errno` :math:`2`)
On entry, :math:`\textit{ne} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \textit{ne}\leq n\times \mathrm{m}`.
(`errno` :math:`2`)
On entry, :math:`\textit{lenc} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \textit{lenc}\leq n`.
(`errno` :math:`2`)
An error has occurred in the basis package, perhaps indicating incorrect setup of arrays :math:`\mathrm{inda}` and :math:`\mathrm{loca}`. Set the option 'Print File' and examine the output carefully for further information.
(`errno` :math:`2`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ncolh}\leq n`.
(`errno` :math:`2`)
On entry, :math:`\textit{nname} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nname} = 1` or :math:`n+\mathrm{m}`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{iobj}\leq \mathrm{m}`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{loca}[0] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{loca}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{ne} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{loca}[0] = 1` or :math:`\mathrm{loca}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{ne}+1`.
(`errno` :math:`2`)
On entry, row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\mathrm{inda}[\langle\mathit{\boldsymbol{value}}\rangle]` is outside the range :math:`1` to :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`\textit{ne}` is not equal to the number of nonzeros in :math:`\mathrm{acol}`. :math:`\textit{ne} = \langle\mathit{\boldsymbol{value}}\rangle`, nonzeros in :math:`\mathrm{acol} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{bl}` and :math:`\mathrm{bu}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite: :math:`\mathrm{bl} = \mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{bl}` and :math:`\mathrm{bu}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle`:math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{bl} = \mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle`:math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
Basis file dimensions do not match this problem.
(`errno` :math:`2`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`12`)
Internal memory allocation failed when attempting to obtain workspace sizes :math:`\langle\mathit{\boldsymbol{value}}\rangle`, :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle`. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`13`)
Internal memory allocation was insufficient. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`14`)
An error has occurred in the basis package, perhaps indicating incorrect setup of arrays :math:`\mathrm{inda}` and :math:`\mathrm{loca}`. Set the option 'Print File' and examine the output carefully for further information.
(`errno` :math:`15`)
An unexpected error has occurred. Set the option 'Print File' and examine the output carefully for further information.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`4`)
Weak solution found -- the solution is not unique.
(`errno` :math:`5`)
The linear constraints appear to be infeasible.
(`errno` :math:`5`)
The problem appears to be infeasible. The linear equality constraints could not be satisfied.
(`errno` :math:`5`)
The problem appears to be infeasible. Nonlinear infeasibilites have been minimized.
(`errno` :math:`5`)
The problem appears to be infeasible. Infeasibilites have been minimized.
(`errno` :math:`6`)
The problem appears to be unbounded. The objective function is unbounded.
(`errno` :math:`6`)
The problem appears to be unbounded. The constraint violation limit has been reached.
**NagAlgorithmicMajorWarning**
(`errno` :math:`3`)
The requested accuracy could not be achieved.
(`errno` :math:`7`)
Iteration limit reached.
(`errno` :math:`7`)
Major iteration limit reached.
(`errno` :math:`8`)
The value of the option 'Superbasics Limit' is too small.
(`errno` :math:`9`)
The basis is singular after several attempts to factorize it (and add slacks where necessary).
(`errno` :math:`10`)
Numerical difficulties have been encountered and no further progress can be made.
(`errno` :math:`11`)
Error in :math:`\mathrm{qphx}`: the QP Hessian is indefinite.
.. _e04nq-py2-py-notes:
**Notes**
``qpconvex2_sparse_solve`` is designed to solve large-scale `linear` or `quadratic programming problems` of the form:
.. math::
\mathrm{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }l\leq \begin{pmatrix}x\\Ax\end{pmatrix}\leq u\text{,}
where :math:`x` is an :math:`n`-vector of variables, :math:`l` and :math:`u` are constant lower and upper bounds, :math:`A` is an :math:`m\times n` sparse matrix and :math:`f\left(x\right)` is a linear or quadratic objective function that may be specified in a variety of ways, depending upon the particular problem being solved.
The option 'Maximize' may be used to specify a problem in which :math:`f\left(x\right)` is maximized instead of minimized.
Upper and lower bounds are specified for all variables and constraints.
This form allows full generality in specifying various types of constraint.
In particular, the :math:`j`\ th constraint may be defined as an equality by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` may be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
The possible forms for the function :math:`f\left(x\right)` are summarised in Table [label omitted].
The most general form for :math:`f\left(x\right)` is
.. math::
f\left(x\right) = q+c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx = q+\sum_{{j = 1}}^nc_jx_j+\frac{1}{2}\sum_{{i = 1}}^n\sum_{{j = 1}}^nx_iH_{{ij}}x_j
where :math:`q` is a constant, :math:`c` is a constant :math:`n`-vector and :math:`H` is a constant symmetric :math:`n\times n` matrix with elements :math:`\left\{H_{{ij}}\right\}`.
In this form, :math:`f` is a quadratic function of :math:`x` and `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#eqn1>`__ is known as a `quadratic program` (QP). ``qpconvex2_sparse_solve`` is suitable for all `convex` quadratic programs.
The defining feature of a `convex` QP is that the matrix :math:`H` must be `positive semidefinite`, i.e., it must satisfy :math:`x^\mathrm{T}Hx\geq 0` for all :math:`x`.
If not, :math:`f\left(x\right)` is nonconvex and ``qpconvex2_sparse_solve`` will terminate with the error indicator :math:`\mathrm{errno}` = 11.
If :math:`f\left(x\right)` is nonconvex it may be more appropriate to call :meth:`handle_solve_ssqp` instead.
+------------+-------------------------------------------------+-------------------------------+
|Problem type|Objective function :math:`f\left(x\right)` |Hessian matrix :math:`H` |
+============+=================================================+===============================+
|FP |Not applicable |:math:`q = c = H = 0` |
+------------+-------------------------------------------------+-------------------------------+
|LP |:math:`q+c^\mathrm{T}x` |:math:`H = 0` |
+------------+-------------------------------------------------+-------------------------------+
|QP |:math:`q+c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx`|Symmetric positive semidefinite|
+------------+-------------------------------------------------+-------------------------------+
If :math:`H = 0`, then :math:`f\left(x\right) = q+c^\mathrm{T}x` and the problem is known as a `linear program` (LP).
In this case, rather than defining an :math:`H` with zero elements, you can define :math:`H` to have no columns by setting :math:`\mathrm{ncolh} = 0` (see :ref:`Parameters <e04nq-py2-py-parameters>`).
If :math:`H = 0`, :math:`q = 0`, and :math:`c = 0`, there is no objective function and the problem is a `feasible point problem` (FP), which is equivalent to finding a point that satisfies the constraints on :math:`x`.
In the situation where no feasible point exists, several options are available for finding a point that minimizes the constraint violations (see the description of the option 'Elastic Mode').
``qpconvex2_sparse_solve`` is suitable for large LPs and QPs in which the matrix :math:`A` is `sparse`, i.e., when the number of zero elements is sufficiently large that it is worthwhile using algorithms which avoid computations and storage involving zero elements.
The matrix :math:`A` is input to ``qpconvex2_sparse_solve`` by means of the three array arguments :math:`\mathrm{acol}`, :math:`\mathrm{inda}` and :math:`\mathrm{loca}`.
This allows you to specify the pattern of nonzero elements in :math:`A`.
``qpconvex2_sparse_solve`` exploits structure in :math:`H` by requiring :math:`H` to be defined `implicitly` in a function that computes the product :math:`Hx` for any given vector :math:`x`.
In many cases, the product :math:`Hx` can be computed very efficiently for any given :math:`x`, e.g., :math:`H` may be a sparse matrix, or a sum of matrices of rank-one.
For problems in which :math:`A` can be treated as a `dense` matrix, it is usually more efficient to use :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve`.
There is considerable flexibility allowed in the definition of :math:`f\left(x\right)` in Table [label omitted].
The vector :math:`c` defining the linear term :math:`c^\mathrm{T}x` can be input in three ways: as a sparse row of :math:`A`; as an explicit dense vector :math:`c`; or as both a sparse row and an explicit vector (in which case, :math:`c^\mathrm{T}x` will be the sum of two linear terms).
When stored in :math:`A`, :math:`c` is the :math:`\mathrm{iobj}`\ th row of :math:`A`, which is known as the `objective row`.
The objective row must always be a `free` row of :math:`A` in the sense that its lower and upper bounds must be :math:`{-\infty }` and :math:`{+\infty }`.
Storing :math:`c` as part of :math:`A` is recommended if :math:`c` is a sparse vector.
Storing :math:`c` as an explicit vector is recommended for a sequence of problems, each with a different objective (see arguments :math:`\mathrm{c}` and :math:`\textit{lenc}`).
The upper and lower bounds on the :math:`m` elements of :math:`Ax` are said to define the `general constraints` of the problem.
Internally, ``qpconvex2_sparse_solve`` converts the general constraints to equalities by introducing a set of `slack variables` :math:`s`, where :math:`s = \left(s_1,s_2,\ldots,s_m\right)^\mathrm{T}`.
For example, the linear constraint :math:`5\leq 2x_1+3x_2\leq +\infty` is replaced by :math:`2x_1+3x_2-s_1 = 0`, together with the bounded slack :math:`5\leq s_1\leq {+\infty }`.
The problem defined by `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#eqn1>`__ can, therefore, be re-written in the following equivalent form:
.. math::
\mathrm{minimize}_{{x \in R^n,s \in R^m}}\text{}\text{}f\left(x\right)\quad \text{ subject to }Ax-s = 0\text{, }\quad l\leq \begin{pmatrix}x\\s\end{pmatrix}\leq u\text{.}
Since the slack variables :math:`s` are subject to the same upper and lower bounds as the elements of :math:`Ax`, the bounds on :math:`x` and :math:`Ax` can simply be thought of as bounds on the combined vector :math:`\left(x, s\right)`. (In order to indicate their special role in QP problems, the original variables :math:`x` are sometimes known as 'column variables', and the slack variables :math:`s` are known as 'row variables'.)
Each LP or QP problem is solved using a two-phase iterative procedure (in which the general constraints are satisfied throughout): a `feasibility phase` (`Phase 1`), in which the sum of infeasibilities with respect to the bounds on :math:`x` and :math:`s` is minimized to find a feasible point that satisfies all constraints within a specified feasibility tolerance; and an `optimality phase` (`Phase 2`), in which :math:`f\left(x\right)` is minimized (or maximized) by constructing a sequence of iterates that lies within the feasible region.
Phase 1 involves solving a linear program of the form
.. rst-class:: nag-rules-none nag-frame-border
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+
|Phase 1| |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| |:math:`\mathrm{minimize}_{{x,s,v,w}}\sum_{{j = 1}}^{{n+m}}\left({v_j+w_j}\right)` |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+
| |:math:`\text{subject to }Ax-s = 0\text{, }\quad l\leq \begin{pmatrix}x\\s\end{pmatrix}{-v}+w\leq u\text{, }\quad v\geq 0\text{, }\quad w\geq 0`|
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+
which is equivalent to minimizing the sum of the constraint violations.
If the constraints are feasible (i.e., at least one feasible point exists), eventually a point will be found at which both :math:`v` and :math:`w` are zero.
Then the associated value of :math:`\left(x, s\right)` satisfies the original constraints and is used as the starting point for the Phase 2 iterations for minimizing :math:`f\left(x\right)`.
If the constraints are infeasible (i.e., :math:`v\neq 0` or :math:`w\neq 0` at the end of Phase 1), no solution exists for `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#eqn1>`__ and you have the option of either terminating or continuing in so-called `elastic mode` (see the discussion of the option 'Elastic Mode').
In elastic mode, a 'relaxed' or 'perturbed' problem is solved in which :math:`f\left(x\right)` is minimized while allowing some of the bounds to become 'elastic', i.e., to change from their specified values.
Variables subject to elastic bounds are known as `elastic variables`.
An elastic variable is free to violate one or both of its original upper or lower bounds.
You are able to assign which bounds will become elastic if elastic mode is ever started (see the argument :math:`\mathrm{helast}` in :ref:`Parameters <e04nq-py2-py-parameters>`).
To make the relaxed problem meaningful, ``qpconvex2_sparse_solve`` minimizes :math:`f\left(x\right)` while (in some sense) finding the 'smallest' violation of the elastic variables.
In the situation where all the variables are elastic, the relaxed problem has the form
.. rst-class:: nag-rules-none nag-frame-border
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
|Phase 2 (:math:`\gamma`)| |
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| |:math:`\mathrm{minimize}_{{x,s,v,w}}f\left(x\right)+\gamma \sum_{{j = 1}}^{{n+m}}\left({v_j+w_j}\right)` |
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| |:math:`\text{subject to }Ax-s = 0\text{, }\quad l\leq \begin{pmatrix}x\\s\end{pmatrix}{-v}+w\leq u\text{, }\quad v\geq 0\text{, }\quad w\geq 0`,|
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
where :math:`\gamma` is a non-negative argument known as the `elastic weight` (see the description of the option 'Elastic Weight'), and :math:`f\left(x\right)+\gamma \sum_j\left(v_j+w_j\right)` is called the `composite objective`.
In the more general situation where only a subset of the bounds are elastic, the :math:`v`'s and :math:`w`'s for the non-elastic bounds are fixed at zero.
The elastic weight can be chosen to make the composite objective behave like the original objective :math:`f\left(x\right)`, the sum of infeasibilities, or anything in-between.
If :math:`\gamma = 0`, ``qpconvex2_sparse_solve`` will attempt to minimize :math:`f` subject to the (true) upper and lower bounds on the non-elastic variables (and declare the problem infeasible if the non-elastic variables cannot be made feasible).
At the other extreme, choosing :math:`\gamma` sufficiently large will have the effect of minimizing the sum of the violations of the elastic variables subject to the original constraints on the non-elastic variables.
Choosing a large value of the elastic weight is useful for defining a 'least-infeasible' point for an infeasible problem.
In Phase 1 and elastic mode, all calculations involving :math:`v` and :math:`w` are done implicitly in the sense that an elastic variable :math:`x_j` is allowed to violate its lower bound (say) and an explicit value of :math:`v` can be recovered as :math:`v_j = l_j-x_j`.
A constraint is said to be `active` or `binding` at :math:`x` if the associated element of either :math:`x` or :math:`Ax` is equal to one of its upper or lower bounds.
Since an active constraint in :math:`Ax` has its associated slack variable at a bound, the status of both simple and general upper and lower bounds can be conveniently described in terms of the status of the variables :math:`\left(x, s\right)`.
A variable is said to be `nonbasic` if it is temporarily fixed at its upper or lower bound.
It follows that regarding a general constraint as being `active` is equivalent to thinking of its associated slack as being `nonbasic`.
At each iteration of an active-set method, the constraints :math:`Ax-s = 0` are (conceptually) partitioned into the form
.. math::
Bx_B+Sx_S+Nx_N = 0\text{,}
where :math:`x_N` consists of the nonbasic elements of :math:`\left(x, s\right)` and the `basis matrix` :math:`B` is square and nonsingular.
The elements of :math:`x_B` and :math:`x_S` are called the `basic` and `superbasic` variables respectively; with :math:`x_N` they are a permutation of the elements of :math:`x` and :math:`s`.
At a QP solution, the basic and superbasic variables will lie somewhere between their upper or lower bounds, while the nonbasic variables will be equal to one of their bounds.
At each iteration, :math:`x_S` is regarded as a set of independent variables that are free to move in any desired direction, namely one that will improve the value of the objective function (or sum of infeasibilities).
The basic variables are then adjusted in order to ensure that :math:`\left(x, s\right)` continues to satisfy :math:`Ax-s = 0`.
The number of superbasic variables (:math:`n_S` say), therefore, indicates the number of degrees of freedom remaining after the constraints have been satisfied.
In broad terms, :math:`n_S` is a measure of `how nonlinear` the problem is.
In particular, :math:`n_S` will always be zero for FP and LP problems.
If it appears that no improvement can be made with the current definition of :math:`B`, :math:`S` and :math:`N`, a nonbasic variable is selected to be added to :math:`S`, and the process is repeated with the value of :math:`n_S` increased by one.
At all stages, if a basic or superbasic variable encounters one of its bounds, the variable is made nonbasic and the value of :math:`n_S` is decreased by one.
Associated with each of the :math:`m` equality constraints :math:`Ax-s = 0` is a `dual variable` :math:`\pi_i`.
Similarly, each variable in :math:`\left(x, s\right)` has an associated `reduced gradient` :math:`d_j` (also known as a `reduced cost`).
The reduced gradients for the variables :math:`x` are the quantities :math:`g-A^\mathrm{T}\pi`, where :math:`g` is the gradient of the QP objective function, and the reduced gradients for the slack variables :math:`s` are the dual variables :math:`\pi`.
The QP subproblem is optimal if :math:`d_j\geq 0` for all nonbasic variables at their lower bounds, :math:`d_j\leq 0` for all nonbasic variables at their upper bounds and :math:`d_j = 0` for all superbasic variables.
In practice, an `approximate` QP solution is found by slightly relaxing these conditions on :math:`d_j` (see the description of the option 'Optimality Tolerance').
The process of computing and comparing reduced gradients is known as `pricing` (a term first introduced in the context of the simplex method for linear programming).
To 'price' a nonbasic variable :math:`x_j` means that the reduced gradient :math:`d_j` associated with the relevant active upper or lower bound on :math:`x_j` is computed via the formula :math:`d_j = g_j-a_j^\mathrm{T}\pi`, where :math:`a_j` is the :math:`j`\ th column of :math:`\begin{pmatrix}A&-I\end{pmatrix}`. (The variable selected by such a process and the corresponding value of :math:`d_j` (i.e., its reduced gradient) are the quantities +SBS and dj in the monitoring file output; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#fc-printedoutput>`__.) If :math:`A` has significantly more columns than rows (i.e., :math:`n≫m`), pricing can be computationally expensive.
In this case, a strategy known as `partial pricing` can be used to compute and compare only a subset of the :math:`d_j`\ s.
``qpconvex2_sparse_solve`` is based on SQOPT, which is part of the SNOPT package described in Gill `et al.` (2005a).
It uses stable numerical methods throughout and includes a reliable basis package (for maintaining sparse :math:`LU` factors of the basis matrix :math:`B`), a practical anti-degeneracy procedure, efficient handling of linear constraints and bounds on the variables (by an active-set strategy), as well as automatic scaling of the constraints.
Further details can be found in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nqf.html#algdetails>`__.
.. _e04nq-py2-py-references:
**References**
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E and Murray, W, 1978, `Numerically stable methods for quadratic programming`, Math. Programming (14), 349--372
Gill, P E, Murray, W and Saunders, M A, 1995, `User's guide for QPOPT 1.0: a Fortran package for quadratic programming`, Report SOL 95-4, Department of Operations Research, Stanford University
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SQOPT 7: a Fortran package for large-scale linear and quadratic programming`, Report NA 05-1, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sqdoc7.pdf
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SNOPT 7.1: a Fortran package for large-scale linear nonlinear programming`, Report NA 05-2, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sndoc7.pdf
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1987, `Maintaining` :math:`LU` `factors of a general sparse matrix`, Linear Algebra and its Applics. (88/89), 239--270
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1991, `Inertia-controlling methods for general quadratic programming`, SIAM Rev. (33), 1--36
Hall, J A J and McKinnon, K I M, 1996, `The simplest examples where the simplex method cycles and conditions where EXPAND fails to prevent cycling`, Report MS, 96--100, Department of Mathematics and Statistics, University of Edinburgh
See Also
--------
:meth:`naginterfaces.library.examples.opt.qpconvex2_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_file(ispecs, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_file`` may be used to supply options to :meth:`qpconvex2_sparse_solve` from an external file.
The initialization function :meth:`qpconvex2_sparse_init` **must** have been called before calling ``qpconvex2_sparse_option_file``.
.. _e04nr-py2-py-doc:
For full information please refer to the NAG Library document for e04nr
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nrf.html
.. _e04nr-py2-py-parameters:
**Parameters**
**ispecs** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the option file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nr-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
At least one line of the options file is invalid.
(`errno` :math:`2`)
Could not read options file on unit :math:`\mathrm{ispecs} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nr-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_file`` may be used to supply values for options to :meth:`qpconvex2_sparse_solve`. ``qpconvex2_sparse_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Option settings are preserved following a call to :meth:`qpconvex2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_string(string, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_string`` may be used to supply individual options to :meth:`qpconvex2_sparse_solve`.
The initialization function :meth:`qpconvex2_sparse_init` **must** have been called before calling ``qpconvex2_sparse_option_string``.
.. _e04ns-py2-py-doc:
For full information please refer to the NAG Library document for e04ns
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nsf.html
.. _e04ns-py2-py-parameters:
**Parameters**
**string** : str
A single valid option string (see :ref:`Notes <e04ns-py2-py-notes>` in ``qpconvex2_sparse_option_string`` and :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ns-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ns-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_string`` may be used to supply values for options to :meth:`qpconvex2_sparse_solve`.
It is only necessary to call ``qpconvex2_sparse_option_string`` for those arguments whose values are to be different from their default values.
One call to ``qpconvex2_sparse_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`qpconvex2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_integer_set(string, ivalue, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_integer_set`` may be used to supply individual `int` options to :meth:`qpconvex2_sparse_solve`.
The initialization function :meth:`qpconvex2_sparse_init` **must** have been called before calling ``qpconvex2_sparse_option_integer_set``.
.. _e04nt-py2-py-doc:
For full information please refer to the NAG Library document for e04nt
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ntf.html
.. _e04nt-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`).
**ivalue** : int
An `int` value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nt-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ivalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nt-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_integer_set`` may be used to supply values for `int` options to :meth:`qpconvex2_sparse_solve`.
It is only necessary to call ``qpconvex2_sparse_option_integer_set`` for those arguments whose values are to be different from their default values.
One call to ``qpconvex2_sparse_option_integer_set`` sets one argument value.
Each `int` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{ivalue}`.
Option settings are preserved following a call to :meth:`qpconvex2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_double_set(string, rvalue, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_double_set`` may be used to supply individual `float` options to :meth:`qpconvex2_sparse_solve`.
The initialization function :meth:`qpconvex2_sparse_init` **must** have been called before calling ``qpconvex2_sparse_option_double_set``.
.. _e04nu-py2-py-doc:
For full information please refer to the NAG Library document for e04nu
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nuf.html
.. _e04nu-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`).
**rvalue** : float
The value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nu-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{rvalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nu-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_double_set`` may be used to supply values for `float` options to :meth:`qpconvex2_sparse_solve`.
It is only necessary to call ``qpconvex2_sparse_option_double_set`` for those arguments whose values are to be different from their default values.
One call to ``qpconvex2_sparse_option_double_set`` sets one argument value.
Each `float` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{rvalue}`.
Option settings are preserved following a call to :meth:`qpconvex2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`qpconvex2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_integer_get(string, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_integer_get`` is used to get the value of an `int` option. ``qpconvex2_sparse_option_integer_get`` can be used before or after calling :meth:`qpconvex2_sparse_solve`.
.. _e04nx-py2-py-doc:
For full information please refer to the NAG Library document for e04nx
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nxf.html
.. _e04nx-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**ivalue** : int
The `int` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04nx-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option string is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04nx-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_integer_get`` obtains the current value of an integer option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def qpconvex2_sparse_option_double_get(string, comm, io_manager=None):
r"""
``qpconvex2_sparse_option_double_get`` is used to get the value of a `float` option. ``qpconvex2_sparse_option_double_get`` can be used before or after calling :meth:`qpconvex2_sparse_solve`.
.. _e04ny-py2-py-doc:
For full information please refer to the NAG Library document for e04ny
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04nyf.html
.. _e04ny-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`qpconvex2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**rvalue** : float
The `float` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04ny-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`qpconvex2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ny-py2-py-notes:
**Notes**
``qpconvex2_sparse_option_double_get`` obtains the current value of a `float` option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for qpconvex2_sparse_solve <e04nq-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def bnd_lin_lsq(a, b, bl, bu, itype=1, tol=0.0):
r"""
``bnd_lin_lsq`` solves a linear least squares problem subject to fixed lower and upper bounds on the variables.
.. _e04pc-py2-py-doc:
For full information please refer to the NAG Library document for e04pc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04pcf.html
.. _e04pc-py2-py-parameters:
**Parameters**
**a** : float, array-like, shape :math:`\left(m, n\right)`
The :math:`m\times n` matrix :math:`A`.
**b** : float, array-like, shape :math:`\left(m\right)`
The right-hand side vector :math:`b`.
**bl** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bl}[\textit{i}-1]` and :math:`\mathrm{bu}[\textit{i}-1]` must specify the lower and upper bounds, :math:`l_i` and :math:`u_i` respectively, to be imposed on the solution vector :math:`x_i`.
**bu** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bl}[\textit{i}-1]` and :math:`\mathrm{bu}[\textit{i}-1]` must specify the lower and upper bounds, :math:`l_i` and :math:`u_i` respectively, to be imposed on the solution vector :math:`x_i`.
**itype** : int, optional
Provides the choice of returning a regularized solution if the matrix is not of full rank.
:math:`\mathrm{itype} = 0`
Specifies that a regularized solution is to be computed.
:math:`\mathrm{itype} = 1`
Specifies that no regularization is to take place.
**tol** : float, optional
:math:`\mathrm{tol}` specifies a parameter used to determine the relative linear dependence of a column vector for a variable moved from its initial value. It determines the computational rank of the matrix. Increasing its value from :math:`\sqrt{\text{machine precision}}` will increase the likelihood of additional elements of :math:`x` being set to zero. It may be worth experimenting with increasing values of :math:`\mathrm{tol}` to determine whether the nature of the solution, :math:`x`, changes significantly. In practice a value of :math:`\sqrt{\text{machine precision}}` is recommended (see :meth:`machine.precision <naginterfaces.library.machine.precision>`).
If on entry :math:`\mathrm{tol} < \sqrt{\text{machine precision}}`, :math:`\sqrt{\text{machine precision}}` is used.
**Returns**
**a** : float, ndarray, shape :math:`\left(m, n\right)`
If :math:`\mathrm{itype} = 1`, :math:`\mathrm{a}` contains the product matrix :math:`QA`, where :math:`Q` is an :math:`m\times m` orthogonal matrix generated by ``bnd_lin_lsq``; otherwise, :math:`\mathrm{a}` is unchanged.
**b** : float, ndarray, shape :math:`\left(m\right)`
If :math:`\mathrm{itype} = 1`, the product of :math:`Q` times the original vector :math:`b`, where :math:`Q` is as described in argument :math:`\mathrm{a}`; otherwise, :math:`\mathrm{b}` is unchanged.
**x** : float, ndarray, shape :math:`\left(n\right)`
The solution vector :math:`x`.
**rnorm** : float
The Euclidean norm of the residual vector :math:`b-Ax`.
**nfree** : int
Indicates the number of components of the solution vector that are not at one of the constraints.
**w** : float, ndarray, shape :math:`\left(n\right)`
Contains the dual solution vector. The magnitude of :math:`\mathrm{w}[i-1]` gives a measure of the improvement in the objective value if the corresponding bound were to be relaxed so that :math:`x_i` could take different values.
A value of :math:`\mathrm{w}[i-1]` equal to the special value :math:`-999.0` is indicative of the matrix :math:`A` not having full rank.
It is only likely to occur when :math:`\mathrm{itype} = 1`.
However a matrix may have less than full rank without :math:`\mathrm{w}[i-1]` being set to :math:`-999.0`.
If :math:`\mathrm{itype} = 1`, then the values contained in :math:`\mathrm{w}` (other than those set to :math:`-999.0`) may be unreliable; the corresponding values in :math:`\mathrm{indx}` may likewise be unreliable.
If you have any doubts set :math:`\mathrm{itype} = 0`.
Otherwise, the values of :math:`\mathrm{w}[i-1]` have the following meaning:
:math:`\mathrm{w}[i-1] = 0`
if :math:`x_i` is unconstrained.
:math:`\mathrm{w}[i-1] < 0`
if :math:`x_i` is constrained by its lower bound.
:math:`\mathrm{w}[i-1] > 0`
if :math:`x_i` is constrained by its upper bound.
:math:`\mathrm{w}[i-1]`
may be any value if :math:`l_i = u_i`.
**indx** : int, ndarray, shape :math:`\left(n\right)`
The contents of this array describe the components of the solution vector as follows:
:math:`\mathrm{indx}[\textit{i}-1]`, for :math:`\textit{i} = 1,2,\ldots,\mathrm{nfree}`
These elements of the solution have not hit a constraint; i.e., :math:`\mathrm{w}[i-1] = 0`.
:math:`\mathrm{indx}[\textit{i}-1]`, for :math:`\textit{i} = \mathrm{nfree}+1,\ldots,k`
These elements of the solution have been constrained by either the lower or upper bound.
:math:`\mathrm{indx}[\textit{i}-1]`, for :math:`\textit{i} = k+1,\ldots,n`
These elements of the solution are fixed by the bounds; i.e., :math:`\mathrm{bl}[i-1] = \mathrm{bu}[i-1]`.
Here :math:`k` is determined from :math:`\mathrm{nfree}` and the number of fixed components. (Often the latter will be :math:`0`, so :math:`k` will be :math:`n-\mathrm{nfree}`.)
.. _e04pc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 0`.
(`errno` :math:`1`)
On entry, :math:`m = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`m\geq 0`.
(`errno` :math:`1`)
On entry, when :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[i-1]\leq \mathrm{bu}[i-1]`.
(`errno` :math:`1`)
On entry, :math:`m = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{lda} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lda}\geq m`.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`2`)
The function failed to converge in :math:`3\times n` iterations. This is not expected. Please contact `NAG <https://www.nag.com>`__.
.. _e04pc-py2-py-notes:
**Notes**
Given an :math:`m\times n` matrix :math:`A`, an :math:`n`-vector :math:`l` of lower bounds, an :math:`n`-vector :math:`u` of upper bounds, and an :math:`m`-vector :math:`b`, ``bnd_lin_lsq`` computes an :math:`n`-vector :math:`x` that solves the least squares problem :math:`Ax = b` subject to :math:`x_i` satisfying :math:`l_i\leq x_i\leq u_i`.
A facility is provided to return a 'regularized' solution, which will closely approximate a minimal length solution whenever :math:`A` is not of full rank.
A minimal length solution is the solution to the problem which has the smallest Euclidean norm.
The algorithm works by applying orthogonal transformations to the matrix and to the right-hand side to obtain within the matrix an upper triangular matrix :math:`R`.
In general the elements of :math:`x` corresponding to the columns of :math:`R` will be the candidate nonzero solutions.
If a diagonal element of :math:`R` is small compared to the other members of :math:`R` then this is undesirable. :math:`R` will be nearly singular and the equations for :math:`x` thus ill-conditioned.
You may specify the tolerance used to determine the relative linear dependence of a column vector for a variable moved from its initial value.
.. _e04pc-py2-py-references:
**References**
Lawson, C L and Hanson, R J, 1974, `Solving Least Squares Problems`, Prentice--Hall
"""
raise NotImplementedError
[docs]def handle_solve_socp_ipm(handle, x=None, u=None, uc=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_socp_ipm`` is a solver from the NAG optimization modelling suite for large-scale Second-order Cone Programming (SOCP) problems.
It is based on an interior point method (IPM).
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04pt-py2-py-doc:
For full information please refer to the NAG Library document for e04pt
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html
.. _e04pt-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_socp_ipm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : None or float, array-like, shape :math:`\left(\textit{nvar}\right)`, optional
The input of :math:`\mathrm{x}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the bound constraints and linear constraints. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
The input of :math:`\mathrm{u}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**uc** : None or float, array-like, shape :math:`\left(\textit{nnzuc}\right)`, optional
Note: if :math:`\textit{nnzuc} > 0`, :math:`\mathrm{uc}` holds Lagrange multipliers (dual variables) for second-order cones as defined by :meth:`handle_set_group`. If :math:`\textit{nnzuc} = 0`, :math:`\mathrm{uc}` will not be referenced.
The input of :math:`\mathrm{uc}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**monit** : None or callable monit(handle, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the option 'SOCP Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**handle** : Handle
The handle to the problem as provided on entry to ``handle_solve_socp_ipm``. It may be used to query the model during the solve, and extract the current approximation of the solution by :meth:`handle_set_get_real`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`, however, elements :math:`3`, :math:`5`, :math:`9`, :math:`10` and :math:`15` refer to the quantities in the last iteration rather than accumulated over all iterations through the whole algorithm run.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final values of the variables :math:`u`.
**uc** : float, ndarray, shape :math:`\left(\textit{nnzuc}\right)`
The final values of the variables :math:`\textit{uc}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators of the algorithm (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdetails>`__ for details) as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Value of the primal objective. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Value of the dual objective. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Flag indicating the system formulation used by the solver, :math:`0`: augmented system, :math:`1`: normal equation. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Factorization type, :math:`3`: Cholesky, :math:`4`: Bunch--Parlett. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4`--:math:`13` |Not referenced in this solver. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Relative primal infeasibility, see `Convergence-optimal termination <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdet_sc_cot>`__.|
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15` |Relative duality gap, see `Convergence-optimal termination <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdet_sc_cot>`__. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`16` |Relative dual infeasibility, see `Convergence-optimal termination <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdet_sc_cot>`__. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`17` |Accuracy, see `Convergence-optimal termination <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdet_sc_cot>`__. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`18` |:math:`\tau`, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#hsd>`__. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`19` |:math:`\kappa`, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#hsd>`__. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`20` |Step length. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`21`--:math:`99`|Reserved for future use. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics as given in the table below. Note that times are measured in seconds, see option 'Stats Time'.
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Not referenced. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of iterative refinements performed. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Value of the perturbation added to the diagonal in the normal equation formulation or the augmented system formulation.|
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Total number of factorizations performed. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Total time spent in the solver. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Time spent in the presolve phase. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Time spent in the last iteration. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent factorizing the system matrix. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent backsolving the system matrix. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Not referenced. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Time spent in the initialization phase. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Number of nonzeros in the system matrix. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`13` |Number of nonzeros in the system matrix factor. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Maximum error of the backsolve. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`15` |Number of columns in :math:`A` considered dense by the solver. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`16` |Number of conic constraints considered dense by the solver. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
|:math:`17`--:math:`99`|Reserved for future use. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------+
.. _e04pt-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any argument value given with this keyword will be ignored.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options;
- problem statistics, the iteration log, and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log, and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the primal and dual objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but each iteration line is longer, including step lengths and errors |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'SOCP Factorization Method'** : str
Default :math:`= \texttt{'MA86'}`
If the value of 'SOCP System Formulation' is 'AUGMENTED SYSTEM', then this parameter controls whether Harwell packages 'MA86' or 'MA97' is used for the sparse linear algebra factorization.
Note that if the option value 'SOCP System Formulation' is set to 'AUTO' or 'NORMAL EQUATIONS', then specifying 'MA86' with this option will allow the solver to use this package in the case that the solver switches to the augmented system formulation.
Constraint: :math:`\text{‘SOCP Factorization Method'} = \texttt{'MA97'}` or :math:`\texttt{'MA86'}`.
**'SOCP Iteration Limit'** : int
Default :math:`\text{} = 100`
The maximum number of iterations to be performed by ``handle_solve_socp_ipm``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘SOCP Iteration Limit'} \geq 1`.
**'SOCP Monitor Frequency'** : int
Default :math:`\text{} = 0`
This argument defines the frequency of how often function :math:`\mathrm{monit}` is called.
If :math:`i > 0`, the solver calls :math:`\mathrm{monit}` at the end of every :math:`i`\ th iteration.
If it is set to :math:`0`, the function is not called at all.
Constraint: :math:`\text{‘SOCP Monitor Frequency'} \geq 0`.
**'SOCP Presolve'** : str
Default :math:`= \texttt{'FULL'}`
This argument allows you to reduce the level of presolving of the problem or turn it off completely.
If the presolver is turned off, the solver will try to handle the problem as given by you.
In such a case, the presence of fixed variables or linear dependencies in the constraint matrix can cause numerical instabilities to occur.
In normal circumstances, it is recommended to use the full presolve which is the default.
Constraint: :math:`\text{‘SOCP Presolve'} = \texttt{'FULL'}`, :math:`\texttt{'BASIC'}` or :math:`\texttt{'NO'}`.
**'SOCP Scaling'** : str
Default :math:`= \texttt{'NONE'}`
This argument controls the type of scaling to be applied on the constraint matrix :math:`A` before solving the problem.
More precisely, the scaling procedure will try to find diagonal matrices :math:`D_1` and :math:`D_2` such that the values in :math:`D_1AD_2` are of a similar order of magnitude.
The solver is less likely to run into numerical difficulties when the constraint matrix is well scaled.
Constraint: :math:`\text{‘SOCP Scaling'} = \texttt{'ARITHMETIC'}`, :math:`\texttt{'GEOMETRIC'}` or :math:`\texttt{'NONE'}`.
**'SOCP Stop Tolerance'** : float
Default :math:`= \sqrt{\epsilon }`
This argument sets the value :math:`\epsilon_1` which is the tolerance for the convergence measures in the stopping criteria, see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘SOCP Stop Tolerance'} > \epsilon`.
**'SOCP Stop Tolerance 2'** : float
Default :math:`= \sqrt{\epsilon }`
This argument sets the additional tolerance :math:`\epsilon_2` used in the stopping criteria, see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘SOCP Stop Tolerance 2'} > \epsilon`.
**'SOCP System Formulation'** : str
Default :math:`= \texttt{'AUGMENTED SYSTEM'}`
As described in `Solving the KKT System <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdetails_system>`__, ``handle_solve_socp_ipm`` can internally work either with the normal equations formulation `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#ne>`__ or with the augmented system `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#as1>`__ and `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#as1>`__.
A brief discussion of advantages and disadvantages is presented in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#as2>`__.
Setting the option value to 'AUTO' leaves the decision to the solver based on the structure of the constraints.
This will typically lead to the normal equations formulation unless there are many dense columns or the system is significantly cheaper to factorize as the augmented system.
Note that in some cases even if :math:`\text{‘SOCP System Formulation'} = \texttt{'NORMAL EQUATIONS'}` the solver might switch the formulation through the computation to the augmented system due to numerical instabilities or computational cost.
Constraint: :math:`\text{‘SOCP System Formulation'} = \texttt{'AUTO'}`, :math:`\texttt{'AUGMENTED SYSTEM'}`, :math:`\texttt{'AS'}`, :math:`\texttt{'NORMAL EQUATIONS'}` or :math:`\texttt{'NE'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04pt-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is either :math:`0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is :math:`0` for no constraints.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzuc} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzuc}` does not match the size of the Lagrangian multipliers for second-order cone constraints.
The correct value is either :math:`0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzuc} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzuc}` does not match the size of the Lagrangian multipliers for second-order cone constraints.
:math:`\textit{nnzuc} = 0` when there are no second-order cone constraints.
(`errno` :math:`51`)
The problem was found to be primal infeasible.
(`errno` :math:`52`)
The problem was found to be dual infeasible.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Suboptimal solution.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, stopping early.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04pt-py2-py-notes:
**Notes**
``handle_solve_socp_ipm`` solves a large-scale SOCP optimization problem in the following form
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x \\ \text{subject to} & l_A \leq Ax \leq u_A \text{,} \\& l_x \leq x \leq u_x \text{,} \\& x \in \mathcal{K} \text{,} \end{array}
where :math:`\mathcal{K} = \mathcal{K}^{n_1}\times \cdots \times \mathcal{K}^{n_r}\times ℝ^{n_l}` is a Cartesian product of :math:`r` quadratic (second-order type) cones and :math:`n_l`-dimensional real space, and :math:`n = \sum_{{i = 1}}^rn_i+n_l` is the number of decision variables.
Here :math:`c`, :math:`x`, :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors, :math:`A` is an :math:`m\times n` sparse matrix, and :math:`l_A` and :math:`u_A` are :math:`m`-dimensional vectors.
Note that :math:`x \in \mathcal{K}` partitions subsets of variables into quadratic cones and each :math:`\mathcal{K}^{n_i}` can be either a quadratic cone or a rotated quadratic cone.
These are defined as follows:
- Quadratic cone:
.. math::
\mathcal{K}_q^{n_i} \mathrel{:=} \left\{z = \left(z_1, z_2, \ldots, z_{n_i}\right) \in \mathbb{R}^{n_i}\quad \text{ : }\quad z_1^2\geq \sum_{{j = 2}}^{n_i}z_j^2\text{, }\quad z_1\geq 0\right\}\text{.}
- Rotated quadratic cone:
.. math::
\mathcal{K}_r^{n_i} \mathrel{:=} \left\{z = \left(z_1, z_2, \ldots, z_{n_i}\right) \in \mathbb{R}^{n_i}\quad \text{ : }\quad 2z_1z_2\geq \sum_{{j = 3}}^{n_i}z_j^2\text{, }\quad z_1\geq 0\text{, }\quad z_2\geq 0\right\}\text{.}
``handle_solve_socp_ipm`` solves SOCP problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_group`, :meth:`handle_set_linobj`, :meth:`handle_set_qconstr`, :meth:`handle_set_qconstr_fac`, :meth:`handle_set_quadobj`, :meth:`handle_set_simplebounds` or :meth:`handle_set_linconstr` may be called to formulate the quadratic cones, linear objective function, quadratic objective function, quadratic constraints, bounds of the variables, and the block of linear constraints, respectively.
Alternatively, the whole model can be loaded from a file by :meth:`handle_read_file`.
When the handle is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver method can be modified by various options (see :ref:`Other Parameters <e04pt-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various options.
The option 'Task' may be used to switch the problem to maximization or to ignore the objective function and find only a feasible point.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended that you experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ptf.html#algdetails>`__ and :ref:`Other Parameters <e04pt-py2-py-other_params>` for further details.
**Structure of the Lagrangian Multipliers**
The algorithm works internally with estimates of both the decision variables, denoted by :math:`x`, and the Lagrangian multipliers (dual variables), denoted by :math:`u` for bound and linear constraints, and :math:`uc` for quadratic cone constraints.
If the simple bounds have been defined (:meth:`handle_set_simplebounds` was successfully called), the first :math:`2n` elements of :math:`u` belong to the corresponding Lagrangian multipliers, interleaving a multiplier for the lower and the upper bound for each :math:`x_i`.
If any of the bounds were set to infinity, the corresponding Lagrangian multipliers are set to :math:`0` and may be ignored.
Similarly, the following :math:`2m` elements of :math:`u` belong to multipliers for the linear constraints (if :meth:`handle_set_linconstr` has been successfully called).
The organization is the same, i.e., the multipliers for each constraint for the lower and upper bounds are alternated and zeros are used for any missing (infinite bound) constraints.
If convex quadratic constraints have been defined successfully by :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac`, denote the number of such constraints as :math:`\textit{nq}`, then the following :math:`2\textit{nq}` elements of :math:`u` belong to multipliers for the convex quadratic constraints.
The organization is the same as linear constraints.
Some solvers merge multipliers for both lower and upper inequality into one element whose sign determines the inequality.
Negative multipliers are associated with the upper bounds and positive with the lower bounds.
An equivalent result can be achieved with this storage scheme by subtracting the upper bound multiplier from the lower one.
This is also consistent with equality constraints.
Finally, the elements of :math:`uc` are the corresponding Lagrangian multipliers for the variables in the quadratic cone constraints that have been defined by :meth:`handle_set_group`.
All multipliers are stored next to each other in array :math:`\mathrm{uc}` in the same order as the cone constraints were defined by :meth:`handle_set_group`.
For example, if the first cone constraint contains variables :math:`x_4`, :math:`x_2`, :math:`x_3` and the second cone constraint contains variables :math:`x_1`, :math:`x_7`, :math:`x_6`, :math:`x_5`, then the dimension of array :math:`\mathrm{uc}` must be :math:`7` and the first :math:`3` elements are the corresponding Lagrangian multipliers for the cone composed of :math:`x_4`, :math:`x_2`, :math:`x_3`, followed by :math:`4` elements that are the corresponding Lagrangian multipliers for the cone of :math:`x_1`, :math:`x_7`, :math:`x_6`, :math:`x_5`.
.. _e04pt-py2-py-references:
**References**
Alizadeh, F and Goldfarb, D, 2003, `Second-order cone programming`, Mathematical programming (95(1)), 3--51
Andersen, E D, Roos, C and Terlaky, T, 2003, `On implementing a primal-dual interior-point method for conic quadratic optimization`, Mathematical programming (95(2)), 249--277
Goldfarb, D and Scheinberg, K, 2005, `Product-form Cholesky factorization in interior point methods for second-order cone programming`, Mathematical programming (103(1)), 153--179
Goldman, A J and Tucker, A W, 1956, `Theory of linear programming`, Linear inequalities and related systems (38), 53--97
Hogg, J D and Scott, J A, 2010, `An indefinite sparse direct solver for large problems on multicore machines`, RAL Technical Report. RAL-TR-2010-011
Hogg, J D and Scott, J A, 2011, `HSL MA97: a bit-compatible multifrontal code for sparse symmetric systems`, RAL Technical Report. RAL-TR-2011-024
HSL, `a collection of Fortran codes for large-scale scientific computation`, http://www.hsl.rl.ac.uk/
Karypis, G and Kumar, V, 1998, `A fast and high quality multilevel scheme for partitioning irregular graphs`, SIAM J. Sci. Comput. (20(1)), 359--392
Lobo, M S, Vandenberghe, L, Boyd, S and Levret, H, 1998, `Applications of second-order cone programming`, Linear Algebra and its Applications (284(1-3)), 193--228
Lustig, I J, Marsten, R E and Shanno, D F, 1992, `On implementing Mehrotra's predictor--corrector interior-point method for linear programming`, SIAM J. Optim. (2(3)), 435--449
Mehrotra, S, 1992, `On the implementation of a primal-dual interior point method`, SIAM J. Optim. (2), 575--601
Nesterov, Y E and Todd, M J, 1997, `Self-scaled barriers and interior-point methods for convex programming`, Mathematics of Operations research (22(1)), 1--42
Nesterov, Y E and Todd, M J, 1998, `Primal-dual interior-point methods for self-scaled cones`, SIAM J. Optim. (8(2)), 324--364
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
Sturm, J F, 2002, `Implementation of Interior Point Methods for Mixed Semidefinite and Second Order Cone Optimization Problems`, Optimization Methods and Software (17(6)), 151--171
Xu, X, Hung, P-F and Ye, Y, 1996, `A simplified homogeneous and self-dual linear programming algorithm and its implementation`, Annals of Operations Research (62(1)), 151--171
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_socp_ipm_ex.main`
"""
raise NotImplementedError
[docs]def handle_init(nvar):
r"""
``handle_init`` initializes a data structure for the NAG optimization modelling suite for problems such as, Linear Programming (LP), Quadratic Programming (QP), Nonlinear Programming (NLP), Least Squares (LSQ) problems, Second-order Cone Programming (SOCP), linear Semidefinite Programming (SDP) and Semidefinite Programming with Bilinear Matrix Inequalities (BMI-SDP).
.. _e04ra-py2-py-doc:
For full information please refer to the NAG Library document for e04ra
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04raf.html
.. _e04ra-py2-py-parameters:
**Parameters**
**nvar** : int
:math:`n`, the initial number of decision variables defining the problem.
**Returns**
**handle** : Handle
Holds a handle to the internal data structure where an empty problem with :math:`\mathrm{nvar}` variables is defined.
.. _e04ra-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`6`)
On entry, :math:`\mathrm{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nvar}\geq 0`.
.. _e04ra-py2-py-notes:
**Notes**
``handle_init`` initializes an empty problem, adds :math:`n\geq 0` decision variables, :math:`x`, and returns a handle to the data structure.
Note that further variables may be added to the problem later by :meth:`handle_add_vars` and it is also possible to temporarily disable and enable individual variables in the model by :meth:`handle_disable` and :meth:`handle_enable`, respectively. This handle may then be passed to some of the functions of the suite to formulate the problem (define or edit the variables, the objective function and constraints). Once the problem is fully defined, the handle may be passed to a suitable solver from the suite (:meth:`handle_solve_dfls`, :meth:`handle_solve_dfls_rcomm`, :meth:`handle_solve_bxnl`, :meth:`handle_solve_nldf`, :meth:`handle_solve_dfno`, :meth:`handle_solve_dfno_rcomm`, :meth:`handle_solve_bounds_foas`, :meth:`handle_solve_lp_ipm`, :meth:`handle_solve_socp_ipm`, :meth:`handle_solve_ssqp`, :meth:`handle_solve_ipopt`, :meth:`handle_solve_pennon` or :meth:`mip.handle_solve_milp <naginterfaces.library.mip.handle_solve_milp>`). Afterwards, the problem may be further modified or passed to another solver of the suite.
The handle **must not** be changed between calls to the functions of the suite.
When the handle is no longer needed, :meth:`handle_free` must be called to destroy it and deallocate all the allocated memory and data within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.glopt.handle_solve_mcs_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
(and others)
"""
raise NotImplementedError
[docs]def handle_set_group(handle, gtype, idgroup, group=None):
r"""
``handle_set_group`` is a part of the NAG optimization modelling suite and modifies a model by either adding a new, or replacing, or deleting an existing quadratic or rotated quadratic cone constraint.
.. _e04rb-py2-py-doc:
For full information please refer to the NAG Library document for e04rb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rbf.html
.. _e04rb-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**gtype** : str
The type of the cone constraint. :math:`\mathrm{gtype}` is case insensitive.
:math:`\mathrm{gtype} = \texttt{'QUAD'}` or :math:`\texttt{'Q'}`
The group defines a quadratic cone.
:math:`\mathrm{gtype} = \texttt{'RQUAD'}` or :math:`\texttt{'R'}`
The group defines a rotated quadratic cone.
**idgroup** : int
:math:`\mathrm{idgroup} = 0`
A new cone constraint is created.
:math:`\mathrm{idgroup} > 0`
:math:`i`, the ID number of the existing constraint to be deleted or replaced.
**group** : None or int, array-like, shape :math:`\left(\textit{lgroup}\right)`, optional
:math:`G^i`, the indices of the variables in the constraint. If :math:`\textit{lgroup} = 0`, :math:`\mathrm{group}` is not referenced.
**Returns**
**idgroup** : int
If :math:`\mathrm{idgroup} = 0` on entry, the ID number of the new cone constraint is returned. By definition, this is the number of cone constraints already defined plus one. Otherwise, :math:`\mathrm{idgroup}` remains unchanged.
.. _e04rb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idgroup} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idgroup}` does not match with any cone constraint already defined.
(`errno` :math:`5`)
On entry, :math:`\textit{lgroup} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lgroup} \geq 0`.
(`errno` :math:`5`)
On entry, :math:`\mathrm{gtype} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{lgroup} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{gtype} = \texttt{'QUAD'}` or :math:`\texttt{'Q'}`, :math:`\textit{lgroup} = 0` or :math:`\textit{lgroup} \geq 2`.
(`errno` :math:`5`)
On entry, :math:`\mathrm{gtype} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{lgroup} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{gtype} = \texttt{'RQUAD'}` or :math:`\texttt{'R'}`, :math:`\textit{lgroup} = 0` or :math:`\textit{lgroup} \geq 3`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{gtype} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{gtype} = \texttt{'QUAD'}`, :math:`\texttt{'Q'}`, :math:`\texttt{'RQUAD'}` or :math:`\texttt{'R'}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{idgroup} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idgroup} \geq 0`.
(`errno` :math:`8`)
On entry, :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{group}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{group}[k-1]\leq n`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{group}[i-1] = \mathrm{group}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` for :math:`i = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: elements in :math:`\mathrm{group}` cannot repeat.
.. _e04rb-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_group`` may be used to edit a model by adding, replacing, or deleting a cone constraint :math:`i` of dimension :math:`m_i`.
The supported cones are quadratic cone and rotated quadratic cone, also known as second-order cones, which are defined as follows:
- Quadratic cone:
.. math::
\mathcal{K}_q^{m_i} \mathrel{:=} \left\{z = \left(z_1, z_2, \ldots, z_{m_i}\right) \in \mathbb{R}^{m_i}\quad \text{ : }\quad z_1^2\geq \sum_{{j = 2}}^{m_i}z_j^2\text{, }\quad z_1\geq 0\right\}\text{.}
- Rotated quadratic cone:
.. math::
\mathcal{K}_r^{m_i} \mathrel{:=} \left\{z = \left(z_1, z_2, \ldots, z_{m_i}\right) \in \mathbb{R}^{m_i}\quad \text{ : }\quad 2z_1z_2\geq \sum_{{j = 3}}^{m_i}z_j^2\text{, }\quad z_1\geq 0\text{, }\quad z_2\geq 0\right\}\text{.}
The cone constraint is defined by its type and a subset (group) of variables.
Let index set :math:`G^i⊆\left\{1, 2, \ldots, n\right\}` denote variable indices, then :math:`x_{G^i}` will denote the subvector of variables :math:`x \in \mathbb{R}^n`.
For example, if :math:`m_i = 3` and :math:`G^i = \left\{4, 1, 2\right\}`, then a quadratic cone constraint
.. math::
x_{G^i} = \left(x_4, x_1, x_2\right) \in \mathcal{K}_q^3
implies the inequality constraints
.. math::
x_4^2\geq x_1^2+x_2^2\text{, }\quad x_4\geq 0\text{.}
Typically, this function will be used to build Second-order Cone Programming (SOCP) problems which might be formulated in the following way:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to} &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\\&x_{G^i} \in \mathcal{K}^{m_i},\quad \text{ }\quad i = 1,\ldots,r\text{,}&\quad \text{ (d)}\end{array}
where :math:`\mathcal{K}^{m_i}` is either a quadratic cone or a rotated quadratic cone of dimension :math:`m_i`.
``handle_set_group`` can be called repeatedly to add, replace or delete one cone constraint at a time.
Note that it is also possible to temporarily disable and enable individual cone constraints in the model by calling :meth:`handle_disable` and :meth:`handle_enable`, respectively.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_socp_ipm_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_property(handle, ptype, idx):
r"""
``handle_set_property`` is a part of the NAG optimization modelling suite and sets a given property to a subset of the decision variables, particularly, it specifies if the variables are integral or continuous.
.. _e04rc-py2-py-doc:
For full information please refer to the NAG Library document for e04rc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rcf.html
.. _e04rc-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**ptype** : str
Indicates the property to set for the specified variables. :math:`\mathrm{ptype}` is case insensitive.
Class of properties referring to integrality:
:math:`\mathrm{ptype} = \texttt{'CONTINUOUS'}` or :math:`\texttt{'CONT'}`
The variables will be continuous (the default property).
:math:`\mathrm{ptype} = \texttt{'INTEGER'}` or :math:`\texttt{'INT'}`
The variables become integer.
:math:`\mathrm{ptype} = \texttt{'BINARY'}` or :math:`\texttt{'BIN'}`
The variables become binary so they will be able to hold only values 0 or 1. Note that any existing box bounds are preserved. For example, if variable :math:`x_{42}` has its bounds set as :math:`1.0\leq x_{42}\leq 5.3` and becomes binary, its box bounds are not removed and, therefore, the only allowed value will be :math:`x_{42} = 1.0`.
Class of properties referring to linearity.
:math:`\mathrm{ptype} = \texttt{'GENERIC'}` or :math:`\texttt{'GEN'}`
The variables have no special property. If they appear in at least one nonlinear expression (constraints or the objective), they will be assumed to be nonlinear (the default property).
:math:`\mathrm{ptype} = \texttt{'LINEAR'}` or :math:`\texttt{'LIN'}`
The variables will be assumed to be linear in all nonlinear constraints and the objective they might appear.
**idx** : int, array-like, shape :math:`\left(\textit{lidx}\right)`
The index set of variables whose properties will be modified. The elements may be supplied in any order.
.. _e04rc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`5`)
On entry, :math:`\textit{lidx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lidx} \geq 1`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{ptype} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ptype} = \texttt{'BIN'}`, :math:`\texttt{'CONT'}`, :math:`\texttt{'GEN'}`, :math:`\texttt{'INT'}` or :math:`\texttt{'LIN'}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idx}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idx}[i-1]\leq n`.
.. _e04rc-py2-py-notes:
**Notes**
After the model has been fully or partially built by calling functions of the NAG optimization modelling suite, ``handle_set_property`` may be used to change properties of a subset of the variables in the model.
The property is given by :math:`\mathrm{ptype}` which refers to the `integrality` of the variables (`continuous, integer` or `binary`) or their `linearity` (if the variables are `linear` or `generic`, see `Linearity of the Variables <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rcf.html#varlinearity>`__).
Both classes of properties can be set independently, however, only one property per class of properties can be chosen (i.e., a variable can be linear and integer but it cannot be continuous and binary at the same time).
The properties can be changed multiple times and the last setting is the one used. :meth:`handle_print` may be used to display the properties and further details of the problem.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
**Linearity of the Variables**
It is sometimes useful in Nonlinear Programming (NLP) problems to distinguish between `linear` and `nonlinear` variables and thus give the solvers an extra piece of information to utilize.
A variable is said to be `linear` if its involvement in the whole optimization model is linear, i.e., it appears only in linear expressions.
All other variables, if used at all, are `nonlinear`.
Let's assume that we have a problem with seven variables with two nonlinear constraints declared by :meth:`handle_set_nlnconstr` and defined as follows:
.. math::
g\left(x\right) = \begin{pmatrix} 3x_1 + e^{x_2} x_4 + 4x_4 - x_3 + x_5 \\ x_2 + x_3^2 - 3x_5 \end{pmatrix}
and with a linear objective function, linear constraints.
Variables :math:`x_6` and :math:`x_7` are not used at all in the definition of :math:`g\left(x\right)` and, therefore, they are clearly linear which is automatically detected by the solver.
Although :math:`x_1` and :math:`x_5` appear in the nonlinear constraints :math:`g\left(x\right)`, they are involved only in linear expressions and, therefore, are also linear.
These two cannot be detected by the solver because the solver can only see the sparsity pattern defined by :meth:`handle_set_nlnconstr` and a Black Box call-back.
In this case, it might be beneficial to set their property as `linear` using this function.
If no property is set, it is assumed that the variables are `generic`, meaning that they have no special property, and their linearity is auto detected based on the constraints they appear in.
Solvers which can exploit this property will describe it in their documentation.
See Also
--------
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ssqp_ex.main`
"""
raise NotImplementedError
[docs]def sdp_read_sdpa(infile, maxnvar, maxnblk, maxnnz, filelst=0, io_manager=None):
r"""
``sdp_read_sdpa`` reads in a linear semidefinite programming problem (SDP) from a file in sparse SDPA format and returns it in the form which is usable by functions :meth:`handle_init` (initialization), :meth:`handle_set_linobj` (linear objective function), :meth:`handle_set_linmatineq` (linear matrix constraints), :meth:`handle_solve_pennon` (solver) and :meth:`handle_free` (deallocation) from the NAG optimization modelling suite.
.. _e04rd-py2-py-doc:
For full information please refer to the NAG Library document for e04rd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rdf.html
.. _e04rd-py2-py-parameters:
**Parameters**
**infile** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) associated with the sparse SDPA data file. **Note:** that the file needs to be opened in read mode.
**maxnvar** : int
The upper limit for the number of variables in the problem. If :math:`\mathrm{maxnvar} = 0`, :math:`\mathrm{cvec}` and :math:`\mathrm{nnza}` will not be referenced.
**maxnblk** : int
The upper limit for the number of matrix constraints (i.e., the number of diagonal blocks within the matrix). If :math:`\mathrm{maxnblk} = 0`, :math:`\mathrm{blksizea}` will not be referenced.
**maxnnz** : int
The upper limit on the sum of nonzeros in all matrices :math:`A_{\textit{i}}^{\textit{k}}`, for :math:`\textit{k} = 1,2,\ldots,\mathrm{nblk}`, for :math:`\textit{i} = 0,1,\ldots,\mathrm{nvar}`. If :math:`\mathrm{maxnnz} = 0`, :math:`\mathrm{irowa}`, :math:`\mathrm{icola}` and :math:`\mathrm{a}` will not be referenced.
**filelst** : int, optional
If :math:`\mathrm{filelst} \neq 0`, a listing of the input data is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`). This can be useful for debugging the data file.
If :math:`\mathrm{filelst} = 0`, no listing is produced.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**nvar** : int
The actual number of the variables :math:`n`, matrix constraints :math:`m_A` and number of nonzeros of the problem in the file. This also indicates the exact memory needed in :math:`\mathrm{cvec}`, :math:`\mathrm{nnza}`, :math:`\mathrm{irowa}`, :math:`\mathrm{icola}`, :math:`\mathrm{a}` and :math:`\mathrm{blksizea}`.
**nblk** : int
The actual number of the variables :math:`n`, matrix constraints :math:`m_A` and number of nonzeros of the problem in the file. This also indicates the exact memory needed in :math:`\mathrm{cvec}`, :math:`\mathrm{nnza}`, :math:`\mathrm{irowa}`, :math:`\mathrm{icola}`, :math:`\mathrm{a}` and :math:`\mathrm{blksizea}`.
**nnz** : int
The actual number of the variables :math:`n`, matrix constraints :math:`m_A` and number of nonzeros of the problem in the file. This also indicates the exact memory needed in :math:`\mathrm{cvec}`, :math:`\mathrm{nnza}`, :math:`\mathrm{irowa}`, :math:`\mathrm{icola}`, :math:`\mathrm{a}` and :math:`\mathrm{blksizea}`.
**cvec** : float, ndarray, shape :math:`\left(\mathrm{maxnvar}\right)`
:math:`\mathrm{cvec}[\textit{i}-1]`, for :math:`\textit{i} = 1,2,\ldots,\mathrm{nvar}`, stores the dense vector :math:`c` of the linear objective function.
**nnza** : int, ndarray, shape :math:`\left(\mathrm{maxnvar}+1\right)`
:math:`\mathrm{nnza}[\textit{i}]`, for :math:`\textit{i} = 0,1,\ldots,\mathrm{nvar}`, stores the number of nonzero elements in matrices :math:`A_i`.
**irowa** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
:math:`\mathrm{irowa}`, :math:`\mathrm{icola}` and :math:`\mathrm{a}` store the nonzeros in the upper triangle of matrices :math:`A_{\textit{i}}`, for :math:`\textit{i} = 0,1,\ldots,\mathrm{nvar}`, in the coordinate storage, i.e., :math:`\mathrm{irowa}[\textit{j}-1]` are one-based row indices, :math:`\mathrm{icola}[\textit{j}-1]` are one-based column indices and :math:`\mathrm{a}[\textit{j}-1]` are the values of the nonzero elements, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nnz}`. See `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rdf.html#fcomments>`__.
**icola** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
:math:`\mathrm{irowa}`, :math:`\mathrm{icola}` and :math:`\mathrm{a}` store the nonzeros in the upper triangle of matrices :math:`A_{\textit{i}}`, for :math:`\textit{i} = 0,1,\ldots,\mathrm{nvar}`, in the coordinate storage, i.e., :math:`\mathrm{irowa}[\textit{j}-1]` are one-based row indices, :math:`\mathrm{icola}[\textit{j}-1]` are one-based column indices and :math:`\mathrm{a}[\textit{j}-1]` are the values of the nonzero elements, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nnz}`. See `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rdf.html#fcomments>`__.
**a** : float, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
:math:`\mathrm{irowa}`, :math:`\mathrm{icola}` and :math:`\mathrm{a}` store the nonzeros in the upper triangle of matrices :math:`A_{\textit{i}}`, for :math:`\textit{i} = 0,1,\ldots,\mathrm{nvar}`, in the coordinate storage, i.e., :math:`\mathrm{irowa}[\textit{j}-1]` are one-based row indices, :math:`\mathrm{icola}[\textit{j}-1]` are one-based column indices and :math:`\mathrm{a}[\textit{j}-1]` are the values of the nonzero elements, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nnz}`. See `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rdf.html#fcomments>`__.
**blksizea** : int, ndarray, shape :math:`\left(\mathrm{maxnblk}\right)`
:math:`\mathrm{blksizea}[\textit{k}-1]`, for :math:`\textit{k} = 1,2,\ldots,\mathrm{nblk}`, stores the sizes of the diagonal blocks in matrices :math:`A_i` from the top to the bottom.
.. _e04rd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`2`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` to :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not recognized as a valid integer.
(`errno` :math:`3`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` to :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not recognized as a valid real number.
(`errno` :math:`4`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` starting at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` was too long and was not recognized.
(`errno` :math:`5`)
An invalid number of variables was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be at least :math:`1`.
(`errno` :math:`6`)
An invalid number of blocks was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be at least :math:`1`.
(`errno` :math:`7`)
An invalid size of the block number :math:`\langle\mathit{\boldsymbol{value}}\rangle` was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be nonzero.
(`errno` :math:`8`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying block sizes.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying the objective function.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`10`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying nonzero matrix elements.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`11`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given matrix number is out of bounds. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`12`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given block number is out of bounds. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`13`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given row index is out of bounds, it must respect the size of the block. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`14`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given column index is out of bounds, it must respect the size of the block. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`15`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The specified nonzero element is not in the upper triangle.
The row index is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`16`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The specified element belongs to a diagonal block but is not diagonal.
The row index is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`17`)
An entry in the constraints with :math:`\boldsymbol{matno} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\boldsymbol{blkno} = \langle\mathit{\boldsymbol{value}}\rangle`, row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle` was defined more than once. All entries need to be unique.
(`errno` :math:`18`)
A premature end of the input stream. The part defining the number of variables was not found.
(`errno` :math:`18`)
A premature end of the input stream. The part defining the number of blocks was not found.
(`errno` :math:`18`)
A premature end of the input stream. The part defining the dimensions of the blocks was not found.
(`errno` :math:`18`)
A premature end of the input stream. The part defining the objective function was not found.
(`errno` :math:`18`)
A premature end of the input stream. The part defining the nonzero entries was not found.
(`errno` :math:`19`)
The input stream seems to be empty. No data was read.
(`errno` :math:`20`)
Reading from the stream caused an unknown error on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`21`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{infile}\geq 0`.
(`errno` :math:`21`)
On entry, :math:`\mathrm{maxnvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxnvar}\geq 0`.
(`errno` :math:`21`)
On entry, :math:`\mathrm{maxnblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxnblk}\geq 0`.
(`errno` :math:`21`)
On entry, :math:`\mathrm{maxnnz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxnnz}\geq 0`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
At least one of :math:`\mathrm{maxnvar}`, :math:`\mathrm{maxnblk}` or :math:`\mathrm{maxnnz}` is too small.
:math:`\mathrm{maxnvar}` should be at least :math:`\langle\mathit{\boldsymbol{value}}\rangle`, was :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\mathrm{maxnblk}` should be at least :math:`\langle\mathit{\boldsymbol{value}}\rangle`, was :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\mathrm{maxnnz}` should be at least :math:`\langle\mathit{\boldsymbol{value}}\rangle`, was :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04rd-py2-py-notes:
**Notes**
``sdp_read_sdpa`` is capable of reading linear semidefinite programming problems (SDP) from a text file in sparse SDPA format.
The problem is captured and returned in the following form:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i-A_0} ⪰ 0 \text{,} &\quad \text{ (b)}\end{array}
where :math:`A_i` denotes symmetric matrices and :math:`c` is a vector.
The expression :math:`S⪰0` stands for a constraint on the eigenvalues of a symmetric matrix :math:`S`, namely, all the eigenvalues should be non-negative, i.e., the matrix :math:`S` should be positive semidefinite.
Please note that this form covers even general linear SDP formulations with multiple linear matrix inequalities and a set of standard linear constraints.
A set of :math:`m_A` linear matrix inequalities
.. math::
\sum_{1}^{n}{x_iA_i^k-A_0^k}⪰0\text{, }\quad k = 1,\ldots,m_A
can be equivalently expressed as one matrix inequality `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rdf.html#eqn3>`__\ (b) in the following block diagonal form where the matrices :math:`A_i^1,A_i^2,\ldots,A_i^{m_A}` create the diagonal blocks of :math:`A_i`:
.. math::
\sum_{1}^{n}{x_i}\begin{pmatrix}A_i^1&&&\\&A_i^2&&\\&&⋱&\\&&&A_i^{m_A}\end{pmatrix}-\begin{pmatrix}A_0^1&&&\\&A_0^2&&\\&&⋱&\\&&&A_0^{m_A}\end{pmatrix}⪰0\text{.}
In addition, notice that if all matrices :math:`A_i^k` belonging to the same block, say block :math:`k`, are themselves diagonal matrices (or have dimension :math:`1\times 1`), the associated matrix inequality
.. math::
\sum_{1}^{n}{x_iA_i^k-A_0^k}⪰0
actually defines a standard linear constraint
.. math::
Bx\geq l
where :math:`l` and columns of the matrix :math:`B` are formed by the diagonals of matrices :math:`A_0^k` and :math:`A_1^k,\ldots,A_n^k`, respectively.
Precisely, :math:`l_i = \left(A_0^k\right)_{{ii}}` and :math:`b_{{ij}} = \left(A_j^k\right)_{{ii}}`.
Alternatively, :meth:`handle_read_file` may be used to read the data file directly into a handle of the NAG optimization modelling suite.
**Sparse SDPA file format**
The problem data is written in an ASCII input file in the SDPA sparse format which was first introduced in Fujisawa `et al.` (1998).
In the description below we follow closely the specification from Borchers (1999).
The format is line oriented.
If more elements are required on the line they need to be separated by a space, a tab or any of the special characters ',', '(', ')', '{' or '}'.
The file consists of six sections:
(1) Comments. The file can begin with an arbitrary number of lines of comment. Each line of comment must begin with ''' or '*'.
(#) The first line after the comments contains integer :math:`n`, the number of variables. The rest of this line is ignored.
(#) The second line after the comments contains integer :math:`m_A`, the number of blocks in the block diagonal structure of the matrices. Additional text on this line after :math:`m_A` is ignored.
(#) The third line after the comments contains a vector of :math:`m_A` integers that give the sizes of the individual blocks. Negative numbers may be used to indicate that a block is actually a diagonal submatrix. Thus a block size of ':math:`-5`' indicates a :math:`5\times 5` block in which only the diagonal elements are nonzero.
(#) The fourth line after the comments contains an :math:`n`-dimensional real vector defining the objective function vector :math:`c`.
(#) The remaining lines of the file contain nonzero entries in the constraint matrices, with one entry per line. The format for each line is
.. math::
\boldsymbol{matno}\quad \text{ }\quad \boldsymbol{blkno}\quad \text{ }\quad \boldsymbol{i}\quad \text{ }\quad \boldsymbol{j}\quad \text{ }\quad \boldsymbol{entry}
where :math:`\boldsymbol{matno}` is the number :math:`\left(0, \ldots, n\right)` of the matrix to which this entry belongs and :math:`\boldsymbol{blkno}` specifies the block number :math:`k = 1,2,\ldots,m_A` within this matrix. Together, they uniquely identify the block :math:`A_{{\boldsymbol{matno}}}^{{\boldsymbol{blkno}}}`. Integers :math:`\boldsymbol{i}` and :math:`\boldsymbol{j}` are one-based indices which specify a location of the entry within the block. Note that since all matrices are assumed to be symmetric, only entries in the upper triangle of a matrix should be supplied. Finally, :math:`\boldsymbol{entry}` should give the real value of the entry in the matrix. Precisely, :math:`\left(A_{\boldsymbol{matno}}^{\boldsymbol{blkno}}\right)_{{\boldsymbol{i}\boldsymbol{j}}} = \left(A_{\boldsymbol{matno}}^{\boldsymbol{blkno}}\right)_{{\boldsymbol{j}\boldsymbol{i}}} = \boldsymbol{entry}`.
In the text below and in the file listing (:math:`\mathrm{filelst}`) we use the word 'token' as a reference to a group of contiguous characters without a space or any other delimeters.
**Recommendation on how best to use sdp_read_sdpa**
(a) The input file with the problem needs to be opened for reading by the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj` (:math:`{\textit{mode}} = 0`). In this way we avoid possible limitations of maximal lengths of lines inherited by Fortran I/O (the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj` uses the formatted stream access mode to bypass the restriction). If the file is opened by other means or standard input is used instead, lines within the file might be truncated which would produce a file format error message. This would most likely happen on the line defining the objective function. Setting :math:`\mathrm{filelst} = 1` might help with possible file formatting errors.
(#) Unless the dimension of the problem (or its overestimate) is known in advance, initially call ``sdp_read_sdpa`` with :math:`\mathrm{maxnvar} = 0`, :math:`\mathrm{maxnblk} = 0` and :math:`\mathrm{maxnnz} = 0`. In this case, the exact size of the problem is computed and returned in :math:`\mathrm{nvar}`, :math:`\mathrm{nblk}` and :math:`\mathrm{nnz}`. No other data will be stored and the arrays :math:`\mathrm{cvec}`, :math:`\mathrm{nnza}`, :math:`\mathrm{irowa}`, :math:`\mathrm{icola}`, :math:`\mathrm{a}`, :math:`\mathrm{blksizea}` will not be referenced. Then the exact storage can be allocated and the file reopened. When ``sdp_read_sdpa`` is called for the second time, the problem is read in and stored in appropriate arrays.
(#) A typical sequence of calls to solve the problem read in by ``sdp_read_sdpa`` might be as follows. First, an empty handle needs to be initialized by :meth:`handle_init` with :math:`\textit{nvar}` variables. This should be followed by calls to :meth:`handle_set_linobj` and :meth:`handle_set_linmatineq` to formulate the objective function and the constraints, respectively. The arguments of both functions use the same naming and storage as in ``sdp_read_sdpa`` so the variables can be passed unchanged; only :math:`\textit{dima}` in :meth:`handle_set_linmatineq` is new and should be equal to :math:`\mathrm{SUM}\left(\mathrm{blksizea}[0:\mathrm{nblk}]\right)`. :math:`\textit{nnzasum}` in :meth:`handle_set_linmatineq` is the same as :math:`\mathrm{nnz}` in ``sdp_read_sdpa``. You may at this point want to modify option settings using :meth:`handle_opt_set`. If dual variables (Lagrangian multipliers) are required from the solver, sufficient space needs to be allocated. The size is equal to the sum of the number of elements of dense triangular matrices for each block. For further details, see the argument :math:`\textit{ua}` in the solver :meth:`handle_solve_pennon`. The solver should be called and then followed, finally, by a call to :meth:`handle_free` to deallocate memory associated with the problem.
.. _e04rd-py2-py-references:
**References**
Borchers, B, 1999, `SDPLIB 1.2, A Library of semidefinite programming test problems`, Optimization Methods and Software (11(1)), 683--690, http://euler.nmt.edu/~brian/sdplib/
Fujisawa, K, Kojima, M and Nakata, K, 1998, `SDPA (Semidefinite Programming Algorithm) User's Manual`, Technical Report B-308, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.
"""
raise NotImplementedError
[docs]def handle_set_linobj(handle, cvec):
r"""
``handle_set_linobj`` is a part of the NAG optimization modelling suite and defines or redefines the objective function of the problem to be linear.
.. _e04re-py2-py-doc:
For full information please refer to the NAG Library document for e04re
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ref.html
.. _e04re-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**cvec** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
The dense vector :math:`c` of the objective function.
.. _e04re-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
.. _e04re-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_linobj`` may be used to define the objective function of the problem as a linear function :math:`c^\mathrm{T}x` using a dense vector :math:`c`.
If the objective function has already been defined, it will be overwritten.
If :math:`c = 0`, any existing objective function is removed, no new one is added and the problem will be solved as a feasible point problem. It is recommended to use :meth:`handle_set_quadobj` if the vector :math:`c` is sparse. :meth:`handle_set_linobj_coeff` may be used to set individual elements :math:`c_i` of the linear objective.
This will typically be used for Linear Programming (LP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
Second-order Cone Programming (SOCP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to} &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\\&x_{G^i} \in \mathcal{K}^{m_i},i = 1,\ldots,r\text{,}&\quad \text{ (d)}\end{array}
linear Semidefinite Programming problems (SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\& l_B\leq Bx\leq u_B \text{,} &\quad \text{ (c)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (d)}\end{array}
or SDP with bilinear matrix inequalities (BMI-SDP) where the objective function has only linear terms.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_bmi_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_socp_ipm_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_quadobj(handle, idxc=None, c=None, irowh=None, icolh=None, h=None):
r"""
``handle_set_quadobj`` is a part of the NAG optimization modelling suite and defines or redefines the objective function of the problem to be linear or quadratic.
.. _e04rf-py2-py-doc:
For full information please refer to the NAG Library document for e04rf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rff.html
.. _e04rf-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**idxc** : None or int, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of the sparse vector :math:`c`. :math:`\mathrm{idxc}[i-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`. The elements must be stored in ascending order. Note that :math:`n` is the current number of variables in the model.
**c** : None or float, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of the sparse vector :math:`c`. :math:`\mathrm{idxc}[i-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`. The elements must be stored in ascending order. Note that :math:`n` is the current number of variables in the model.
**irowh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
Arrays :math:`\mathrm{irowh}`, :math:`\mathrm{icolh}` and :math:`\mathrm{h}` store the nonzeros of the upper triangle of the matrix :math:`H` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowh}` specifies one-based row indices, :math:`\mathrm{icolh}` specifies one-based column indices and :math:`\mathrm{h}` specifies the values of the nonzero elements in such a way that :math:`h_{{ij}} = \mathrm{h}[l-1]` where :math:`i = \mathrm{irowh}[l-1]`, :math:`j = \mathrm{icolh}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzh}`. No particular order is expected, but elements should not repeat.
**icolh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
Arrays :math:`\mathrm{irowh}`, :math:`\mathrm{icolh}` and :math:`\mathrm{h}` store the nonzeros of the upper triangle of the matrix :math:`H` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowh}` specifies one-based row indices, :math:`\mathrm{icolh}` specifies one-based column indices and :math:`\mathrm{h}` specifies the values of the nonzero elements in such a way that :math:`h_{{ij}} = \mathrm{h}[l-1]` where :math:`i = \mathrm{irowh}[l-1]`, :math:`j = \mathrm{icolh}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzh}`. No particular order is expected, but elements should not repeat.
**h** : None or float, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
Arrays :math:`\mathrm{irowh}`, :math:`\mathrm{icolh}` and :math:`\mathrm{h}` store the nonzeros of the upper triangle of the matrix :math:`H` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowh}` specifies one-based row indices, :math:`\mathrm{icolh}` specifies one-based column indices and :math:`\mathrm{h}` specifies the values of the nonzero elements in such a way that :math:`h_{{ij}} = \mathrm{h}[l-1]` where :math:`i = \mathrm{irowh}[l-1]`, :math:`j = \mathrm{icolh}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzh}`. No particular order is expected, but elements should not repeat.
.. _e04rf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzh}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzc}\geq 0`.
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idxc}[i] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idxc}[\textit{i}-1] < \mathrm{idxc}[i]` (ascending order).
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxc}[i-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowh}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolh}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icolh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{irowh}[\textit{i}-1]\leq \mathrm{icolh}[\textit{i}-1]` (elements within the upper triangle).
(`errno` :math:`8`)
On entry, more than one element of :math:`\mathrm{h}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{h}` must have a unique row and column index.
.. _e04rf-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_quadobj`` may be used to define the objective function of the problem as a quadratic function :math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx` or a sparse linear function :math:`c^\mathrm{T}x`.
If the objective function has already been defined, it will be overwritten.
If ``handle_set_quadobj`` is called with no nonzeroes in either :math:`c` or :math:`H`, any existing objective function is removed, no new one is added and the problem will be solved as a feasible point problem. :meth:`handle_set_linobj_coeff` may be used to set individual elements :math:`c_i` of the linear objective.
This objective function will typically be used for
Linear Programming (LP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
Quadratic Programming problems (QP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\\text{subject to}&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\&l_x\leq x\leq u_x\text{,}&\quad \text{ (c)}\end{array}
or for Semidefinite Programming problems with bilinear matrix inequalities (BMI-SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_ix_jQ_{{ij}}^k} + \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\&l_x\leq x\leq u_x\text{.}&\quad \text{ (d)}\end{array}
The matrix :math:`H` is a sparse symmetric :math:`n\times n` matrix.
It does not need to be positive definite.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_lmi_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_nlnobj(handle, idxfd=None):
r"""
``handle_set_nlnobj`` is a part of the NAG optimization modelling suite and declares the objective function of the problem as a nonlinear function with a particular gradient sparsity structure.
.. _e04rg-py2-py-doc:
For full information please refer to the NAG Library document for e04rg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rgf.html
.. _e04rg-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**idxfd** : None or int, array-like, shape :math:`\left(\textit{nnzfd}\right)`, optional
The one-based indices of the nonzero elements of the sparse gradient vector. The indices must be stored in ascending order. Note that :math:`n` is the current number of decision variables in the model.
If :math:`\textit{nnzfd} = 0`, the objective is assumed to be zero and the array :math:`\mathrm{idxfd}` will not be referenced.
.. _e04rg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzfd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzfd}\geq 0`.
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxfd}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idxfd}[i] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idxfd}[\textit{i}-1] < \mathrm{idxfd}[i]` (ascending order).
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxfd}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxfd}[i-1]\leq \textit{n}`.
.. _e04rg-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_nlnobj`` may be used to declare the objective function of the problem as a nonlinear function and define the sparsity pattern (list of nonzero elements) of its gradient.
If the objective function has already been defined, it will be overwritten and its Hessian (or the Hessian of the Lagrangian) will be removed.
If ``handle_set_nlnobj`` is called with no nonzeroes in the sparsity pattern, any existing objective function is removed, no new one is added and the problem will be solved as a feasible point problem.
This objective function will typically be used for nonlinear programming problems (NLP) of the kind:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\quad \text{ (a)}\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{.}&\quad \text{ (c)}\\& l_x\leq x\leq u_x &\quad \text{ (d)}\end{array}
The values of the nonlinear objective function :math:`f\left(x\right)` and the nonzero values of its gradient :math:`\frac{{\partial f}}{{\partial x_i}}` (matching the sparsity pattern) evaluated at particular points in the decision variable space will be communicated to the NLP solver by user-supplied functions (e.g., :math:`\textit{objfun}` and :math:`\textit{objgrd}` for :meth:`handle_solve_ipopt`).
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.glopt.handle_solve_mcs_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ssqp_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_simplebounds(handle, bl, bu):
r"""
``handle_set_simplebounds`` is a part of the NAG optimization modelling suite and sets bounds on the variables of the problem.
.. _e04rh-py2-py-doc:
For full information please refer to the NAG Library document for e04rh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rhf.html
.. _e04rh-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**bl** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`\mathrm{bl}` defines the lower bounds on the variables
**bu** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`\mathrm{bu}` defines the upper bounds on the variables
.. _e04rh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1]\leq \mathrm{bu}[j-1]`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1] < \textit{bigbnd}`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bu}[j-1] > -\textit{bigbnd}`.
.. _e04rh-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_simplebounds`` may be used to define the variable bounds :math:`l_x\leq x\leq u_x` of the problem.
If the bounds have already been defined, they will be overwritten.
Individual bounds may also be set by :meth:`handle_set_bound`.
This will typically be used for problems, such as:
Linear Programming (LP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
Quadratic Programming (QP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\\text{subject to}&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (c)}\end{array}
Nonlinear Programming (NLP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\quad \text{ (a)}\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (d)}\end{array}
or linear Semidefinite Programming (SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\& l_B\leq Bx\leq u_B\text{,} &\quad \text{ (c)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (d)}\end{array}
where :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors.
Note that upper and lower bounds are specified for all the variables.
This form allows full generality in specifying various types of constraint.
In particular, the :math:`j`\ th variable may be fixed by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements of :math:`l_x` or :math:`u_x` may be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
See the description of the option 'Infinite Bound Size' which is common among all solvers in the suite.
Its value is denoted as :math:`\textit{bigbnd}` further in this text.
Note that the bounds are interpreted based on its value at the time of calling this function and any later alterations to 'Infinite Bound Size' will not affect these constraints.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
.. _e04rh-py2-py-references:
**References**
Candes, E and Recht, B, 2009, `Exact matrix completion via convex optimization`, Foundations of Computation Mathematics (Volume 9), 717--772
See Also
--------
:meth:`naginterfaces.library.examples.glopt.handle_solve_mcs_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
(and others)
"""
raise NotImplementedError
[docs]def handle_set_linconstr(handle, bl=None, bu=None, irowb=None, icolb=None, b=None, idlc=0):
r"""
``handle_set_linconstr`` is a part of the NAG optimization modelling suite and adds a new block of linear constraints to the problem or modifies an individual linear constraint.
.. _e04rj-py2-py-doc:
For full information please refer to the NAG Library document for e04rj
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rjf.html
.. _e04rj-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**bl** : None or float, array-like, shape :math:`\left(\textit{nclin}\right)`, optional
:math:`\mathrm{bl}` defines the lower bounds of the linear constraints, :math:`l_B`
**bu** : None or float, array-like, shape :math:`\left(\textit{nclin}\right)`, optional
:math:`\mathrm{bu}` defines the upper bounds of the linear constraints, :math:`u_B`
**irowb** : None or int, array-like, shape :math:`\left(\textit{nnzb}\right)`, optional
:math:`\mathrm{irowb}` specifies one-based row indices of the :math:`\textit{nnzb}` nonzeros of the sparse matrix :math:`B` in coordinate storage (CS) format.
**icolb** : None or int, array-like, shape :math:`\left(\textit{nnzb}\right)`, optional
:math:`\mathrm{icolb}` specifies one-based column indices of the :math:`\textit{nnzb}` nonzeros of the sparse matrix :math:`B` in coordinate storage (CS) format.
**b** : None or float, array-like, shape :math:`\left(\textit{nnzb}\right)`, optional
:math:`\mathrm{b}` stores the :math:`\textit{nnzb}` nonzeros of the sparse matrix :math:`B` in coordinate storage (CS) format.
**idlc** : int, optional
If :math:`\mathrm{idlc} = 0`, a new block of linear constraints is added to the model; otherwise, :math:`\mathrm{idlc} > 0` refers to the number of an existing linear constraint which will be replaced and :math:`\textit{nclin}` must be set to one.
**Returns**
**idlc** : int
If :math:`\mathrm{idlc} = 0`, the number of the last linear constraint added. By definition, it is the number of linear constraints already defined plus :math:`\textit{nclin}`. Otherwise, :math:`\mathrm{idlc} > 0` stays unchanged.
.. _e04rj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idlc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idlc}\geq 0`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idlc} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idlc}` does not match with any existing linear constraint.
The maximum :math:`\mathrm{idlc}` is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzb} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzb}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{idlc} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: If :math:`\mathrm{idlc} > 0`, :math:`\textit{nclin} = 1`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowb}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowb}[\textit{i}-1]\leq \textit{nclin}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolb}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolb}[\textit{i}-1]\leq n`.
(`errno` :math:`8`)
On entry, more than one element of :math:`\mathrm{b}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{b}` must have a unique row and column index.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1]\leq \mathrm{bu}[j-1]`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1] < \textit{bigbnd}`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bu}[j-1] > -\textit{bigbnd}`.
.. _e04rj-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_linconstr`` may be used to add to the problem a new block of :math:`m_B` linear constraints
.. math::
l_B\leq Bx\leq u_B
where :math:`B` is a general :math:`m_B\times n` rectangular matrix, :math:`n` is the current number of decision variables in the model and :math:`l_B` and :math:`u_B` are :math:`m_B`-dimensional vectors defining the lower and upper bounds, respectively.
The call can be repeated to add multiple blocks to the model.
Note that the bounds are specified for all the constraints of this block.
This form allows full generality in specifying various types of constraint.
In particular, the :math:`j`\ th constraint may be defined as an equality by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements of :math:`l_B` or :math:`u_B` may be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
See the description of the option 'Infinite Bound Size' which is common among all solvers in the suite.
Its value is denoted as :math:`\textit{bigbnd}` further in this text.
Note that the bounds are interpreted based on its value at the time of calling this function and any later alterations to 'Infinite Bound Size' will not affect these constraints.
The linear constraints can be edited.
To identify the individual constraints, they are numbered starting with :math:`1`, see :math:`\mathrm{idlc}`.
A single constraint (i.e., a single row of the matrix :math:`B`) can be modified (replaced) by ``handle_set_linconstr`` by referring to its :math:`\mathrm{idlc}`.
An individual coefficient :math:`b_{\textit{ij}}` of the matrix :math:`B` can be set or modified by :meth:`handle_set_linconstr_coeff` and bounds of a single constraint can be set or modified by :meth:`handle_set_bound`.
Note that it is also possible to temporarily disable and enable individual constraints in the model by :meth:`handle_disable` and :meth:`handle_enable`, respectively.
Linear constraints may be present in many different types of problems, for simplicity of the notation, only one block of linear constraints is presented.
For example,
Linear Programming (LP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
Quadratic Programming (QP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
Quadratically Constrained Quadratic Programming (QCQP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \frac{1}{2} x^\mathrm{T}Q_kx + r_k^\mathrm{T}x + s_k\leq 0 \text{, }\quad k = 1,\ldots,m_Q \text{,} &\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (d)}\end{array}
Nonlinear Programming (NLP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\quad \text{ (a)}\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\quad \text{ (b)}\\& \frac{1}{2} x^\mathrm{T}Q_kx + r_k^\mathrm{T}x + s_k\leq 0 \text{, }\quad k = 1,\ldots,m_Q \text{,} &\quad \text{ (c)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (d)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (e)}\end{array}
or linear Semidefinite Programming (SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\& l_B\leq Bx\leq u_B\text{,} &\quad \text{ (c)}\\& l_x\leq x\leq u_x\text{.} &\quad \text{ (d)}\end{array}
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_nldf_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_socp_ipm_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ssqp_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_nlnconstr(handle, bl=None, bu=None, irowgd=None, icolgd=None):
r"""
``handle_set_nlnconstr`` is a part of the NAG optimization modelling suite and defines or overwrites nonlinear constraints in the model, their number as well as the sparsity structure of their first derivatives.
.. _e04rk-py2-py-doc:
For full information please refer to the NAG Library document for e04rk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rkf.html
.. _e04rk-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**bl** : None or float, array-like, shape :math:`\left(\textit{ncnln}\right)`, optional
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` define lower and upper bounds of the nonlinear constraints, :math:`l_g` and :math:`u_g`, respectively. To define the :math:`j`\ th constraint as equality, set :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1] = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. To specify a nonexistent lower bound (i.e., :math:`l_j = {-\infty }`), set :math:`\mathrm{bl}[j-1]\leq -\textit{bigbnd}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq \textit{bigbnd}`.
**bu** : None or float, array-like, shape :math:`\left(\textit{ncnln}\right)`, optional
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` define lower and upper bounds of the nonlinear constraints, :math:`l_g` and :math:`u_g`, respectively. To define the :math:`j`\ th constraint as equality, set :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1] = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. To specify a nonexistent lower bound (i.e., :math:`l_j = {-\infty }`), set :math:`\mathrm{bl}[j-1]\leq -\textit{bigbnd}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq \textit{bigbnd}`.
**irowgd** : None or int, array-like, shape :math:`\left(\textit{nnzgd}\right)`, optional
Arrays :math:`\mathrm{irowgd}` and :math:`\mathrm{icolgd}` store the sparsity structure (pattern) of the Jacobian matrix as :math:`\textit{nnzgd}` nonzeros in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). The matrix has dimensions :math:`\textit{ncnln}\times n`. :math:`\mathrm{irowgd}` specifies one-based row indices and :math:`\mathrm{icolgd}` specifies one-based column indices. No particular order of elements is expected, but elements should not repeat and the same order should be used when the Jacobian is evaluated for the solver, e.g., the value of :math:`\frac{{\partial g_i}}{{\partial x_j}}` where :math:`i = \mathrm{irowgd}[l-1]` and :math:`j = \mathrm{icolgd}[\textit{l}-1]` should be stored in :math:`{\textit{gdx}}[\textit{l}-1]` in :math:`\textit{congrd}` in :meth:`handle_solve_ipopt`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzgd}`.
**icolgd** : None or int, array-like, shape :math:`\left(\textit{nnzgd}\right)`, optional
Arrays :math:`\mathrm{irowgd}` and :math:`\mathrm{icolgd}` store the sparsity structure (pattern) of the Jacobian matrix as :math:`\textit{nnzgd}` nonzeros in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). The matrix has dimensions :math:`\textit{ncnln}\times n`. :math:`\mathrm{irowgd}` specifies one-based row indices and :math:`\mathrm{icolgd}` specifies one-based column indices. No particular order of elements is expected, but elements should not repeat and the same order should be used when the Jacobian is evaluated for the solver, e.g., the value of :math:`\frac{{\partial g_i}}{{\partial x_j}}` where :math:`i = \mathrm{irowgd}[l-1]` and :math:`j = \mathrm{icolgd}[\textit{l}-1]` should be stored in :math:`{\textit{gdx}}[\textit{l}-1]` in :math:`\textit{congrd}` in :meth:`handle_solve_ipopt`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzgd}`.
.. _e04rk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzgd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzgd} > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{ncnln}\geq 0`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowgd}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowgd}[\textit{i}-1]\leq \textit{ncnln}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolgd}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolgd}[\textit{i}-1]\leq n`.
(`errno` :math:`8`)
On entry, more than one element of structural Jacobian matrix has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of structural Jacobian matrix must have a unique row and column index.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1]\leq \mathrm{bu}[j-1]`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[j-1] < \textit{bigbnd}`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bu}[j-1] > -\textit{bigbnd}`.
.. _e04rk-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_nlnconstr`` may be used to define the nonlinear constraints :math:`l_g\leq g\left(x\right)\leq u_g` of the problem.
If the nonlinear constraints have already been defined, they will be overwritten and their Hessians (or the Hessian of the Lagrangian) will be removed.
If ``handle_set_nlnconstr`` is called with :math:`m_g = 0`, existing nonlinear constraints are removed and no new ones are added.
This will typically be used for nonlinear programming problems (NLP) of the kind:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\quad \text{ (a)}\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (d)}\end{array}
where :math:`n` is the number of the decision variables :math:`x`, :math:`m_g` is the number of the nonlinear constraints (in `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rkf.html#e04raeqn2b>`__\ (b)) and :math:`g\left(x\right)`, :math:`l_g` and :math:`u_g` are :math:`m_g`-dimensional vectors.
Note that upper and lower bounds are specified for all the constraints.
This form allows full generality in specifying various types of constraint.
In particular, the :math:`j`\ th constraint may be defined as an equality by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements :math:`l_j` or :math:`u_j` may be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
See the description of the option 'Infinite Bound Size' which is common among all solvers in the suite.
Its value is denoted as :math:`\textit{bigbnd}` further in this text.
Note that the bounds are interpreted based on its value at the time of calling this function and any later alterations to 'Infinite Bound Size' will not affect these constraints.
Since each nonlinear constraint is most likely to involve a small subset of the decision variables, the partial derivatives of the constraint functions with respect to those variables are best expressed as a sparse Jacobian matrix of :math:`m_g` rows and :math:`n` columns.
The row and column positions of all the nonzero derivatives must be registered with the handle through ``handle_set_nlnconstr``.
The values of the nonlinear constraint functions and their nonzero gradients at particular points in the decision variable space will be communicated to the NLP solver by user-supplied functions (e.g., :math:`\textit{confun}` and :math:`\textit{congrd}` for :meth:`handle_solve_ipopt`).
An individual set of bounds can be also modified by :meth:`handle_set_bound`.
Note that it is also possible to temporarily disable and enable individual constraints in the model by calling :meth:`handle_disable` and :meth:`handle_enable`, respectively.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ssqp_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_nlnhess(handle, idf, irowh=None, icolh=None):
r"""
``handle_set_nlnhess`` is a part of the NAG optimization modelling suite and defines or redefines either the structure of the Hessians of the nonlinear objective and constraints or the structure of the Hessian of the Lagrangian.
.. _e04rl-py2-py-doc:
For full information please refer to the NAG Library document for e04rl
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rlf.html
.. _e04rl-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**idf** : int
Specifies the functions for which a Hessian sparsity structure is provided in :math:`\textit{nnzh}`, :math:`\mathrm{irowh}` and :math:`\mathrm{icolh}`.
:math:`\mathrm{idf} = -1`
The sparsity structure of the Hessian of the Lagrangian is provided.
:math:`\mathrm{idf} = 0`
The sparsity structure of the Hessian of the objective function is provided.
:math:`\mathrm{idf} > 0`
The sparsity structure of the Hessian of the :math:`\mathrm{idf}`\ th constraint function is provided.
The value of :math:`\mathrm{idf}` will also determine how an NLP solver will call the user-supplied functions that evaluate these nonzeros at particular points of the decision variable space, i.e., whether the solver will expect the nonzero values of the objective and constraint Hessians in separate calls or merged in the Lagrangian Hessian, in one call.
See, for example, :math:`\textit{hess}` of :meth:`handle_solve_ipopt`.
**irowh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{icolh}` store the nonzeros of the upper triangle of the matrix :math:`H` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowh}` specifies one-based row indices, :math:`\mathrm{icolh}` specifies one-based column indices and specifies the values of the nonzero elements in such a way that :math:`h_{{ij}} = \textit{H}[l-1]` where :math:`i = \mathrm{irowh}[l-1]` and :math:`j = \mathrm{icolh}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzh}`. No particular order is expected, but elements should not repeat.
**icolh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{icolh}` store the nonzeros of the upper triangle of the matrix :math:`H` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowh}` specifies one-based row indices, :math:`\mathrm{icolh}` specifies one-based column indices and specifies the values of the nonzero elements in such a way that :math:`h_{{ij}} = \textit{H}[l-1]` where :math:`i = \mathrm{irowh}[l-1]` and :math:`j = \mathrm{icolh}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzh}`. No particular order is expected, but elements should not repeat.
.. _e04rl-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`2`)
Neither nonlinear objective nor nonlinear constraints are present. The structure of the Hessian cannot be defined.
(`errno` :math:`2`)
No nonlinear objective has been defined, its Hessian cannot be set.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzh} > 0`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{idf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`-1\leq \mathrm{idf}\leq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowh}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolh}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icolh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{irowh}[\textit{i}-1]\leq \mathrm{icolh}[\textit{i}-1]` (elements within the upper triangle).
(`errno` :math:`8`)
On entry, more than one element of structural matrix :math:`H` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of structural matrix :math:`H` must have a unique row and column index.
.. _e04rl-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), and a nonlinear objective function :math:`f\left(x\right)` and/or the :math:`i`\ th (:math:`1\leq i\leq m_g`) nonlinear constraint function :math:`g_i\left(x\right)` has been registered with :meth:`handle_set_nlnconstr` and :meth:`handle_set_nlnobj`, then ``handle_set_nlnhess`` may be used to define the sparsity structure (pattern) of the Hessians of those functions or of their Lagrangian function.
Define:
:math:`\nabla^2f\left(x\right)\equiv \left(\begin{array}{cccc} \frac{{\partial^2f\left(x\right)}}{{\partial^2x_1}} & \frac{{\partial^2f\left(x\right)}}{{\partial x_2\partial x_1}} &\ldots & \frac{{\partial^2f\left(x\right)}}{{\partial x_n\partial x_1}} \\ \frac{{\partial^2f\left(x\right)}}{{\partial x_1\partial x_2}} & \frac{{\partial^2f\left(x\right)}}{{\partial^2x_2}}&\ldots & \frac{{\partial^2f\left(x\right)}}{{\partial x_n\partial x_2}} \\ \vdots & \vdots &⋱& \vdots \\ \frac{{\partial^2f\left(x\right)}}{{\partial x_1\partial x_n}} & \frac{{\partial^2f\left(x\right)}}{{\partial x_2\partial x_n}} &\ldots & \frac{{\partial^2f\left(x\right)}}{{\partial^2x_n}} \end{array}\right)`, and :math:`\nabla^2g_i\left(x\right)\equiv \left(\begin{array}{cccc} \frac{{\partial^2g_i\left(x\right)}}{{\partial^2x_1}} & \frac{{\partial^2g_i\left(x\right)}}{{\partial x_2\partial x_1}} &\ldots & \frac{{\partial^2g_i\left(x\right)}}{{\partial x_n\partial x_1}} \\ \frac{{\partial^2g_i\left(x\right)}}{{\partial x_1\partial x_2}} & \frac{{\partial^2g_i\left(x\right)}}{{\partial^2x_2}}&\ldots & \frac{{\partial^2g_i\left(x\right)}}{{\partial x_n\partial x_2}} \\ \vdots & \vdots &⋱& \vdots \\ \frac{{\partial^2g_i\left(x\right)}}{{\partial x_1\partial x_n}} & \frac{{\partial^2g_i\left(x\right)}}{{\partial x_2\partial x_n}} &\ldots & \frac{{\partial^2g_i\left(x\right)}}{{\partial^2x_n}} \end{array}\right)` for :math:`1\leq i\leq m_g`
``handle_set_nlnhess`` can be used to define the following sparsity structures:
the Hessian of the Lagrangian function :math:`\sigma \nabla^2f\left(x\right)+\sum_{{i = 1}}^{m_g}\lambda_i\nabla^2g_i\left(x\right)`,
the Hessian of the objective function :math:`\nabla^2f\left(x\right)`, or
the Hessian of the :math:`i`\ th constraint function :math:`\nabla^2g_i\left(x\right)` with :math:`1\leq i\leq m_g`.
In general, each of the symmetric :math:`n\times n` Hessian matrices will have its own sparsity structure.
These structures can be given in separate ``handle_set_nlnhess`` calls, or merged together in the Lagrangian and given in one call.
The nonzero values of the Hessians at particular points will be communicated to the NLP solver by user-supplied functions (e.g., :math:`\textit{hess}` for :meth:`handle_solve_ipopt`).
The values will need to be provided in the order matching the sparsity pattern.
Note that the Hessians are automatically deleted whenever the underlying functions change.
For example, if :meth:`handle_set_nlnconstr` is called to redefine the nonlinear constraints, all individual constraints Hessians or Hessian of the Lagrangian would be deleted.
If a nonlinear objective function was changed to linear, the Hessian of the objective function or of the Lagrangian would be deleted. ``handle_set_nlnhess`` can work either with individual Hessians or with the Hessian of the Lagrangian but not both.
Therefore, if the Hessian of the Lagrangian was defined and ``handle_set_nlnhess`` was called to define an individual Hessian of the constraint, the Hessian of the Lagrangian would be removed, and vice versa.
Hessians can be redefined by multiple calls of ``handle_set_nlnhess``.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_nlnls(handle, nres, irowrd=None, icolrd=None):
r"""
``handle_set_nlnls`` is a part of the NAG optimization modelling suite and defines the residual functions for nonlinear regression problems (such as nonlinear least squares and general nonlinear data fitting) with the given number of residuals and, optionally, the sparsity structure of their first derivatives.
.. _e04rm-py2-py-doc:
For full information please refer to the NAG Library document for e04rm
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rmf.html
.. _e04rm-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**nres** : int
:math:`m_r`, the number of residuals in the objective function.
If :math:`\mathrm{nres} = 0`, no objective function will be defined and :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` will not be referenced.
**irowrd** : None or int, array-like, shape :math:`\left(\textit{nnzrd}\right)`, optional
Arrays :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` store the sparsity structure (pattern) of the first derivative matrix as :math:`\textit{nnzrd}` nonzeros in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). The matrix has dimensions :math:`n\times m_r`. :math:`\mathrm{irowrd}` specifies one-based row indices and :math:`\mathrm{icolrd}` specifies one-based column indices. No particular order of elements is expected, but elements should not repeat and the same order should be used when the first derivative matrix is evaluated for the solver.
If :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` are both **None** then the first derivative matrix is considered dense and :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` will not be referenced.
The ordering is assumed to be column-wise, namely the function will behave as if :math:`\textit{nnzrd} = n\times m_r` and the vectors :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` filled as:
:math:`\mathrm{irowrd} = \left(1, 2, …, n, 1, 2, …, n, …, 1, 2, …, n\right)`;
:math:`\mathrm{icolrd} = \left(1, 1, …, 1, 2, 2, …, 2, …, m_r, m_r, …, m_r\right)`.
**icolrd** : None or int, array-like, shape :math:`\left(\textit{nnzrd}\right)`, optional
Arrays :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` store the sparsity structure (pattern) of the first derivative matrix as :math:`\textit{nnzrd}` nonzeros in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). The matrix has dimensions :math:`n\times m_r`. :math:`\mathrm{irowrd}` specifies one-based row indices and :math:`\mathrm{icolrd}` specifies one-based column indices. No particular order of elements is expected, but elements should not repeat and the same order should be used when the first derivative matrix is evaluated for the solver.
If :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` are both **None** then the first derivative matrix is considered dense and :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` will not be referenced.
The ordering is assumed to be column-wise, namely the function will behave as if :math:`\textit{nnzrd} = n\times m_r` and the vectors :math:`\mathrm{irowrd}` and :math:`\mathrm{icolrd}` filled as:
:math:`\mathrm{irowrd} = \left(1, 2, …, n, 1, 2, …, n, …, 1, 2, …, n\right)`;
:math:`\mathrm{icolrd} = \left(1, 1, …, 1, 2, 2, …, 2, …, m_r, m_r, …, m_r\right)`.
.. _e04rm-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nres} \geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzrd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzrd} > 0`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowrd}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowrd}[\textit{i}-1]\leq n`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolrd}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolrd}[\textit{i}-1]\leq \mathrm{nres}`.
(`errno` :math:`8`)
On entry, more than one element of first derivative matrix has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of first derivative matrix must have a unique row and column index.
.. _e04rm-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_nlnls`` may be used to define the residual functions in the objective function of nonlinear least squares or general nonlinear data fitting problems.
If the objective function has already been defined, it will be overwritten.
It will typically be used in data fitting or calibration problems of the form
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) = \sum_{{j = 1}}^{m_r} \chi \left(r_j\left(x\right)\right) &\quad \text{ }\quad \\\text{subject to}& l_x\leq x\leq u_x \text{,} &\quad \text{ }\quad \end{array}
where :math:`x` is an :math:`n`-dimensional variable vector, :math:`r_j\left(x\right)` are nonlinear residuals (see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optclasses>`__), and :math:`\chi` is a type of loss function.
For example, the model of a least squares problem can be written as
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) = \sum_{{j = 1}}^{m_r} {r_j\left(x\right)}^2 &\quad \text{ }\quad \\\text{subject to}& l_x\leq x\leq u_x \text{,} &\quad \text{ }\quad \end{array}
The values of the residuals, and possibly their derivatives, will be communicated to the solver by a user-supplied function. ``handle_set_nlnls`` also allows the structured first derivative matrix
.. math::
\left[\frac{{\partial r_j\left(x\right)}}{{\partial x_i}}\right]_{{i = 1,\ldots,n\text{, }j = 1,\ldots,m_r}}
to be declared as being dense or sparse.
If declared as sparse, its sparsity structure must be specified by ``handle_set_nlnls``.
If ``handle_set_nlnls`` is called with :math:`m_r = 0`, any existing objective function is removed, no new one is added and the problem will be solved as a feasible point problem. Note that it is possible to temporarily disable and enable individual residuals in the model by :meth:`handle_disable` and :meth:`handle_enable`, respectively.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_nldf_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_linmatineq(handle, dima, nnza, irowa, icola, a, blksizea=None, idblk=0):
r"""
``handle_set_linmatineq`` is a part of the NAG optimization modelling suite and defines one or more linear matrix constraints of the problem.
.. _e04rn-py2-py-doc:
For full information please refer to the NAG Library document for e04rn
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rnf.html
.. _e04rn-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**dima** : int
:math:`d`, the dimension of the matrices :math:`A_{\textit{i}}`, for :math:`\textit{i} = 0,1,\ldots,\textit{nvar}`.
**nnza** : int, array-like, shape :math:`\left(\textit{nvar}+1\right)`
:math:`\mathrm{nnza}[\textit{i}]`, for :math:`\textit{i} = 0,1,\ldots,\textit{nvar}`, gives the number of nonzero elements in the upper triangle of matrix :math:`A_i`. To define :math:`A_i` as a zero matrix, set :math:`\mathrm{nnza}[i] = 0`. However, there must be at least one matrix with at least one nonzero.
**irowa** : int, array-like, shape :math:`\left(\textit{nnzasum}\right)`
Nonzero elements in upper triangle of matrices :math:`A_i` stored in coordinate storage. The first :math:`\mathrm{nnza}[0]` elements belong to :math:`A_0`, the following :math:`\mathrm{nnza}[1]` elements belong to :math:`A_1`, etc. See explanation above.
**icola** : int, array-like, shape :math:`\left(\textit{nnzasum}\right)`
Nonzero elements in upper triangle of matrices :math:`A_i` stored in coordinate storage. The first :math:`\mathrm{nnza}[0]` elements belong to :math:`A_0`, the following :math:`\mathrm{nnza}[1]` elements belong to :math:`A_1`, etc. See explanation above.
**a** : float, array-like, shape :math:`\left(\textit{nnzasum}\right)`
Nonzero elements in upper triangle of matrices :math:`A_i` stored in coordinate storage. The first :math:`\mathrm{nnza}[0]` elements belong to :math:`A_0`, the following :math:`\mathrm{nnza}[1]` elements belong to :math:`A_1`, etc. See explanation above.
**blksizea** : None or int, array-like, shape :math:`\left(\textit{nblk}\right)`, optional
If :math:`\textit{nblk} > 1`, sizes :math:`d_k` of the diagonal blocks.
If :math:`\textit{nblk} = 1`, :math:`\mathrm{blksizea}` is not referenced.
**idblk** : int, optional
If :math:`\mathrm{idblk} = 0`, new matrix inequalities are created. This is the only value allowed at the moment; nonzero values are reserved for future releases of the NAG Library.
**Returns**
**idblk** : int
The number of the last matrix inequality added. By definition, it is the number of matrix inequalities already defined plus :math:`\textit{nblk}`.
.. _e04rn-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idblk} = 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{dima} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{dima} > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzasum} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{sum}\left(\mathrm{nnza}\right) = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzasum}\geq \textit{sum}\left(\mathrm{nnza}\right)`.
(`errno` :math:`6`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{nnza}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nnza}[i-1]\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{sum}\left(\mathrm{nnza}\right) = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{sum}\left(\mathrm{nnza}\right)\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\textit{nblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nblk} > 0`.
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{blksizea}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{blksizea}[i-1]\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{dima} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{sum}\left(\mathrm{blksizea}\right) = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{sum}\left(\mathrm{blksizea}\right) = \mathrm{dima}`.
(`errno` :math:`8`)
An error occurred in matrix :math:`A_i`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowa}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{dima} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowa}[j-1]\leq \mathrm{dima}`.
(`errno` :math:`8`)
An error occurred in matrix :math:`A_i`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icola}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{dima} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icola}[j-1]\leq \mathrm{dima}`.
(`errno` :math:`8`)
An error occurred in matrix :math:`A_i`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowa}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icola}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{irowa}[\textit{j}-1]\leq \mathrm{icola}[\textit{j}-1]` (elements within the upper triangle).
(`errno` :math:`8`)
An error occurred in matrix :math:`A_i`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
On entry, more than one element of :math:`A_i` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`A_i` must have a unique row and column index.
(`errno` :math:`8`)
An error occurred in matrix :math:`A_i`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowa}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icola}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`. Maximum column index in this row given by the block structure defined by :math:`\mathrm{blksizea}` is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: all elements of :math:`A_i` must respect the block structure given by :math:`\mathrm{blksizea}`.
.. _e04rn-py2-py-notes:
**Notes**
After the initialization function :meth:`handle_init` has been called, ``handle_set_linmatineq`` may be used to add one or more linear matrix inequalities
.. math::
\sum_{1}^{n}{x_iA_i-A_0}⪰0
to the problem definition.
Here :math:`A_i` are :math:`d\times d` symmetric matrices.
The expression :math:`S⪰0` stands for a constraint on eigenvalues of a symmetric matrix :math:`S`, namely, all the eigenvalues should be non-negative, i.e., the matrix :math:`S` should be positive semidefinite.
Typically, this will be used in linear semidefinite programming problems (SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A &\quad \text{ (b)}\\& l_B\leq Bx\leq u_B &\quad \text{ (c)}\\& l_x\leq x\leq u_x &\quad \text{ (d)}\end{array}
or to define the linear part of bilinear matrix inequalities `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rnf.html#e04raeqn2>`__\ (b) in (BMI-SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_ix_jQ_{{ij}}^k} + \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A &\quad \text{ (b)}\\&l_B\leq Bx\leq u_B&\quad \text{ (c)}\\&l_x\leq x\leq u_x&\quad \text{ (d)}\end{array}
``handle_set_linmatineq`` can be called repeatedly to accumulate more matrix inequalities.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
**Input data organization**
All the matrices :math:`A_{\textit{i}}`, for :math:`\textit{i} = 0,1,\ldots,n`, are symmetric and thus only their upper triangles are passed to the function.
They are stored in sparse coordinate storage format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__), i.e., every nonzero from the upper triangles is coded as a triplet of row index, column index and the numeric value.
These triplets of all (upper triangle) nonzeros from all :math:`A_i` matrices are passed to the function in three arrays: :math:`\mathrm{irowa}` for row indices, :math:`\mathrm{icola}` for column indices and :math:`\mathrm{a}` for the values.
No particular order of nonzeros within one matrix is enforced but all nonzeros from :math:`A_0` must be stored first, followed by all nonzero from :math:`A_1`, followed by :math:`A_2`, etc.
The number of stored nonzeros from each :math:`A_i` matrix is given in :math:`\mathrm{nnza}[i]`, thus this array indicates which section of arrays :math:`\mathrm{irowa}`, :math:`\mathrm{icola}` and :math:`\mathrm{a}` belongs to which :math:`A_i` matrix.
See Table [label omitted] and the example in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rnf.html#fcomments>`__.
See also :meth:`sdp_read_sdpa` which uses the same data organization.
.. rst-class:: nag-rules-cols
+----------------------+---------------------------------------+---------------------------------------+--------------+---------------------------------------+
|:math:`\mathrm{irowa}`|upper triangle |upper triangle | |upper triangle |
+----------------------+---------------------------------------+---------------------------------------+--------------+---------------------------------------+
|:math:`\mathrm{icola}`|nonzeros |nonzeros |:math:`\cdots`|nonzeros |
+----------------------+---------------------------------------+---------------------------------------+--------------+---------------------------------------+
|:math:`\mathrm{a}` |:math:`\underbrace{ \text{from } A_0 }`|:math:`\underbrace{ \text{from } A_1 }`| |:math:`\underbrace{ \text{from } A_n }`|
+----------------------+---------------------------------------+---------------------------------------+--------------+---------------------------------------+
| |:math:`\mathrm{nnza}[0]` |:math:`\mathrm{nnza}[1]` | |:math:`\mathrm{nnza}[n]` |
+----------------------+---------------------------------------+---------------------------------------+--------------+---------------------------------------+
There are two possibilities for defining more matrix inequality constraints
.. math::
\sum_{1}^{n}{x_iA_i^k-A_0^k}⪰0\text{, }\quad k = 1,2,\ldots,m_A
to the problem.
The first is to call ``handle_set_linmatineq`` :math:`m_A` times and define a single matrix inequality at a time.
This might be more straightforward and, therefore, it is recommended.
Alternatively, it is possible to merge all :math:`m_A` constraints into one inequality and pass them in a single call to ``handle_set_linmatineq``.
It is easy to see that `(4) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rnf.html#eqn4>`__ can be equivalently expressed as one bigger matrix inequality with the following block diagonal structure
.. math::
\sum_{1}^{n}{x_i}\begin{pmatrix}A_i^1&&&\\&A_i^2&&\\&&⋱&\\&&&A_i^{m_A}\end{pmatrix}-\begin{pmatrix}A_0^1&&&\\&A_0^2&&\\&&⋱&\\&&&A_0^{m_A}\end{pmatrix}⪰0\text{.}
If :math:`d_k` denotes the dimension of inequality :math:`k`, the new merged inequality has dimension :math:`d = \sum_{1}^{m_A}{d_k}` and each of the :math:`A_i` matrices is formed by :math:`A_i^1,A_i^2,\ldots,A_i^{m_A}` stored as :math:`m_A` diagonal blocks.
In such a case, :math:`\textit{nblk}` is set to :math:`m_A` and :math:`\mathrm{blksizea}[k-1]` to :math:`d_k`, the size of the :math:`k`\ th diagonal blocks.
This might be useful in connection with :meth:`sdp_read_sdpa`.
On the other hand, if there is no block structure and just one matrix inequality is provided, :math:`\textit{nblk}` should be set to :math:`1` and :math:`\mathrm{blksizea}` is not referenced.
**Definition of Bilinear Matrix Inequalities (BMI)**
``handle_set_linmatineq`` is designed to be used together with :meth:`handle_set_quadmatineq` to define bilinear matrix inequalities `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rnf.html#e04raeqn2>`__\ (b). ``handle_set_linmatineq`` sets the linear part of the constraint and :meth:`handle_set_quadmatineq` expands it by higher order terms.
To distinquish which linear matrix inequality (or more precisely, which block) is to be expanded, :meth:`handle_set_quadmatineq` needs the number of the block, :math:`\mathrm{idblk}`.
The blocks are numbered as they are added, starting from :math:`1`.
Whenever a matrix inequality (or a set of them expressed as diagonal blocks) is stored, the function returns :math:`\mathrm{idblk}` of the last inequality added. :math:`\mathrm{idblk}` is just the order of the inequality amongst all matrix inequalities accumulated through the calls.
The first inequality has :math:`\mathrm{idblk} = 1`, the second one :math:`\mathrm{idblk} = 2`, etc.
Therefore, if you call ``handle_set_linmatineq`` for the very first time with :math:`\textit{nblk} = 42`, it adds :math:`42` inequalities with :math:`\mathrm{idblk}` from :math:`1` to :math:`42` and the function returns :math:`\mathrm{idblk} = 42` (the number of the last one).
A subsequent call with :math:`\textit{nblk} = 1` would add only one inequality, this time with :math:`\mathrm{idblk} = 43` which would be returned.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_bmi_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_lmi_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_quadmatineq(handle, qi, qj, dimq, nnzq, irowq, icolq, q, idblk):
r"""
``handle_set_quadmatineq`` is a part of the NAG optimization modelling suite and defines bilinear matrix terms either in a new matrix constraint or adds them to an existing linear matrix inequality.
.. _e04rp-py2-py-doc:
For full information please refer to the NAG Library document for e04rp
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rpf.html
.. _e04rp-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**qi** : int, array-like, shape :math:`\left(\textit{nq}\right)`
The index pairs :math:`i,j` of the nonzero matrices :math:`Q_{{ij}}` in any order.
**qj** : int, array-like, shape :math:`\left(\textit{nq}\right)`
The index pairs :math:`i,j` of the nonzero matrices :math:`Q_{{ij}}` in any order.
**dimq** : int
:math:`d`, the dimension of matrices :math:`Q_{{ij}}`.
**nnzq** : int, array-like, shape :math:`\left(\textit{nq}\right)`
The numbers of nonzero elements in the upper triangles of :math:`Q_{{ij}}` matrices.
**irowq** : int, array-like, shape :math:`\left(\textit{nnzqsum}\right)`
The nonzero elements of the upper triangles of matrices :math:`Q_{{ij}}` stored in coordinate storage format. The first :math:`\mathrm{nnzq}[0]` elements belong to the first :math:`Q_{{i_1j_1}}`, the following :math:`\mathrm{nnzq}[1]` to :math:`Q_{{i_2j_2}}`, etc.
**icolq** : int, array-like, shape :math:`\left(\textit{nnzqsum}\right)`
The nonzero elements of the upper triangles of matrices :math:`Q_{{ij}}` stored in coordinate storage format. The first :math:`\mathrm{nnzq}[0]` elements belong to the first :math:`Q_{{i_1j_1}}`, the following :math:`\mathrm{nnzq}[1]` to :math:`Q_{{i_2j_2}}`, etc.
**q** : float, array-like, shape :math:`\left(\textit{nnzqsum}\right)`
The nonzero elements of the upper triangles of matrices :math:`Q_{{ij}}` stored in coordinate storage format. The first :math:`\mathrm{nnzq}[0]` elements belong to the first :math:`Q_{{i_1j_1}}`, the following :math:`\mathrm{nnzq}[1]` to :math:`Q_{{i_2j_2}}`, etc.
**idblk** : int
If :math:`\mathrm{idblk} = 0`, a new matrix constraint is created; otherwise, :math:`\mathrm{idblk} = k > 0`, the number of the existing linear matrix constraint to be expanded with the bilinear terms.
**Returns**
**idblk** : int
If :math:`\mathrm{idblk} = 0` on entry, the number of the new matrix constraint is returned. By definition, it is the number of matrix inequalities already defined plus one. Otherwise, :math:`\mathrm{idblk} > 0` stays unchanged.
.. _e04rp-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`3`)
On entry, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
Bilinear terms of the matrix inequality block with the given :math:`\mathrm{idblk}` have already been defined.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idblk}` refers to a nonexistent matrix inequality block.
No matrix inequalities have been added yet.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idblk}` does not match with any existing matrix inequality block.
The maximum :math:`\mathrm{idblk}` is currently :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\mathrm{dimq} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
The correct dimension of the given :math:`\mathrm{idblk}` is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{dimq}` must match the dimension of the block supplied earlier.
(`errno` :math:`6`)
On entry, :math:`\mathrm{idblk} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idblk}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nq} > 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{dimq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{dimq} > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzqsum} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{sum}\left(\mathrm{nnzq}\right) = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzqsum}\geq \textit{sum}\left(\mathrm{nnzq}\right)`.
(`errno` :math:`6`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{nnzq}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nnzq}[i-1] > 0`.
(`errno` :math:`8`)
On entry, an error occurred in matrix :math:`Q_{{ij}}` of index :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qi}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qj}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
For :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowq}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{dimq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowq}[j-1]\leq \mathrm{dimq}`.
(`errno` :math:`8`)
On entry, an error occurred in matrix :math:`Q_{{ij}}` of index :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qi}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qj}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
For :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolq}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{dimq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolq}[j-1]\leq \mathrm{dimq}`.
(`errno` :math:`8`)
On entry, an error occurred in matrix :math:`Q_{{ij}}` of index :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qi}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qj}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
For :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowq}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icolq}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{irowq}[\textit{j}-1]\leq \mathrm{icolq}[\textit{j}-1]` (elements within the upper triangle).
(`errno` :math:`8`)
On entry, an error occurred in matrix :math:`Q_{{ij}}` of index :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qi}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qj}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
More than one element of :math:`Q_{{ij}}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`Q_{{ij}}` must have a unique row and column index.
(`errno` :math:`9`)
On entry, :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qi}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{qi}[k-1]\leq \textit{n}`.
(`errno` :math:`9`)
On entry, :math:`k = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{qj}[k-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{qj}[k-1]\leq \textit{n}`.
(`errno` :math:`9`)
On entry, index pair with :math:`\mathrm{qi} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{qj} = \langle\mathit{\boldsymbol{value}}\rangle` repeats.
Constraint: each index pair :math:`\mathrm{qi}`, :math:`\mathrm{qj}` must be unique.
.. _e04rp-py2-py-notes:
**Notes**
After the initialization function :meth:`handle_init` has been called, ``handle_set_quadmatineq`` may be used to define bilinear matrix terms.
It may be used in two ways, either to add to the problem formulation a new bilinear matrix inequality (BMI) which does not have linear terms:
.. math::
\sum_{1}^{n}{x_ix_jQ_{{ij}}}⪰0
or to extend an existing linear matrix inequality constraint by bilinear terms:
.. math::
\sum_{1}^{n}{x_ix_jQ_{{ij}}^k}\text{.}
Here :math:`Q_{{ij}}^k` are :math:`d\times d` (sparse) symmetric matrices and :math:`k`, if present, is the number of the existing constraint.
This function will typically be used on semidefinite programming problems with bilinear matrix constraints (BMI-SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_ix_jQ_{{ij}}^k} + \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\& l_x\leq x\leq u_x \text{.} &\quad \text{ (d)}\end{array}
The function can be called multiple times to define an additional matrix inequality or to extend an existing one, but it cannot be called twice to extend the same matrix inequality.
The argument :math:`\mathrm{idblk}` is used to distinguish whether a new matrix constraint should be added (:math:`\mathrm{idblk} = 0`) or if an existing linear matrix inequality should be extended (:math:`\mathrm{idblk} > 0`).
In the latter case, :math:`\mathrm{idblk}` should be set to :math:`k`, the number of the existing inequality.
See :meth:`handle_set_linmatineq` for details about formulation of linear matrix constraints and their numbering and a further description of :math:`\mathrm{idblk}`.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
In the further text, the index :math:`k` will be omitted.
**Input data organization**
It is expected that only some of the matrices :math:`Q_{{ij}}` will be nonzero, therefore, only their index pairs :math:`i,j` are listed in arrays :math:`\mathrm{qi}` and :math:`\mathrm{qj}`.
Note that a pair :math:`i,j` should not repeat, i.e., a matrix :math:`Q_{{ij}}` should not be defined more than once.
No particular ordering of pairs :math:`i,j` is expected but other input arrays :math:`\mathrm{irowq}`, :math:`\mathrm{icolq}`, :math:`\mathrm{q}` and :math:`\mathrm{nnzq}` need to respect the chosen order.
**Note:** the dimension of :math:`Q_{{ij}}` must respect the size of the linear matrix inequality if they are supposed to expand it (case :math:`\mathrm{idblk} > 0`).
Matrices :math:`Q_{{ij}}` are symmetric and thus only their upper triangles are passed to the function.
They are stored in sparse coordinate storage format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__), i.e., every nonzero from the upper triangles is coded as a triplet of row index, column index and the numeric value.
All these triplets from all :math:`Q_{{ij}}` matrices are passed to the function in three arrays: :math:`\mathrm{irowq}` for row indices, :math:`\mathrm{icolq}` for column indices and :math:`\mathrm{q}` for the values.
No particular order of nonzeros within one matrix is enforced but all nonzeros belonging to one :math:`Q_{{ij}}` matrix need to be stored next to each other.
The first :math:`\mathrm{nnzq}[0]` nonzeros belong to :math:`Q_{{i_1j_1}}` where :math:`i_1 = \mathrm{qi}[0]`, :math:`j_1 = \mathrm{qj}[0]`, the following :math:`\mathrm{nnzq}[1]` nonzeros to the next one given by :math:`\mathrm{qi}`, :math:`\mathrm{qj}` and so on.
The array :math:`\mathrm{nnzq}` thus splits arrays :math:`\mathrm{irowq}`, :math:`\mathrm{icolq}` and :math:`\mathrm{q}` into sections so that each section defines one :math:`Q_{{ij}}` matrix.
See Table [label omitted] below. Functions :meth:`handle_set_linmatineq` and :meth:`sdp_read_sdpa` use the same data organization so further examples can be found there.
.. rst-class:: nag-rules-cols
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
|:math:`\mathrm{irowq}`|upper triangle |upper triangle | |upper triangle |
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
|:math:`\mathrm{icolq}`|nonzeros |nonzeros |:math:`\cdots`|nonzeros |
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
|:math:`\mathrm{q}` |:math:`\underbrace{ \text{from } Q_{{i_1j_1}} }`|:math:`\underbrace{ \text{from } Q_{{i_2j_2}} }`| |:math:`\underbrace{ \text{from } Q_{{i_{\textit{nq}}j_{\textit{nq}}}} }`|
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
| |:math:`\mathrm{nnzq}[0]` |:math:`\mathrm{nnzq}[1]` | |:math:`\mathrm{nnzq}[\textit{nq}-1]` |
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
| |:math:`i_1 = \mathrm{qi}[0]` |:math:`i_2 = \mathrm{qi}[1]` | |:math:`i_{\textit{nq}} = \mathrm{qi}[\textit{nq}-1]` |
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
| |:math:`j_1 = \mathrm{qj}[0]` |:math:`j_2 = \mathrm{qj}[1]` | |:math:`j_{\textit{nq}} = \mathrm{qj}[\textit{nq}-1]` |
+----------------------+------------------------------------------------+------------------------------------------------+--------------+------------------------------------------------------------------------+
.. _e04rp-py2-py-references:
**References**
Syrmos, V L, Abdallah, C T, Dorato, P and Grigoriadis, K, 1997, `Static output feedback -- a survey`, Journal Automatica (Journal of IFAC) (Volume 33) (2), 125--137
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_bmi_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_qconstr(handle, s, idqc, idxr=None, r=None, irowq=None, icolq=None, q=None):
r"""
``handle_set_qconstr`` is a part of the NAG optimization modelling suite and defines a new, or edits an existing, quadratic objective function or constraint of the problem.
.. _e04rs-py2-py-doc:
For full information please refer to the NAG Library document for e04rs
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rsf.html
.. _e04rs-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**s** : float
The constant term in quadratic constraint.
If :math:`\mathrm{idqc} = -1`, :math:`\mathrm{s}` will not be referenced.
**idqc** : int
:math:`\mathrm{idqc} = 0`
A new quadratic constraint is created.
:math:`\mathrm{idqc} > 0`
Specifies the index of an existing constraint to be replaced. i.e., replaces the :math:`\mathrm{idqc}`\ th constraint.
:math:`\mathrm{idqc} = -1`
A new quadratic objective is created and will replace any previously defined objective function.
**idxr** : None or int, array-like, shape :math:`\left(\textit{nnzr}\right)`, optional
The nonzero elements of the sparse vector :math:`r`. :math:`\mathrm{idxr}[i-1]` must contain the index of :math:`\mathrm{r}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzr}`. No particular order is expected, but elements should not repeat. Note that :math:`n` is the current number of variables in the problem.
**r** : None or float, array-like, shape :math:`\left(\textit{nnzr}\right)`, optional
The nonzero elements of the sparse vector :math:`r`. :math:`\mathrm{idxr}[i-1]` must contain the index of :math:`\mathrm{r}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzr}`. No particular order is expected, but elements should not repeat. Note that :math:`n` is the current number of variables in the problem.
**irowq** : None or int, array-like, shape :math:`\left(\textit{nnzq}\right)`, optional
Arrays :math:`\mathrm{irowq}`, :math:`\mathrm{icolq}` and :math:`\mathrm{q}` store the nonzeros of the upper triangle of the matrix :math:`Q` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowq}` specifies one-based row indices, :math:`\mathrm{icolq}` specifies one-based column indices and :math:`\mathrm{q}` specifies the values of the nonzero elements in such a way that :math:`Q_{{ij}} = \mathrm{q}[l-1]` where :math:`i = \mathrm{irowq}[l-1]`, :math:`j = \mathrm{icolq}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzq}`. No particular order is expected, but elements should not repeat.
**icolq** : None or int, array-like, shape :math:`\left(\textit{nnzq}\right)`, optional
Arrays :math:`\mathrm{irowq}`, :math:`\mathrm{icolq}` and :math:`\mathrm{q}` store the nonzeros of the upper triangle of the matrix :math:`Q` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowq}` specifies one-based row indices, :math:`\mathrm{icolq}` specifies one-based column indices and :math:`\mathrm{q}` specifies the values of the nonzero elements in such a way that :math:`Q_{{ij}} = \mathrm{q}[l-1]` where :math:`i = \mathrm{irowq}[l-1]`, :math:`j = \mathrm{icolq}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzq}`. No particular order is expected, but elements should not repeat.
**q** : None or float, array-like, shape :math:`\left(\textit{nnzq}\right)`, optional
Arrays :math:`\mathrm{irowq}`, :math:`\mathrm{icolq}` and :math:`\mathrm{q}` store the nonzeros of the upper triangle of the matrix :math:`Q` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowq}` specifies one-based row indices, :math:`\mathrm{icolq}` specifies one-based column indices and :math:`\mathrm{q}` specifies the values of the nonzero elements in such a way that :math:`Q_{{ij}} = \mathrm{q}[l-1]` where :math:`i = \mathrm{irowq}[l-1]`, :math:`j = \mathrm{icolq}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzq}`. No particular order is expected, but elements should not repeat.
**Returns**
**idqc** : int
If :math:`\mathrm{idqc} = 0` on entry, then :math:`\mathrm{idqc}` is overwritten with the index of the new quadratic constraint. By definition, this is the number of quadratic constraints already defined plus one. Otherwise, :math:`\mathrm{idqc}` stays unchanged.
.. _e04rs-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idqc} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idqc}` does not match with any quadratic constraint already defined.
(`errno` :math:`5`)
On entry, :math:`\mathrm{idqc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idqc} \geq -1`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzq}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzr} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzr}\geq 0`.
(`errno` :math:`6`)
:math:`\textit{nnzr}` and :math:`\textit{nnzq}` cannot be zero at the same time.
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxr}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxr}[i-1]\leq \textit{n}`.
(`errno` :math:`7`)
On entry, more than one element of :math:`\mathrm{idxr}` has index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{idxr}` must have a unique index.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowq}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowq}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolq}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolq}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowq}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{icolq}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{irowq}[\textit{i}-1]\leq \mathrm{icolq}[\textit{i}-1]` (elements within the upper triangle).
(`errno` :math:`8`)
On entry, more than one element of :math:`\mathrm{q}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{q}` must have a unique row and column index.
.. _e04rs-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_qconstr`` may be used to edit a model by adding or replacing a quadratic objective function or constraint of the form
.. math::
\frac{1}{2}x^\mathrm{T}Qx+r^\mathrm{T}x
and
.. math::
\frac{1}{2}x^\mathrm{T}Qx+r^\mathrm{T}x+s\leq 0\text{,}
respectively.
If a factor :math:`F` of :math:`Q` such that :math:`Q = F^\mathrm{T}F` is available, :meth:`handle_set_qconstr_fac` should be called instead.
The matrix :math:`Q` is a sparse symmetric :math:`n\times n` matrix.
Typically ``handle_set_qconstr`` would be used together with Second-order Cone Programming (SOCP) solver :meth:`handle_solve_socp_ipm` if :math:`Q` is positive semidefinite which implies the problem is convex.
Otherwise, if the problem is nonconvex, a general nonlinear optimization solver (such as :meth:`handle_solve_ipopt`) may be used.
It is also acceptable if :math:`Q` is a zero matrix, in which case the corresponding objective function or constraint becomes linear.
Note that it is possible to temporarily disable and enable individual constraints in the model by :meth:`handle_disable` and :meth:`handle_enable`, respectively.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
[docs]def handle_set_qconstr_fac(handle, s, mf, idqc, idxr=None, r=None, irowf=None, icolf=None, f=None):
r"""
``handle_set_qconstr_fac`` is a part of the NAG optimization modelling suite and defines a new, or edits an existing, quadratic objective function or constraint of the problem using a factor of the quadratic coefficient matrix.
.. _e04rt-py2-py-doc:
For full information please refer to the NAG Library document for e04rt
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rtf.html
.. _e04rt-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**s** : float
The constant term in quadratic constraint.
If :math:`\mathrm{idqc} = -1`, :math:`\mathrm{s}` will not be referenced.
**mf** : int
:math:`m`, row dimension of matrix :math:`F`.
**idqc** : int
:math:`\mathrm{idqc} = 0`
A new quadratic constraint is created.
:math:`\mathrm{idqc} > 0`
Specifies the index of an existing constraint to be replaced. i.e., replaces the :math:`\mathrm{idqc}`\ th constraint.
:math:`\mathrm{idqc} = -1`
A new quadratic objective is created and will replace any previously defined objective function.
**idxr** : None or int, array-like, shape :math:`\left(\textit{nnzr}\right)`, optional
The nonzero elements of the sparse vector :math:`r`. :math:`\mathrm{idxr}[i-1]` must contain the index of :math:`\mathrm{r}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzr}`. No particular order is expected, but elements should not repeat. Note that :math:`n` is the current number of variables in the problem.
**r** : None or float, array-like, shape :math:`\left(\textit{nnzr}\right)`, optional
The nonzero elements of the sparse vector :math:`r`. :math:`\mathrm{idxr}[i-1]` must contain the index of :math:`\mathrm{r}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzr}`. No particular order is expected, but elements should not repeat. Note that :math:`n` is the current number of variables in the problem.
**irowf** : None or int, array-like, shape :math:`\left(\textit{nnzf}\right)`, optional
Arrays :math:`\mathrm{irowf}`, :math:`\mathrm{icolf}` and :math:`\mathrm{f}` store the nonzeros of the matrix :math:`F` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowf}` specifies one-based row indices, :math:`\mathrm{icolf}` specifies one-based column indices and :math:`\mathrm{f}` specifies the values of the nonzero elements in such a way that :math:`F_{{ij}} = \mathrm{f}[l-1]` where :math:`i = \mathrm{irowf}[l-1]`, :math:`j = \mathrm{icolf}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzf}`. No particular order is expected, but elements should not repeat.
**icolf** : None or int, array-like, shape :math:`\left(\textit{nnzf}\right)`, optional
Arrays :math:`\mathrm{irowf}`, :math:`\mathrm{icolf}` and :math:`\mathrm{f}` store the nonzeros of the matrix :math:`F` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowf}` specifies one-based row indices, :math:`\mathrm{icolf}` specifies one-based column indices and :math:`\mathrm{f}` specifies the values of the nonzero elements in such a way that :math:`F_{{ij}} = \mathrm{f}[l-1]` where :math:`i = \mathrm{irowf}[l-1]`, :math:`j = \mathrm{icolf}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzf}`. No particular order is expected, but elements should not repeat.
**f** : None or float, array-like, shape :math:`\left(\textit{nnzf}\right)`, optional
Arrays :math:`\mathrm{irowf}`, :math:`\mathrm{icolf}` and :math:`\mathrm{f}` store the nonzeros of the matrix :math:`F` in coordinate storage (CS) format (see `the F11 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f11/f11intro.html#background10>`__). :math:`\mathrm{irowf}` specifies one-based row indices, :math:`\mathrm{icolf}` specifies one-based column indices and :math:`\mathrm{f}` specifies the values of the nonzero elements in such a way that :math:`F_{{ij}} = \mathrm{f}[l-1]` where :math:`i = \mathrm{irowf}[l-1]`, :math:`j = \mathrm{icolf}[\textit{l}-1]`, for :math:`\textit{l} = 1,2,\ldots,\textit{nnzf}`. No particular order is expected, but elements should not repeat.
**Returns**
**idqc** : int
If :math:`\mathrm{idqc} = 0` on entry, then :math:`\mathrm{idqc}` is overwritten with the index of the new quadratic constraint. By definition, this is the number of quadratic constraints already defined plus one. Otherwise, :math:`\mathrm{idqc}` stays unchanged.
.. _e04rt-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`4`)
On entry, :math:`\mathrm{idqc} = \langle\mathit{\boldsymbol{value}}\rangle`.
The given :math:`\mathrm{idqc}` does not match with any quadratic constraint already defined.
(`errno` :math:`5`)
On entry, :math:`\mathrm{idqc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idqc} \geq -1`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzf}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nnzr} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzr}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{mf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{mf} > 0`.
(`errno` :math:`6`)
:math:`\textit{nnzr}` and :math:`\textit{nnzf}` cannot be zero at the same time.
(`errno` :math:`7`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxr}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxr}[i-1]\leq \textit{n}`.
(`errno` :math:`7`)
On entry, more than one element of :math:`\mathrm{idxr}` has index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{idxr}` must have a unique index.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowf}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{mf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowf}[\textit{i}-1]\leq \mathrm{mf}`.
(`errno` :math:`8`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{icolf}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolf}[\textit{i}-1]\leq \textit{n}`.
(`errno` :math:`8`)
On entry, more than one element of :math:`\mathrm{f}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{f}` must have a unique row and column index.
.. _e04rt-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_set_qconstr_fac`` may be used to edit a model by adding or replacing a quadratic objective function or constraint of the form
.. math::
\frac{1}{2}x^\mathrm{T}F^\mathrm{T}Fx+r^\mathrm{T}x
and
.. math::
\frac{1}{2}x^\mathrm{T}F^\mathrm{T}Fx+r^\mathrm{T}x+s\leq 0\text{,}
respectively.
The matrix :math:`F` is a sparse :math:`m\times n` matrix.
It can be viewed as the factor of the symmetric matrix :math:`Q = F^\mathrm{T}F` in a general quadratic function
.. math::
\frac{1}{2}x^\mathrm{T}Qx+r^\mathrm{T}x+s\text{.}
It is also acceptable if :math:`F` is a zero matrix, in which case the corresponding objective function or constraint becomes linear.
If you have the full matrix :math:`Q` as input data, please call function :meth:`handle_set_qconstr` instead.
Note that it is possible to temporarily disable and enable individual constraints in the model by :meth:`handle_disable` and :meth:`handle_enable`, respectively.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
[docs]def handle_set_get_integer(handle, cmdstr, iarr=None):
r"""
``handle_set_get_integer`` is a part of the NAG optimization modelling suite.
It allows an Integer piece of information to be read from or written to the problem stored in the handle.
For example, it may be used to extract the current problem sizes and statistics.
.. _e04rw-py2-py-doc:
For full information please refer to the NAG Library document for e04rw
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rwf.html
.. _e04rw-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized by :meth:`handle_init` and **must not** be changed between calls to the NAG optimization modelling suite.
**cmdstr** : str
A string which identifies the item within the handle to be read or written. The string is case insensitive and space tolerant.
**iarr** : None or int, array-like, shape :math:`\left(\textit{liarr}\right)`, optional
If :math:`\mathrm{iarr}` is not **None** (write mode), :math:`\mathrm{iarr}` must contain the information to be written to the handle.
**Returns**
**liarr** : int
In write mode and when the size of the input :math:`\mathrm{iarr}` does not match the item identified by :math:`\mathrm{cmdstr}`, :math:`\mathrm{liarr}` is the correct expected size. Otherwise, it is merely the actual size of the relevant :math:`\mathrm{iarr}`.
**iarr** : None or int, ndarray, shape :math:`\left(\textit{liarr}\right)`
If :math:`\mathrm{iarr}` was **None** on entry (read mode), :math:`\mathrm{iarr}` contains the information requested by :math:`\mathrm{cmdstr}`.
.. _e04rw-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`3`)
The provided :math:`\mathrm{cmdstr}` is not recognised.
(`errno` :math:`4`)
Reading mode is not supported for the given :math:`\mathrm{cmdstr}`.
(`errno` :math:`5`)
Writing mode is not supported for the given :math:`\mathrm{cmdstr}`.
(`errno` :math:`6`)
The requested information is not available.
(`errno` :math:`6`)
The request cannot be processed at this phase.
.. _e04rw-py2-py-notes:
**Notes**
``handle_set_get_integer`` adds an additional means of communication to functions within the NAG optimization modelling suite.
It allows you to either read or write a piece of information in the handle in the form of an integer array.
The item is identified by :math:`\mathrm{cmdstr}` and the direction of the communication is set by the presence or absence of :math:`\mathrm{iarr}`.
The functionality is limited in this release of the NAG Library to the retrieval of the problem size and statistics.
General information on available :math:`\mathrm{cmdstr}` is as follows.
.. rst-class:: nag-rules-none
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+
|:math:`\mathrm{cmdstr}`|Description |Direction of communication|:math:`\textit{liarr}`|
+=======================+==========================================================================================================================================+==========================+======================+
|BASIS |The current state of the variables and slacks of the problem. It can also be referred as **WARM START BASIS**. |Read and write |:math:`*` |
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+
|PINFO |The current problem sizes and statistics as :math:`\textit{pinfo}` defined in :meth:`handle_read_file` |Read |:math:`100` |
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+
|MCS Number Points |Used to provide a custom initialization list for the solver :meth:`glopt.handle_solve_mcs <naginterfaces.library.glopt.handle_solve_mcs>`.|Write |:math:`\textit{nvar}` |
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+
|MCS Initial Points |Used to provide a custom initialization list for the solver :meth:`glopt.handle_solve_mcs <naginterfaces.library.glopt.handle_solve_mcs>`.|Write |:math:`\textit{nvar}` |
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_get_real(handle, cmdstr, rarr=None):
r"""
``handle_set_get_real`` is a part of the NAG optimization modelling suite.
It allows a piece of real information to be read from or written to the problem stored in the handle.
For example, it may be used to extract the current approximation of the solution during a monitoring step.
.. _e04rx-py2-py-doc:
For full information please refer to the NAG Library document for e04rx
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rxf.html
.. _e04rx-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized by :meth:`handle_init` and **must not** be changed between calls to the NAG optimization modelling suite.
**cmdstr** : str
A string which identifies the item within the handle to be read or written. The string is case insensitive and space tolerant.
**rarr** : None or float, array-like, shape :math:`\left(\textit{lrarr}\right)`, optional
If :math:`\mathrm{rarr}` is not **None** (write mode), :math:`\mathrm{rarr}` must contain the information to be written to the handle.
**Returns**
**lrarr** : int
In write mode and when the size of the input :math:`\mathrm{rarr}` does not match the item identified by :math:`\mathrm{cmdstr}`, :math:`\mathrm{lrarr}` is the correct expected size. Otherwise, it is merely the actual size of the relevant :math:`\mathrm{rarr}`.
**rarr** : None or float, ndarray, shape :math:`\left(\textit{lrarr}\right)`
If :math:`\mathrm{rarr}` was **None** on entry (read mode), :math:`\mathrm{rarr}` contains the information requested by :math:`\mathrm{cmdstr}`.
.. _e04rx-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`3`)
The provided :math:`\mathrm{cmdstr}` is not recognised.
(`errno` :math:`4`)
Reading mode is not supported for the given :math:`\mathrm{cmdstr}`.
(`errno` :math:`5`)
Writing mode is not supported for the given :math:`\mathrm{cmdstr}`.
(`errno` :math:`6`)
The requested information is not available.
(`errno` :math:`6`)
The request cannot be processed at this phase.
.. _e04rx-py2-py-notes:
**Notes**
``handle_set_get_real`` adds an additional means of communication to functions within the NAG optimization modelling suite.
It allows you to either read or write a piece of information in the handle in the form of a real array.
The item is identified by :math:`\mathrm{cmdstr}` and the direction of the communication is set by the presence or absence of :math:`\mathrm{rarr}`.
The following :math:`\mathrm{cmdstr}` are available:
**Primal Variables** or **X**
The current value of the primal variables.
**Dual Variables** or **U**
The current value of the dual variables (Lagrangian multipliers for bounds, linear, quadratic and nonlinear constraints).
**Dual Cone Variables** or **UC**
The current value of the dual cone variables (Lagrangian multipliers for quadratic cone constraints).
**Dual Matrix Variables** or **UA**
The current value of the dual matrix variables (Lagrangian multipliers for matrix inequality constraints).
**Residual Weights** or **RW**
Array of residual weights.
**Covariance Matrix** or **COV**
Covariance matrix of the primal variables.
**Hessian Matrix** or **HES**
The current value of the Hessian matrix.
**Variance** or **VAR**
Variance for the primal variables.
**MCS List** or **MCSL**
Used to provide a custom initialization list for the solver :meth:`glopt.handle_solve_mcs <naginterfaces.library.glopt.handle_solve_mcs>`.
**Slack** or **SLK**
Slack variables.
The functionality is supported by all solvers within the NAG optimization modelling suite for the retrieval of their final solutions. ``handle_set_get_real``, moreover, can also be used by a selection of the solvers in the NAG Library, such as, :meth:`handle_solve_lp_ipm` and :meth:`handle_solve_socp_ipm` for the retrieval of the approximate solutions within the monitoring step.
See the documentation of the individual solvers.
In addition, you can retrieve solutions from a specific solver by adding the solver name after the available :math:`\mathrm{cmdstr}` mentioned above, e.g., keywords **X e04mt** or **X handle_solve_lp_ipm** return the value of primal variables only from the linear programming solver :meth:`handle_solve_lp_ipm`.
This is useful when a problem is solved by several different solvers.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
[docs]def handle_print(handle, nout, cmdstr, io_manager=None):
r"""
``handle_print`` is a part of the NAG optimization modelling suite.
It allows you to print information about the problem, stored as a handle, such as which parts have already been defined or details of the constraints.
.. _e04ry-py2-py-doc:
For full information please refer to the NAG Library document for e04ry
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ryf.html
.. _e04ry-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**nout** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) which identifies the file to be written to.
**cmdstr** : str
A command string which contains one or more keywords which identify the piece of information about the handle to be printed. Keywords are case-insensitive and space tolerant. Multiple keywords in :math:`\mathrm{cmdstr}` must be separated by commas or semicolons.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ry-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`6`)
On entry, :math:`\mathrm{nout} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nout}\geq 0`.
(`errno` :math:`7`)
Keyword number :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not recognized.
(`errno` :math:`7`)
Keyword number :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not recognized, it is too long.
(`errno` :math:`7`)
:math:`\mathrm{cmdstr}` does not contain any keywords or is empty.
(`errno` :math:`199`)
An error occurred when writing to output.
.. _e04ry-py2-py-notes:
**Notes**
``handle_print`` prints information on a problem handle which has been previously correctly initialized (e.g., by :meth:`handle_init`).
Various pieces of information can be retrieved and printed to the given output unit.
This can be helpful when the function is interfaced from interactive environments, for debugging purposes or to help familiarize you with the NAG optimization modelling suite.
The printer is guided by a command string which contains one or more of the following keywords:
**Overview**
Gives a brief overview of the problem handle, particularly, the problem dimension and which parts of the problem have already been set.
**Objective**
Prints the objective function as it was defined by :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac` when it is linear or quadratic. Prints the sparsity structure of the objective function as it was defined by :meth:`handle_set_nlnobj` when it is nonlinear or nonlinear least squares by :meth:`handle_set_nlnls`.
**Simple bounds**
Prints the variable bounds as defined by :meth:`handle_set_simplebounds`. This might help you understand the effect of the option 'Infinite Bound Size' on the bounds.
**Linear constraints bounds** **Linear constraints detailed**
Prints the bounds or the linear constraint matrix as defined by :meth:`handle_set_linconstr`.
**Quadratic constraints detailed**
Prints all the quadratic constraints including all nonzeros of quadratic, linear and constant terms as defined by :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac`.
**Cone constraints** **Cone constraints detailed**
Prints all second-order cone constraints as defined by :meth:`handle_set_group`. For each cone constraint, its :math:`\textit{idgroup}`, dimension and type are printed. If details are requested, the variables composing the cone constraint are also listed.
**Matrix constraints**
Gives a list of the matrix constraints as defined by :meth:`handle_set_linmatineq` and :meth:`handle_set_quadmatineq`. For each matrix constraint its :math:`\textit{idblk}`, dimension and order (e.g., linear, bilinear) are printed.
**Matrix constraints detailed**
Prints all the matrix constraints including all nonzeros of all the matrices as formulated by :meth:`handle_set_linmatineq` and :meth:`handle_set_quadmatineq`.
**Nonlinear constraints bounds** **Nonlinear constraints detailed**
Prints the bounds or sparsity structure of the nonlinear constraints as defined by :meth:`handle_set_nlnconstr`.
**Multipliers sizes**
Prints the expected dimensions of arrays storing the Lagrangian multipliers, such as :math:`\textit{u}` and :math:`\textit{ua}` of the solver :meth:`handle_solve_pennon` which store the Lagrangian multipliers for standard and matrix constraints, respectively. This might be helpful with **Overview** and **Matrix constraints** to check the way the sizes of the arrays are derived.
**Options** **Options** `solver`
Prints all the current options. It flags whether the argument is set to its default, whether you have set it or whether it is chosen by the solver (for example, options left on 'AUTO' setting after the solver has been called). In addition, you can print optional parameters belonging to a specific `solver` by adding the solver name after **Options**, e.g., keywords **Options e04mt** or **Options handle_solve_lp_ipm** print options only belonging to the linear programming solver :meth:`handle_solve_lp_ipm`.
Note that the output data might not match your input exactly.
The sparse matrices are typically transposed, sorted and explicit zeros removed and in certain cases transformed as needed (for example, matrices :math:`Q_{{ij}}` and :math:`Q_{{ji}}` are merged by :meth:`handle_set_quadmatineq`).
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
[docs]def handle_free(handle):
r"""
``handle_free`` is a part of the NAG optimization modelling suite.
It is used to deallocate the memory used within the problem handle and to destroy the handle itself.
.. _e04rz-py2-py-doc:
For full information please refer to the NAG Library document for e04rz
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rzf.html
.. _e04rz-py2-py-parameters:
**Parameters**
**handle** : Handle, modified in place
`On entry`: the handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
`On exit`: the handle is destroyed and set to a null Handle.
.. _e04rz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
.. _e04rz-py2-py-notes:
**Notes**
Each initialized problem handle (e.g., by :meth:`handle_init`) should be deallocated to avoid memory leaks.
Therefore, ``handle_free`` should be called on all the handles which are no longer needed, typically after obtaining results from the solver.
Please note that passing a handle which has not been properly initialized might cause unpredictable behaviour, including a crash of your program.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.glopt.handle_solve_mcs_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_simplex_ex.main`
(and others)
"""
raise NotImplementedError
[docs]def handle_read_file(file, ftype):
r"""
``handle_read_file`` initializes a data structure for the NAG optimization modelling suite from a data file for problems such as, Linear Programming (LP), Quadratic Programming (QP), Second-order Cone Programming (SOCP), or linear Semidefinite Programming (SDP).
.. _e04sa-py2-py-doc:
For full information please refer to the NAG Library document for e04sa
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04saf.html
.. _e04sa-py2-py-parameters:
**Parameters**
**file** : str
The name of the file to be opened.
**ftype** : str
The expected file format of the input file. :math:`\mathrm{ftype}` is case insensitive.
:math:`\mathrm{ftype} = \texttt{'MPS'}` or :math:`\texttt{'M'}`
The input file is in MPS format.
:math:`\mathrm{ftype} = \texttt{'SDPA'}` or :math:`\texttt{'S'}`
The input file is in SDPA format.
**Returns**
**handle** : Handle
Holds a handle to the internal data structure. It **must not** be changed between calls to the NAG optimization modelling suite.
**pinfo** : int, ndarray, shape :math:`\left(100\right)`
Problem sizes and statistics as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of variables (:math:`\textit{nvar}`). |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Number of integer variables. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Number of binary variables. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Number of Lagrange multipliers (dual variables) for the bound constraints and linear constraints (:math:`\textit{nnzu}`). |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Number of Lagrange multipliers (dual variables) for the cone constraints (:math:`\textit{nnzuc}`). |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Number of nonzeros in all Lagrange multipliers for the matrix constraints (:math:`\textit{nnzua}`). |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`20` |Flag indicating the objective function type, :math:`0`: no objective, :math:`1`: linear objective, :math:`2`: quadratic without linear term, :math:`3`: quadratic with linear term, :math:`4`: general nonlinear, :math:`5`: nonlinear least square with dense Jacobian, :math:`6`: nonlinear least square with sparse Jacobian.|
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`21` |Number of residuals defined in a nonlinear least square problem. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`30` |Definition of box constraints, :math:`0`: not defined, :math:`1`: defined. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`31` |Number of linear constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`32` |Number of nonlinear constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`33` |Definition of Hessian for nonlinear objective and constraints, :math:`0`: not defined, :math:`-1`: defined as Lagrangian, :math:`1`: defined as individual Hessians of the objective and the constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`40` |Number of cone constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`41` |Number of matrix inequality constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`42` |Number of quadratic constraints. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|otherwise |Reserved for future use. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`42`--:math:`99`|Reserved for future use. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
:math:`\textit{nvar}`, :math:`\textit{nnzu}`, :math:`\textit{nnzuc}` and :math:`\textit{nnzua}` define the problem size and are required by other functions such as :meth:`handle_solve_lp_ipm`, :meth:`handle_solve_socp_ipm` and :meth:`handle_solve_pennon` in the optimization suite.
.. _e04sa-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ftype} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ftype} = \texttt{'MPS'}`, :math:`\texttt{'M'}`, :math:`\texttt{'SDPA'}` or :math:`\texttt{'S'}`.
(`errno` :math:`2`)
Cannot open file :math:`\langle\mathit{\boldsymbol{value}}\rangle` for reading.
(`errno` :math:`3`)
Cannot close file.
(`errno` :math:`4`)
The input file seems to be empty. No data was read.
(`errno` :math:`5`)
Reading from the file caused an unknown error on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`50`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` to :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not recognized as a valid integer.
(`errno` :math:`51`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` to :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not recognized as a valid real number.
(`errno` :math:`52`)
The token on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` starting at position :math:`\langle\mathit{\boldsymbol{value}}\rangle` was too long and was not recognized.
(`errno` :math:`53`)
An invalid number of variables was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be at least :math:`1`.
(`errno` :math:`54`)
An invalid number of blocks was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be at least :math:`1`.
(`errno` :math:`55`)
An invalid size of the block number :math:`\langle\mathit{\boldsymbol{value}}\rangle` was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The number stated there is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and needs to be nonzero.
(`errno` :math:`56`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying block sizes.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`57`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying the objective function.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`58`)
Not enough data was given on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` specifying nonzero matrix elements.
Expected :math:`\langle\mathit{\boldsymbol{value}}\rangle` tokens but found only :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`59`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given matrix number is out of bounds. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`60`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given block number is out of bounds. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`61`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given row index is out of bounds, it must respect the size of the block. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`62`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The given column index is out of bounds, it must respect the size of the block. Its value :math:`\langle\mathit{\boldsymbol{value}}\rangle` must be between :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle` (inclusive).
(`errno` :math:`63`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The specified nonzero element is not in the upper triangle.
The row index is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`64`)
Invalid structural data found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
The specified element belongs to a diagonal block but is not diagonal.
The row index is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`65`)
An entry in the constraints with :math:`\boldsymbol{matno} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\boldsymbol{blkno} = \langle\mathit{\boldsymbol{value}}\rangle`, row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle` was defined more than once. All entries need to be unique.
(`errno` :math:`66`)
A premature end of the input stream. The part defining the number of variables was not found.
(`errno` :math:`66`)
A premature end of the input stream. The part defining the number of blocks was not found.
(`errno` :math:`66`)
A premature end of the input stream. The part defining the dimensions of the blocks was not found.
(`errno` :math:`66`)
A premature end of the input stream. The part defining the objective function was not found.
(`errno` :math:`66`)
A premature end of the input stream. The part defining the nonzero entries was not found.
(`errno` :math:`100`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
OBJNAME indicator line found after ROWS indicator line.
(`errno` :math:`101`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
COLUMNS indicator line found before ROWS indicator line.
(`errno` :math:`102`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RHS indicator line found before COLUMNS indicator line.
(`errno` :math:`103`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RANGES indicator line found before RHS indicator line.
(`errno` :math:`104`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
BOUNDS indicator line found before COLUMNS indicator line.
(`errno` :math:`105`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before BOUNDS indicator line.
(`errno` :math:`106`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before COLUMNS indicator line.
(`errno` :math:`107`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
CSECTION indicator line found before COLUMNS indicator line.
(`errno` :math:`108`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown indicator line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`109`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Indicator line ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been found more than once in the MPS file.
(`errno` :math:`110`)
End of file found before ENDATA indicator line.
(`errno` :math:`111`)
At least one mandatory section not found in MPS file.
(`errno` :math:`112`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: An illegal line was detected in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
This is neither a comment nor a valid data line.
(`errno` :math:`113`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown inequality key :math:`\langle\mathit{\boldsymbol{value}}\rangle` in ROWS section.
Expected '``N``', '``G``', '``L``' or '``E``'.
(`errno` :math:`114`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Empty ROWS section.
Neither the objective row nor the constraints were defined.
(`errno` :math:`115`)
The supplied name in OBJNAME of the objective row was not found among the free rows in the ROWS section.
(`errno` :math:`116`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal row name.
Row names must consist of printable characters only.
(`errno` :math:`117`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal column name.
Column names must consist of printable characters only.
(`errno` :math:`118`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` has been defined more than once in the ROWS section.
(`errno` :math:`119`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been defined more than once in the COLUMNS section. Column definitions must be continuous. (See `COLUMNS Section <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04saf.html#columnssection>`__).
(`errno` :math:`120`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` marker within ``'INTORG'`` to ``'INTEND'`` range.
(`errno` :math:`121`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTEND'`` marker without previous marker being ``'INTORG'``.
(`errno` :math:`122`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` but not ``'INTEND'`` before the end of the COLUMNS section.
(`errno` :math:`123`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal marker type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Should be either ``'INTORG'`` or ``'INTEND'``.
(`errno` :math:`124`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All row names must be specified in the ROWS section.
(`errno` :math:`125`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All column names must be specified in the COLUMNS section.
(`errno` :math:`126`)
Inconsistent bounds for row ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`126`)
Inconsistent bounds for column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`126`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown bound type :math:`\langle\mathit{\boldsymbol{value}}\rangle` in BOUNDS section.
(`errno` :math:`127`)
More than one nonzero of :math:`A` has row name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' and column name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' in the COLUMNS section.
(`errno` :math:`128`)
Field :math:`\langle\mathit{\boldsymbol{value}}\rangle` did not contain a number (see `MPS input format <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04saf.html#mpsinput>`__).
(`errno` :math:`129`)
Both quadratic objective and cone constraints found, not supported.
(`errno` :math:`130`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Second-order cone :math:`\langle\mathit{\boldsymbol{value}}\rangle` should have at least :math:`2` variables.
(`errno` :math:`130`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Rotated second-order cone :math:`\langle\mathit{\boldsymbol{value}}\rangle` should have at least :math:`3` variables.
(`errno` :math:`131`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown cone type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`132`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal cone name.
Cone names must consist of printable characters only.
(`errno` :math:`133`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Cone name :math:`\langle\mathit{\boldsymbol{value}}\rangle` has been defined more than once.
(`errno` :math:`134`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` has been defined more than once in cone constraint.
.. _e04sa-py2-py-notes:
**Notes**
``handle_read_file`` reads a file in one of the supported file formats and stores the data in a new handle.
The supported formats are MPS and its derivatives (for LP, QP, SOCP) and sparse SDPA for linear semidefinite programming problems.
See `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04saf.html#fcomments>`__ for more details on the supported file formats.
It might be useful to refer to the file format you use to fully understand the possible error messages.
Once the file is successfuly processed, the problem can be solved by a compatible solver from the suite or queried via the printing function :meth:`handle_print`.
Once the problem is not needed any more, :meth:`handle_free` should be called to destroy the handle and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
Also see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#classification>`__ for more details on the standard formulations of the optimization problems (e.g., LP, QP, SOCP and SDP) for better understanding on the notations used in this document.
.. _e04sa-py2-py-references:
**References**
Borchers, B, 1999, `SDPLIB 1.2, A Library of semidefinite programming test problems`, Optimization Methods and Software (11(1)), 683--690, http://euler.nmt.edu/~brian/sdplib/
Fujisawa, K, Kojima, M and Nakata, K, 1998, `SDPA (Semidefinite Programming Algorithm) User's Manual`, Technical Report B-308, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def handle_solve_ssqp(handle, x, objfun=None, objgrd=None, confun=None, congrd=None, hess=None, monit=None, u=None, data=None, io_manager=None):
r"""
``handle_solve_ssqp`` is a solver from the NAG optimization modelling suite for large-scale Nonlinear Programming (NLP) problems based on an active-set Sequential Quadratic Programming (SQP) method, using limited-memory quasi-Newton approximations to the Hessian of the Lagrangian.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_init`, :meth:`handle_opt_get`.
.. _e04sr-py2-py-doc:
For full information please refer to the NAG Library document for e04sr
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html
.. _e04sr-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_ssqp``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` must calculate the value of the nonlinear objective function :math:`f\left(x\right)` at a specified point :math:`x`.
If there is no nonlinear objective (i.e., :meth:`handle_set_nlnobj` was not called), :math:`\mathrm{objfun}` will never be called by ``handle_solve_ssqp`` and :math:`\mathrm{objfun}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
May be used to indicate that the function cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. The algorithm will try to recover if the objective cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 21 or :math:`\mathrm{errno}` = 25.
**objgrd** : None or callable inform = objgrd(x, fdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objgrd}` must calculate the values of the nonlinear objective function gradient :math:`\frac{{\partial f}}{{\partial x}}` at a specified point :math:`x`.
If there is no nonlinear objective (i.e., :meth:`handle_set_nlnobj` was not called), :math:`\mathrm{objgrd}` will never be called by ``handle_solve_ssqp`` and :math:`\mathrm{objgrd}` may be **None**.
If the option :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'ONLYOBJ'}`, then after returning from :math:`\mathrm{objgrd}` the gradient vector is checked for missing entries which you have not supplied.
Missing entries will be estimated using a finite difference method, see option 'SSQP Estimate Derivatives' description for more details.
Note: :math:`\mathrm{objgrd}` should not return floating-point NaN (Not a Number) or infinity values, if your code inadvertently returns any NaNs or infinities, ``handle_solve_ssqp`` is likely to terminate with :math:`\mathrm{errno}` = 21, 24 or 25.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function gradient is to be evaluated.
**fdx** : float, ndarray, shape :math:`\left(\textit{nnzfd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the values of the nonzero elements in the sparse gradient vector of the objective function, in the order specified by :math:`\textit{idxfd}` in a previous call to :meth:`handle_set_nlnobj`. :math:`\mathrm{fdx}[\textit{i}-1]` will store the gradient element :math:`\frac{{\partial f}}{{\partial x_j}}`, where :math:`j = {\textit{idxfd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
If :math:`\mathrm{inform} = 100`, then this is a special `presolve` call.
This call is used to fill-in the constant elements of the objective gradient, e.g., when there are linear variables in the model.
Note that this call may occur with an iterate :math:`\mathrm{x}` that may not be feasible with respect to the linear constraints but it will always be feasible with respect to the variable bounds.
Furthermore, this call **only** requires to return the constant elements of the gradient, :math:`\frac{{\partial f}}{{\partial x_{\textit{i}}}}`, where :math:`f\left(x\right)` is linear in :math:`x_{\textit{i}}`.
If you do not provide these elements, subsequent calls to :math:`\mathrm{objfun}` with :math:`\mathrm{inform}`:math:`= 0` may occur in the vicinity of :math:`\mathrm{x}` with the purpose of estimating the missing derivatives using finite differences.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to inform that the gradient cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 21 or :math:`\mathrm{errno}` = 25.
**confun** : None or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the values of the nonlinear constraints :math:`g\left(x\right)` at a specified point :math:`x`.
If there are no nonlinear constraints then :math:`\mathrm{confun}` will never be called by ``handle_solve_ssqp`` and it may be **None**.
Note: :math:`\mathrm{confun}` should not return floating-point NaN (Not a Number) or infinity values, if your code inadvertently returns any NaNs or infinities, ``handle_solve_ssqp`` is likely to terminate with :math:`\mathrm{errno}` = 21, 24 or 25.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the constraint functions are to be evaluated.
**ncnln** : int
:math:`m_g`, the number of nonlinear constraints, as specified in an earlier call to :meth:`handle_set_nlnconstr`.
**inform** : int
A non-negative value.
If :math:`\mathrm{inform} = 100`, then this is a special `presolve` call.
This call is used to quickly identify if the problem is infeasible, e.g., due to fixed variables.
Note that this call can occur with an iterate :math:`\mathrm{x}` that may not be feasible with respect to the linear constraints but it will always be feasible with respect to the variable bounds.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**gx** : float, array-like, shape :math:`\left(\mathrm{ncnln}\right)`
The :math:`m_g` values of the nonlinear constraint functions at :math:`x`.
**inform** : int
May be used to inform that the constraint values cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 21 or :math:`\mathrm{errno}` = 25.
**congrd** : None or callable inform = congrd(x, gdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{congrd}` must calculate the Jacobian of the nonlinear constraints :math:`\frac{{\partial g}}{{\partial x}}` at a specified point :math:`x`.
If there are no nonlinear constraints, :math:`\mathrm{congrd}` will never be called by ``handle_solve_ssqp`` and :math:`\mathrm{congrd}` may be **None**.
If the option :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'ONLYCON'}`, then after returning from :math:`\mathrm{congrd}` the Jacobian is checked for missing entries which you have not supplied.
Missing entries will be estimated using a finite difference method, see option 'SSQP Estimate Derivatives' description for more details.
Note: :math:`\mathrm{congrd}` should not return floating-point NaN (Not a Number) or infinity values, if your code inadvertently returns any NaNs or infinities, ``handle_solve_ssqp`` is likely to terminate with :math:`\mathrm{errno}` = 21, 24 or 25.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Jacobian of the constraint functions is to be evaluated.
**gdx** : float, ndarray, shape :math:`\left(\textit{nnzgd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by :math:`\textit{irowgd}` and :math:`\textit{icolgd}` in an earlier call to :meth:`handle_set_nlnconstr`. :math:`\mathrm{gdx}[\textit{i}-1]` will be the gradient :math:`\frac{{\partial g_j}}{{\partial x_k}}`, where :math:`j = {\textit{irowgd}}[\textit{i}-1]` and :math:`k = {\textit{icolgd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
If :math:`\mathrm{inform} = 100`, then this is a special `presolve` call.
This call is used to fill-in the constant elements of the Jacobian, e.g., when there are linear variables in the model.
Note that this call may occur with an iterate :math:`\mathrm{x}` that may not be feasible with respect to the linear constraints but it will always be feasible with respect to the variable bounds.
Furthermore, this call **only** requires to return the constant elements of the Jacobian, :math:`\frac{{\partial g_{\textit{j}}}}{{\partial x_{\textit{i}}}}`, where :math:`g_{\textit{j}}\left(x\right)` is linear in :math:`x_{\textit{i}}`.
If the user does not provide these elements, subsequent calls to :math:`\mathrm{confun}` with :math:`\mathrm{inform}`:math:`= 0` may occur in the vicinity of :math:`\mathrm{x}` with the purpose of estimating the missing derivatives using finite differences.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to inform that the Jacobian of the constraints cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 21 or :math:`\mathrm{errno}` = 25.
**hess** : None or callable inform = hess(x, idf, sigma, lamda, hx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{hess}` is reserved for future releases of the NAG Library which will allow calculation of the second derivatives.
It will never be called in the current implementation and :math:`\mathrm{hess}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Hessian functions are to be evaluated.
**idf** : int
Specifies the quantities to be computed in :math:`\mathrm{hx}`.
:math:`\mathrm{idf} = -1`
The values of the Hessian of the Lagrangian will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
:math:`\mathrm{idf} = 0`
The values of the Hessian of the objective function will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
:math:`\mathrm{idf} > 0`
The values of the Hessian of the constraint function with index :math:`\mathrm{idf}` will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
**sigma** : float
If :math:`\mathrm{idf} = -1`, the value of the :math:`\sigma` quantity in the definition of the Hessian of the Lagrangian. Otherwise, :math:`\mathrm{sigma}` should not be referenced.
**lamda** : float, ndarray, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\mathrm{idf} = -1`, the values of the :math:`\lambda_i` quantities in the definition of the Hessian of the Lagrangian. Otherwise, :math:`\mathrm{lamda}` should not be referenced.
**hx** : float, ndarray, shape :math:`\left(\textit{nnzh}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the requested Hessian evaluated at :math:`x`. For each value of :math:`\mathrm{idf}`, the ordering of nonzeros must follow the sparsity structure registered in the :math:`\mathrm{handle}` by earlier calls to :meth:`handle_set_nlnhess` through the arguments :math:`\textit{irowh}` and :math:`\textit{icolh}`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{hess}`. Specifically, if the value is negative the solution of the current problem will terminate immediately with :math:`\mathrm{errno}` = 21 or :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**monit** : None or callable monit(x, u, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization and optionally to terminate the solver early if necessary.
It is invoked at the end of every :math:`i`\ th major iteration where :math:`i` is given by the 'SSQP Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
:math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x^i`, the values of the decision variables :math:`x` at the current iteration.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
If :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds the values of Lagrange multipliers (dual variables, see `Minor Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-minoriterations>`__) for the constraints at the current iteration. See `Lagrangian Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#lagrangianmultipliers>`__ for layout information.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the constraints. See `Lagrangian Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#lagrangianmultipliers>`__ for layout information. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
Optionally provides the initial estimates of Lagrange multipliers, :math:`\left(z_0,\pi_0\right)`. These values are only referenced if a `warm start` is requested (`Warm Starting <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#warmstart>`__). If there are no initial estimates available, then set to zero.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**u** : None or float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final value of Lagrange multipliers :math:`\left(z^*,\pi^*\right)`, see `Minor Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-minoriterations>`__.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Constraint violation (primal infeasibility), see `Accuracy <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#accuracy>`__. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Dual infeasibility. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`--:math:`6` |Reserved for future use. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Step size for the primal variables, see **Iteration log** in `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#fcomments1>`__. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Estimate of the condition number of :math:`R^\mathrm{T}R`, itself an estimate of :math:`Z^\mathrm{T}HZ`, the reduced Hessian of the Lagrangian. See `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#fcomments1>`__.|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Largest nonlinear constraint violation. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Largest nonlinear constraint violation, normalized using :math:`1+\left\lVert x_k\right\rVert`, with :math:`x_k` the latest iterate. See `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn12>`__. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Sum of the infeasibilities of constraints that lie outside one of their bounds by more than the option 'SSQP Minor Feasibility Tol' `before the solution is unscaled`. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |+---------+---------------------------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`j`|Description | |
| |+=========+=============================================================================================================================================+ |
| ||0 |none of the conditions are satisfied; | |
| |+---------+---------------------------------------------------------------------------------------------------------------------------------------------+ |
| ||1 |condition on primal infeasibility satisfied, :math:`\mathrm{rinfo}`:math:`\left(2\right)` is within 'SSQP Major Feasibility Tol' tolerance; | |
| |+---------+---------------------------------------------------------------------------------------------------------------------------------------------+ |
| ||2 |condition on dual infeasibility satisfied, :math:`\mathrm{rinfo}`:math:`\left(3\right)` is within 'SSQP Minor Feasibility Tol' tolerance; and| |
| |+---------+---------------------------------------------------------------------------------------------------------------------------------------------+ |
| ||3 |both conditions are satisfied. | |
| |+---------+---------------------------------------------------------------------------------------------------------------------------------------------+ |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13` |The size of the penalty parameter :math:`D` of `(8) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn8>`__. See `The Merit Function <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-meritfunction>`__. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Flag indicating whether the current iterate is feasible with respect to the linear constraints, if any. A nonzero value indicates feasibility. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15`--:math:`99`|Reserved for future use. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of major iterations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of minor iterations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Number of minor iterations performed on the last major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Number of Hessian evaluations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Number of gradient evaluations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of infeasibilities of constraints that lie outside one of their bounds by more than the option 'SSQP Minor Feasibility Tol' `before the solution is unscaled`. If any `linear` constraints are infeasible, it contains the number of variables and linear constraints lying outside their upper or lower bounds. The nonlinear constraints are not evaluated. Otherwise, it contains the number of components of :math:`g\left(x\right)` lying outside their bounds by more than the option 'SSQP Minor Feasibility Tol'. Again this is `before the solution is unscaled`.|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Total time spent in presolver (including user-supplied function calls). |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Total time spent in solver (including user-supplied function calls). |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent evaluating constraint function. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent evaluating constraint Jacobian. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Total time spent evaluating objective function. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Total time spent evaluating objective gradient. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Total time spent evaluating Hessian. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13`--:math:`17`|Reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`18` |Number of function evaluations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`19` |Number of constraint function evaluations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`20` |Number of constraint Jacobian evaluations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`21`--:math:`23`|Reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`24` |Number of superbasic variables. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`25` |Number of column swaps between :math:`B` and :math:`S`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`26` |Number of elements in the :math:`LU` factorization of :math:`B`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`27`--:math:`28`|Reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`29` |Number of slack variables. See `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-constraintsandslacks>`__. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`30`--:math:`99`|Reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. _e04sr-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options;
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` and additionally inner QP iteration log is reported every 'SSQP Print Frequency' |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each minor and major iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the final values of the solution vector are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'SSQP Crash Option'** : str
Default :math:`\text{} = \texttt{'TRIPLE-PASS'}`
Except on restarts :math:`\text{‘SSQP Start Type'} = \texttt{'COLD'}`, a crash procedure is used to select an initial basis :math:`\mathcal{B}` (see `(7) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn7>`__ in `Minor Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-minoriterations>`__) from certain rows and columns of the constraint matrix :math:`\begin{pmatrix}A&{-I}\end{pmatrix}`.
The 'SSQP Crash Option' determines which rows and columns of :math:`A` (see `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn5>`__ in `Major Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-majoriterations>`__) are eligible initially, and how many times crash is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+-------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`a` |Meaning |
+=============+=============================================================================================================================================================================================================================================================================================================================================================================================================+
|'ONLY-SLACKS'|The initial basis contains only slack variables: :math:`\mathcal{B} = I` (see `(7) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn7>`__ in `Minor Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-minoriterations>`__). |
+-------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|'SINGLE-PASS'|crash is called once, looking for a triangular basis in all rows and columns of :math:`A`. |
+-------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|'DOUBLE-PASS'|crash is called twice (if there are nonlinear constraints). The first call looks for a triangular basis in linear rows, and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and crash is called again to find a triangular basis in the nonlinear rows (retaining the current basis for linear rows).|
+-------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|'TRIPLE-PASS'|crash is called up to three times (if there are nonlinear constraints). The first two calls treat linear equalities and linear inequalities separately. As before, the last call treats nonlinear rows before the first major iteration. |
+-------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘SSQP Crash Option'} \neq \texttt{'ONLY-SLACKS'}`, certain slacks on inequality rows are selected for the basis first, furthermore if :math:`\text{‘SSQP Crash Option'} = \texttt{'DOUBLE-PASS'}` or :math:`\texttt{'TRIPLE-PASS'}`, numerical values are used to exclude slacks that are close to a bound. crash then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to `pivot` on a particular row if the column contains a suitably large element in a row that has not yet been assigned (the pivot elements ultimately form the diagonals of the triangular basis.
For remaining unassigned rows, slack variables are inserted to complete the basis.
Note: Each crash pass is counted as a minor iteration..
Constraint: :math:`\text{‘SSQP Crash Option'} = \texttt{'ONLY-SLACKS'}`, :math:`\texttt{'SINGLE-PASS'}`, :math:`\texttt{'DOUBLE-PASS'}` or :math:`\texttt{'TRIPLE-PASS'}`.
**'SSQP Crash Tolerance'** : float
Default :math:`= 0.1`
The 'SSQP Crash Tolerance' :math:`r` allows the starting procedure crash to ignore certain `small` nonzeros in each column of :math:`A` (see `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn5>`__ in `Major Iterations <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-majoriterations>`__).
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros :math:`a_{{ij}}` in the column are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`.
To be meaningful, :math:`r` needs to be in the range :math:`0\leq r\leq 1`.)
When :math:`r > 0`, the basis obtained by crash may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks (columns of :math:`{-I}`).
A feasible solution may be reached sooner on some problems.
When :math:`r = {-1}` (or :math:`r < 0`) the solver will automatically choose an appropriate tolerance.
Constraint: 'SSQP Crash Tolerance':math:`\geq 0` or 'SSQP Crash Tolerance':math:`= -1.0`.
**'SSQP Estimate Derivatives'** : str
Default :math:`\text{} = \texttt{'NO'}`
This option specifies which nonlinear function gradients (objective and nonlinear constraints) are known analytically and will be supplied to the solver by you functions :math:`\mathrm{objgrd}` and :math:`\mathrm{congrd}`.
This option has the following alternatives:
.. rst-class:: nag-rules-none nag-align-left
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`a` |Meaning |
+==============+==============================================================================================================================================+
|'NO' (default)|Indicates to the solver that you are providing **full derivative information**, that is, **all objective and constraint gradients** are known.|
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+
|'ONLYOBJ' |Indicates that some or all components of the objective gradient will be missing but **all constraint gradients** are known. |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+
|'ONLYCON' |Conversely to 'ONLYOBJ', this indicated that some or all of the constraint gradients will be missing but the **objective gradient** is known. |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+
|'YES' |Indicates that the user is not providing derivatives. |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+
The value :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'NO'}` should be used whenever possible.
It is the most reliable and will usually be the most efficient.
If :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'ONLYOBJ'}`, ``handle_solve_ssqp`` will estimate the missing components of the objective gradient, using finite differences.
This may simplify the coding of function :math:`\mathrm{objgrd}`.
However, it could increase the total run-time substantially (since a special call to :math:`\mathrm{objfun}` is required for each missing element), and there is less assurance that an acceptable solution will be located.
If the nonlinear variables are not well scaled, it may be necessary to specify a nonstandard difference interval changing 'SSQP Finite Diff Ctrl Interval' or 'SSQP Finite Diff Interval'.
Conversely, if :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'ONLYCON'}`, ``handle_solve_ssqp`` will estimate missing elements of the Jacobian.
For **each column** of the Jacobian, one call to :math:`\mathrm{confun}` is needed to estimate all missing elements in that column, if any.
At times, central differences are used rather than forward differences.
Twice as many calls to :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` are then needed. (This is not under your control.)
Note: If :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'NO'}`, :math:`\texttt{'ONLYCON'}` or :math:`\texttt{'ONLYOBJ'}` and the user does not provide one or more of the required gradient (or Jacobian) elements then these will be assumed to be constant, this might not be the desired behaviour. Under this circumstance, ``handle_solve_ssqp`` will inform the user with a message.
Constraint: :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'NO'}`, :math:`\texttt{'ONLYOBJ'}`, :math:`\texttt{'ONLYCON'}` or :math:`\texttt{'YES'}`.
**'SSQP Finite Diff Ctrl Interval'** : float
Default :math:`≈1.7\times \epsilon^{0.27}`
When :math:`\text{‘SSQP Estimate Derivatives'} = \texttt{'YES'}`, :math:`\texttt{'ONLYOBJ'}` or :math:`\texttt{'ONLYCON'}`, the central-difference interval :math:`r` is used near an optimal solution to obtain more accurate (but more expensive) estimates of gradients.
Twice as many function evaluations are required compared to forward differencing.
The interval used for the :math:`j`\ th variable is :math:`h_j = r\left(1+\left\lvert x_j\right\rvert \right)`.
The resulting derivative estimates should be accurate to :math:`\mathrm{O}\left(r^2\right)`, unless the functions are badly scaled.
Constraint: :math:`0 < r < 0.1`.
**'SSQP Finite Diff Interval'** : float
Default :math:`≈1.7\times \epsilon^{0.4}`
This alters the interval :math:`r` used to estimate gradients by forward differences.
It does so in the following circumstances:
- in the 'cheap' phase of verifying the problem derivatives, when :math:`\text{‘Verify Derivatives'} = \texttt{'CHEAP'}`;
- for verifying the problem derivatives (:math:`\text{‘Verify Derivatives'} = \texttt{'YES'}`);
- for estimating missing derivatives.
In all cases, a derivative with respect to :math:`x_j` is estimated by perturbing that component of :math:`x` to the value :math:`x_j+r\left(1+\left\lvert x_j\right\rvert \right)`, and then evaluating the objective or constraint functions at the perturbed point.
The resulting gradient estimates should be accurate to :math:`\mathrm{O}\left(r\right)` unless the functions are badly scaled.
Judicious alteration of :math:`r` may sometimes lead to greater accuracy.
Constraint: :math:`0 < r < 0.1`.
**'SSQP Function Precision'** : float
Default :math:`≈1.7\times \epsilon^{0.8}`
The `relative function precision` :math:`\epsilon_r` is intended to be a measure of the relative accuracy with which the nonlinear functions can be computed.
For example, if :math:`f\left(x\right)` is computed as :math:`1000.56789` for some relevant :math:`x` and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-6`.
Ideally, the functions :math:`f\left(x\right)` and :math:`g_i\left(x\right)` should have a magnitude of order :math:`1`.
If all functions are substantially `less` than :math:`1` in magnitude, :math:`\epsilon_r` should be the `absolute` precision.
For example, if :math:`f\left(x\right) = 1.23456789e-4` at some point and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-10`.)
The default value of :math:`\epsilon_r` is appropriate for simple analytic functions.
In some cases the function values will be the result of extensive computation, possibly involving a costly iterative procedure that can provide few digits of precision.
Specifying an appropriate 'SSQP Function Precision' may lead to savings, by allowing the linesearch procedure to terminate when the difference between function values along the search direction becomes as small as the absolute error in the values.
Constraint: :math:`0 < \epsilon_r < 0.1`.
**'SSQP Hessian'** : str
Default :math:`\text{} = \texttt{'AUTO'}`
This option selects the method for storing and updating the approximate Hessian. (``handle_solve_ssqp`` uses a quasi-Newton approximation to the Hessian of the Lagrangian.
A BFGS update is applied after each major iteration.)
If :math:`\text{‘SSQP Hessian'} = \texttt{'FULL-MEMORY'}` is specified, the approximate Hessian is treated as a dense matrix and the BFGS updates are applied explicitly.
This option is most efficient when the number of variables :math:`n` is not too large (say, less than :math:`75`).
In this case, the storage requirement is fixed and one can expect :math:`1`-step Q-superlinear convergence to the solution.
:math:`\text{‘SSQP Hessian'} = \texttt{'LIMITED-MEMORY'}` should be used on problems where :math:`n` is very large.
In this case, a limited-memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times.
Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal, see option 'SSQP Hessian Updates'.
If :math:`\text{‘SSQP Hessian'} = \texttt{'AUTO'}` is specified, then the quasi-Newton method used is chosen based on a heuristic involving the number of nonlinear variables (see `Note regarding nonlinear functions <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#nonlinearfunctions>`__) in the problem.
Constraint: :math:`\text{‘SSQP Hessian'} = \texttt{'AUTO'}`, :math:`\texttt{'FULL-MEMORY'}` or :math:`\texttt{'LIMITED-MEMORY'}`.
**'SSQP Hessian Updates'** : int
Default :math:`\text{} = 10`
If option :math:`\text{‘SSQP Hessian'} = \texttt{'LIMITED-MEMORY'}` is in effect and :math:`i` BFGS updates have already been carried out, all but the diagonal elements of the accumulated updates are discarded and the updating process starts again.
Broadly speaking, the more updates stored, the better the quality of the approximate Hessian.
However, the more vectors stored, the greater the cost of each QP iteration.
The default value is likely to give a robust algorithm without significant expense, but faster convergence can sometimes be obtained with significantly fewer updates (e.g., :math:`i = 5`).
Constraint: :math:`0 < \text{‘SSQP Hessian Updates'}`.
**'SSQP Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(10000, {10\mathrm{max}\left(\textit{nvar}, m\right)}\right)`
The value of :math:`i` specifies the maximum number of minor iterations allowed (i.e., iterations of the simplex method or the QP algorithm), summed over all major iterations. (See also the description of the option 'SSQP Minor Iteration Limit'.) For the default value, :math:`m` is the number of linear, quadratic and nonlinear constraints defined in the model.
Constraint: :math:`0 < \text{‘SSQP Iteration Limit'}`.
**'SSQP Major Feasibility Tol'** : float
Default :math:`≈\epsilon^{0.38}`
This tolerance, :math:`r`, specifies how accurately the nonlinear constraints should be satisfied.
The default value is appropriate when the linear and nonlinear constraints contain data to about that accuracy.
Let :math:`v_{\mathrm{max}}` be the maximum nonlinear constraint violation, normalized by the size of the solution, which is required to satisfy
.. math::
v_{\mathrm{max}} = \mathrm{max}_iv_i\text{ }/\text{ }\left\lVert x\right\rVert \leq r\text{,}
where :math:`v_{\textit{i}}` is the violation of the :math:`\textit{i}`\ th nonlinear constraint, for :math:`\textit{i} = 1,2,\ldots,\mathrm{ncnln}`.
In the major iteration log (see `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#fcomments1>`__), :math:`v_{\mathrm{max}}` appears as the quantity labelled 'Feasible'.
If some of the problem functions are known to be of low accuracy, a larger 'SSQP Major Feasibility Tol' may be appropriate.
Constraint: :math:`0 < r\leq 0.1`.
**'SSQP Major Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(1000, {3\mathrm{max}\left(\textit{nvar}, m\right)}\right)`
This is the maximum number of major iterations allowed.
It is intended to guard against an excessive number of linearizations of the constraints.
For the default value, :math:`m` is the number of linear, quadratic and nonlinear constraints defined in the model.
Constraint: :math:`0 < \text{‘SSQP Major Iteration Limit'}`.
**'SSQP Major Optimality Tol'** : float
Default :math:`≈\epsilon^{0.38}`
This tolerance, :math:`r`, specifies the final accuracy of the dual variables.
On successful termination, ``handle_solve_ssqp`` will have computed a solution :math:`\left(x, s, \pi \right)` such that
.. math::
c_{\mathrm{max}} = \mathrm{max}_jc_j\text{ }/\text{ }\left\lVert \pi \right\rVert \leq r\text{,}
where :math:`c_{\textit{j}}` is an estimate of the complementarity slackness for variable :math:`\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n+m`.
Here :math:`m` represents the number of constraints.
The values :math:`c_i` are computed from the final QP solution using the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j` (where :math:`g_j` is the :math:`j`\ th component of the objective gradient, :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`, and :math:`\pi` is the set of QP dual variables):
.. math::
c_j = \left\{\begin{array}{cc} d_j \mathrm{min}\left({x_j-l_j}, 1\right) & \text{if } d_j\geq 0 \text{;} \\ {-d_j} \mathrm{min}\left({u_j-x_j}, 1\right) & \text{if } d_j < 0 \text{.} \end{array}\right)
In the major iteration log, :math:`c_{\mathrm{max}}` appears as the quantity labelled 'Optimal', See section `Major Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#majoriteration>`__.
Constraint: :math:`0 < r\leq 0.1`.
**'SSQP Minor Feasibility Tol'** : float
Default :math:`≈\epsilon^{0.38}`
``handle_solve_ssqp`` tries to ensure that all variables eventually satisfy their upper and lower bounds to within this tolerance, :math:`r`.
This includes slack variables.
Hence, general linear constraints should also be satisfied to within :math:`r`.
Feasibility with respect to nonlinear constraints is judged by the option 'SSQP Major Feasibility Tol' (not by 'SSQP Minor Feasibility Tol').
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
If :math:`\mathrm{rinfo}`:math:`\left(12\right)` is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Nonlinear functions will be evaluated only at points that satisfy the bounds and linear constraints, except perhaps in presolve where the solver may require to evaluate :math:`\mathrm{confun}`, :math:`\mathrm{objgrd}` or :math:`\mathrm{congrd}` at a bound-feasible point.
If there are regions where a function is undefined, every attempt should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both variables.
If :math:`r = 1.0e-6`, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious.
In general, keep :math:`x` as far away from singularities as possible.)
If :math:`\text{‘SSQP Scale Option'} = \texttt{'ALL'}` or :math:`\texttt{'VARS AND LC'}`, feasibility is defined in terms of the `scaled` problem (since it is then more likely to be meaningful).
In reality, ``handle_solve_ssqp`` uses :math:`r` as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible. ``handle_solve_ssqp`` then activates `elastic mode` for the rest of the solving process (with only the linearized nonlinear constraints defined to be `elastic`).
For further details see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#ad-treatment>`__.
Constraint: :math:`0 < r\leq 0.1`.
**'SSQP Minor Iteration Limit'** : int
Default :math:`\text{} = 500`
If the number of minor iterations for the optimality phase of the QP subproblem exceeds :math:`i`, then all nonbasic QP variables that have not yet moved are frozen at their current values and the reduced QP is solved to optimality.
Note that more than :math:`i` minor iterations may be necessary to solve the reduced QP to optimality.
These extra iterations are necessary to ensure that the terminated point gives a suitable direction for the linesearch.
In the major iteration log (see `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#fcomments1>`__) the flag t at the end of a line indicates that the corresponding QP was artificially terminated using the limit :math:`i`.
Compare with the option 'SSQP Iteration Limit', which defines an independent `absolute` limit on the `total` number of minor iterations (summed over all QP subproblems).
Constraint: :math:`0 < \text{‘SSQP Minor Iteration Limit'}`.
**'SSQP Monitor Frequency'** : int
Default :math:`= 0`
This argument specifies the frequency on which to call the monitor function :math:`\mathrm{monit}`.
If zero, the monitor function will not be called.
Constraint: :math:`\text{‘SSQP Monitor Frequency'} \geq 0`.
**'SSQP Penalty Parameter'** : float
Default :math:`\text{} = 0.0`
Defines the initial value of the penalty parameter and sets :math:`D = \text{diag}\left(r\right)` in `(8) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn8>`__.
Constraint: :math:`0.0\leq \text{‘SSQP Penalty Parameter'}`.
**'SSQP Print Frequency'** : int
Default :math:`\text{} = 1`
Defines how frequently to print log information for the inner iterations.
Only relevant if 'Print Level':math:`\geq 3` (with 'Print File':math:`\neq -1`) or 'Monitoring Level':math:`\geq 3` (with 'Monitoring File':math:`\neq -1`)
Constraint: :math:`0\leq \text{‘SSQP Print Frequency'}`.
**'SSQP Scale Option'** : str
Default :math:`\text{} = \texttt{'NONE'}`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`a` |Meaning |
+=============+================================================================================================================================================================================================================================================================================================================================================================+
|'NONE' |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|'VARS AND LC'|The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|'ALL' |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`\text{‘SSQP Scale Option'} = \texttt{'NONE'}`, :math:`\texttt{'VARS AND LC'}` or :math:`\texttt{'ALL'}`.
**'SSQP Scale Print'** : str
Default :math:`\text{} = \texttt{'NO'}`
Defines whether to print the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)`.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)` and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/r\left(j-n\right)` if :math:`j > n`.
Constraint: :math:`\text{‘SSQP Scale Print'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'SSQP Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
This argument affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
Constraint: :math:`\epsilon \leq \text{‘SSQP Scale Tolerance'}`.
**'SSQP Start Type'** : str
Default :math:`\text{} = \texttt{'COLD'}`
Defines whether to perform a `cold` or `warm` start.
If warm start data is not provided or is considered to have an unexpected size or content, then the solver will revert to perform a cold start on the problem.
See `Warm Starting <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#warmstart>`__ on how to correctly warm start a problem.
Constraint: :math:`\text{‘SSQP Start Type'} = \texttt{'COLD'}` or :math:`\texttt{'WARM'}`.
**'SSQP Superbasics Limit'** : int
Default :math:`\text{} = -1`
This option places a limit on the storage allocated for superbasic variables.
Ideally, :math:`i` should be set slightly larger than the 'number of degrees of freedom' expected at an optimal solution.
For nonlinear problems, the number of degrees of freedom is often called the 'number of independent variables'.
Normally, :math:`i` need not be greater than :math:`n+1`.
If :math:`i = -1` (the default), then ``handle_solve_ssqp`` automatically sets the limit to the number of nonlinear variables.
For many problems, :math:`i` may be considerably smaller than :math:`n`.
This will save storage if :math:`n` is very large.
If the limit set is too low then the solver will terminate with :math:`\mathrm{errno}` = 27 error message.
Constraint: :math:`-1\leq \text{‘SSQP Superbasics Limit'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on the timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
This argument specifies a limit in seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23 error message.
Constraint: :math:`\text{‘Time Limit'} > 0`.
**'Verify Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument specifies whether the function should perform numerical checks on the consistency of the user-supplied gradient functions :math:`\mathrm{objgrd}` and :math:`\mathrm{congrd}`.
If any discrepancies are found, :math:`\mathrm{errno}` = 26 is returned.
It is recommended that such checks are enabled when first developing the formulation of the problem, however, the verification process results in a significant increase in the number of the function evaluations and thus it shouldn't be used in production code.
.. rst-class:: nag-rules-none nag-align-left
+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|a |Meaning |
+========================+=============================================================================================================================================================================================================================+
|:math:`\texttt{'CHEAP'}`|Only a 'cheap' test will be performed. |
+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\texttt{'YES'}` |Individual columns of the problem Jacobian, as well as individual gradient elements, will be checked (with a more reliable test). A key of the form OK or Bad? indicates whether or not each component appears to be correct.|
+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\texttt{'NO'}` |Derivative checking is disabled. |
+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
:math:`\text{‘Verify Derivatives'} = \texttt{'YES'}` should be specified whenever a new :math:`\mathrm{objgrd}` or :math:`\mathrm{congrd}` is being developed.
Constraint: :math:`\text{‘Verify Derivatives'} = \texttt{'CHEAP'}`, :math:`\texttt{'YES'}` or :math:`\texttt{'NO'}`.
.. _e04sr-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle` or :math:`0`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: no constraints present, so :math:`\textit{nnzu}` must be :math:`0`.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objgrd}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{confun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{congrd}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{hess}` function.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`50`)
Problem was solved to an acceptable level, full accuracy was not achieved.
(`errno` :math:`51`)
The problem was found to be infeasible during preprocessing.
(`errno` :math:`53`)
The problem seems to be infeasible, the algorithm was stopped.
(`errno` :math:`54`)
The problem seems to be unbounded and the algorithm was stopped.
**NagAlgorithmicMajorWarning**
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid value detected in user function and recovery failed.
(`errno` :math:`26`)
User-provided derivatives are likely to be incorrect.
(`errno` :math:`27`)
Maximum number of superbasic variables reached. Increase value of the option 'SSQP Superbasics Limit'.
(`errno` :math:`90`)
Memory limit is too small.
.. _e04sr-py2-py-notes:
**Notes**
``handle_solve_ssqp`` is designed to minimize (maximize) a linear or nonlinear function possibly subject to a set of constraints including bounds on the variables and sparse linear and nonlinear constraints.
It is suitable for large-scale general nonlinear programming (NLP) problems, as well as for linearly constrained optimization.
It is particularly efficient if only some of the variables enter nonlinearly, or there are relatively few degrees of freedom at a solution (i.e., many constraints are active).
However, there is no limit on the number of degrees of freedom.
``handle_solve_ssqp`` solves a nonlinear programming problem in the following form
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\\& \frac{1}{2} x^\mathrm{T}Q_ix + r_i^\mathrm{T}x + s_i\leq 0 \text{, }\quad 1\leq i\leq m_Q\text{,} & \\&l_B\leq Bx\leq u_B\text{,}&\\& l_x\leq x\leq u_x\text{,} &\end{array}
where
:math:`n` is the number of the decision variables,
:math:`m_g` is the number of the nonlinear constraints and :math:`g\left(x\right)`, :math:`l_g` and :math:`u_g` are :math:`m_g`-dimensional vectors,
:math:`m_Q` is the number of the quadratic constraints,
:math:`m_B` is the number of the linear constraints and :math:`B` is an :math:`m_B\times n` matrix, :math:`l_B` and :math:`u_B` are :math:`m_B`-dimensional vectors,
there are :math:`n` box constraints and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors.
``handle_solve_ssqp`` serves as a solver for problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized, typically by calling :meth:`handle_init`.
Then some of the components of the model need to be defined by calling one or more of the following routines based on the component type:
the objective function :math:`f\left(x\right)`: linear (:meth:`handle_set_linobj` or :meth:`handle_set_quadobj`), quadratic (:meth:`handle_set_quadobj`, :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac`) or nonlinear (:meth:`handle_set_nlnobj`),
bound constraints on the variables :math:`l_x,u_x` (:meth:`handle_set_simplebounds`),
one or more blocks of linear constraints :math:`l_B,B,u_B` (:meth:`handle_set_linconstr`),
quadratic constraints (:meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac` for quadratics in a factorized form),
general nonlinear constraints :math:`l_g,g\left(x\right),u_g` (:meth:`handle_set_nlnconstr`).
Once the problem is fully described, the handle may be passed to the solver ``handle_solve_ssqp``.
When the handle is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
There are other ways that the problem model can be defined and/or updated, see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The algorithm behaviour can be modified by various options (see :ref:`Other Parameters <e04sr-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle and a call to the solver.
Once the solver has finished, options or the model may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
Option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
The option 'Task' may be used to switch the problem to maximization.
In certain situations the solver can be sped up by `warm starting`, see `Warm Starting <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#warmstart>`__ and option 'SSQP Start Type' for details.
Several options may have a `significant impact` on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended that you experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#algdetails>`__ and :ref:`Other Parameters <e04sr-py2-py-other_params>` for further details.
**Note regarding nonlinear functions**
In the presence of a nonlinear objective function or constraint function, the structure (sparsity pattern) of these functions are defined by :meth:`handle_set_nlnobj` and :meth:`handle_set_nlnconstr` respectively; the function values and the first derivatives (gradients) are computed at requested points by the corresponding supplied functions :math:`\mathrm{objfun}`, :math:`\mathrm{objgrd}`, :math:`\mathrm{confun}` and :math:`\mathrm{congrd}`.
Ideally, the first derivatives should be known and coded but ``handle_solve_ssqp`` estimates any missing ones by finite differences, see 'SSQP Estimate Derivatives'.
The solver can strongly benefit from the correct identification of `linear variables`; these are the variables which appear only in linear expressions in the whole optimization model.
The solver cannot autodetect if any variable present in nonlinear constraints or the nonlinear objective is linear thus there is a mechanism to declare the variable as linear by :meth:`handle_set_property` and it is highly recommended to do so whenever possible.
See `Linearity of the Variables for handle_set_property <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04rcf.html#varlinearity>`__ for a detailed explanation and examples.
The main advantage is that linear variables have constant first partial derivatives and must be zero in second derivatives, thus the internal approximation of the Hessian doesn't need to take them into account which speeds up the solver.
**Alternative solvers**
``handle_solve_ssqp`` is a generic solver which can solve a great variety of problems, however, a dedicated solver might work better in the following cases.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#dtree>`__.
If :math:`f\left(x\right)` is linear, :math:`m_Q = 0`, and :math:`g\left(x\right)` is absent, `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04srf.html#eqn1>`__ is a linear program (LP) and :meth:`handle_solve_lp_ipm` is recommended.
If the problem is a convex quadratic or quadratically constrained problem, try :meth:`handle_solve_socp_ipm`.
If the problem is unconstrained or only box-constrained, see :meth:`handle_solve_bounds_foas`, or :meth:`handle_solve_dfno` if the problem is small and derivatives cannot be provided.
A general nonlinear problem lacking the sparse structure (which is typically the case for problems with a relatively small number of variables and constraints) might be better handled by :meth:`nlp1_solve` or :meth:`nlp2_solve`.
An alternative solver for the same class of problems but based on a different algorithm (interior point method) is :meth:`handle_solve_ipopt`, see `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#background44>`__ for a discussion of the advantages of each approach.
.. _e04sr-py2-py-references:
**References**
Dantzig, G B, 1963, `Linear Programming and Extensions`, Princeton University Press
Eldersveld, S K, 1991, `Large-scale sequential quadratic programming algorithms`, PhD Thesis, Department of Operations Research, Stanford University, Stanford
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E, Murray, W and Saunders, M A, 2002, `SNOPT: An SQP Algorithm for Large-scale Constrained Optimization` (12), 979--1006, SIAM J. Optim.
Gill, P E, Murray, W and Saunders, M A, 2015, `Users' guide for SNOPT 7.5: Software for large-scale linear nonlinear programming`, Report SNOPT, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sndoc7.pdf
Gill, P E, Murray, W and Saunders, M A, 2016, `Users' guide for SQOPT 7.5: Software for large-scale linear and quadratic programming`, Report SQOPT, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sqdoc7.pdf
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1987, `Maintaining` :math:`LU` `factors of a general sparse matrix`, Linear Algebra and its Applics. (88/89), 239--270
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1992, `Some theoretical properties of an augmented Lagrangian merit function`, Advances in Optimization and Parallel Computing, (ed P M Pardalos), 101--128, North Holland
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Murtagh, B A and Saunders, M A, 1978, `Large-scale linearly constrained optimization`, 14, 41--72, Math. Programming
Murtagh, B A and Saunders, M A, 1982, `A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints`, Math. Program. Stud. (16), 84--118
Murtagh, B A and Saunders, M A, 1995, `MINOS 5.4 users' guide`, Report SOL 83-20R, Department of Operations Research, Stanford University
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_ssqp_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_ipopt(handle, x, objfun=None, objgrd=None, confun=None, congrd=None, hess=None, monit=None, u=None, data=None, io_manager=None):
r"""
``handle_solve_ipopt`` is a solver from the NAG optimization modelling suite for constrained large-scale Nonlinear Programming (NLP) problems.
It is an interior point method optimization solver based on the IPOPT software package.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04st-py2-py-doc:
For full information please refer to the NAG Library document for e04st
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html
.. _e04st-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_ipopt``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x^0`, the initial estimates of the variables :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` must calculate the value of the nonlinear objective function :math:`f\left(x\right)` at a specified value of the :math:`n`-element vector of :math:`x` variables.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac` was called to define a linear or quadratic objective function), :math:`\mathrm{objfun}` will never be called by ``handle_solve_ipopt`` and :math:`\mathrm{objfun}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{objfun}`. Specifically, if the value is negative, then the value of :math:`\mathrm{fx}` will be discarded and the solver will either attempt to find a different trial point or terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, the solver will proceed normally.
**objgrd** : None or callable inform = objgrd(x, fdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objgrd}` must calculate the values of the nonlinear objective function gradients :math:`\frac{{\partial f}}{{\partial x}}` at a specified value of the :math:`n`-element vector of :math:`x` variables.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac` was called to define a linear or quadratic objective function), :math:`\mathrm{objgrd}` will never be called by ``handle_solve_ipopt`` and :math:`\mathrm{objgrd}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function gradient is to be evaluated.
**fdx** : float, ndarray, shape :math:`\left(\textit{nnzfd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the values of the nonzero elements in the sparse gradient vector of the objective function, in the order specified by :math:`\textit{idxfd}` in a previous call to :meth:`handle_set_nlnobj`. :math:`\mathrm{fdx}[\textit{i}-1]` will be the gradient :math:`\frac{{\partial f}}{{\partial x_{{\textit{idxfd}}[\textit{i}-1]}}}`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{objgrd}`. Specifically, if the value is negative then the value of :math:`\mathrm{fdx}` will be discarded and the solver will either attempt to find a different trial point or will terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**confun** : None or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the values of the :math:`m_g`-element vector :math:`g_i\left(x\right)` of nonlinear constraint functions at a specified value of the :math:`n`-element vector of :math:`x` variables.
If there are no nonlinear constraints then :math:`\mathrm{confun}` will never be called by ``handle_solve_ipopt`` and it may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the constraint functions are to be evaluated.
**ncnln** : int
:math:`m_g`, the number of nonlinear constraints, as specified in an earlier call to :meth:`handle_set_nlnconstr`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**gx** : float, array-like, shape :math:`\left(\mathrm{ncnln}\right)`
The :math:`m_g` values of the nonlinear constraint functions at :math:`x`.
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{confun}`. Specifically, if the value is negative, then the value of :math:`\mathrm{gx}` will be discarded and the solver will either attempt to find a different trial point or terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, the solver will proceed normally.
**congrd** : None or callable inform = congrd(x, gdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{congrd}` must calculate the nonzero values of the sparse Jacobian of the nonlinear constraint functions :math:`\frac{{\partial g_i}}{{\partial x}}` at a specified value of the :math:`n`-element vector of :math:`x` variables.
If there are no nonlinear constraints, :math:`\mathrm{congrd}` will never be called by ``handle_solve_ipopt`` and :math:`\mathrm{congrd}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Jacobian of the constraint functions is to be evaluated.
**gdx** : float, ndarray, shape :math:`\left(\textit{nnzgd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by :math:`\textit{irowgd}` and :math:`\textit{icolgd}` in an earlier call to :meth:`handle_set_nlnconstr`. :math:`\mathrm{gdx}[\textit{i}-1]` will be the gradient :math:`\frac{{\partial g_j}}{{\partial x_k}}`, where :math:`j = {\textit{irowgd}}[\textit{i}-1]` and :math:`k = {\textit{icolgd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{congrd}`. Specifically, if the value is negative the solution of the current problem will terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**hess** : None or callable inform = hess(x, idf, sigma, lamda, hx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{hess}` must calculate the nonzero values of one of a set of second derivative quantities:
the Hessian of the Lagrangian function :math:`\sigma \nabla^2f\left(x\right)+\sum_{{i = 1}}^{m_g}\lambda_i\nabla^2g_i\left(x\right)`,
the Hessian of the objective function :math:`\nabla^2f\left(x\right)`,
the Hessian of the :math:`i`\ th constraint function :math:`\nabla^2g_i\left(x\right)`.
The value of argument :math:`\mathrm{idf}` determines which one of these is to be computed and this, in turn, is determined by earlier calls to :meth:`handle_set_nlnhess`, when the nonzero sparsity structure of these Hessians was registered.
Please note that it is not possible to only supply a subset of the Hessians (see :math:`\mathrm{errno}` = 6).
If there were no calls to :meth:`handle_set_nlnhess`, :math:`\mathrm{hess}` will never be called by ``handle_solve_ipopt`` and :math:`\mathrm{hess}` may be **None**. In this case, the Hessian of the Lagrangian will be approximated by a limited-memory quasi-Newton method (L-BFGS).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Hessian functions are to be evaluated.
**idf** : int
Specifies the quantities to be computed in :math:`\mathrm{hx}`.
:math:`\mathrm{idf} = -1`
The values of the Hessian of the Lagrangian will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
:math:`\mathrm{idf} = 0`
The values of the Hessian of the objective function will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
:math:`\mathrm{idf} > 0`
The values of the Hessian of the constraint function with index :math:`\mathrm{idf}` will be computed in :math:`\mathrm{hx}`. This will be the case if :meth:`handle_set_nlnhess` has been called with :math:`\mathrm{idf}` of the same value.
**sigma** : float
If :math:`\mathrm{idf} = -1`, the value of the :math:`\sigma` quantity in the definition of the Hessian of the Lagrangian. Otherwise, :math:`\mathrm{sigma}` should not be referenced.
**lamda** : float, ndarray, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\mathrm{idf} = -1`, the values of the :math:`\lambda_i` quantities in the definition of the Hessian of the Lagrangian. Otherwise, :math:`\mathrm{lamda}` should not be referenced.
**hx** : float, ndarray, shape :math:`\left(\textit{nnzh}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the requested Hessian evaluated at :math:`x`. For each value of :math:`\mathrm{idf}`, the ordering of nonzeros must follow the sparsity structure registered in the :math:`\mathrm{handle}` by earlier calls to :meth:`handle_set_nlnhess` through the arguments :math:`\textit{irowh}` and :math:`\textit{icolh}`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{hess}`. Specifically, if the value is negative the solution of the current problem will terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**monit** : None or callable monit(x, u, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
:math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x^i`, the values of the decision variables :math:`x` at the current iteration.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
If :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds the values of Lagrange multipliers (dual variables) for the constraints at the current iteration. See `Structure of the Lagrange Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#lagrangianmultipliers>`__ for layout information.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#printedoutputdesc>`__.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration. It reports only the iteration count and the number of backtracking trial steps taken. See `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#printedoutputdesc>`__.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
The input of :math:`\mathrm{u}` is reserved for future releases of the NAG Library and it is ignored at the moment.
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the constraints. See `Structure of the Lagrange Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#lagrangianmultipliers>`__ for layout information. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final value of Lagrange multipliers :math:`\left(z,\lambda \right)`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the final iteration as given in the list below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Constraint violation (primal infeasibility), see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#eqn3b>`__. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Dual infeasibility, see `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#eqn3a>`__. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Complementarity. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Karush--Kuhn--Tucker error. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Regularization term for the Hessian of the Lagrangian. This value is only available in :math:`\mathrm{monit}`, see **Iteration log** in `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#printedoutputdesc>`__.|
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Step size for the dual variables. This value is only available in :math:`\mathrm{monit}`, see **Iteration log** in `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#printedoutputdesc>`__. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Step size for the primal variables. This value is only available in :math:`\mathrm{monit}`, see **Iteration log** in `Description of the Printed Output <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#printedoutputdesc>`__. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8`--:math:`99`|Reserved for future use. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the final iteration as given in the list below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------+
|:math:`0` |Number of the iterations. |
+----------------------+------------------------------------------+
|:math:`1` |Reserved for future use. |
+----------------------+------------------------------------------+
|:math:`2` |Number of backtracking trial steps. |
+----------------------+------------------------------------------+
|:math:`3` |Number of Hessian evaluations. |
+----------------------+------------------------------------------+
|:math:`4` |Number of objective gradient evaluations. |
+----------------------+------------------------------------------+
|:math:`5`, :math:`6` |Reserved for future use. |
+----------------------+------------------------------------------+
|:math:`7` |Total wall clock time elapsed. |
+----------------------+------------------------------------------+
|:math:`8`--:math:`17` |Reserved for future use. |
+----------------------+------------------------------------------+
|:math:`18` |Number of objective function evaluations. |
+----------------------+------------------------------------------+
|:math:`19` |Number of constraint function evaluations.|
+----------------------+------------------------------------------+
|:math:`20` |Number of constraint Jacobian evaluations.|
+----------------------+------------------------------------------+
|:math:`21`--:math:`99`|Reserved for future use. |
+----------------------+------------------------------------------+
.. _e04st-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Hessian Mode'** : str
Default :math:`= \texttt{'AUTO'}`
This argument specifies whether the Hessian will be user-supplied (in :math:`\mathrm{hx}`) or approximated by ``handle_solve_ipopt`` using a limited-memory quasi-Newton L-BFGS method.
In the 'AUTO' setting, if no Hessian structure has been registered in the problem with a call to :meth:`handle_set_nlnhess` and there are general nonlinear objective or constraints, then the Hessian will be approximated.
Otherwise :math:`\mathrm{hess}` will be called if and only if any of :meth:`handle_set_nlnobj` or :meth:`handle_set_nlnconstr` have been used to define the problem.
Approximating the Hessian is likely to require more iterations to achieve convergence but will reduce the time spent in user-supplied functions.
Constraint: :math:`\text{‘Hessian Mode'} = \texttt{'AUTO'}`, :math:`\texttt{'EXACT'}` or :math:`\texttt{'APPROXIMATE'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
It also serves as a limit for the objective function to be considered unbounded (:math:`\mathrm{errno}` = 54).
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Matrix Ordering'** : str
Default :math:`= \texttt{'AUTO'}`
If 'NLP Factorization Method' is 'MA97', this argument specifies the ordering to be used by the internal sparse linear algebra solver.
It affects the number of nonzeros in the factorized matrix and thus influences the cost per iteration.
:math:`\text{‘Matrix Ordering'} = \texttt{'AUTO'}`
A heuristic is used to choose automatically between METIS and AMD orderings.
:math:`\text{‘Matrix Ordering'} = \texttt{'BEST'}`
Both AMD and METIS orderings are computed at the beginning of the solve and the one with the fewest nonzeros in the factorized matrix is selected.
:math:`\text{‘Matrix Ordering'} = \texttt{'AMD'}`
An approximate minimum degree (AMD) ordering is used.
:math:`\text{‘Matrix Ordering'} = \texttt{'METIS'}`
METIS ordering is used.
**Note:** if 'NLP Factorization Method' is 'MA86', this parameter is ignored and AMD ordering will be used.
Constraint: :math:`\text{‘Matrix Ordering'} = \texttt{'AUTO'}`, :math:`\texttt{'BEST'}`, :math:`\texttt{'AMD'}` or :math:`\texttt{'METIS'}`.
**'NLP Factorization Method'** : str
Default :math:`= \texttt{'MA97'}`
This parameter controls whether Harwell package 'MA97' or 'MA86' is used for the sparse linear algebra factorization.
To determine which best suits your application, it is recommended to try both 'MA97' and 'MA86'.
Constraint: :math:`\text{‘NLP Factorization Method'} = \texttt{'MA97'}` or :math:`\texttt{'MA86'}`.
**'Outer Iteration Limit'** : int
Default :math:`\text{} = 100`
The maximum number of iterations to be performed by ``handle_solve_ipopt``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘Outer Iteration Limit'} \geq 0`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+--------------------+------------------------------------------------------------------------------------+
|:math:`i` |Output |
+====================+====================================================================================+
|:math:`0` |No output from the solver |
+--------------------+------------------------------------------------------------------------------------+
|:math:`1` |Additionally, derivative check information, the Header and Summary. |
+--------------------+------------------------------------------------------------------------------------+
|:math:`2` |Additionally, the Iteration log. |
+--------------------+------------------------------------------------------------------------------------+
|:math:`3`, :math:`4`|Additionally, details of each iteration with scalar quantities printed. |
+--------------------+------------------------------------------------------------------------------------+
|:math:`5` |Additionally, individual components of arrays are printed resulting in large output.|
+--------------------+------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stop Tolerance 1'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This option sets the value :math:`\epsilon_{\textit{tol}}` of `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#eqn5>`__ which is used for optimality and complementarity tests from KKT conditions.
See `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04stf.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘Stop Tolerance 1'} > \epsilon`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with a corresponding error message.
Constraint: :math:`\text{‘Time Limit'} > 0`.
**'Verify Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument specifies whether the function should perform numerical checks on the consistency of the user-supplied functions.
It is recommended that such checks are enabled when first developing the formulation of the problem, however, the derivative check results in a significant increase of the number of the function evaluations and thus it shouldn't be used in production code.
Constraint: :math:`\text{‘Verify Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
.. _e04st-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`-199`)
``handle_solve_ipopt`` is not available in this implementation.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle` or :math:`0`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: no constraints present, so :math:`\textit{nnzu}` must be :math:`0`.
(`errno` :math:`6`)
On entry, a nonlinear objective function has been defined but no objective Hessian sparsity structure has been defined through :meth:`handle_set_nlnhess`.
(`errno` :math:`6`)
On entry, a nonlinear constraint function has been defined but no constraint Hessian sparsity structure has been defined through :meth:`handle_set_nlnhess`, for constraint number :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
Either change the option 'Hessian Mode' or provide a proper :math:`\mathrm{hess}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objgrd}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{confun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{congrd}` function.
(`errno` :math:`23`)
The solver terminated with not enough degrees of freedom.
(`errno` :math:`23`)
The solver terminated due to an invalid problem definition.
(`errno` :math:`23`)
The solver terminated due to an invalid option.
(`errno` :math:`51`)
The solver detected an infeasible problem.
(`errno` :math:`54`)
The solver terminated due to diverging iterates.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`22`)
Maximum number of iterations exceeded.
(`errno` :math:`50`)
The solver reports NLP solved to acceptable level.
**NagAlgorithmicMajorWarning**
(`errno` :math:`23`)
The solver terminated after failure in the restoration phase.
(`errno` :math:`23`)
The solver terminated after an error in the step computation.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
The solver terminated after the search direction became too small.
(`errno` :math:`25`)
Invalid number detected in user function.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04st-py2-py-notes:
**Notes**
``handle_solve_ipopt`` is typically used to solve the following nonlinear programming problem
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & f\left(x\right) &\\\text{subject to}&l_g\leq g\left(x\right)\leq u_g\text{,}&\\& \frac{1}{2} x^\mathrm{T}Q_ix + r_i^\mathrm{T}x + s_i\leq 0 \text{, }\quad 1\leq i\leq m_Q\text{,} &\\&l_B\leq Bx\leq u_B\text{,}&\\& l_x\leq x\leq u_x\text{,} &\end{array}
where
:math:`n` is the number of the decision variables,
:math:`m_g` is the number of the nonlinear constraints and :math:`g\left(x\right)`, :math:`l_g` and :math:`u_g` are :math:`m_g`-dimensional vectors,
:math:`m_Q` is the number of the quadratic constraints,
:math:`m_B` is the number of the linear constraints and :math:`B` is a :math:`m_B\times n` matrix, :math:`l_B` and :math:`u_B` are :math:`m_B`-dimensional vectors,
there are :math:`n` box constraints and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors.
The objective :math:`f\left(x\right)` can be specified in a number of ways: :meth:`handle_set_linobj` for a dense linear function, :meth:`handle_set_quadobj` (and :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac`) for a sparse linear or quadratic function, :meth:`handle_set_qconstr_fac` is used to provide the quadratic function in factorized form, and :meth:`handle_set_nlnobj` for a general nonlinear function.
In the last case, :math:`\mathrm{objfun}` and :math:`\mathrm{objgrd}` will be used to compute values and gradients of the objective function.
Variable box bounds :math:`l_x,u_x` can be specified with :meth:`handle_set_simplebounds`.
The special case of linear constraints :math:`l_B,B,u_B` is handled by :meth:`handle_set_linconstr`, quadratic constraints are handled by :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac` while general nonlinear constraints :math:`l_g,g\left(x\right),u_g` are specified by :meth:`handle_set_nlnconstr` (all can be specified).
Again, in the last case, :math:`\mathrm{confun}` and :math:`\mathrm{congrd}` will be used to compute values and gradients of the nonlinear constraint functions.
Finally, if it is viable to calculate second derivatives, the sparsity structure of the second partial derivatives of a general nonlinear objective and/or of any general nonlinear constraints is specified by :meth:`handle_set_nlnhess` and the values of these derivatives themselves will be computed by user-supplied :math:`\mathrm{hess}`.
While there is an option (see 'Hessian Mode') that forces internal approximation of second derivatives, no such option exists for first derivatives which must be computed accurately.
If :meth:`handle_set_nlnhess` has been called and :math:`\mathrm{hess}` is used to calculate values for second derivatives, both the nonlinear objective and all the nonlinear constraints must be included; it is not possible to provide a subset of these.
If the problem has only linear or quadratic objective and constraints, then :math:`\mathrm{hess}` is never called since the required Hessian information is already provided by the calls to :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_linconstr`, :meth:`handle_set_qconstr_fac` and :meth:`handle_set_qconstr`.
If :meth:`handle_set_nlnhess` is not called, then internal approximation of second derivatives will take place.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
**Structure of the Lagrange Multipliers**
For a problem consisting of :math:`n` variable bounds, :math:`m_B` linear constraints, :math:`m_Q` quadratic constraints, and :math:`m_g` nonlinear constraints, the number of Lagrange multipliers, and consequently the correct value for :math:`\mathrm{nnzu}`, will be :math:`q = 2*n+2*m_B+2*m_Q+2*m_g`.
The order these will be found in the :math:`\mathrm{u}` array is
:math:`\begin{array}{l} z_{1_L}, z_{1_U}, z_{2_L}, z_{2_U}\ldots z_{n_L}, z_{n_U}, \\ \lambda_{1_L}, \lambda_{1_U}, \lambda_{2_L}, \lambda_{2_U}\ldots \lambda_{{m_B}_L}, \lambda_{{m_B}_U}, \\ \lambda_{\left(m_B+1\right)_L}, \lambda_{\left(m_B+1\right)_U}, \lambda_{\left(m_B+2\right)_L}, \lambda_{\left(m_B+2\right)_U}\ldots \lambda_{\left(m_B+m_Q\right)_L}, \lambda_{\left(m_B+m_Q\right)_U}, \\ \lambda_{\left(m_B+m_Q+1\right)_L}, \lambda_{\left(m_B+m_Q+1\right)_U}, \lambda_{\left(m_B+m_Q+2\right)_L}, \lambda_{\left(m_B+m_Q+2\right)_U}\ldots \lambda_{\left(m_B+m_Q+m_g\right)_L}, \lambda_{\left(m_B+m_Q+m_g\right)_U}. \end{array}`
where the :math:`L` and :math:`U` subscripts refer to lower and upper bounds, respectively, and the variable bound constraint multipliers come first (if present, i.e., if :meth:`handle_set_simplebounds` was called), followed by the linear constraint multipliers (if present, i.e., if :meth:`handle_set_linconstr` was called), followed by the quadratic constraint multipliers (if present, i.e., if :meth:`handle_set_qconstr` or :meth:`handle_set_qconstr_fac` were called), and the nonlinear constraint multipliers (if present, i.e., if :meth:`handle_set_nlnconstr` was called).
Significantly nonzero values for any of these, after the solver has terminated, indicates that the corresponding constraint is active.
Significance is judged in the first instance by the relative scale of any value compared to the smallest among them.
.. _e04st-py2-py-references:
**References**
Byrd, R H, Gilbert, J Ch and Nocedal, J, 2000, `A trust region method based on interior point techniques for nonlinear programming`, Mathematical Programming (89), 149--185
Byrd, R H, Liu, G and Nocedal, J, 1997, `On the local behavior of an interior point method for nonlinear programming`, Numerical Analysis, (eds D F Griffiths and D J Higham), Addison--Wesley
Conn, A R, Gould, N I M, Orban, D and Toint, Ph L, 2000, `A primal-dual trust-region algorithm for non-convex nonlinear programming`, Mathematical Programming (87 (2)), 215--249
Conn, A R, Gould, N I M and Toint, Ph L, 2000, `Trust Region Methods`, SIAM, Philadephia
Fiacco, A V and McCormick, G P, 1990, `Nonlinear Programming: Sequential Unconstrained Minimization Techniques`, SIAM, Philadelphia
Gould, N I M, Orban, D, Sartenaer, A and Toint, Ph L, 2001, `Superlinear convergence of primal-dual interior point algorithms for nonlinear programming`, SIAM Journal on Optimization (11 (4)), 974--1002
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Hogg, J D and Scott, J A, 2010, `An indefinite sparse direct solver for large problems on multicore machines`, RAL Technical Report. RAL-TR-2010-011
Hogg, J D and Scott, J A, 2011, `HSL MA97: a bit-compatible multifrontal code for sparse symmetric systems`, RAL Technical Report. RAL-TR-2011-024
Wächter, A and Biegler, L T, 2006, `On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming`, Mathematical Programming (106(1)), 25--57
Williams, P and Lang, B, 2013, `A framework for the` :math:`MR^3` `Algorithm: theory and implementation`, SIAM J. Sci. Comput. (35), 740--766
Yamashita, H, 1998, `A globally convergent primal-dual interior-point method for constrained optimization`, Optimization Methods and Software (10), 443--469
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_pennon(handle, x, inform, u=None, uc=None, ua=None, io_manager=None):
r"""
``handle_solve_pennon`` is a solver from the NAG optimization modelling suite for problems such as, Quadratic Programming (QP), linear Semidefinite Programming (SDP) and semidefinite programming with bilinear matrix inequalities (BMI-SDP).
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04sv-py2-py-doc:
For full information please refer to the NAG Library document for e04sv
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html
.. _e04sv-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_pennon``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
Note: intermediate stops take place only if :math:`\text{‘Monitor Frequency'} > 0`.
If :math:`\text{‘Initial X'} = \texttt{'USER'}` (the default), :math:`x^0`, the initial estimate of the variables :math:`x`; otherwise, :math:`\mathrm{x}` need not be set.
`On intermediate entry`: the input is ignored.
**inform** : int
Note: intermediate stops take place only if :math:`\text{‘Monitor Frequency'} > 0`.
`On initial entry`: no effect.
`On intermediate entry`: if set to :math:`0`, solving the current problem is terminated and the function returns :math:`\mathrm{errno}` = 20; otherwise, the function continues.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: intermediate stops take place only if :math:`\text{‘Monitor Frequency'} > 0`.
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrangian multipliers (dual variables) for (standard) constraints, i.e., simple bounds defined by :meth:`handle_set_simplebounds` and a set of :math:`m_B` linear constraints defined by :meth:`handle_set_linconstr`. Either their initial estimates, intermediate approximations or final values, see `Structure of the Lagrangian Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#lagrangianmultipliers>`__.
Note: if :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
If :math:`\text{‘Initial U'} = \texttt{'USER'}` (the default is 'AUTOMATIC'), :math:`u^0`, the initial estimate of the Lagrangian multipliers :math:`u`; otherwise, :math:`\mathrm{u}` need not be set.
`On intermediate entry`: the input is ignored.
**uc** : None or float, array-like, shape :math:`\left(\textit{nnzuc}\right)`, optional
:math:`\mathrm{uc}` is reserved for future releases of the NAG Library which will allow definition of second-order cone constraints.
It is not referenced at the moment.
**ua** : None or float, array-like, shape :math:`\left(\textit{nnzua}\right)`, optional
Note: intermediate stops take place only if :math:`\text{‘Monitor Frequency'} > 0`.
If :math:`\textit{nnzua} > 0`, :math:`\mathrm{ua}` holds the Lagrangian multipliers for matrix constraints defined by :meth:`handle_set_linmatineq` and :meth:`handle_set_quadmatineq`, see `Structure of the Lagrangian Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#lagrangianmultipliers>`__.
If :math:`\textit{nnzua} = 0`, :math:`\mathrm{ua}` will not be referenced.
If :math:`\text{‘Initial U'} = \texttt{'USER'}` (the default is 'AUTOMATIC'), :math:`U^0`, the initial estimate of the matrix Lagrangian multipliers :math:`U`; otherwise, :math:`\mathrm{ua}` need not be set.
`On intermediate entry`: the input is ignored.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
`On intermediate exit`: the value of the variables :math:`x` at the end of the current outer iteration.
`On final exit`: the final value of the variables :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
`On intermediate exit`: the estimate of the multipliers :math:`u` at the end of the current outer iteration.The final value of multipliers :math:`u`.
**uc** : float, ndarray, shape :math:`\left(\textit{nnzuc}\right)`
:math:`\mathrm{uc}` is reserved for future releases of the NAG Library which will allow definition of second-order cone constraints.
It is not referenced at the moment.
**ua** : float, ndarray, shape :math:`\left(\textit{nnzua}\right)`
`On intermediate exit`: the estimate of the matrix multipliers :math:`U` at the end of the outer iteration.
`On final exit`: the final estimate of the multipliers :math:`U`.
**rinfo** : float, ndarray, shape :math:`\left(32\right)`
`On intermediate or final entry`: error measures and various indicators (see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algdetails>`__ for details) at the end of the current (or final) outer iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |objective function value :math:`f\left(x\right)` |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |optimality `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn12>`__ |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |feasibility `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn13>`__ |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |complementarity `(4) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn14>`__ |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |minimum penalty |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |relative precision `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn11>`__ |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |relative duality gap `(0) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn10>`__ |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |precision :math:`\left\lvert f\left(x^{\ell }\right)-f\left(x^{{\ell +1}}\right)\right\rvert` |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |duality gap |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |minimum penalty for (standard) inequalities :math:`p` |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |minimum penalty for matrix inequalities :math:`P` |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |feasibility of equality constraints |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |feasibility of (standard) inequalities |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13` |feasibility of matrix inequalities |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |complementarity of equality constraints |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15` |complementarity of (standard) inequalities |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`16` |complementarity of matrix inequalities |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`17`--:math:`22`|DIMACS error measures `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn16a>`__ (only if turned on by 'DIMACS Measures')|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`23`--:math:`31`|reserved for future use |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(32\right)`
`On intermediate or final exit`: solver statistics at the end of the current (or final) outer iteration as given in the table below. Note that time statistics is provided only if 'Stats Time' is set (the default is 'NO'), the measured time is returned in seconds.
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of the outer iterations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of the inner iterations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of the linesearch steps. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Number of evaluations of the augmented Lagrangian :math:`F\left(\right)`, (see `(8) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn8>`__).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Number of evaluations of :math:`\nabla F\left(\right)`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of evaluations of :math:`\nabla^2F\left(\right)`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Total running time of the solver. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total running time of the solver without evaluations of the user's functions and monitoring stops. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Time spent in the inner iterations. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Time spent in Lagrangian multipliers updates. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Time spent in penalty parameters updates. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Time spent in matrix feasibility computation. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13` |Time of evaluations of :math:`F\left(\right)`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Time of evaluations of :math:`\nabla F\left(\right)`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15` |Time of evaluations of :math:`\nabla^2F\left(\right)`. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`16` |Time of factorizations of the Newton system. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`17` |Time of factorizations of the matrix constraints. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`18`--:math:`31`|reserved for future use. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**inform** : int
Note: intermediate stops take place only if :math:`\text{‘Monitor Frequency'} > 0`.
`On intermediate exit`: :math:`\mathrm{inform} = 1`.
`On final exit`: :math:`\mathrm{inform} = 0`.
.. _e04sv-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DIMACS Measures'** : str
Default :math:`= \texttt{'CHECK'}`
If the problem is a linear semidefinite programming problem, this argument specifies if DIMACS error measures (see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#stoppingcriteria>`__) should be computed and/or checked.
In other cases, this option reverts to 'NO' automatically.
Constraint: :math:`\text{‘DIMACS Measures'} = \texttt{'COMPUTE'}`, :math:`\texttt{'CHECK'}` or :math:`\texttt{'NO'}`.
**'Hessian Density'** : str
Default :math:`= \texttt{'AUTO'}`
This option guides the solver on how the Hessian matrix of augmented Lagrangian :math:`F\left(x, u, v, U, p, P\right)` should be built.
Option 'AUTO' leaves the decision to the solver and it is the recommended option.
Setting it to 'DENSE' bypasses the autodetection and the Hessian is always built as a dense matrix.
Option 'SPARSE' instructs the solver to use a sparse storage and factorization of the matrix if possible.
Constraint: :math:`\text{‘Hessian Density'} = \texttt{'AUTO'}`, :math:`\texttt{'DENSE'}` or :math:`\texttt{'SPARSE'}`
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'Initial P'** : str
Default :math:`= \texttt{'AUTOMATIC'}`
This option defines the choice of the penalty options :math:`p^0`, :math:`P^0`, see `Algorithm 1 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm1>`__.
:math:`\text{‘Initial P'} = \texttt{'AUTOMATIC'}`
The penalty options are chosen automatically as set by option 'Init Value P', 'Init Value Pmat' and subject to automatic scaling. Note that :math:`P^0` might be increased so that the penalty function :math:`\Phi_P\left(\right)` is defined for all matrix constraints at the starting point.
:math:`\text{‘Initial P'} = \texttt{'KEEP PREVIOUS'}`
The penalty options are kept from the previous run of the solver if possible. If not, this options reverts to 'AUTOMATIC'. Note that even if the matrix penalty options are the same as in the previous run, they are still subject to a possible increase so that the penalty function :math:`\Phi_P\left(\right)` is well defined at the starting point.
Constraint: :math:`\text{‘Initial P'} = \texttt{'AUTOMATIC'}` or :math:`\texttt{'KEEP PREVIOUS'}`.
**'Initial U'** : str
Default :math:`= \texttt{'AUTOMATIC'}`
This argument guides the solver on which initial Lagrangian multipliers are to be used.
:math:`\text{‘Initial U'} = \texttt{'AUTOMATIC'}`
The Lagrangian multipliers are chosen automatically as set by automatic scaling.
:math:`\text{‘Initial U'} = \texttt{'USER'}`
The values of arrays :math:`\mathrm{u}` and :math:`\mathrm{ua}` (if provided) are used as the initial Lagrangian multipliers subject to automatic adjustments. If one or the other array is not provided, the choice for missing data is as in 'AUTOMATIC'.
:math:`\text{‘Initial U'} = \texttt{'KEEP PREVIOUS'}`
The Lagrangian multipliers are kept from the previous run of the solver. If this option is set for the first run or options change the approach of the solver, the choice automatically reverts to 'AUTOMATIC'. This might be useful if the solver is hot started, for example, to achieve higher precision of the solution.
Constraint: :math:`\text{‘Initial U'} = \texttt{'AUTOMATIC'}`, :math:`\texttt{'USER'}` or :math:`\texttt{'KEEP PREVIOUS'}`.
**'Initial X'** : str
Default :math:`= \texttt{'USER'}`
This argument guides which starting point :math:`x^0` is to be used.
:math:`\text{‘Initial X'} = \texttt{'AUTOMATIC'}`
The starting point is chosen automatically so that it satisfies simple bounds on the variables or as a zero vector. Input of argument :math:`\mathrm{x}` is ignored.
:math:`\text{‘Initial X'} = \texttt{'USER'}`
Initial values of argument :math:`\mathrm{x}` are used as a starting point.
Constraint: :math:`\text{‘Initial X'} = \texttt{'AUTOMATIC'}` or :math:`\texttt{'USER'}`.
**'Init Value P'** : float
Default :math:`= 1.0`
This argument defines the value :math:`p^0`, the initial penalty option for (standard) inequalities.
A low value of the penalty causes the solution of the inner problem to be closer to the feasible region and thus to the desirable result.
However, it also increases ill-conditioning of the system.
It is not advisable to set the penalty too low unless a good starting point is provided.
Constraint: :math:`\sqrt[4]{\epsilon }\leq \text{‘Init Value P'}\leq 10^4`.
**'Init Value Pmat'** : float
Default :math:`= 1.0`
The value of this option suggests :math:`P^0`, the initial penalty option for matrix inequalities.
It is similar to 'Init Value P' (and the same advice applies), however, :math:`P^0` gets increased automatically if the matrix constraints are more infeasible than the actual penalty option.
Constraint: :math:`\sqrt[4]{\epsilon }\leq \text{‘Init Value Pmat'}\leq 10^4`.
**'Inner Iteration Limit'** : int
Default :math:`= 100`
The maximum number of the inner iterations (Newton steps) to be performed by `Algorithm 2 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm2>`__ in each outer iteration.
Setting the option too low might lead to :math:`\mathrm{errno}` = 23.
Values higher than :math:`100` are unlikely to improve convergence.
Constraint: :math:`\text{‘Inner Iteration Limit'} > 0`.
**'Inner Stop Criteria'** : str
Default :math:`= \texttt{'HEURISTIC'}`
The precision :math:`\alpha` for the solution of the inner subproblem is determined in `Algorithm 1 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm1>`__ and under typical circumstances `Algorithm 2 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm2>`__ is expected to reach this precision within the given 'Inner Iteration Limit'.
If any problems are detected and :math:`\text{‘Inner Stop Criteria'} = \texttt{'HEURISTIC'}`, `Algorithm 2 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm2>`__ is allowed to stop before reaching the requested precision or the 'Inner Iteration Limit'.
This usually saves many unfruitful iterations and the solver may recover in the following iterations.
If you suspect that the heuristic problem detection is not suitable for your problem, setting :math:`\text{‘Inner Stop Criteria'} = \texttt{'STRICT'}` disallows such behaviour.
Constraint: :math:`\text{‘Inner Stop Criteria'} = \texttt{'HEURISTIC'}` or :math:`\texttt{'STRICT'}`.
**'Inner Stop Tolerance'** : float
Default :math:`= 10^{-2}`
This option sets the required precision :math:`\alpha^0` for the first inner problem solved by `Algorithm 2 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm2>`__.
The precison of the solution of the inner problem does not need to be very high in the first outer iterations and it is automatically adjusted through the outer iterations to reach the optimality limit :math:`\epsilon_2` in the last one.
Setting :math:`\alpha^0` too restrictive (too low) causes an increase of the number of inner iterations needed in the first outer iterations and might lead to :math:`\mathrm{errno}` = 23.
In certain cases it might be helpful to use a more relaxed (higher) :math:`\alpha^0` and increase 'P Update Speed' which should reduce the number of inner iterations needed at the beginning of the computation in exchange for a possibly higher number of the outer iterations.
Constraint: :math:`\epsilon < \text{‘Inner Stop Tolerance'}\leq 10^3`.
**'Linesearch Mode'** : str
Default :math:`= \texttt{'AUTO'}`
This controls the step size selection in `Algorithm 2 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm2>`__.
If :math:`\text{‘Linesearch Mode'} = \texttt{'FULLSTEP'}` (the default for linear problems), unit steps are taken where possible and the step shortening takes place only to avoid undefined regions for the matrix penalty function :math:`\Phi_P\left(\right)` (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn16>`__).
This may be used for linear problems but it is not recommended for any nonlinear ones.
If :math:`\text{‘Linesearch Mode'} = \texttt{'ARMIJO'}`, Armijo backtracking linesearch is used instead which is a fairly basic linesearch.
If :math:`\text{‘Linesearch Mode'} = \texttt{'GOLDSTEIN'}`, a cubic safe guarded linesearch based on Goldstein condition is employed, this is the recommended (and default) choice for nonlinear problems.
Constraint: :math:`\text{‘Linesearch Mode'} = \texttt{'AUTO'}`, :math:`\texttt{'FULLSTEP'}`, :math:`\texttt{'ARMIJO'}` or :math:`\texttt{'GOLDSTEIN'}`.
**'List'** : str
Default :math:`= \texttt{'NO'}`
This argument may be set to 'YES' if you wish to turn on printing of each option specification as it is supplied.
Constraint: :math:`\text{‘List'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`
**'Monitor Frequency'** : int
Default :math:`= 0`
If :math:`\text{‘Monitor Frequency'} > 0`, the solver returns to you at the end of every :math:`i`\ th outer iteration.
During these intermediate exits, the current point :math:`\mathrm{x}` and Lagrangian multipliers :math:`\mathrm{u}`, :math:`\mathrm{ua}` (if requested) are provided as well as the statistics and error measures (:math:`\mathrm{rinfo}`, :math:`\mathrm{stats}`). Argument :math:`\mathrm{inform}` helps to distinguish between intermediate and final exits and also allows immediate termination.
If :math:`\text{‘Monitor Frequency'} = 0`, the solver stops only once on the final point and no intermediate exits are made.
Constraint: :math:`\text{‘Monitor Frequency'}\geq 0`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options;
- problem statistics, the iteration log and the final status as set by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Outer Iteration Limit'** : int
Default :math:`\text{} = 100`
The maximum number of the outer iterations to be performed by `Algorithm 1 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm1>`__.
If :math:`\text{‘Outer Iteration Limit'} = 0`, no iteration is performed, only quantities needed in the stopping criteria are computed and returned in :math:`\mathrm{rinfo}`.
This might be useful in connection with :math:`\text{‘Initial X'} = \texttt{'USER'}` and :math:`\text{‘Initial U'} = \texttt{'USER'}` to check optimality of the given point.
However, note that the rules for possible modifications of the starting point still apply, see :math:`\mathrm{u}` and :math:`\mathrm{ua}`.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘Outer Iteration Limit'} \geq 0`.
**'P Min'** : float
Default :math:`= \sqrt{\epsilon }`
This controls :math:`p_{\mathrm{min}}`, the lowest possible penalty value :math:`p` used for (standard) inequalities.
In general, very small values of the penalty options cause ill-conditioning which might lead to numerical difficulties.
On the other hand, very high :math:`p_{\mathrm{min}}` prevents the algorithm from reaching the requested accuracy on the feasibility.
Under normal circumstances, the default value is recommended.
Constraint: :math:`\epsilon \leq \text{‘P Min'}\leq 10^{-2}`.
**'Pmat Min'** : float
Default :math:`= \sqrt{\epsilon }`
This is an equivalent of 'P Min' for the minimal matrix penalty option :math:`P_{\mathrm{min}}`.
The same advice applies.
Constraint: :math:`\epsilon \leq \text{‘Pmat Min'}\leq 10^{-2}`.
**'Preference'** : str
Default :math:`= \texttt{'SPEED'}`
This option affects how contributions from the matrix constraints `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn16>`__ to the system Hessian matrix are computed.
The default option of :math:`\text{‘Preference'} = \texttt{'SPEED'}` should be suitable in most cases.
However, dealing with matrix constraints of a very high dimension may cause noticable memory overhead and switching to :math:`\text{‘Preference'} = \texttt{'MEMORY'}` may be required.
Constraint: :math:`\text{‘Preference'} = \texttt{'SPEED'}` or :math:`\texttt{'MEMORY'}`.
**'Presolve Block Detect'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Presolve Block Detect'} = \texttt{'YES'}`, the matrix constraints are checked during preprocessoring to determine if they can be split into smaller independent ones, thus speeding up the solver.
Constraint: :math:`\text{‘Presolve Block Detect'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+--------------------+---------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+====================+=====================================================================================================================+
|:math:`0` |No output from the solver |
+--------------------+---------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the objective value |
+--------------------+---------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per outer iteration showing the progress of the solution, final status and statistics |
+--------------------+---------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but detailed output of the outer iterations is provided and brief overview of the inner iterations|
+--------------------+---------------------------------------------------------------------------------------------------------------------+
|:math:`4`, :math:`5`|As level :math:`3` but details of the inner iterations are printed as well |
+--------------------+---------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'P Update Speed'** : int
Default :math:`= 12`
This option affects the rate at which the penalty options :math:`p,P` are updated (`Algorithm 1 <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algorithm1>`__, step \(3)) and thus indirectly influences the overall number of outer iterations.
Its value can be interpretted as the typical number of outer iterations needed to get from the initial penalty values :math:`p^0`, :math:`P^0` half-way to the :math:`p_{\mathrm{min}}` and :math:`P_{\mathrm{min}}`.
Values smaller than :math:`3` causes a very agressive penalty update strategy which might lead to the increased number of inner iterations and possibly to numerical difficulties.
On the other hand, values higher than :math:`15` produce a relatively conservative approach which leads to a higher number of the outer iterations.
If the solver encounters difficulties on your problem, a higher value might help.
If your problem is working fine, setting a lower value might increase the speed.
Constraint: :math:`1\leq \text{‘P Update Speed'}\leq 100`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Stop Criteria'** : str
Default :math:`= \texttt{'SOFT'}`
If :math:`\text{‘Stop Criteria'} = \texttt{'SOFT'}`, the solver is allowed to stop prematurely with a suboptimal solution, :math:`\mathrm{errno}` = 50, if it predicts that a better estimate of the solution cannot be reached.
This is the recommended option.
Constraint: :math:`\text{‘Stop Criteria'} = \texttt{'SOFT'}` or :math:`\texttt{'STRICT'}`.
**'Stop Tolerance 1'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This option defines :math:`\epsilon_1` used as a tolerance for the relative duality gap `(0) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn10>`__ and the relative precision `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn11>`__, see `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘Stop Tolerance 1'} > \epsilon`.
**'Stop Tolerance 2'** : float
Default :math:`= \mathrm{max}\left(10^{-7}, \sqrt{\epsilon }\right)`
This option sets the value :math:`\epsilon_2` which is used for optimality `(2) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn12>`__ and complementarity `(4) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn14>`__ tests from KKT conditions or if :math:`\text{‘DIMACS Measures'} = \texttt{'Check'}` for all DIMACS error measures instead.
See `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘Stop Tolerance 2'} > \epsilon`.
**'Stop Tolerance Feasibility'** : float
Default :math:`= \mathrm{max}\left(10^{-7}, \sqrt{\epsilon }\right)`
This argument places an acceptance limit on the feasibility of the solution `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn13>`__, :math:`\epsilon_{\mathrm{feas}}`.
See `Stopping Criteria <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘Stop Tolerance Feasibility'} > \epsilon`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function was set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Transform Constraints'** : str
Default :math:`= \texttt{'AUTO'}`
This argument controls how equality constraints are treated by the solver.
If :math:`\text{‘Transform Constraints'} = \texttt{'EQUALITIES'}`, all equality constraints :math:`h_k\left(x\right) = 0` from `(4) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#eqn4>`__ are treated as two inequalities :math:`h_k\left(x\right)\leq 0` and :math:`h_k\left(x\right)\geq 0`, see `Solution of the inner problem <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#innerproblem>`__.
This is the default and the only option in this release for equality constrained problems.
Constraint: :math:`\text{‘Transform Constraints'} = \texttt{'AUTO'}`, :math:`\texttt{'NO'}` or :math:`\texttt{'EQUALITIES'}`.
**'U Update Restriction'** : float
Default :math:`= 0.5`
This defines the value :math:`\mu_g` giving the bounds on the updates of Lagrangian multipliers for (standard) inequalities between the outer iterations.
Values close to :math:`1` limit the changes of the multipliers and serve as a kind of smoothing, lower values allow more significant changes.
Based on numerical experience, big variation in the multipliers may lead to a large number of iterations in the subsequent step and might disturb the convergence due to ill-conditioning.
It might be worth experimenting with the value on your particular problem.
Mid range values are recommended over the more extremal ones.
Constraint: :math:`\epsilon < \text{‘U Update Restriction'} < 1`.
**'Umat Update Restriction'** : float
Default :math:`= 0.3`
This is an equivalent of 'U Update Restriction' for matrix constraints, denoted as :math:`\mu_A` in `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#overview>`__.
The advice above applies equally.
Constraint: :math:`\epsilon < \text{‘Umat Update Restriction'} < 1`.
.. _e04sv-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`3`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for (standard) constraints.
:math:`\textit{nnzu} = 0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for (standard) constraints.
:math:`\textit{nnzu} = 0` when there are no (standard) constraints.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzua} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzua}` does not match the size of the Lagrangian multipliers for matrix constraints.
:math:`\textit{nnzua} = 0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzua} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzua}` does not match the size of the Lagrangian multipliers for matrix constraints.
:math:`\textit{nnzua} = 0` when there are no matrix constraints.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzuc} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzuc}` does not match the size of the Lagrangian multipliers for second-order cone constraints.
:math:`\textit{nnzuc} = 0` when there are no second-order cone constraints.
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`51`)
The problem was found to be infeasible during preprocessing.
(`errno` :math:`52`)
The problem was found unbounded during preprocessing.
(`errno` :math:`53`)
The problem seems to be infeasible, the algorithm was stopped.
(`errno` :math:`54`)
The problem seems to be unbounded, the algorithm was stopped.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The algorithm converged to a suboptimal solution.
The full accuracy was not achieved. The solution should still be usable.
**NagAlgorithmicMajorWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`22`)
Outer iteration limit has been reached.
The requested accuracy is not achieved.
(`errno` :math:`23`)
The inner subproblem could not be solved to the required accuracy.
Inner iteration limit has been reached.
(`errno` :math:`23`)
The inner subproblem could not be solved to the required accuracy.
Limited progress in the inner subproblem triggered a stop (heuristic inner stop criteria).
(`errno` :math:`23`)
The inner subproblem could not be solved to the required accuracy.
Line search or another internal component failed.
(`errno` :math:`24`)
Unable to make progress, the algorithm was stopped.
.. _e04sv-py2-py-notes:
**Notes**
``handle_solve_pennon`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_simplebounds`, :meth:`handle_set_linconstr`, :meth:`handle_set_linmatineq` or :meth:`handle_set_quadmatineq` may be used to formulate the objective function, (standard) constraints and matrix constraints of the problem.
Once the problem is fully set, the handle may be passed to the solver.
When the handle is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
Problems which can be defined this way are, for example, (generally nonconvex) Quadratic Programming (QP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\\text{subject to}&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
linear semidefinite programming problems (SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\& l_B\leq Bx\leq u_B\text{,} &\quad \text{ (c)}\\& l_x\leq x\leq u_x\text{,} &\quad \text{ (d)}\end{array}
or semidefinite programming problems with bilinear matrix inequalities (BMI-SDP)
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & \frac{1}{2} x^\mathrm{T}Hx + c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad & \sum_{1}^{n}{x_ix_jQ_{{ij}}^k} + \sum_{1}^{n}{x_iA_i^k-A_0^k} ⪰ 0 \text{, }\quad k = 1,\ldots,m_A \text{,} &\quad \text{ (b)}\\&l_B\leq Bx\leq u_B\text{,}&\quad \text{ (c)}\\&l_x\leq x\leq u_x \text{.} &\quad \text{ (d)}\end{array}
Here :math:`c`, :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors, :math:`H` is a symmetric :math:`n\times n` matrix, :math:`l_B`, :math:`u_B` are :math:`m_B`-dimensional vectors, :math:`B` is a general :math:`m_B\times n` rectangular matrix and :math:`A_i^k`, :math:`Q_{{ij}}^k` are symmetric matrices.
The expression :math:`S⪰0` stands for a constraint on eigenvalues of a symmetric matrix :math:`S`, namely, all the eigenvalues should be non-negative, i.e., the matrix should be positive semidefinite.
See relevant functions in the suite for more details on the problem formulation.
The solver is based on a generalized Augmented Lagrangian method with a suitable choice of standard and matrix penalty functions.
For a detailed description of the algorithm see `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#algdetails>`__.
Under standard assumptions on the problem (Slater constraint qualification, boundedness of the objective function on the feasible set, see Stingl (2006) for details) the algorithm converges to a local solution.
In case of convex problems such as linear SDP or convex QP, this is the global solution.
The solver is suitable for both small dense and large-scale sparse problems.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04sv-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
There are several options with a multiple choice where the default choice is 'AUTO' (for example, 'Hessian Density').
This value means that the decision over the option is left to the solver based on the structure of the problem.
Option getter :meth:`handle_opt_get` can be called to retrieve the choice of these options as well as on any other options.
Option 'Task' may be used to switch the problem to maximization or to ignore the objective function and find only a feasible point.
Option 'Monitor Frequency' may be used to turn on the monitor mode of the solver.
The solver invoked in this mode pauses regularly even before the optimal point is found to allow monitoring the progress from the calling program.
All the important error measures and statistics are available in the calling program which may terminate the solver early if desired (see argument :math:`\mathrm{inform}`).
**Structure of the Lagrangian Multipliers**
The algorithm works internally with estimates of both the decision variables, denoted by :math:`x`, and the Lagrangian multipliers (dual variables) for standard and matrix constraints, denoted by :math:`u` and :math:`U`, respectively.
You may provide initial estimates, request approximations during the run (the monitor mode turned on) and obtain the final values.
The Lagrangian multipliers are split into two arrays, the multipliers :math:`u` for (standard) constraints are stored in array :math:`\mathrm{u}` and multipliers :math:`U` for matrix constraints in array :math:`\mathrm{ua}`.
Both arrays need to conform to the structure of the constraints.
If the simple bounds were defined (:meth:`handle_set_simplebounds` was successfully called), the first :math:`2n` elements of :math:`\mathrm{u}` belong to the corresponding Lagrangian multipliers, interleaving a multiplier for the lower and for the upper bound for each :math:`x_i`.
If any of the bounds were set to infinity, the corresponding Lagrangian multipliers are set to :math:`0` and may be ignored.
Similarly, the following :math:`2m_B` elements of :math:`\mathrm{u}` belong to multipliers for the linear constraints, if formulated by :meth:`handle_set_linconstr`.
The organization is the same, i.e., the multipliers for each constraint for the lower and upper bounds are alternated and zeroes are used for any missing (infinite bound) constraint.
A Lagrangian multiplier for a matrix constraint (one block) of dimension :math:`d\times d` is a dense symmetric matrix of the same dimension.
All multipliers :math:`U` are stored next to each other in array :math:`\mathrm{ua}` in the same order as the matrix constraints were defined by :meth:`handle_set_linmatineq` and :meth:`handle_set_quadmatineq`.
The lower triangle of each is stored in the packed column order (see `the F07 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/f07/f07intro.html#recomm_32>`__).
For example, if there are four matrix constraints of dimensions :math:`7`, :math:`3`, :math:`1`, :math:`1`, the dimension of array :math:`\mathrm{ua}` should be :math:`36`.
The first :math:`28` elements :math:`\left(d_1\times \left(d_1+1\right)/2\right)` belong to the packed lower triangle of :math:`U_1`, followed by six elements of :math:`U_2` and one element for each :math:`U_3` and :math:`U_4`.
**Approximation of the Lagrangian Multipliers**
By the nature of the algorithm, all inequality Lagrangian multiplier :math:`u,U` are always kept positive during the computational process.
This applies even to Lagrangian multipliers of inactive constraints at the solution.
They will only be close to zero although they would normally be equal to zero exactly.
This is one of the major differences between results from solvers based on the active set method (such as :meth:`qpconvex2_sparse_solve`) and others, such as, ``handle_solve_pennon`` or interior point methods.
As a consequence, the initial estimate of the multipliers (if provided) might be adjusted by the solver to be sufficiently positive, also the estimates returned during the intermediate exits might only be a very crude approximation to their final values as they do not satisfy all the Karush--Kuhn--Tucker (KKT) conditions.
Another difference is that :meth:`qpconvex2_sparse_solve` merges multipliers for both lower and upper inequality into one element whose sign determines the inequality because there can be at most one active constraint and multiplier for the inactive is exact zero.
Negative multipliers are associated with the upper bounds and positive with the lower bounds.
On the other hand, ``handle_solve_pennon`` works with both multipliers at the same time so they are returned in two elements, one for the lower bound, the other for the upper bound (see `Structure of the Lagrangian Multipliers <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04svf.html#lagrangianmultipliers>`__).
An equivalent result can be achieved by subtracting the upper bound multiplier from the lower one.
This holds even when equalities are interpreted as two inequalities (see option 'Transform Constraints').
.. _e04sv-py2-py-references:
**References**
Ben--Tal, A and Zibulevsky, M, 1997, `Penalty/barrier multiplier methods for convex programming problems`, SIAM Journal on Optimization (7), 347--366
Fujisawa, K, Kojima, M, Nakata, K, 1997, `Exploiting sparsity in primal-dual interior-point method for semidefinite programming`, Math. Programming (79), 235--253
Hogg, J D and Scott, J A, 2011, `HSL MA97: a bit-compatible multifrontal code for sparse symmetric systems`, RAL Technical Report. RAL-TR-2011-024
Kočvara, M and Stingl, M, 2003, `PENNON -- a code for convex nonlinear and semidefinite programming`, Optimization Methods and Software (18(3)), 317--333
Kočvara, M and Stingl, M, 2007, `On the solution of large-scale SDP problems by the modified barrier method using iterative solvers`, Math. Programming (Series B) (109(2--3)), 413--444
Mittelmann, H D, 2003, `An independent benchmarking of SDP and SOCP solvers`, Math. Programming (95), 407--430
Stingl, M, 2006, `On the Solution of Nonlinear Semidefinite Programs by Augmented Lagrangian Methods, PhD thesis`, Institute of Applied Mathematics II, Friedrich--Alexander University of Erlangen--Nuremberg
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_bmi_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_pennon_lmi_ex.main`
"""
raise NotImplementedError
[docs]def handle_add_vars(handle, nadd):
r"""
``handle_add_vars`` is a part of the NAG optimization modelling suite and adds new variables to the problem.
.. _e04ta-py2-py-doc:
For full information please refer to the NAG Library document for e04ta
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04taf.html
.. _e04ta-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**nadd** : int
:math:`n_{\textit{add}}`, the number of decision variables to add to the problem.
**Returns**
**nvar** : int
:math:`n`, the new total number of the variables in the problem.
.. _e04ta-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\mathrm{nadd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nadd} > 0`.
.. _e04ta-py2-py-notes:
**Notes**
After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), ``handle_add_vars`` may be used to define :math:`n_{\textit{add}}` new unconstrained continuous variables to the problem.
If :math:`n_{\textit{now}}` denotes the current number of variables in the model, the total number of variables will be :math:`n = n_{\textit{now}}+n_{\textit{add}}` and the new variables will be indexed by :math:`n_{\textit{now}}+1,\ldots,n_{\textit{now}}+n_{\textit{add}}`.
The objective function or constraints, if already defined in the model, won't be affected as they will be naturally extended as if the new variables were not referred during their definition (e.g., for a linear objective function the coefficients for the new variables would be set to zero).
If the new variables should enter any already defined parts of the problem, you should modify them with the appropriate functions from the suite.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
"""
raise NotImplementedError
[docs]def handle_enable(handle, comp, idx):
r"""
``handle_enable`` is a part of the NAG optimization modelling suite and allows you to `enable` various components of the existing model which were previously `disabled` by :meth:`handle_disable`.
.. _e04tb-py2-py-doc:
For full information please refer to the NAG Library document for e04tb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04tbf.html
.. _e04tb-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**comp** : str
The type of the component of the model to be enabled. :math:`\mathrm{comp}` is case insensitive.
:math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'VAR'}` or :math:`\texttt{'VARIABLE'}`
Decision variables :math:`x`.
:math:`\mathrm{comp} = \texttt{'LC'}` or :math:`\texttt{'LINEAR CONSTRAINT'}`
Linear constraints (see :meth:`handle_set_linconstr`).
:math:`\mathrm{comp} = \texttt{'QC'}` or :math:`\texttt{'QUADRATIC CONSTRAINT'}`
Quadratic constraints (see :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac`).
:math:`\mathrm{comp} = \texttt{'NLC'}` or :math:`\texttt{'NONLINEAR CONSTRAINT'}`
Nonlinear constraints (see :meth:`handle_set_nlnconstr`).
:math:`\mathrm{comp} = \texttt{'CN'}` or :math:`\texttt{'CONE'}`
Quadratic or rotated quadratic cones (see :meth:`handle_set_group`).
:math:`\mathrm{comp} = \texttt{'MI'}` or :math:`\texttt{'MATRIX INEQUALITY'}`
Matrix inequality constraints (see :meth:`handle_set_linmatineq`).
:math:`\mathrm{comp} = \texttt{'NLS'}` or :math:`\texttt{'RESIDUAL'}`
Nonlinear residuals :math:`r_j\left(x\right)` in the nonlinear least squares objective function (see :meth:`handle_set_nlnls`).
**idx** : int, array-like, shape :math:`\left(\textit{lidx}\right)`
The index set of components :math:`\mathrm{comp}` to be enabled. The elements may be supplied in any order.
.. _e04tb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`5`)
On entry, :math:`\textit{lidx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lidx} \geq 1`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'LC'}`, :math:`\texttt{'QC'}`, :math:`\texttt{'NLC'}`, :math:`\texttt{'CN'}`, :math:`\texttt{'MI'}` or :math:`\texttt{'NLS'}`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idx}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`M = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idx}[i-1]\leq M`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idx}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
This component has been deleted.
.. _e04tb-py2-py-notes:
**Notes**
:meth:`handle_disable` and ``handle_enable`` form a pair of functions which allow you to temporarily disable and then re-enable parts of a model.
This is particularly useful when a sequence of similar problems needs to be solved, to identify how a particular constraint or variable affects the solution, or to switch between previously defined constraints which are somewhat related to each other.
``handle_enable`` may be used to re-enable a component of the model previously disabled by a call to :meth:`handle_disable`.
The components to be re-enabled are identified by supplying the same value of :math:`\mathrm{comp}` and :math:`\mathrm{idx}` as used in the call to :meth:`handle_disable` when they were disabled.
All newly created components of the model are enabled.
Calling this function on enabled components is not an error but has no effect.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
"""
raise NotImplementedError
[docs]def handle_disable(handle, comp, idx):
r"""
``handle_disable`` is a part of the NAG optimization modelling suite and allows you to `disable` various components of an existing model.
.. _e04tc-py2-py-doc:
For full information please refer to the NAG Library document for e04tc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04tcf.html
.. _e04tc-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**comp** : str
The type of the component of the model to be disabled. :math:`\mathrm{comp}` is case insensitive.
:math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'VAR'}` or :math:`\texttt{'VARIABLE'}`
Decision variables :math:`x`.
:math:`\mathrm{comp} = \texttt{'LC'}` or :math:`\texttt{'LINEAR CONSTRAINT'}`
Linear constraints (see :meth:`handle_set_linconstr`).
:math:`\mathrm{comp} = \texttt{'QC'}` or :math:`\texttt{'QUADRATIC CONSTRAINT'}`
Quadratic constraints (see :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac`).
:math:`\mathrm{comp} = \texttt{'NLC'}` or :math:`\texttt{'NONLINEAR CONSTRAINT'}`
Nonlinear constraints (see :meth:`handle_set_nlnconstr`).
:math:`\mathrm{comp} = \texttt{'CN'}` or :math:`\texttt{'CONE'}`
Quadratic or rotated quadratic cones (see :meth:`handle_set_group`).
:math:`\mathrm{comp} = \texttt{'MI'}` or :math:`\texttt{'MATRIX INEQUALITY'}`
Matrix inequality constraints (see :meth:`handle_set_linmatineq`).
:math:`\mathrm{comp} = \texttt{'NLS'}` or :math:`\texttt{'RESIDUAL'}`
Nonlinear residuals :math:`r_i\left(x\right)` in the nonlinear least squares objective function (see :meth:`handle_set_nlnls`).
**idx** : int, array-like, shape :math:`\left(\textit{lidx}\right)`
The index set of components :math:`\mathrm{comp}` to be disabled. The elements may be supplied in any order.
.. _e04tc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`5`)
On entry, :math:`\textit{lidx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lidx} \geq 1`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'LC'}`, :math:`\texttt{'QC'}`, :math:`\texttt{'NLC'}`, :math:`\texttt{'CN'}`, :math:`\texttt{'MI'}` or :math:`\texttt{'NLS'}`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idx}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`M = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idx}[i-1]\leq M`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idx}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`.
This component has been deleted.
.. _e04tc-py2-py-notes:
**Notes**
``handle_disable`` and :meth:`handle_enable` form a pair of functions which allow you to quickly remove parts of the model and add them back.
This is particularly useful when a sequence of similar problems needs to be solved, to identify how a particular constraint or variable affects the solution, or to switch between previously defined constraints which are somewhat related to each other.
After the model has been fully or partially built by calling functions of the NAG optimization modelling suite, ``handle_disable`` may be used to `disable` some components (variables or constraints) from the model.
To identify the part of the model to be disabled, the component type needs to be specified in :math:`\mathrm{comp}` together with the appropriate component indices, such as variable or linear constraint numbers, in an index set :math:`\mathrm{idx}`. :meth:`handle_enable` may be used to bring them back (to `enable` them).
Note that a disabled component may still be edited but would not be automatically re-enabled; a disabled component can only be enabled by calling :meth:`handle_enable`.
By default, all newly created components of the model are enabled.
Calling this function on disabled components is not an error but has no effect.
In general, disabled components are still present in the model with respect to the numbering of the variables and constraints (e.g., disabling a variable will not renumber the remaining variables) but the solver will ignore them and work internally with a smaller model formulation.
The solver will communicate with you (e.g., return results) in arrays matching the full size of the model and the elements corresponding to the disabled components will be set to NaN, which can be tested for via a call to :meth:`ieee.is_nan <naginterfaces.library.ieee.is_nan>`. The effect of disabling components is described in detail as follows:
**Decision variables**
As a rule, a disabled variable behaves as if the existing parts of the model were defined without the variable in the first place. Matching elements in all other model components are ignored. In particular, the following applies:
- elements in the linear or quadratic objective function corresponding to the disabled variables are ignored, as if they were set to 0;
- the same applies to the linear and quadratic constraints as well as to the matrix inequalities, however, note that this can generate infeasible constraints (for instance, :math:`-2\leq x+y\leq -1` with disabled :math:`x` and :math:`y` is interpreted as :math:`-2\leq 0\leq -1` and thus would be infeasible);
- the box bounds of the disabled variables are ignored;
- if a disabled variable is present in a cone constraint and the cone constraint becomes invalid (e.g., the length of the cone is just one or the first cone variable is disabled), the whole constraint is disabled as well;
- any callbacks for nonlinear function evaluation will have on input the full :math:`x` vector, however, all disabled variables are set to NaN and the matching elements in the returned derivatives are not referenced.
An alternative approach is to fix the variable to a predefined value by changing its simple bounds using :meth:`handle_set_bound` instead of disabling it.
**Linear constraints, Quadratic constraints, Second-order cones, Matrix inequalities**
Disabled constraints do not restrict the feasible set, they are eliminated before the model is presented to the solver. As the constraints are not renumbered, the position for Lagrangian multipliers (:math:`\textit{u}`, :math:`\textit{uc}`, :math:`\textit{ua}`) is unchanged and NaNs are returned in place of disabled constraints.
**Nonlinear constraints**
The disabled nonlinear constraints do not affect the feasible set and their Lagrangian multipliers are returned as NaNs in their expected location. The values of the nonlinear constraints and their partial derivatives are passed to the solver via callbacks, such as, :math:`\textit{objfun}` and :math:`\textit{objgrd}`. The size of all array arguments in the callbacks remain the same, irrespective of the number of disabled constraints. On exit from the callback any elements in these arrays corresponding to the disabled constraints are not referenced.
**Nonlinear residuals**
The disabled residuals in the nonlinear least squares objective function :math:`\sum r_j^2\left(x\right)` are skipped in the sum and their values are not referenced.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_bound(handle, comp, idx, bli, bui):
r"""
``handle_set_bound`` is a part of the NAG optimization modelling suite and sets or modifies the bounds of a single constraint, either the simple variable bounds, linear constraint or nonlinear constraint of the problem.
It can be also used to fix any given variable.
.. _e04td-py2-py-doc:
For full information please refer to the NAG Library document for e04td
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04tdf.html
.. _e04td-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**comp** : str
The type of the component of the model whose bounds will be modified. :math:`\mathrm{comp}` is case insensitive.
:math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'VAR'}` or :math:`\texttt{'VARIABLE'}`
Simple bounds of a decision variable :math:`x_i` (see :meth:`handle_set_simplebounds`).
:math:`\mathrm{comp} = \texttt{'LC'}` or :math:`\texttt{'LINEAR CONSTRAINT'}`
Linear constraint (see :meth:`handle_set_linconstr`).
:math:`\mathrm{comp} = \texttt{'NLC'}` or :math:`\texttt{'NONLINEAR CONSTRAINT'}`
Nonlinear constraint (see :meth:`handle_set_nlnconstr`).
**idx** : int
:math:`i`, the index of the component :math:`\mathrm{comp}` whose bounds will be changed.
**bli** : float
:math:`\mathrm{bli}` defines the new lower bound
**bui** : float
:math:`\mathrm{bui}` defines the new upper bound
.. _e04td-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`6`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{comp} = \texttt{'X'}`, :math:`\texttt{'LC'}` or :math:`\texttt{'NLC'}`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{comp} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`M = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idx}\leq M`.
(`errno` :math:`10`)
On entry, :math:`\mathrm{bli} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bui} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bli}\leq \mathrm{bui}`.
(`errno` :math:`10`)
On entry, :math:`\mathrm{bli} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bli} < \textit{bigbnd}`.
(`errno` :math:`10`)
On entry, :math:`\mathrm{bui} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bui} > -\textit{bigbnd}`.
.. _e04td-py2-py-notes:
**Notes**
After the model has been fully or partially built by calling functions of the NAG optimization modelling suite, ``handle_set_bound`` may be used to define a new lower bound, :math:`l_i`, and an upper bound :math:`u_i`, of a single constraint identified by the component type :math:`\mathrm{comp}` together with the appropriate component index :math:`i` given in :math:`\mathrm{idx}`.
Note that both upper and lower bounds need to be specified.
This form allows full generality in specifying various types of constraint.
In particular, the model component may be set as an equality by setting :math:`l_i = u_i` (e.g., to fix a variable).
If certain bounds are not present, the associated value :math:`l_i` or :math:`u_i` may be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
See the description of the option 'Infinite Bound Size' which is common among all solvers in the suite.
Its value is denoted as :math:`\textit{bigbnd}` further in this text.
Note that the bounds are interpreted based on its value at the time of calling this function and any later alterations to 'Infinite Bound Size' will not affect these constraints.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_linobj_coeff(handle, idxci, ci):
r"""
``handle_set_linobj_coeff`` is a part of the NAG optimization modelling suite and sets or modifies a single element in the linear objective function of the problem.
.. _e04te-py2-py-doc:
For full information please refer to the NAG Library document for e04te
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04tef.html
.. _e04te-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**idxci** : int
:math:`i`, the index of the element :math:`c_i` in the linear objective function.
**ci** : float
:math:`c_i`, the new value of the element in the linear objective function.
.. _e04te-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`3`)
The objective function is not linear.
(`errno` :math:`8`)
On entry, :math:`\mathrm{idxci} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxci}\leq n`.
.. _e04te-py2-py-notes:
**Notes**
After the model has been fully or partially built by calling functions from the NAG optimization modelling suite, ``handle_set_linobj_coeff`` may be used to set or modify a single element :math:`c_i` in the (possibly sparse) linear objective function :math:`c^\mathrm{T}x`.
This might be particularly helpful when the objective function needs to be extended after new variables have been added to the model by :meth:`handle_add_vars`.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
"""
raise NotImplementedError
[docs]def handle_set_linconstr_coeff(handle, idlc, icolbj, bij):
r"""
``handle_set_linconstr_coeff`` is a part of the NAG optimization modelling suite and sets or modifies a single element in the linear constraint matrix of the problem.
.. _e04tj-py2-py-doc:
For full information please refer to the NAG Library document for e04tj
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04tjf.html
.. _e04tj-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**idlc** : int
:math:`i`, the number of the linear constraint, i.e., the row index of the element :math:`b_{\textit{ij}}` in the linear constraint matrix to be changed.
**icolbj** : int
:math:`j`, the column (variable) index of the element :math:`b_{\textit{ij}}`.
**bij** : float
:math:`b_{\textit{ij}}`, the new value of the element in the linear constraint matrix :math:`B`.
.. _e04tj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The problem cannot be modified right now, the solver is running.
(`errno` :math:`3`)
There are no linear constraints in the problem.
(`errno` :math:`8`)
On entry, :math:`\mathrm{icolbj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{icolbj}\leq n`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{idlc} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`m_B = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idlc}\leq m_B`.
.. _e04tj-py2-py-notes:
**Notes**
After the model has been fully or partially built by calling functions of the NAG optimization modelling suite, ``handle_set_linconstr_coeff`` may be used to set or modify a single element :math:`b_{\textit{ij}}` in the sparse linear constraint matrix :math:`B` previously defined by :meth:`handle_set_linconstr`.
This might be particularly helpful when a linear constraint needs to be extended after new variables have been added to the model by :meth:`handle_add_vars`.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
"""
raise NotImplementedError
[docs]def nlp1_solve(a, bl, bu, objfun, x, comm, confun=None, istate=None, cjac=None, clamda=None, r=None, data=None, io_manager=None, spiked_sorder='C'):
r"""
``nlp1_solve`` is designed to minimize an arbitrary smooth function subject to constraints (which may include simple bounds on the variables, linear constraints and smooth nonlinear constraints) using a Sequential Quadratic Programming (SQP) method.
As many first derivatives as possible should be supplied by you; any unspecified derivatives are approximated by finite differences.
It is not intended for large sparse problems.
``nlp1_solve`` may also be used for unconstrained, bound-constrained and linearly constrained optimization.
``nlp1_solve`` uses **forward** **communication** for evaluating the objective function, the nonlinear constraint functions, and any of their derivatives.
Note: this function uses optional algorithmic parameters, see also: :meth:`nlp1_option_file`, :meth:`nlp1_option_string`, :meth:`nlp1_init`.
.. _e04uc-py2-py-doc:
For full information please refer to the NAG Library document for e04uc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html
.. _e04uc-py2-py-parameters:
**Parameters**
**a** : float, array-like, shape :math:`\left(\textit{nclin}, :\right)`
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` contains the :math:`\textit{i}`\ th row of the matrix :math:`A_L` of general linear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn1>`__. That is, the :math:`\textit{i}`\ th row contains the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nclin}`.
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**objfun** : callable (objf, objgrd) = objfun(mode, x, objgrd, nstate, data=None)
:math:`\mathrm{objfun}` must calculate the objective function :math:`F\left(x\right)` and (optionally) its gradient :math:`g\left(x\right) = \frac{{\partial F}}{{\partial x}}` for a specified :math:`n`-vector :math:`x`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{objf}`.
:math:`\mathrm{mode} = 1`
All available elements of :math:`\mathrm{objgrd}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{objf}` and all available elements of :math:`\mathrm{objgrd}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the objective function and/or all available elements of its gradient are to be evaluated.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`
The elements of :math:`\mathrm{objgrd}` are set to special values which enable ``nlp1_solve`` to detect whether they are changed by :math:`\mathrm{objfun}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``nlp1_solve`` is calling :math:`\mathrm{objfun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**objf** : float
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{objf}` must be set to the value of the objective function at :math:`x`.
**objgrd** : float, array-like, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 1` or :math:`2`, :math:`\mathrm{objgrd}` must return the available elements of the gradient evaluated at :math:`x`.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**confun** : None or callable (c, cjac) = confun(mode, needc, x, cjac, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the vector :math:`c\left(x\right)` of nonlinear constraint functions and (optionally) its Jacobian (:math:`\text{} = \frac{{\partial c}}{{\partial x}}`) for a specified :math:`n`-element vector :math:`x`.
If there are no nonlinear constraints (i.e., :math:`\textit{ncnln} = 0`), :math:`\mathrm{confun}` will never be called by ``nlp1_solve`` and :math:`\mathrm{confun}` may be **None**. If there are nonlinear constraints, the first call to :math:`\mathrm{confun}` will occur before the first call to :math:`\mathrm{objfun}`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{confun}`. Only the following values need be assigned, for each value of :math:`i` such that :math:`\mathrm{needc}[i-1] > 0`:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{c}[i-1]`.
:math:`\mathrm{mode} = 1`
All available elements in the :math:`i`\ th row of :math:`\mathrm{cjac}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{c}[i-1]` and all available elements in the :math:`i`\ th row of :math:`\mathrm{cjac}`.
**needc** : int, ndarray, shape :math:`\left(\textit{ncnln}\right)`
The indices of the elements of :math:`\mathrm{c}` and/or :math:`\mathrm{cjac}` that must be evaluated by :math:`\mathrm{confun}`. If :math:`\mathrm{needc}[i-1] > 0`, the :math:`i`\ th element of :math:`\mathrm{c}` and/or the available elements of the :math:`i`\ th row of :math:`\mathrm{cjac}` (see argument :math:`\mathrm{mode}`) must be evaluated at :math:`x`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the constraint functions and/or the available elements of the constraint Jacobian are to be evaluated.
**cjac** : float, ndarray, shape :math:`\left(\max\left(1,\textit{ncnln}\right), n\right)`
Unless :math:`\text{‘Derivative Level'} = 2` or :math:`3`, the elements of :math:`\mathrm{cjac}` are set to special values which enable ``nlp1_solve`` to detect whether they are changed by :math:`\mathrm{confun}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``nlp1_solve`` is calling :math:`\mathrm{confun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**c** : float, array-like, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{c}[i-1]` must contain the value of the :math:`i`\ th constraint at :math:`x`. The remaining elements of :math:`\mathrm{c}`, corresponding to the non-positive elements of :math:`\mathrm{needc}`, are ignored.
**cjac** : float, array-like, shape :math:`\left(\max\left(1,\textit{ncnln}\right), n\right)`
If :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 1` or :math:`2`, the :math:`i`\ th row of :math:`\mathrm{cjac}` must contain the available elements of the vector :math:`\nabla c_i` given by
.. math::
\nabla c_i = \left(\frac{{\partial c_i}}{{\partial x_1}},\frac{{\partial c_i}}{{\partial x_2}},\ldots,\frac{{\partial c_i}}{{\partial x_n}}\right)^\mathrm{T}\text{,}
where :math:`\frac{{\partial c_i}}{{\partial x_j}}` is the partial derivative of the :math:`i`\ th constraint with respect to the :math:`j`\ th variable, evaluated at the point :math:`x`. See also the argument :math:`\mathrm{nstate}`. The remaining rows of :math:`\mathrm{cjac}`, corresponding to non-positive elements of :math:`\mathrm{needc}`, are ignored.
If all elements of the constraint Jacobian are known (i.e., :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any constant elements may be assigned to :math:`\mathrm{cjac}` one time only at the start of the optimization.
An element of :math:`\mathrm{cjac}` that is not subsequently assigned in :math:`\mathrm{confun}` will retain its initial value throughout.
Constant elements may be loaded into :math:`\mathrm{cjac}` either before the call to ``nlp1_solve`` or during the first call to :math:`\mathrm{confun}` (signalled by the value :math:`\mathrm{nstate} = 1`).
The ability to preload constants is useful when many Jacobian elements are identically zero, in which case :math:`\mathrm{cjac}` may be initialized to zero and nonzero elements may be reset by :math:`\mathrm{confun}`.
Note that constant nonzero elements do affect the values of the constraints.
Thus, if :math:`\mathrm{cjac}[i-1,j-1]` is set to a constant value, it need not be reset in subsequent calls to :math:`\mathrm{confun}`, but the value :math:`\mathrm{cjac}[i-1,j-1]\times \mathrm{x}[j-1]` must nonetheless be added to :math:`\mathrm{c}[i-1]`.
For example, if :math:`\mathrm{cjac}[0,0] = 2` and :math:`\mathrm{cjac}[0,1] = -5`, then the term :math:`2\times \mathrm{x}[0]-5\times \mathrm{x}[1]` must be included in the definition of :math:`\mathrm{c}[0]`.
It must be emphasized that, if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{cjac}` are not treated as constant; they are estimated by finite differences, at nontrivial expense.
If you do not supply a value for the option 'Difference Interval', an interval for each element of :math:`x` is computed automatically at the start of the optimization.
The automatic procedure can usually identify constant elements of :math:`\mathrm{cjac}`, which are then computed once only by finite differences.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, the elements of :math:`\mathrm{istate}` corresponding to the bounds and linear constraints define the initial working set for the procedure that finds a feasible point for the linear constraints and bounds.
The active set at the conclusion of this procedure and the elements of :math:`\mathrm{istate}` corresponding to nonlinear constraints then define the initial working set for the first QP subproblem.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, the next :math:`n_L` elements refer to the upper and lower bounds on :math:`A_Lx`, and the next :math:`n_N` elements refer to the upper and lower bounds on :math:`c\left(x\right)`.
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+===================================================================================================================================================+
|0 |The corresponding constraint is `not` in the initial QP working set. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |This inequality constraint should be in the working set at its lower bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |This inequality constraint should be in the working set at its upper bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |This equality constraint should be in the initial working set. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be modified by the function.
If ``nlp1_solve`` has been called previously with the same values of :math:`\textit{n}`, :math:`\textit{nclin}` and :math:`\textit{ncnln}`, :math:`\mathrm{istate}` already contains satisfactory information.
The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**cjac** : None or float, array-like, shape :math:`\left(\textit{ncnln}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{ncnln} > 0`: :math:`n`; otherwise: :math:`1`.
In general, :math:`\mathrm{cjac}` need not be initialized before the call to ``nlp1_solve``. However, if :math:`\text{‘Derivative Level'} = 2` or :math:`3`, you may optionally set the constant elements of :math:`\mathrm{cjac}` (see argument :math:`\mathrm{nstate}` in the description of :math:`\mathrm{confun}`). Such constant elements need not be re-assigned on subsequent calls to :math:`\mathrm{confun}`.
**clamda** : None or float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{clamda}[\textit{j}-1]` must contain a multiplier estimate for each nonlinear constraint with a sign that matches the status of the constraint specified by the :math:`\mathrm{istate}` array, for :math:`\textit{j} = n+\textit{nclin}+1,\ldots,n+\textit{nclin}+\textit{ncnln}`.
The remaining elements need not be set.
Note that if the :math:`j`\ th constraint is defined as 'inactive' by the initial value of :math:`\mathrm{istate}` array (i.e., :math:`\mathrm{istate}[j-1] = 0`), :math:`\mathrm{clamda}[j-1]` should be zero; if the :math:`j`\ th constraint is an inequality active at its lower bound (i.e., :math:`\mathrm{istate}[j-1] = 1`), :math:`\mathrm{clamda}[j-1]` should be non-negative; if the :math:`j`\ th constraint is an inequality active at its upper bound (i.e., :math:`\mathrm{istate}[j-1] = 2`), :math:`\mathrm{clamda}[j-1]` should be non-positive.
If necessary, the function will modify :math:`\mathrm{clamda}` to match these rules.
**r** : None or float, array-like, shape :math:`\left(n, n\right)`, optional
Need not be initialized if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{r}` must contain the upper triangular Cholesky factor :math:`R` of the initial approximation of the Hessian of the Lagrangian function, with the variables in the natural order.
Elements not in the upper triangular part of :math:`\mathrm{r}` are assumed to be zero and need not be assigned.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**spiked_sorder** : str, optional
If :math:`\mathrm{a}` and :math:`\mathrm{r}` are spiked (i.e., have unit extent in all but one dimension, or have size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with them in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
Two-dimensional arrays returned from callback functions in this routine must then use the same storage order.
**Returns**
**itera** : int
The number of major iterations performed.
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
The status of the constraints in the QP working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================================================================+
|:math:`-2` |This constraint violates its lower bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |This constraint violates its upper bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the QP working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the QP working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the QP working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the QP working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**c** : None or float, ndarray, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{c}[\textit{i}-1]` contains the value of the :math:`\textit{i}`\ th nonlinear constraint function :math:`c_{\textit{i}}` at the final iterate, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
If :math:`\textit{ncnln} = 0`, :math:`\mathrm{c}` is returned as **None**.
**cjac** : None or float, ndarray, shape :math:`\left(\textit{ncnln}, :\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{cjac}` contains the Jacobian matrix of the nonlinear constraint functions at the final iterate, i.e., :math:`\mathrm{cjac}[\textit{i}-1,\textit{j}-1]` contains the partial derivative of the :math:`\textit{i}`\ th constraint function with respect to the :math:`\textit{j}`\ th variable, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`. (See the discussion of argument :math:`\mathrm{cjac}` under :math:`\mathrm{confun}`.)
If :math:`\textit{ncnln} = 0`, :math:`\mathrm{cjac}` is returned as **None**.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
The values of the QP multipliers from the last QP subproblem. :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1` and non-positive if :math:`\mathrm{istate}[j-1] = 2`.
**objf** : float
The value of the objective function at the final iterate.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`
The gradient of the objective function at the final iterate (or its finite difference approximation).
**r** : float, ndarray, shape :math:`\left(n, n\right)`
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`Q^\mathrm{T}\tilde{H}Q`, an estimate of the transformed and reordered Hessian of the Lagrangian at :math:`x` (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn6>`__ in `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__). If :math:`\text{‘Hessian'} = \texttt{'YES'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`H`, the approximate (untransformed) Hessian of the Lagrangian, with the variables in the natural order.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final estimate of the solution.
.. _e04uc-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default values are computed
If the algorithm switches to central differences because the forward-difference approximation is not sufficiently accurate, the value of :math:`r` is used as the difference interval for every element of :math:`x`.
The switch to central differences is indicated by C at the end of each line of intermediate printout produced by the major iterations (see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__).
The use of finite differences is discussed further under the option 'Difference Interval'.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Cold Start'** : valueless
Default
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp1_solve`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04uc-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp1_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp1_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp1_solve`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04uc-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp1_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp1_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``nlp1_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Level'** : int
Default :math:`\text{} = 3`
This argument indicates which derivatives are provided in functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+====================================================================================================================+
|3 |All elements of the objective gradient and the constraint Jacobian are provided. |
+---------+--------------------------------------------------------------------------------------------------------------------+
|2 |All elements of the constraint Jacobian are provided, but some elements of the objective gradient are not specified.|
+---------+--------------------------------------------------------------------------------------------------------------------+
|1 |All elements of the objective gradient are provided, but some elements of the constraint Jacobian are not specified.|
+---------+--------------------------------------------------------------------------------------------------------------------+
|0 |Some elements of both the objective gradient and the constraint Jacobian are not specified. |
+---------+--------------------------------------------------------------------------------------------------------------------+
The value :math:`i = 3` should be used whenever possible, since ``nlp1_solve`` is more reliable (and will usually be more efficient) when all derivatives are exact.
If :math:`i = 0` or :math:`2`, ``nlp1_solve`` will estimate the unspecified elements of the objective gradient, using finite differences.
The computation of finite difference approximations usually increases the total run-time, since a call to :math:`\mathrm{objfun}` is required for each unspecified element.
Furthermore, less accuracy can be attained in the solution (see Module 8 of Gill `et al.` (1981), for a discussion of limiting accuracy).
If :math:`i = 0` or :math:`1`, ``nlp1_solve`` will approximate unspecified elements of the constraint Jacobian.
One call to :math:`\mathrm{confun}` is needed for each variable for which partial derivatives are not available.
For example, if the Jacobian has the form
.. math::
\begin{pmatrix}\text{*}&\text{*}&\text{*}&\text{*}\\\text{*}&\text{?}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{*}&\text{*}\end{pmatrix}
where ':math:`*`' indicates an element provided by you and '?' indicates an unspecified element, ``nlp1_solve`` will call :math:`\mathrm{confun}` twice: once to estimate the missing element in column 2, and again to estimate the two missing elements in column :math:`3`. (Since columns :math:`1` and :math:`4` are known, they require no calls to :math:`\mathrm{confun}`.)
At times, central differences are used rather than forward differences, in which case twice as many calls to :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` are needed. (The switch to central differences is not under your control.)
If :math:`i < 0` or :math:`i > 3`, the default value is used.
**'Difference Interval'** : float
Default values are computed
This option defines an interval used to estimate derivatives by finite differences in the following circumstances:
(a) For verifying the objective and/or constraint gradients (see the description of the option 'Verify').
(#) For estimating unspecified elements of the objective gradient or the constraint Jacobian.
In general, a derivative with respect to the :math:`j`\ th variable is approximated using the interval :math:`\delta_j`, where :math:`\delta_j = r\left(1+\left\lvert \hat{x}_j\right\rvert \right)`, with :math:`\hat{x}` the first point feasible with respect to the bounds and linear constraints.
If the functions are well scaled, the resulting derivative approximation should be accurate to :math:`\mathrm{O}\left(r\right)`.
See Gill `et al.` (1981) for a discussion of the accuracy in finite difference approximations.
If a difference interval is not specified by you, a finite difference interval will be computed automatically for each variable by a procedure that requires up to six calls of :math:`\mathrm{confun}` and :math:`\mathrm{objfun}` for each element.
This option is recommended if the function is badly scaled or you wish to have ``nlp1_solve`` determine constant elements in the objective and constraint gradients (see the descriptions of :math:`\mathrm{confun}` and :math:`\mathrm{objfun}` in :ref:`Parameters <e04uc-py2-py-parameters>`).
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The scalar :math:`r` defines the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a constraint is considered satisfied if its violation does not exceed :math:`r`.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
Using this keyword sets both options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance' to :math:`r`, if :math:`\epsilon \leq r < 1`. (Additional details are given under the descriptions of these options.)
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.9}`
This argument defines :math:`\epsilon_r`, which is intended to be a measure of the accuracy with which the problem functions :math:`F\left(x\right)` and :math:`c\left(x\right)` can be computed.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
The value of :math:`\epsilon_r` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`; i.e., :math:`\epsilon_r` acts as a relative precision when :math:`\left\lvert F\right\rvert` is large and as an absolute precision when :math:`\left\lvert F\right\rvert` is small.
For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
In contrast, if :math:`F\left(x\right)` is typically of order :math:`10^{-4}` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
However, when the accuracy of the computed function values is known to be significantly worse than full precision, the value of :math:`\epsilon_r` should be large enough so that ``nlp1_solve`` will not attempt to distinguish between function values that differ by less than the error inherent in the calculation.
**'Hessian'** : str
Default :math:`= \mathrm{NO}`
This option controls the contents of the upper triangular matrix :math:`R` (see :ref:`Parameters <e04uc-py2-py-parameters>`). ``nlp1_solve`` works exclusively with the `transformed and reordered` Hessian :math:`H_Q` `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn6>`__, and hence extra computation is required to form the Hessian itself.
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the Cholesky factor of the transformed and reordered Hessian.
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, the Cholesky factor of the approximate Hessian itself is formed and stored in :math:`\mathrm{r}`.
You should select :math:`\text{‘Hessian'} = \texttt{'YES'}` if a 'Warm Start' will be used for the next call to ``nlp1_solve``.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that is treated as a step to an unbounded solution.
If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Line Search Tolerance'** : float
Default :math:`\text{} = 0.9`
The value :math:`r` (:math:`0\leq r < 1`) controls the accuracy with which the step :math:`\alpha` taken during each iteration approximates a minimum of the merit function along the search direction (the smaller the value of :math:`r`, the more accurate the linesearch).
The default value :math:`r = 0.9` requests an inaccurate search and is appropriate for most problems, particularly those with any nonlinear constraints.
If there are no nonlinear constraints, a more accurate search may be appropriate when it is desirable to reduce the number of major iterations -- for example, if the objective function is cheap to evaluate, or if a substantial number of derivatives are unspecified.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'Linear Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`, and similarly for a nonlinear constraint and :math:`r_2`.
If :math:`r_{\textit{m}} < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``nlp1_solve``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``nlp1_solve`` (see the discussion of option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'Nonlinear Feasibility Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.33}` or :math:`\sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`, and similarly for a nonlinear constraint and :math:`r_2`.
If :math:`r_{\textit{m}} < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``nlp1_solve``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``nlp1_solve`` (see the discussion of option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Major Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Major Print Level'** : int
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Print Level'** : int
Default for ``nlp1_solve`` :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Minor Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L+n_N\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations for finding a feasible point with respect to the bounds and linear constraints (if any).
The value of :math:`i` also specifies the maximum number of minor iterations for the optimality phase of each QP subproblem.
If :math:`i\leq 0`, the default value is used.
**'Minor Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the minor iterations of ``nlp1_solve`` (i.e., the iterations of the quadratic programming algorithm), as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__ (summary output at each minor iteration and the final QP solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#monitoring>`__ (monitoring information at each minor iteration). (See also the description of the option 'Major Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final QP solution only. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each minor iteration (no printout of the final QP solution).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final QP solution and one line of summary output for each minor iteration. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each minor iteration (no printout of the final QP solution). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each minor iteration, the current estimates of the QP multipliers, the current estimate of the QP search direction, the QP constraint values, and the status of each QP constraint. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each minor iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__) of the QP working set, and the diagonal elements of the Cholesky factor :math:`R` of the transformed Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__).|
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Major Print Level'} \geq 5` or :math:`i\geq 0` and :math:`\text{‘Minor Print Level'} \geq 5`, monitoring information produced by ``nlp1_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Major Print Level'} < 5` and :math:`\text{‘Minor Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon_r^{0.8}`
The argument :math:`r` (:math:`\epsilon_r\leq r < 1`) specifies the accuracy to which you wish the final iterate to approximate a solution of the problem.
Broadly speaking, :math:`r` indicates the number of correct figures desired in the objective function at the solution.
For example, if :math:`r` is :math:`10^{-6}` and ``nlp1_solve`` terminates successfully, the final value of :math:`F` should have approximately six correct figures.
If :math:`r < \epsilon_r` or :math:`r\geq 1`, the default value is used.
``nlp1_solve`` will terminate successfully if the iterative sequence of :math:`x` values is judged to have converged and the final point satisfies the first-order Kuhn--Tucker conditions (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__).
The sequence of iterates is considered to have converged at :math:`x` if
.. math::
\alpha \left\lVert p\right\rVert \leq \sqrt{r}\left(1+\left\lVert x\right\rVert \right)\text{,}
where :math:`p` is the search direction and :math:`\alpha` the step length from `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn3>`__.
An iterate is considered to satisfy the first-order conditions for a minimum if
.. math::
\left\lVert Z^\mathrm{T}g_{\mathrm{FR}}\right\rVert \leq \sqrt{r}\left(1+\mathrm{max}\left({1+\left\lvert F\left(x\right)\right\rvert }, \left\lVert g_{\mathrm{FR}}\right\rVert \right)\right)
and
.. math::
\left\lvert \textit{res}_j\right\rvert \leq \textit{ftol}\quad \text{ for all }\quad j\text{,}
where :math:`Z^\mathrm{T}g_{\mathrm{FR}}` is the projected gradient (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#ad-overview>`__), :math:`g_{\mathrm{FR}}` is the gradient of :math:`F\left(x\right)` with respect to the free variables, :math:`\textit{res}_j` is the violation of the :math:`j`\ th active nonlinear constraint, and :math:`\textit{ftol}` is the 'Nonlinear Feasibility Tolerance'.
**'Start Objective Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Objective Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Start Constraint Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Constraint Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Step Limit'** : float
Default :math:`\text{} = 2.0`
If :math:`r > 0,r` specifies the maximum change in variables at the first step of the linesearch.
In some cases, such as :math:`F\left(x\right) = ae^{{bx}}` or :math:`F\left(x\right) = ax^b`, even a moderate change in the elements of :math:`x` can lead to floating-point overflow.
The argument :math:`r` is, therefore, used to encourage evaluation of the problem functions at meaningful points.
Given any major iterate :math:`x`, the first point :math:`\tilde{x}` at which :math:`F` and :math:`c` are evaluated during the linesearch is restricted so that
.. math::
\left\lVert \tilde{x}-x\right\rVert_2\leq r\left(1+\left\lVert x\right\rVert_2\right)\text{.}
The linesearch may go on and evaluate :math:`F` and :math:`c` at points further from :math:`x` if this will result in a lower value of the merit function (indicated by L at the end of each line of output produced by the major iterations; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#fc-printedoutput>`__).
If L is printed for most of the iterations, :math:`r` should be set to a larger value.
Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at wild values.
The default value :math:`\text{‘Step Limit'} = 2.0` should not affect progress on well-behaved functions, but values such as :math:`0.1` or :math:`0.01` may be helpful when rapidly varying functions are present.
If a small value of 'Step Limit' is selected, a good starting point may be required.
An important application is to the class of nonlinear least squares problems.
If :math:`r\leq 0`, the default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Constraint Check At Variable' and 'Stop Constraint Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_solve`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Constraint Check At Variable' and 'Stop Constraint Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_solve`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Constraint Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Constraint Check At Variable' and 'Stop Constraint Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_solve`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Constraint Check At Variable' and 'Stop Constraint Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_solve`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Objective Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Constraint Check At Variable' and 'Stop Constraint Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_solve`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
.. _e04uc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`7`)
Large errors found in the derivatives.
(`errno` :math:`9`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`9`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`9`)
On entry, :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{ncnln}\geq 0`.
(`errno` :math:`9`)
Not enough workspace to solve problem. Workspace provided is :math:`\textit{iwork}[\langle\mathit{\boldsymbol{value}}\rangle]` and :math:`\textit{work}[\langle\mathit{\boldsymbol{value}}\rangle]`. To solve problem we need :math:`\textit{iwork}[\langle\mathit{\boldsymbol{value}}\rangle]` and :math:`\textit{work}[\langle\mathit{\boldsymbol{value}}\rangle]`.
(`errno` :math:`9`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Optimal solution found, but requested accuracy not achieved.
(`errno` :math:`2`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
No feasible point for the nonlinear constraints.
(`errno` :math:`6`)
Current point cannot be improved upon.
**NagAlgorithmicMajorWarning**
(`errno` :math:`4`)
Too many major iterations.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{mode}` negative in :math:`\mathrm{objfun}` or :math:`\mathrm{confun}`.
.. _e04uc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``nlp1_solve`` is designed to solve the nonlinear programming problem -- the minimization of a smooth nonlinear function subject to a set of constraints on the variables.
The problem is assumed to be stated in the following form:
.. math::
\textit{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad l\leq \begin{pmatrix}x\\A_Lx\\c\left(x\right)\end{pmatrix}\leq u\text{,}
where :math:`F\left(x\right)` (the `objective function`) is a nonlinear function, :math:`A_L` is an :math:`n_L\times n` constant matrix, and :math:`c\left(x\right)` is an :math:`n_N` element vector of nonlinear constraint functions. (The matrix :math:`A_L` and the vector :math:`c\left(x\right)` may be empty.) The objective function and the constraint functions are assumed to be smooth, i.e., at least twice-continuously differentiable. (The method of ``nlp1_solve`` will usually solve `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn1>`__ if there are only isolated discontinuities away from the solution.)
Note that although the bounds on the variables could be included in the definition of the linear constraints, we prefer to distinguish between them for reasons of computational efficiency.
For the same reason, the linear constraints should **not** be included in the definition of the nonlinear constraints.
Upper and lower bounds are specified for all the variables and for all the constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
If there are no nonlinear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn1>`__ and :math:`F` is linear or quadratic, then it will generally be more efficient to use one of :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve`, or :meth:`handle_solve_ssqp` if the problem is large and sparse.
If the problem is large and sparse and does have nonlinear constraints, then :meth:`handle_solve_ssqp` should be used, since ``nlp1_solve`` treats all matrices as dense.
You must supply an initial estimate of the solution to `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#eqn1>`__, together with functions that define :math:`F\left(x\right)`, :math:`c\left(x\right)` and as many first partial derivatives as possible; unspecified derivatives are approximated by finite differences.
The objective function is defined by :math:`\mathrm{objfun}`, and the nonlinear constraints are defined by :math:`\mathrm{confun}`.
On every call, these functions must return appropriate values of the objective and nonlinear constraints.
You should also provide the available partial derivatives.
Any unspecified derivatives are approximated by finite differences; see :ref:`Other Parameters <e04uc-py2-py-other_params>` for a discussion of the option 'Derivative Level'.
Note that if there `are` any nonlinear constraints then the `first` call to :math:`\mathrm{confun}` will precede the `first` call to :math:`\mathrm{objfun}`.
For maximum reliability, it is preferable for you to provide all partial derivatives (see Module 8 of Gill `et al.` (1981), for a detailed discussion).
If all gradients cannot be provided, it is similarly advisable to provide as many as possible.
While developing :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`, the option 'Verify' should be used to check the calculation of any known gradients.
The method used by ``nlp1_solve`` is described in detail in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ucf.html#algdetails>`__.
:meth:`nlp1_rcomm` is an alternative function which uses exactly the same method, but uses **reverse** **communication** for evaluating the objective and constraint functions.
.. _e04uc-py2-py-references:
**References**
Dennis, J E Jr and Moré, J J, 1977, `Quasi-Newton methods, motivation and theory`, SIAM Rev. (19), 46--89
Dennis, J E Jr and Schnabel, R B, 1981, `A new derivation of symmetric positive-definite secant updates`, nonlinear programming, (eds O L Mangasarian, R R Meyer and S M Robinson) (4), 167--199, Academic Press
Dennis, J E Jr and Schnabel, R B, 1983, `Numerical Methods for Unconstrained Optimization and Nonlinear Equations`, Prentice--Hall
Fletcher, R, 1987, `Practical Methods of Optimization`, (2nd Edition), Wiley
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Users' guide for SOL/QPSOL version 3.2`, Report SOL, 84--5, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Some theoretical properties of an augmented Lagrangian merit function`, Report SOL, 86--6R, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Powell, M J D, 1974, `Introduction to constrained optimization`, Numerical Methods for Constrained Optimization, (eds P E Gill and W Murray), 1--28, Academic Press
Powell, M J D, 1983, `Variable metric methods in constrained optimization`, Mathematical Programming: the State of the Art, (eds A Bachem, M Grötschel and B Korte), 288--311, Springer--Verlag
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp1_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp1_option_file(ioptns, comm, io_manager=None):
r"""
``nlp1_option_file`` may be used to supply options to :meth:`nlp1_solve` from an external file.
``nlp1_option_file`` can also be used to supply options to :meth:`nlp1_rcomm`.
.. _e04ud-py2-py-doc:
For full information please refer to the NAG Library document for e04ud
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04udf.html
.. _e04ud-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ud-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04ud-py2-py-notes:
**Notes**
``nlp1_option_file`` may be used to supply values for options to :meth:`nlp1_solve`. ``nlp1_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp1_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp1_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp1_solve <e04uc-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp1_option_string(optstr, comm, io_manager=None):
r"""
``nlp1_option_string`` may be used to supply individual options to :meth:`nlp1_solve`
``nlp1_option_string`` can also be used to supply individual options to :meth:`nlp1_rcomm`.
.. _e04ue-py2-py-doc:
For full information please refer to the NAG Library document for e04ue
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uef.html
.. _e04ue-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04ue-py2-py-notes>` and in :ref:`Other Parameters for nlp1_solve <e04uc-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ue-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ue-py2-py-notes:
**Notes**
``nlp1_option_string`` may be used to supply values for options to :meth:`nlp1_solve`. It is only necessary to call ``nlp1_option_string`` for those arguments whose values are to be different from their default values.
One call to ``nlp1_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp1_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp1_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp1_solve <e04uc-py2-py-other_params>`.
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp1_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp1_rcomm(irevcm, nclin, a, bl, bu, itera, istate, c, cjac, clamda, objf, objgrd, r, x, comm, io_manager=None):
r"""
``nlp1_rcomm`` is designed to minimize an arbitrary smooth function subject to constraints (which may include simple bounds on the variables, linear constraints and smooth nonlinear constraints) using a Sequential Quadratic Programming (SQP) method.
You should supply as many first derivatives as possible; any unspecified derivatives are approximated by finite differences.
It is not intended for large sparse problems.
``nlp1_rcomm`` may also be used for unconstrained, bound-constrained and linearly constrained optimization.
``nlp1_rcomm`` uses **reverse communication** for evaluating the objective function, the nonlinear constraint functions and any of their derivatives.
Note: this function uses optional algorithmic parameters, see also: :meth:`nlp1_option_file`, :meth:`nlp1_option_string`, :meth:`nlp1_init`.
.. _e04uf-py2-py-doc:
For full information please refer to the NAG Library document for e04uf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html
.. _e04uf-py2-py-parameters:
**Parameters**
**irevcm** : int
`On initial entry`: must be set to :math:`0`.
`On intermediate entry`: **must remain unchanged**, unless you wish to terminate the solution to the current problem. In this case :math:`\mathrm{irevcm}` may be set to a negative value and then ``nlp1_rcomm`` will take a final exit with :math:`\textit{errno}` set to this value of :math:`\mathrm{irevcm}`.
**nclin** : int
`On initial entry`: :math:`n_L`, the number of general linear constraints.
**a** : float, array-like, shape :math:`\left(\mathrm{nclin}, :\right)`
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\mathrm{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
`On initial entry`: the :math:`i`\ th row of the matrix :math:`A_L` of general linear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn1>`__ must be stored in :math:`\mathrm{a}[\textit{i}-1,\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\mathrm{nclin}`. That is, the :math:`\textit{i}`\ th row contains the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,\mathrm{nclin}`.
If :math:`\mathrm{nclin} = 0`, the array :math:`\mathrm{a}` is not referenced.
**bl** : float, array-like, shape :math:`\left(n+\mathrm{nclin}+\textit{ncnln}\right)`
On initial entry :math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\mathrm{nclin}+\textit{ncnln}\right)`
On initial entry :math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**itera** : int
`On intermediate entry`: must remain unchanged from a previous call to ``nlp1_rcomm``.
**istate** : int, ndarray, shape :math:`\left(n+\mathrm{nclin}+\textit{ncnln}\right)`, modified in place
`On initial entry`: need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, the elements of :math:`\mathrm{istate}` corresponding to the bounds and linear constraints define the initial working set for the procedure that finds a feasible point for the linear constraints and bounds.
The active set at the conclusion of this procedure and the elements of :math:`\mathrm{istate}` corresponding to nonlinear constraints then define the initial working set for the first QP subproblem.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, the next :math:`n_L` elements refer to the upper and lower bounds on :math:`A_Lx`, and the next :math:`n_N` elements refer to the upper and lower bounds on :math:`c\left(x\right)`.
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+===================================================================================================================================================+
|0 |The corresponding constraint is `not` in the initial QP working set. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |This inequality constraint should be in the working set at its lower bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |This inequality constraint should be in the working set at its upper bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |This equality constraint should be in the initial working set. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be modified by the function.
If ``nlp1_rcomm`` has been called previously with the same values of :math:`\textit{n}`, :math:`\mathrm{nclin}` and :math:`\textit{ncnln}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
`On final exit`: the status of the constraints in the QP working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================================================================+
|:math:`-2` |This constraint violates its lower bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |This constraint violates its upper bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the QP working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the QP working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the QP working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the QP working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**c** : float, ndarray, shape :math:`\left(\textit{ncnln}\right)`, modified in place
`On initial entry`: need not be set.
`On intermediate entry`: if :math:`\mathrm{irevcm} = 4` or :math:`6` and :math:`\mathrm{needc}[i-1] > 0`, :math:`\mathrm{c}[i-1]` must contain the value of the :math:`i`\ th constraint at :math:`x`. The remaining elements of :math:`\mathrm{c}`, corresponding to the non-positive elements of :math:`\mathrm{needc}`, are ignored.
`On final exit`: if :math:`\textit{ncnln} > 0`, :math:`\mathrm{c}[\textit{i}-1]` contains the value of the :math:`\textit{i}`\ th nonlinear constraint function :math:`c_{\textit{i}}` at the final iterate, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
If :math:`\textit{ncnln} = 0`, the array :math:`\mathrm{c}` is not referenced.
**cjac** : float, ndarray, shape :math:`\left(\textit{ncnln}, :\right)`, modified in place
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{ncnln} > 0`: :math:`n`; otherwise: :math:`1`.
`On initial entry`: in general, :math:`\mathrm{cjac}` need not be initialized before the call to ``nlp1_rcomm``. However, if the option :math:`\text{‘Derivative Level'} = 2` or :math:`3`, you may optionally set the constant elements of :math:`\mathrm{cjac}`. Such constant elements need not be re-assigned on subsequent intermediate exits.
If all elements of the constraint Jacobian are known (i.e., :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any constant elements may be assigned to :math:`\mathrm{cjac}` one time only at the start of the optimization.
An element of :math:`\mathrm{cjac}` that is not subsequently assigned during an intermediate exit will retain its initial value throughout.
Constant elements may be loaded into :math:`\mathrm{cjac}` either before the call to ``nlp1_rcomm`` or during the first intermediate exit.
The ability to preload constants is useful when many Jacobian elements are identically zero, in which case :math:`\mathrm{cjac}` may be initialized to zero and nonzero elements may be reset during intermediate exits.
`On intermediate entry`: if :math:`\mathrm{irevcm} = 5` or :math:`6` and :math:`\mathrm{needc}[i-1] > 0`, the :math:`i`\ th row of :math:`\mathrm{cjac}` must contain the available elements of the vector :math:`\nabla c_i` given by
.. math::
\nabla c_i = \left(\frac{{\partial c_i}}{{\partial x_1}},\frac{{\partial c_i}}{{\partial x_2}},\ldots,\frac{{\partial c_i}}{{\partial x_n}}\right)^\mathrm{T}\text{,}
where :math:`\frac{{\partial c_i}}{{\partial x_j}}` is the partial derivative of the :math:`i`\ th constraint with respect to the :math:`j`\ th variable, evaluated at the point :math:`x`. The remaining rows of :math:`\mathrm{cjac}`, corresponding to non-positive elements of :math:`\mathrm{needc}`, are ignored. The :math:`\textit{i}`\ th row of the Jacobian should be stored in elements :math:`\mathrm{cjac}[\textit{i}-1,\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
Note that constant nonzero elements do affect the values of the constraints.
Thus, if :math:`\mathrm{cjac}[i-1,j-1]` is set to a constant value, it need not be reset during subsequent intermediate exits, but the value :math:`\mathrm{cjac}[i-1,j-1]\times \mathrm{x}[j-1]` must nonetheless be added to :math:`\mathrm{c}[i-1]`.
For example, if :math:`\mathrm{cjac}[0,0] = 2` and :math:`\mathrm{cjac}[0,1] = -5`, the term :math:`2\times \mathrm{x}[0]-5\times \mathrm{x}[1]` must be included in the definition of :math:`\mathrm{c}[0]`.
It must be emphasized that, if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{cjac}` are not treated as constant; they are estimated by finite differences, at nontrivial expense.
If you do not supply a value for the option 'Difference Interval', an interval for each element of :math:`x` is computed automatically at the start of the optimization.
The automatic procedure can usually identify constant elements of :math:`\mathrm{cjac}`, which are then computed once only by finite differences.
See also the description of the option 'Verify'.
`On final exit`: if :math:`\textit{ncnln} > 0`, :math:`\mathrm{cjac}` contains the Jacobian matrix of the nonlinear constraint functions at the final iterate, i.e., :math:`\mathrm{cjac}[\textit{i}-1,\textit{j}-1]` contains the partial derivative of the :math:`\textit{i}`\ th constraint function with respect to the :math:`\textit{j}`\ th variable, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
If :math:`\textit{ncnln} = 0`, the array :math:`\mathrm{cjac}` is not referenced.
**clamda** : float, ndarray, shape :math:`\left(n+\mathrm{nclin}+\textit{ncnln}\right)`, modified in place
`On initial entry`: need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{clamda}[\textit{j}-1]` must contain a multiplier estimate for each nonlinear constraint with a sign that matches the status of the constraint specified by the :math:`\mathrm{istate}` array, for :math:`\textit{j} = n+\mathrm{nclin}+1,\ldots,n+\mathrm{nclin}+\textit{ncnln}`.
The remaining elements need not be set.
Note that if the :math:`j`\ th constraint is defined as 'inactive' by the initial value of the :math:`\mathrm{istate}` array (i.e., :math:`\mathrm{istate}[j-1] = 0`), :math:`\mathrm{clamda}[j-1]` should be zero; if the :math:`j`\ th constraint is an inequality active at its lower bound (i.e., :math:`\mathrm{istate}[j-1] = 1`), :math:`\mathrm{clamda}[j-1]` should be non-negative; if the :math:`j`\ th constraint is an inequality active at its upper bound (i.e., :math:`\mathrm{istate}[j-1] = 2`), :math:`\mathrm{clamda}[j-1]` should be non-positive.
If necessary, the function will modify :math:`\mathrm{clamda}` to match these rules.
`On final exit`: the values of the QP multipliers from the last QP subproblem. :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1` and non-positive if :math:`\mathrm{istate}[j-1] = 2`.
**objf** : float
`On initial entry`: need not be set.
`On intermediate entry`: if :math:`\mathrm{irevcm} = 1` or :math:`3`, :math:`\mathrm{objf}` must be set to the value of the objective function at :math:`x`.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`, modified in place
`On initial entry`: need not be set.
`On intermediate entry`: if :math:`\mathrm{irevcm} = 2` or :math:`3`, :math:`\mathrm{objgrd}` must contain the available elements of the gradient evaluated at :math:`x`.
See also the description of the option 'Verify'.
`On final exit`: the gradient of the objective function at the final iterate (or its finite difference approximation).
**r** : float, ndarray, shape :math:`\left(n, n\right)`, modified in place
`On initial entry`: need not be initialized if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{r}` must contain the upper triangular Cholesky factor :math:`R` of the initial approximation of the Hessian of the Lagrangian function, with the variables in the natural order.
Elements not in the upper triangular part of :math:`\mathrm{r}` are assumed to be zero and need not be assigned.
`On final exit`: if :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`Q^\mathrm{T}\tilde{H}Q`, an estimate of the transformed and reordered Hessian of the Lagrangian at :math:`x` (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__ in `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`H`, the approximate (untransformed) Hessian of the Lagrangian, with the variables in the natural order.
**x** : float, ndarray, shape :math:`\left(n\right)`, modified in place
`On initial entry`: an initial estimate of the solution.
`On intermediate exit`: the point :math:`x` at which the objective function, constraint functions or their derivatives are to be evaluated.
`On final exit`: the final estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**irevcm** : int
`On intermediate exit`: specifies what values the calling program must assign to arguments of ``nlp1_rcomm`` before re-entering the function.
:math:`\mathrm{irevcm} = 1`
Set :math:`\mathrm{objf}` to the value of the objective function :math:`F\left(x\right)`.
:math:`\mathrm{irevcm} = 2`
Set :math:`\mathrm{objgrd}[\textit{j}-1]` to the value :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` if available, for :math:`\textit{j} = 1,2,\ldots,n`.
:math:`\mathrm{irevcm} = 3`
Set :math:`\mathrm{objf}` and :math:`\mathrm{objgrd}[j-1]` as for :math:`\mathrm{irevcm} = 1` and :math:`\mathrm{irevcm} = 2`.
:math:`\mathrm{irevcm} = 4`
Set :math:`\mathrm{c}[i-1]` to the value of the constraint function :math:`c_i\left(x\right)`, for each :math:`i` such that :math:`\mathrm{needc}[i-1] > 0`.
:math:`\mathrm{irevcm} = 5`
Set :math:`\mathrm{cjac}[i-1,j-1]` to the value :math:`\frac{{\partial c_i}}{{\partial x_j}}` if available, for each :math:`i` such that :math:`\mathrm{needc}[i-1] > 0` and :math:`j = 1,2,\ldots,n`.
:math:`\mathrm{irevcm} = 6`
Set :math:`\mathrm{c}[i-1]` and :math:`\mathrm{cjac}[i-1,j-1]` as for :math:`\mathrm{irevcm} = 4` and :math:`\mathrm{irevcm} = 5`.
`On final exit`: :math:`\mathrm{irevcm} = 0`.
**itera** : int
`On final exit`: the number of major iterations performed.
**objf** : float
`On final exit`: the value of the objective function at the final iterate.
**needc** : int, ndarray, shape :math:`\left(\max\left(1,\textit{ncnln}\right)\right)`
`On intermediate exit`: if :math:`\mathrm{irevcm}\geq 4`, :math:`\mathrm{needc}` specifies the indices of the elements of :math:`\mathrm{c}` and/or :math:`\mathrm{cjac}` that must be assigned. If :math:`\mathrm{needc}[i-1] > 0`, the :math:`i`\ th element of :math:`\mathrm{c}` and/or the available elements of the :math:`i`\ th row of :math:`\mathrm{cjac}` must be evaluated at :math:`x`.
.. _e04uf-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default values are computed
If the algorithm switches to central differences because the forward-difference approximation is not sufficiently accurate, the value of :math:`r` is used as the difference interval for every element of :math:`x`.
The switch to central differences is indicated by C at the end of each line of intermediate printout produced by the major iterations (see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__).
The use of finite differences is discussed further under the option 'Difference Interval'.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Cold Start'** : valueless
Default
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp1_rcomm`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04uf-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp1_rcomm`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp1_rcomm`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp1_rcomm`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04uf-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp1_rcomm`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp1_rcomm`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``nlp1_rcomm`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Level'** : int
Default :math:`\text{} = 3`
This argument indicates which derivatives are provided during intermediate exits.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+====================================================================================================================+
|3 |All elements of the objective gradient and the constraint Jacobian are provided. |
+---------+--------------------------------------------------------------------------------------------------------------------+
|2 |All elements of the constraint Jacobian are provided, but some elements of the objective gradient are not specified.|
+---------+--------------------------------------------------------------------------------------------------------------------+
|1 |All elements of the objective gradient are provided, but some elements of the constraint Jacobian are not specified.|
+---------+--------------------------------------------------------------------------------------------------------------------+
|0 |Some elements of both the objective gradient and the constraint Jacobian are not specified. |
+---------+--------------------------------------------------------------------------------------------------------------------+
The value :math:`i = 3` should be used whenever possible, since ``nlp1_rcomm`` is more reliable (and will usually be more efficient) when all derivatives are exact.
If :math:`i = 0` or :math:`2`, ``nlp1_rcomm`` will estimate the unspecified elements of the objective gradient, using finite differences.
The computation of finite difference approximations usually increases the total run-time, since an intermediate exit to the calling program is required for each unspecified element.
Furthermore, less accuracy can be attained in the solution (see Module 8 of Gill `et al.` (1981), for a discussion of limiting accuracy).
If :math:`i = 0` or :math:`1`, ``nlp1_rcomm`` will approximate unspecified elements of the constraint Jacobian.
One intermediate exit is needed for each variable for which partial derivatives are not available.
For example, if the Jacobian has the form
.. math::
\begin{pmatrix}\text{*}&\text{*}&\text{*}&\text{*}\\\text{*}&\text{?}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{*}&\text{*}\end{pmatrix}
where ':math:`*`' indicates an element provided and '?' indicates an unspecified element, ``nlp1_rcomm`` will make an intermediate exit to the calling program twice: once to estimate the missing element in column :math:`2`, and again to estimate the two missing elements in column :math:`3`. (Since columns :math:`1` and :math:`4` are known, they require no intermediate exits for information.)
At times, central differences are used rather than forward differences, in which case twice as many intermediate exits are needed. (The switch to central differences is not under your control.)
If :math:`i < 0` or :math:`i > 3`, the default value is used.
**'Difference Interval'** : float
Default values are computed
This option defines an interval used to estimate derivatives by finite differences in the following circumstances:
(a) For verifying the objective and/or constraint gradients (see the description of the option 'Verify').
(#) For estimating unspecified elements of the objective gradient or the constraint Jacobian.
In general, a derivative with respect to the :math:`j`\ th variable is approximated using the interval :math:`\delta_j`, where :math:`\delta_j = r\left(1+\left\lvert \hat{x}_j\right\rvert \right)`, with :math:`\hat{x}` the first point feasible with respect to the bounds and linear constraints.
If the functions are well scaled, then the resulting derivative approximation should be accurate to :math:`\mathrm{O}\left(r\right)`.
See Gill `et al.` (1981) for a discussion of the accuracy in finite difference approximations.
If a difference interval is not specified, then a finite difference interval will be computed automatically for each variable by a procedure that requires up to six intermediate exits for each element.
This option is recommended if the function is badly scaled or you wish to have ``nlp1_rcomm`` determine constant elements in the objective and constraint gradients.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The scalar :math:`r` defines the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a constraint is considered satisfied if its violation does not exceed :math:`r`.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
Using this keyword sets both options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance' to :math:`r`, if :math:`\epsilon \leq r < 1`. (Additional details are given under the descriptions of these options.)
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.9}`
This argument defines :math:`\epsilon_r`, which is intended to be a measure of the accuracy with which the problem functions :math:`F\left(x\right)` and :math:`c\left(x\right)` can be computed.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
The value of :math:`\epsilon_r` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`; i.e., :math:`\epsilon_r` acts as a relative precision when :math:`\left\lvert F\right\rvert` is large and as an absolute precision when :math:`\left\lvert F\right\rvert` is small.
For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
In contrast, if :math:`F\left(x\right)` is typically of order :math:`10^{-4}` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
However, when the accuracy of the computed function values is known to be significantly worse than full precision, the value of :math:`\epsilon_r` should be large enough so that ``nlp1_rcomm`` will not attempt to distinguish between function values that differ by less than the error inherent in the calculation.
**'Hessian'** : valueless
Default :math:`= \texttt{'NO'}`
This option controls the contents of the upper triangular matrix :math:`R` (see :ref:`Parameters <e04uf-py2-py-parameters>`). ``nlp1_rcomm`` works exclusively with the `transformed and reordered` Hessian :math:`H_Q` `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__, and hence extra computation is required to form the Hessian itself.
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the Cholesky factor of the transformed and reordered Hessian.
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, the Cholesky factor of the approximate Hessian itself is formed and stored in :math:`\mathrm{r}`.
You should select :math:`\text{‘Hessian'} = \texttt{'YES'}` if a 'Warm Start' will be used for the next call to ``nlp1_rcomm``.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that is treated as a step to an unbounded solution.
If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Line Search Tolerance'** : float
Default :math:`\text{} = 0.9`
The value :math:`r` (:math:`0\leq r < 1`) controls the accuracy with which the step :math:`\alpha` taken during each iteration approximates a minimum of the merit function along the search direction (the smaller the value of :math:`r`, the more accurate the linesearch).
The default value :math:`r = 0.9` requests an inaccurate search and is appropriate for most problems, particularly those with any nonlinear constraints.
If there are no nonlinear constraints, a more accurate search may be appropriate when it is desirable to reduce the number of major iterations -- for example, if the objective function is cheap to evaluate, or if a substantial number of derivatives are unspecified.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'Linear Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`.
Similarly a nonlinear constraint is considered satisfied if its violation does not exceed :math:`r_2`.
If :math:`r_{\textit{m}} < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``nlp1_rcomm``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``nlp1_rcomm`` (see the discussion of option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'Nonlinear Feasibility Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.33}` or :math:`\sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`.
Similarly a nonlinear constraint is considered satisfied if its violation does not exceed :math:`r_2`.
If :math:`r_{\textit{m}} < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``nlp1_rcomm``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``nlp1_rcomm`` (see the discussion of option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default for ``nlp1_rcomm``
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Major Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Major Print Level'** : int
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_rcomm``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Print Level'** : int
Default for ``nlp1_rcomm`` :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_rcomm``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Minor Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L+n_N\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations for finding a feasible point with respect to the bounds and linear constraints (if any).
The value of :math:`i` also specifies the maximum number of minor iterations for the optimality phase of each QP subproblem.
If :math:`i\leq 0`, the default value is used.
**'Minor Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the minor iterations of ``nlp1_rcomm`` (i.e., the iterations of the quadratic programming algorithm), as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__ (summary output at each minor iteration and the final QP solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#monitoring>`__) (monitoring information at each minor iteration). (See also the description of the option 'Major Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final QP solution only. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each minor iteration (no printout of the final QP solution).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final QP solution and one line of summary output for each minor iteration. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each minor iteration (no printout of the final QP solution). |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each minor iteration, the current estimates of the QP multipliers, the current estimate of the QP search direction, the QP constraint values and the status of each QP constraint. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each minor iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__) of the QP working set and the diagonal elements of the Cholesky factor :math:`R` of the transformed Hessian `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__ (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Major Print Level'} \geq 5` or :math:`i\geq 0` and :math:`\text{‘Minor Print Level'} \geq 5`, monitoring information produced by ``nlp1_rcomm`` at every iteration is sent to a file with logical unit number :math:`i`. If :math:`i < 0` and/or :math:`\text{‘Major Print Level'} < 5` and :math:`\text{‘Minor Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon_r^{0.8}`
The argument :math:`r` (:math:`\epsilon_r\leq r < 1`) specifies the accuracy to which you wish the final iterate to approximate a solution of the problem.
Broadly speaking, :math:`r` indicates the number of correct figures desired in the objective function at the solution.
For example, if :math:`r` is :math:`10^{-6}` and ``nlp1_rcomm`` terminates successfully, the final value of :math:`F` should have approximately six correct figures.
If :math:`r < \epsilon_r` or :math:`r\geq 1`, the default value is used.
``nlp1_rcomm`` will terminate successfully if the iterative sequence of :math:`x` values is judged to have converged and the final point satisfies the first-order Kuhn--Tucker conditions (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).
The sequence of iterates is considered to have converged at :math:`x` if
.. math::
\alpha \left\lVert p\right\rVert \leq \sqrt{r}\left(1+\left\lVert x\right\rVert \right)\text{,}
where :math:`p` is the search direction and :math:`\alpha` the step length from `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn3>`__.
An iterate is considered to satisfy the first-order conditions for a minimum if
.. math::
\left\lVert Z^\mathrm{T}g_{\mathrm{FR}}\right\rVert \leq \sqrt{r}\left(1+\mathrm{max}\left({1+\left\lvert F\left(x\right)\right\rvert }, \left\lVert g_{\mathrm{FR}}\right\rVert \right)\right)
and
.. math::
\left\lvert \textit{res}_j\right\rvert \leq \textit{ftol}\quad \text{ for all }\quad j\text{,}
where :math:`Z^\mathrm{T}g_{\mathrm{FR}}` is the projected gradient (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__), :math:`g_{\mathrm{FR}}` is the gradient of :math:`F\left(x\right)` with respect to the free variables, :math:`\textit{res}_j` is the violation of the :math:`j`\ th active nonlinear constraint, and :math:`\textit{ftol}` is the 'Nonlinear Feasibility Tolerance'.
**'Start Objective Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements and/or Jacobian elements computed by the calling program during intermediate exits.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Objective Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements and/or Jacobian elements computed by the calling program during intermediate exits.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Start Constraint Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements and/or Jacobian elements computed by the calling program during intermediate exits.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Constraint Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements and/or Jacobian elements computed by the calling program during intermediate exits.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Step Limit'** : float
Default :math:`\text{} = 2.0`
If :math:`r > 0,r` specifies the maximum change in variables at the first step of the linesearch.
In some cases, such as :math:`F\left(x\right) = ae^{{bx}}` or :math:`F\left(x\right) = ax^b`, even a moderate change in the elements of :math:`x` can lead to floating-point overflow.
The argument :math:`r` is, therefore, used to encourage evaluation of the problem functions at meaningful points.
Given any major iterate :math:`x`, the first point :math:`\tilde{x}` at which :math:`F` and :math:`c` are evaluated during the linesearch is restricted so that
.. math::
\left\lVert \tilde{x}-x\right\rVert_2\leq r\left(1+\left\lVert x\right\rVert_2\right)\text{.}
The linesearch may go on and evaluate :math:`F` and :math:`c` at points further from :math:`x` if this will result in a lower value of the merit function (indicated by L at the end of each line of output produced by the major iterations; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#fc-printedoutput>`__).
If L is printed for most of the iterations, :math:`r` should be set to a larger value.
Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at wild values.
The default value :math:`\text{‘Step Limit'} = 2.0` should not affect progress on well-behaved functions, but values such as :math:`0.1` or :math:`0.01` may be helpful when rapidly varying functions are present.
If a small value of 'Step Limit' is selected, then a good starting point may be required.
An important application is to the class of nonlinear least squares problems.
If :math:`r\leq 0`, the default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
These keywords refer to finite difference checks on the gradient elements computed by the calling program during intermediate exits. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==============+==============================================================================================================================+
|:math:`-1` |No checks are performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1`|In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+--------------+------------------------------------------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one intermediate exit for the objective function gradients and (if appropriate) one intermediate exit for the partial derivatives of the constraints.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_rcomm`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_rcomm`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i < 10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new calling program is being developed.
**'Verify'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by the calling program during intermediate exits. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==============+==============================================================================================================================+
|:math:`-1` |No checks are performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1`|In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+--------------+------------------------------------------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one intermediate exit for the objective function gradients and (if appropriate) one intermediate exit for the partial derivatives of the constraints.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_rcomm`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_rcomm`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i < 10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new calling program is being developed.
**'Verify Constraint Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by the calling program during intermediate exits. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==============+==============================================================================================================================+
|:math:`-1` |No checks are performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1`|In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+--------------+------------------------------------------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one intermediate exit for the objective function gradients and (if appropriate) one intermediate exit for the partial derivatives of the constraints.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_rcomm`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_rcomm`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i < 10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new calling program is being developed.
**'Verify Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by the calling program during intermediate exits. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==============+==============================================================================================================================+
|:math:`-1` |No checks are performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1`|In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+--------------+------------------------------------------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one intermediate exit for the objective function gradients and (if appropriate) one intermediate exit for the partial derivatives of the constraints.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_rcomm`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_rcomm`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i < 10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new calling program is being developed.
**'Verify Objective Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by the calling program during intermediate exits. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==============+==============================================================================================================================+
|:math:`-1` |No checks are performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed. |
+--------------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1`|In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+--------------+------------------------------------------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
The constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 2` or 'Verify' is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one intermediate exit for the objective function gradients and (if appropriate) one intermediate exit for the partial derivatives of the constraints.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``nlp1_rcomm`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_rcomm`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i < 10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new calling program is being developed.
.. _e04uf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`7`)
Large errors found in the derivatives.
(`errno` :math:`9`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nclin}\geq 0`.
(`errno` :math:`9`)
On entry, :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{ncnln}\geq 0`.
(`errno` :math:`9`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{irevcm} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`6\geq \mathrm{irevcm}\geq 0`.
(`errno` :math:`9`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{irevcm}` negative during an intermediate exit.
(`errno` :math:`1`)
Optimal solution found, but requested accuracy not achieved.
(`errno` :math:`2`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
No feasible point for the nonlinear constraints.
(`errno` :math:`6`)
Current point cannot be improved upon.
**NagAlgorithmicMajorWarning**
(`errno` :math:`4`)
Too many major iterations.
.. _e04uf-py2-py-notes:
**Notes**
``nlp1_rcomm`` is designed to solve the nonlinear programming problem -- the minimization of a smooth nonlinear function subject to a set of constraints on the variables.
The problem is assumed to be stated in the following form:
.. math::
\textit{minimize}_{{x \in R^n}}\text{ }F\left(x\right)\quad \text{ subject to }\quad l\leq \begin{pmatrix}x\\A_Lx\\c\left(x\right)\end{pmatrix}\leq u\text{,}
where :math:`F\left(x\right)` (the `objective function`) is a nonlinear function, :math:`A_L` is an :math:`n_L\times n` constant matrix, and :math:`c\left(x\right)` is an :math:`n_N` element vector of nonlinear constraint functions. (The matrix :math:`A_L` and the vector :math:`c\left(x\right)` may be empty.) The objective function and the constraint functions are assumed to be smooth, i.e., at least twice-continuously differentiable. (The method of ``nlp1_rcomm`` will usually solve `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn1>`__ if there are only isolated discontinuities away from the solution.)
Note that although the bounds on the variables could be included in the definition of the linear constraints, we prefer to distinguish between them for reasons of computational efficiency.
For the same reason, the linear constraints should **not** be included in the definition of the nonlinear constraints.
Upper and lower bounds are specified for all the variables and for all the constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
If there are no nonlinear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn1>`__ and :math:`F` is linear or quadratic then it will generally be more efficient to use one of :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve`, or :meth:`qpconvex2_sparse_solve` if the problem is large and sparse.
If the problem is large and sparse and does have nonlinear constraints, :meth:`handle_solve_ssqp` should be used, since ``nlp1_rcomm`` treats all matrices as dense.
``nlp1_rcomm`` uses reverse communication for evaluating :math:`F\left(x\right)`, :math:`c\left(x\right)` and as many of their first partial derivatives as possible; any remaining derivatives are approximated by finite differences.
See the description of the option 'Derivative Level'.
On initial entry, you must supply an initial estimate of the solution to `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn1>`__.
On intermediate exits, the calling program must compute appropriate values for the objective function, the nonlinear constraints or their derivatives, as specified by the argument :math:`\mathrm{irevcm}`, and then re-enter the function.
For maximum reliability, it is preferable to provide all partial derivatives (see Module 8 of Gill `et al.` (1981), for a detailed discussion).
If they cannot all be provided, it is advisable to provide as many as possible.
While developing code to evaluate the objective function and the constraints, the option 'Verify' should be used to check the calculation of any known derivatives.
The method used by ``nlp1_rcomm`` is described in detail in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#algdetails>`__.
:meth:`nlp2_solve` is an alternative function which uses a similar method, but with **direct communication**: that is, the objective and constraint functions are evaluated by functions, supplied as arguments to the function.
.. _e04uf-py2-py-references:
**References**
Dennis, J E Jr and Moré, J J, 1977, `Quasi-Newton methods, motivation and theory`, SIAM Rev. (19), 46--89
Dennis, J E Jr and Schnabel, R B, 1981, `A new derivation of symmetric positive-definite secant updates`, nonlinear programming, (eds O L Mangasarian, R R Meyer and S M Robinson) (4), 167--199, Academic Press
Dennis, J E Jr and Schnabel, R B, 1983, `Numerical Methods for Unconstrained Optimization and Nonlinear Equations`, Prentice--Hall
Fletcher, R, 1987, `Practical Methods of Optimization`, (2nd Edition), Wiley
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Users' guide for SOL/QPSOL version 3.2`, Report SOL, 84--5, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Some theoretical properties of an augmented Lagrangian merit function`, Report SOL, 86--6R, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Murtagh, B A and Saunders, M A, 1983, `MINOS 5.0 user's guide`, Report SOL 83-20, Department of Operations Research, Stanford University
Powell, M J D, 1974, `Introduction to constrained optimization`, Numerical Methods for Constrained Optimization, (eds P E Gill and W Murray), 1--28, Academic Press
Powell, M J D, 1983, `Variable metric methods in constrained optimization`, Mathematical Programming: the State of the Art, (eds A Bachem, M Grötschel and B Korte), 288--311, Springer--Verlag
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp1_rcomm_ex.main`
"""
raise NotImplementedError
[docs]def nlp1_sparse_solve(m, ncnln, nonln, njnln, iobj, a, ha, ka, bl, bu, start, names, ns, xs, istate, clamda, comm, confun=None, objfun=None, leniz=None, lenz=500, data=None, io_manager=None):
r"""
``nlp1_sparse_solve`` solves sparse nonlinear programming problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`nlp1_sparse_option_file`, :meth:`nlp1_sparse_option_string`, :meth:`nlp1_init`.
.. deprecated:: 28.3.0.0
``nlp1_sparse_solve`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04ug-py2-py-doc:
For full information please refer to the NAG Library document for e04ug
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html
.. _e04ug-py2-py-parameters:
**Parameters**
**m** : int
:math:`m`, the number of general constraints (or slacks). This is the number of rows in :math:`A`, including the free row (if any; see :math:`\mathrm{iobj}`). Note that :math:`A` must contain at least one row. If your problem has no constraints, or only upper and lower bounds on the variables, then you must include a dummy 'free' row consisting of a single (zero) element subject to 'infinite' upper and lower bounds. Further details can be found under the descriptions for :math:`\mathrm{iobj}`, :math:`\textit{nnz}`, :math:`\mathrm{a}`, :math:`\mathrm{ha}`, :math:`\mathrm{ka}`, :math:`\mathrm{bl}` and :math:`\mathrm{bu}`.
**ncnln** : int
:math:`n_N`, the number of nonlinear constraints.
**nonln** : int
:math:`n_1^{\prime }`, the number of nonlinear objective variables. If the objective function is nonlinear, the leading :math:`n_1^{\prime }` columns of :math:`A` belong to the nonlinear objective variables. (See also the description for :math:`\mathrm{njnln}`.)
**njnln** : int
:math:`n_1^{{\prime \prime }}`, the number of nonlinear Jacobian variables. If there are any nonlinear constraints, the leading :math:`n_1^{{\prime \prime }}` columns of :math:`A` belong to the nonlinear Jacobian variables. If :math:`n_1^{\prime } > 0` and :math:`n_1^{{\prime \prime }} > 0`, the nonlinear objective and Jacobian variables overlap. The total number of nonlinear variables is given by :math:`\bar{n} = \mathrm{max}\left({n_1^{\prime }}, {n_1^{{\prime \prime }}}\right)`.
**iobj** : int
If :math:`\mathrm{iobj} > \mathrm{ncnln}`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero elements of the linear part of the objective function.
:math:`\mathrm{iobj} = 0`
There is no free row.
:math:`\mathrm{iobj} = -1`
There is a dummy 'free' row.
**a** : float, array-like, shape :math:`\left(\textit{nnz}\right)`
The nonzero elements of the Jacobian matrix :math:`A`, ordered by increasing column index. Since the constraint Jacobian matrix :math:`J\left(x^{{\prime \prime }}\right)` must always appear in the top left-hand corner of :math:`A`, those elements in a column associated with any nonlinear constraints must come before any elements belonging to the linear constraint matrix :math:`G` and the free row (if any; see :math:`\mathrm{iobj}`).
In general, :math:`\mathrm{a}` is partitioned into a nonlinear part and a linear part corresponding to the nonlinear variables and linear variables in the problem.
Elements in the nonlinear part may be set to any value (e.g., zero) because they are initialized at the first point that satisfies the linear constraints and the upper and lower bounds.
If :math:`\text{‘Derivative Level'} = 2` or :math:`3`, the nonlinear part may also be used to store any constant Jacobian elements.
Note that if :math:`\mathrm{confun}` does not define the constant Jacobian element :math:`\mathrm{fjac}[i-1]`, then the missing value will be obtained directly from :math:`\mathrm{a}[j]` for some :math:`j\geq i`.
If :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{fjac}` are `not` treated as constant; they are estimated by finite differences, at nontrivial expense.
The linear part must contain the nonzero elements of :math:`G` and the free row (if any).
If :math:`\mathrm{iobj} = -1`, set :math:`\mathrm{a}[0] = 0`.
Elements with the same row and column indices are not allowed. (See also the descriptions for :math:`\mathrm{ha}` and :math:`\mathrm{ka}`.)
**ha** : int, array-like, shape :math:`\left(\textit{nnz}\right)`
:math:`\mathrm{ha}[\textit{i}-1]` must contain the row index of the nonzero element stored in :math:`\mathrm{a}[\textit{i}-1]`, for :math:`\textit{i} = 1,2,\ldots,\textit{nnz}`. The row indices for a column may be supplied in any order subject to the condition that those elements in a column associated with any nonlinear constraints must appear before those elements associated with any linear constraints (including the free row, if any). Note that :math:`\mathrm{confun}` must define the Jacobian elements in the same order. If :math:`\mathrm{iobj} = -1`, set :math:`\mathrm{ha}[0] = 1`.
**ka** : int, array-like, shape :math:`\left(n+1\right)`
:math:`\mathrm{ka}[\textit{j}-1]` must contain the index in :math:`\mathrm{a}` of the start of the :math:`\textit{j}`\ th column, for :math:`\textit{j} = 1,2,\ldots,n`. To specify the :math:`\textit{j}`\ th column as empty, set :math:`\mathrm{ka}[\textit{j}-1] = \mathrm{ka}[\textit{j}]`. Note that the first and last elements of :math:`\mathrm{ka}` must be such that :math:`\mathrm{ka}[0] = 1` and :math:`\mathrm{ka}[n] = \textit{nnz}+1`. If :math:`\mathrm{iobj} = -1`, set :math:`\mathrm{ka}[\textit{j}-1] = 2`, for :math:`\textit{j} = 2,3,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`l`, the lower bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bl}` must contain the bounds on the variables :math:`x`, the next :math:`\mathrm{ncnln}` elements the bounds for the nonlinear constraints :math:`F\left(x\right)` (if any) and the next (:math:`\mathrm{m}-\mathrm{ncnln}`) elements the bounds for the linear constraints :math:`Gx` and the free row (if any). To specify a nonexistent lower bound (i.e., :math:`l_j = {-\infty }`), set :math:`\mathrm{bl}[j-1]\leq -\textit{bigbnd}`. To specify the :math:`j`\ th constraint as an `equality`, set :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1] = \beta`, say, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. If :math:`\mathrm{iobj} = -1`, set :math:`\mathrm{bl}[n+\mathrm{abs}\left(\mathrm{iobj}\right)-1]\leq -\textit{bigbnd}`.
**bu** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
:math:`u`, the upper bounds for all the variables and general constraints, in the following order. The first :math:`\textit{n}` elements of :math:`\mathrm{bu}` must contain the bounds on the variables :math:`x`, the next :math:`\mathrm{ncnln}` elements the bounds for the nonlinear constraints :math:`F\left(x\right)` (if any) and the next (:math:`\mathrm{m}-\mathrm{ncnln}`) elements the bounds for the linear constraints :math:`Gx` and the free row (if any). To specify a nonexistent upper bound (i.e., :math:`u_j = {+\infty }`), set :math:`\mathrm{bu}[j-1]\geq \textit{bigbnd}`. To specify the :math:`j`\ th constraint as an `equality`, set :math:`\mathrm{bu}[j-1] = \mathrm{bl}[j-1] = \beta`, say, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`. If :math:`\mathrm{iobj} = -1`, set :math:`\mathrm{bu}[n+\mathrm{abs}\left(\mathrm{iobj}\right)-1]\geq \textit{bigbnd}`.
**start** : str, length 1
Indicates how a starting basis is to be obtained.
:math:`\mathrm{start} = \texttt{'C'}`
An internal Crash procedure will be used to choose an initial basis.
:math:`\mathrm{start} = \texttt{'W'}`
A basis is already defined in :math:`\mathrm{istate}` and :math:`\mathrm{ns}` (probably from a previous call).
**names** : str, length 8, array-like, shape :math:`\left(\textit{nname}\right)`
Specifies the column and row names to be used in the printed output.
If :math:`\textit{nname} = 1`, :math:`\mathrm{names}` is not referenced and the printed output will use default names for the columns and rows.
If :math:`\textit{nname} = n+\mathrm{m}`, the first :math:`\textit{n}` elements must contain the names for the columns, the next :math:`\mathrm{ncnln}` elements must contain the names for the nonlinear rows (if any) and the next :math:`\left(\mathrm{m}-\mathrm{ncnln}\right)` elements must contain the names for the linear rows (if any) to be used in the printed output.
Note that the name for the free row or dummy 'free' row must be stored in :math:`\mathrm{names}[n+\mathrm{abs}\left(\mathrm{iobj}\right)-1]`.
**ns** : int
:math:`n_S`, the number of superbasics. It need not be specified if :math:`\mathrm{start} = \texttt{'C'}`, but must retain its value from a previous call when :math:`\mathrm{start} = \texttt{'W'}`.
**xs** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
The initial values of the variables and slacks :math:`\left(x, s\right)`. (See the description for :math:`\mathrm{istate}`.)
**istate** : int, array-like, shape :math:`\left(n+\mathrm{m}\right)`
If :math:`\mathrm{start} = \texttt{'C'}`, the first :math:`\textit{n}` elements of :math:`\mathrm{istate}` and :math:`\mathrm{xs}` must specify the initial states and values, respectively, of the variables :math:`x`. (The slacks :math:`s` need not be initialized.) An internal Crash procedure is then used to select an initial basis matrix :math:`B`. The initial basis matrix will be triangular (neglecting certain small elements in each column). It is chosen from various rows and columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}`. Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|State of :math:`\mathrm{xs}[j-1]` during Crash procedure |
+============================+=====================================================================+
|:math:`0` or :math:`1` |Eligible for the basis |
+----------------------------+---------------------------------------------------------------------+
|:math:`2` |Ignored |
+----------------------------+---------------------------------------------------------------------+
|:math:`3` |Eligible for the basis (given preference over :math:`0` or :math:`1`)|
+----------------------------+---------------------------------------------------------------------+
|:math:`4` or :math:`5` |Ignored |
+----------------------------+---------------------------------------------------------------------+
If nothing special is known about the problem, or there is no wish to provide special information, you may set :math:`\mathrm{istate}[\textit{j}-1] = 0` and :math:`\mathrm{xs}[\textit{j}-1] = 0.0`, for :math:`\textit{j} = 1,2,\ldots,n`.
All variables will then be eligible for the initial basis.
Less trivially, to say that the :math:`j`\ th variable will probably be equal to one of its bounds, set :math:`\mathrm{istate}[j-1] = 4` and :math:`\mathrm{xs}[j-1] = \mathrm{bl}[j-1]` or :math:`\mathrm{istate}[j-1] = 5` and :math:`\mathrm{xs}[j-1] = \mathrm{bu}[j-1]` as appropriate.
Following the Crash procedure, variables for which :math:`\mathrm{istate}[j-1] = 2` are made superbasic.
Other variables not selected for the basis are then made nonbasic at the value :math:`\mathrm{xs}[j-1]` if :math:`\mathrm{bl}[j-1]\leq \mathrm{xs}[j-1]\leq \mathrm{bu}[j-1]`, or at the value :math:`\mathrm{bl}[j-1]` or :math:`\mathrm{bu}[j-1]` closest to :math:`\mathrm{xs}[j-1]`.
If :math:`\mathrm{start} = \texttt{'W'}`, :math:`\mathrm{istate}` and :math:`\mathrm{xs}` must specify the initial states and values, respectively, of the variables and slacks :math:`\left(x, s\right)`.
If the function has been called previously with the same values of :math:`\textit{n}` and :math:`\mathrm{m}`, :math:`\mathrm{istate}` already contains satisfactory information.
**clamda** : float, array-like, shape :math:`\left(n+\mathrm{m}\right)`
If :math:`\mathrm{ncnln} > 0`, :math:`\mathrm{clamda}[\textit{j}-1]` must contain a Lagrange multiplier estimate for the :math:`\textit{j}`\ th nonlinear constraint :math:`F_{\textit{j}}\left(x\right)`, for :math:`\textit{j} = n+1,\ldots,n+\mathrm{ncnln}`. If nothing special is known about the problem, or there is no wish to provide special information, you may set :math:`\mathrm{clamda}[j-1] = 0.0`. The remaining elements need not be set.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**confun** : None or callable (mode, f, fjac) = confun(mode, ncnln, x, fjac, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the vector :math:`F\left(x\right)` of nonlinear constraint functions and (optionally) its Jacobian :math:`\left(= \frac{{\partial F}}{{\partial x}}\right)` for a specified :math:`n_1^{{\prime \prime }}` (:math:`\text{}\leq n`) element vector :math:`x`.
If there are no nonlinear constraints (i.e., :math:`\mathrm{ncnln} = 0`), :math:`\mathrm{confun}` will never be called by ``nlp1_sparse_solve`` and :math:`\mathrm{confun}` may be **None**. If there are nonlinear constraints, the first call to :math:`\mathrm{confun}` will occur before the first call to :math:`\mathrm{objfun}`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{confun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{f}`.
:math:`\mathrm{mode} = 1`
All available elements of :math:`\mathrm{fjac}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{f}` and all available elements of :math:`\mathrm{fjac}`.
**ncnln** : int
:math:`n_N`, the number of nonlinear constraints. These must be the first :math:`\mathrm{ncnln}` constraints in the problem.
**x** : float, ndarray, shape :math:`\left(\textit{njnln}\right)`
:math:`x`, the vector of nonlinear Jacobian variables at which the nonlinear constraint functions and/or the available elements of the constraint Jacobian are to be evaluated.
**fjac** : float, ndarray, shape :math:`\left(\textit{nnzjac}\right)`
The elements of :math:`\mathrm{fjac}` are set to special values which enable ``nlp1_sparse_solve`` to detect whether they are changed by :math:`\mathrm{confun}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``nlp1_sparse_solve`` is calling :math:`\mathrm{confun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
If :math:`\mathrm{nstate}\geq 2`, ``nlp1_sparse_solve`` is calling :math:`\mathrm{confun}` for the last time.
This argument setting allows you to perform some additional computation on the final solution.
In general, the last call to :math:`\mathrm{confun}` is made with :math:`\mathrm{nstate} = 2+\textit{errno}` (see :ref:`Exceptions <e04ug-py2-py-errors>`).
Otherwise, :math:`\mathrm{nstate} = 0`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**mode** : int
You may set to a negative value as follows:
:math:`\mathrm{mode}\leq -2`
The solution to the current problem is terminated and in this case ``nlp1_sparse_solve`` will terminate with :math:`\textit{errno}` set to :math:`\mathrm{mode}`.
:math:`\mathrm{mode} = -1`
The nonlinear constraint functions cannot be calculated at the current :math:`x`. ``nlp1_sparse_solve`` will then terminate with :math:`\mathrm{errno}` = -1 unless this occurs during the linesearch; in this case, the linesearch will shorten the step and try again.
**f** : float, array-like, shape :math:`\left(\mathrm{ncnln}\right)`
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{f}[i-1]` must contain the value of the :math:`i`\ th nonlinear constraint function at :math:`x`.
**fjac** : float, array-like, shape :math:`\left(\textit{nnzjac}\right)`
If :math:`\mathrm{mode} = 1` or :math:`2`, :math:`\mathrm{fjac}` must return the available elements of the constraint Jacobian evaluated at :math:`x`. These elements must be stored in exactly the same positions as implied by the definitions of the arrays :math:`\mathrm{a}`, :math:`\mathrm{ha}` and :math:`\mathrm{ka}`. If option :math:`\text{‘Derivative Level'} = 2` or :math:`3`, the value of any constant Jacobian element not defined by :math:`\mathrm{confun}` will be obtained directly from :math:`\mathrm{a}`. Note that the function does not perform any internal checks for consistency (except indirectly via the option 'Verify Level'), so great care is essential.
**objfun** : None or callable (mode, objf, objgrd) = objfun(mode, x, objgrd, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` must calculate the nonlinear part of the objective function :math:`f\left(x\right)` and (optionally) its gradient :math:`\left(= \frac{{\partial f}}{{\partial x}}\right)` for a specified :math:`n_1^{\prime }` (:math:`\text{}\leq n`) element vector :math:`x`.
If there are no nonlinear objective variables (i.e., :math:`\mathrm{nonln} = 0`), :math:`\mathrm{objfun}` will never be called by ``nlp1_sparse_solve`` and :math:`\mathrm{objfun}` may be **None**.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{objf}`.
:math:`\mathrm{mode} = 1`
All available elements of :math:`\mathrm{objgrd}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{objf}` and all available elements of :math:`\mathrm{objgrd}`.
**x** : float, ndarray, shape :math:`\left(\textit{nonln}\right)`
:math:`x`, the vector of nonlinear variables at which the nonlinear part of the objective function and/or all available elements of its gradient are to be evaluated.
**objgrd** : float, ndarray, shape :math:`\left(\textit{nonln}\right)`
The elements of :math:`\mathrm{objgrd}` are set to special values which enable ``nlp1_sparse_solve`` to detect whether they are changed by :math:`\mathrm{objfun}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, ``nlp1_sparse_solve`` is calling :math:`\mathrm{objfun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
If :math:`\mathrm{nstate}\geq 2`, ``nlp1_sparse_solve`` is calling :math:`\mathrm{objfun}` for the last time.
This argument setting allows you to perform some additional computation on the final solution.
In general, the last call to :math:`\mathrm{objfun}` is made with :math:`\mathrm{nstate} = 2+\textit{errno}` (see :ref:`Exceptions <e04ug-py2-py-errors>`).
Otherwise, :math:`\mathrm{nstate} = 0`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**mode** : int
You may set to a negative value as follows:
:math:`\mathrm{mode}\leq -2`
The solution to the current problem is terminated and in this case ``nlp1_sparse_solve`` will terminate with :math:`\textit{errno}` set to :math:`\mathrm{mode}`.
:math:`\mathrm{mode} = -1`
The nonlinear part of the objective function cannot be calculated at the current :math:`x`. ``nlp1_sparse_solve`` will then terminate with :math:`\mathrm{errno}` = -1 unless this occurs during the linesearch; in this case, the linesearch will shorten the step and try again.
**objf** : float
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{objf}` must be set to the value of the objective function at :math:`x`.
**objgrd** : float, array-like, shape :math:`\left(\textit{nonln}\right)`
If :math:`\mathrm{mode} = 1` or :math:`2`, :math:`\mathrm{objgrd}` must return the available elements of the gradient evaluated at :math:`x`.
**leniz** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`\max\left(500,{n+\mathrm{m}}\right)`.
The dimension of the internal workspace array :math:`\textit{iz}`.
**lenz** : int, optional
The dimension of the internal workspace array :math:`\textit{z}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**a** : float, ndarray, shape :math:`\left(\textit{nnz}\right)`
Elements in the nonlinear part corresponding to nonlinear Jacobian variables are overwritten.
**ns** : int
The final number of superbasics.
**xs** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final values of the variables and slacks :math:`\left(x, s\right)`.
**istate** : int, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
The final states of the variables and slacks :math:`\left(x, s\right)`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|State of variable :math:`j`|Normal value of :math:`\mathrm{xs}[j-1]` |
+============================+===========================+=============================================================+
|:math:`0` |Nonbasic |:math:`\mathrm{bl}[j-1]` |
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`1` |Nonbasic |:math:`\mathrm{bu}[j-1]` |
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`2` |Superbasic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+----------------------------+---------------------------+-------------------------------------------------------------+
|:math:`3` |Basic |Between :math:`\mathrm{bl}[j-1]` and :math:`\mathrm{bu}[j-1]`|
+----------------------------+---------------------------+-------------------------------------------------------------+
If :math:`\mathrm{ninf} = 0`, basic and superbasic variables may be outside their bounds by as much as the value of the option 'Minor Feasibility Tolerance'.
Note that if scaling is specified, the option 'Minor Feasibility Tolerance' applies to the variables of the `scaled` problem.
In this case, the variables of the original problem may be as much as :math:`0.1` outside their bounds, but this is unlikely unless the problem is very badly scaled.
Very occasionally some nonbasic variables may be outside their bounds by as much as the option 'Minor Feasibility Tolerance' and there may be some nonbasic variables for which :math:`\mathrm{xs}[j-1]` lies strictly between its bounds.
If :math:`\mathrm{ninf} > 0`, some basic and superbasic variables may be outside their bounds by an arbitrary amount (bounded by :math:`\mathrm{sinf}` if scaling was not used).
**clamda** : float, ndarray, shape :math:`\left(n+\mathrm{m}\right)`
A set of Lagrange multipliers for the bounds on the variables (`reduced costs`) and the general constraints (`shadow costs`). More precisely, the first :math:`\textit{n}` elements contain the multipliers for the bounds on the variables, the next :math:`\mathrm{ncnln}` elements contain the multipliers for the nonlinear constraints :math:`F\left(x\right)` (if any) and the next (:math:`\mathrm{m}-\mathrm{ncnln}`) elements contain the multipliers for the linear constraints :math:`Gx` and the free row (if any).
**miniz** : int
The minimum value of :math:`\mathrm{leniz}` required to start solving the problem. If :math:`\mathrm{errno}` = 12, ``nlp1_sparse_solve`` may be called again with :math:`\mathrm{leniz}` suitably larger than :math:`\mathrm{miniz}`. (The bigger the better, since it is not certain how much workspace the basis factors need.)
**minz** : int
The minimum value of :math:`\mathrm{lenz}` required to start solving the problem. If :math:`\mathrm{errno}` = 13, ``nlp1_sparse_solve`` may be called again with :math:`\mathrm{lenz}` suitably larger than :math:`\mathrm{minz}`. (The bigger the better, since it is not certain how much workspace the basis factors need.)
**ninf** : int
The number of constraints that lie outside their bounds by more than the value of the option 'Minor Feasibility Tolerance'.
If the `linear` constraints are infeasible, the sum of the infeasibilities of the linear constraints is minimized subject to the upper and lower bounds being satisfied.
In this case, :math:`\mathrm{ninf}` contains the number of elements of :math:`Gx` that lie outside their upper or lower bounds.
Note that the nonlinear constraints are not evaluated.
Otherwise, the sum of the infeasibilities of the `nonlinear` constraints is minimized subject to the linear constraints and the upper and lower bounds being satisfied.
In this case, :math:`\mathrm{ninf}` contains the number of elements of :math:`F\left(x\right)` that lie outside their upper or lower bounds.
**sinf** : float
The sum of the infeasibilities of constraints that lie outside their bounds by more than the value of the option 'Minor Feasibility Tolerance'.
**obj** : float
The value of the objective function.
.. _e04ug-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default :math:`\text{} = \sqrt[3]{\text{‘Function Precision'}}`
Note that this option does not apply when :math:`\text{‘Derivative Level'} = 3`.
The value of :math:`r` is used near an optimal solution in order to obtain more accurate (but more expensive) estimates of gradients.
This requires twice as many function evaluations as compared to using forward differences (see option 'Forward Difference Interval').
The interval used for the :math:`j`\ th variable is :math:`h_j = r\left(1+\left\lvert x_j\right\rvert \right)`.
The resulting gradient estimates should be accurate to :math:`\mathrm{O}\left(r^2\right)`, unless the functions are badly scaled.
The switch to central differences is indicated by c at the end of each line of intermediate printout produced by the major iterations (see `Major Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-majorprintout>`__).
See Gill `et al.` (1981) for a discussion of the accuracy in finite difference approximations.
If :math:`r\leq 0`, the default value is used.
**'Check Frequency'** : int
Default :math:`\text{} = 60`
Every :math:`i`\ th minor iteration after the most recent basis factorization, a numerical test is made to see if the current solution :math:`\left(x, s\right)` satisfies the general linear constraints (including any linearized nonlinear constraints).
The constraints are of the form :math:`Ax-s = b`, where :math:`s` is the set of slack variables.
If the largest element of the residual vector :math:`r = b-Ax+s` is judged to be too large, the current basis is refactorized and the basic variables recomputed to satisfy the general constraints more accurately.
If :math:`i < 0`, the default value is used.
If :math:`i = 0`, the value :math:`i = 99999999` is used and effectively no checks are made.
**'Crash Option'** : int
Default :math:`\text{} = 0\text{ or }3`
The default value of :math:`i` is :math:`0` if there are any nonlinear constraints and :math:`3` otherwise.
Note that this option does not apply when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04ug-py2-py-parameters>`).
If :math:`\mathrm{start} = \texttt{'C'}`, an internal Crash procedure is used to select an initial basis from various rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The value of :math:`i` determines which rows and columns of :math:`A` are initially eligible for the basis and how many times the Crash procedure is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+===================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|0 |The initial basis contains only slack variables: :math:`B = I`. |
+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The Crash procedure is called once (looking for a triangular basis in all rows and columns of :math:`A`). |
+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The Crash procedure is called twice (if there are any nonlinear constraints). The first call looks for a triangular basis in linear rows and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (whilst retaining the current basis for linear rows).|
+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The Crash procedure is called up to three times (if there are any nonlinear constraints). The first two calls treat linear `equality` constraints and linear `inequality` constraints separately. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (whilst retaining the current basis for linear rows). |
+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i < 0` or :math:`i > 3`, the default value is used.
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound.) The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.1`
The value :math:`r` (:math:`0\leq r < 1`) allows the Crash procedure to ignore certain 'small' nonzero elements in the columns of :math:`A` while searching for a triangular basis.
If :math:`a_{\mathrm{max}}` is the largest element in the :math:`j`\ th column, other nonzeros :math:`a_{{ij}}` in the column are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`.
When :math:`r > 0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached earlier on some problems.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Level'** : int
Default :math:`\text{} = 3`
This argument indicates which nonlinear function gradients are provided in functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=============================================================================================================================+
|3 |All elements of the objective gradient and the constraint Jacobian are provided. |
+---------+-----------------------------------------------------------------------------------------------------------------------------+
|2 |All elements of the constraint Jacobian are provided, but some (or all) elements of the objective gradient are not specified.|
+---------+-----------------------------------------------------------------------------------------------------------------------------+
|1 |All elements of the objective gradient are provided, but some (or all) elements of the constraint Jacobian are not specified.|
+---------+-----------------------------------------------------------------------------------------------------------------------------+
|0 |Some (or all) elements of both the objective gradient and the constraint Jacobian are not specified. |
+---------+-----------------------------------------------------------------------------------------------------------------------------+
The default value :math:`i = 3` should be used whenever possible.
It is the most reliable and will usually be the most efficient.
If :math:`i = 0` or :math:`2`, ``nlp1_sparse_solve`` will `estimate` the unspecified elements of the objective gradient, using finite differences.
This may simplify the coding of :math:`\mathrm{objfun}`.
However, the computation of finite difference approximations usually increases the total run-time substantially (since a call to :math:`\mathrm{objfun}` is required for each unspecified element) and there is less assurance that an acceptable solution will be found.
If :math:`i = 0` or :math:`1`, ``nlp1_sparse_solve`` will approximate unspecified elements of the constraint Jacobian.
For each column of the Jacobian, one call to :math:`\mathrm{confun}` is needed to estimate all unspecified elements in that column (if any).
For example, if the sparsity pattern of the Jacobian has the form
.. math::
\begin{pmatrix}\text{*}&\text{*}&&\text{*}\\&\text{?}&\text{?}&\\\text{*}&&\text{?}&\\&\text{*}&&\text{*}\end{pmatrix}
where ':math:`*`' indicates an element provided and '?' indicates an unspecified element, ``nlp1_sparse_solve`` will call :math:`\mathrm{confun}` twice: once to estimate the missing element in column :math:`2` and again to estimate the two missing elements in column :math:`3`. (Since columns :math:`1` and :math:`4` are known, they require no calls to :math:`\mathrm{confun}`.)
At times, central differences are used rather than forward differences, in which case twice as many calls to :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` are needed. (The switch to central differences is not under your control.)
If :math:`i < 0` or :math:`i > 3`, the default value is used.
**'Derivative Linesearch'** : valueless
Default
At each major iteration, a linesearch is used to improve the value of the Lagrangian merit function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqnlmf>`__.
The default linesearch uses safeguarded cubic interpolation and requires both function and gradient values in order to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided or option 'Nonderivative Linesearch' is specified, a linesearch based upon safeguarded quadratic interpolation (which does not require the evaluation or approximation of any gradients) is used instead.
A nonderivative linesearch can be slightly less robust on difficult problems and it is recommended that the default be used if the functions and their derivatives can be computed at approximately the same cost.
If the gradients are very expensive to compute relative to the functions however, a nonderivative linesearch may result in a significant decrease in the total run-time.
If option 'Nonderivative Linesearch' is selected, ``nlp1_sparse_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` with :math:`\mathrm{mode} = 0`.
Once the linesearch is complete, the nonlinear functions are re-evaluated with :math:`\mathrm{mode} = 2`.
If the potential savings offered by a nonderivative linesearch are to be fully realised, it is essential that :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` be coded so that no derivatives are computed when :math:`\mathrm{mode} = 0`.
**'Nonderivative Linesearch'** : valueless
At each major iteration, a linesearch is used to improve the value of the Lagrangian merit function `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqnlmf>`__.
The default linesearch uses safeguarded cubic interpolation and requires both function and gradient values in order to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided or option 'Nonderivative Linesearch' is specified, a linesearch based upon safeguarded quadratic interpolation (which does not require the evaluation or approximation of any gradients) is used instead.
A nonderivative linesearch can be slightly less robust on difficult problems and it is recommended that the default be used if the functions and their derivatives can be computed at approximately the same cost.
If the gradients are very expensive to compute relative to the functions however, a nonderivative linesearch may result in a significant decrease in the total run-time.
If option 'Nonderivative Linesearch' is selected, ``nlp1_sparse_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` with :math:`\mathrm{mode} = 0`.
Once the linesearch is complete, the nonlinear functions are re-evaluated with :math:`\mathrm{mode} = 2`.
If the potential savings offered by a nonderivative linesearch are to be fully realised, it is essential that :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` be coded so that no derivatives are computed when :math:`\mathrm{mode} = 0`.
**'Elastic Weight'** : float
Default :math:`\text{} = 1.0` or :math:`100.0`
The default value of :math:`r` is :math:`100.0` if there are any nonlinear constraints and :math:`1.0` otherwise.
This option defines the initial weight :math:`\gamma` associated with problem `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#elasticproblem>`__.
At any given major iteration :math:`k`, elastic mode is entered if the QP subproblem is infeasible or the QP dual variables (Lagrange multipliers) are larger in magnitude than :math:`r\times \left(1+\left\lVert g\left(x_k\right)\right\rVert_2\right)`, where :math:`g` is the objective gradient.
In either case, the QP subproblem is resolved in elastic mode with :math:`\gamma = r\times \left(1+\left\lVert g\left(x_k\right)\right\rVert_2\right)`.
Thereafter, :math:`\gamma` is increased (subject to a maximum allowable value) at any point that is optimal for problem `[equation] <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#elasticproblem>`__, but not feasible for problem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn1>`__.
After the :math:`p`\ th increase, :math:`\gamma = r\times 10^p\times \left(1+\left\lVert g\left(x_{k_1}\right)\right\rVert_2\right)`, where :math:`x_{k_1}` is the iterate at which :math:`\gamma` was first needed.
If :math:`r < 0`, the default value is used.
**'Expand Frequency'** : int
Default :math:`\text{} = 10000`
This option is part of the EXPAND anti-cycling procedure due to Gill `et al.` (1989), which is designed to make progress even on highly degenerate problems.
For linear models, the strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of option 'Minor Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``nlp1_sparse_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
For nonlinear models, the same procedure is used for iterations in which there is only one superbasic variable. (Cycling can only occur when the current solution is at a vertex of the feasible region.) Thus, zero steps are allowed if there is more than one superbasic variable, but otherwise positive steps are enforced.
Increasing the value of :math:`i` helps reduce the number of slightly infeasible nonbasic basic variables (most of which are eliminated during the resetting procedure).
However, it also diminishes the freedom to choose a large pivot element (see option 'Pivot Tolerance').
If :math:`i < 0`, the default value is used.
If :math:`i = 0`, the value :math:`i = 99999999` is used and effectively no anti-cycling procedure is invoked.
**'Factorization Frequency'** : int
Default :math:`\text{} = 50\text{ or }100`
The default value of :math:`i` is :math:`50` if there are any nonlinear constraints and :math:`100` otherwise.
If :math:`i > 0`, at most :math:`i` basis changes will occur between factorizations of the basis matrix.
For linear problems, the basis factors are usually updated at every iteration.
The default value :math:`i = 100` is reasonable for typical problems, particularly those that are extremely sparse and well-scaled.
When the objective function is nonlinear, fewer basis updates will occur as the solution is approached.
The number of iterations between basis factorizations will, therefore, increase.
During these iterations a test is made regularly according to the value of option 'Check Frequency' to ensure that the general constraints are satisfied.
If necessary, the basis will be refactorized before the limit of :math:`i` updates is reached.
If :math:`i\leq 0`, the default value is used.
**'Infeasible Exit'** : valueless
Default
Note that this option is ignored if the value of option 'Major Iteration Limit' is exceeded, or the linear constraints are infeasible.
If termination is about to occur at a point that does not satisfy the nonlinear constraints and option 'Feasible Exit' is selected, this option requests that additional iterations be performed in order to find a feasible point (if any) for the nonlinear constraints.
This involves solving a feasible point problem in which the objective function is omitted.
Otherwise, this option requests no additional iterations be performed.
**'Feasible Exit'** : valueless
Note that this option is ignored if the value of option 'Major Iteration Limit' is exceeded, or the linear constraints are infeasible.
If termination is about to occur at a point that does not satisfy the nonlinear constraints and option 'Feasible Exit' is selected, this option requests that additional iterations be performed in order to find a feasible point (if any) for the nonlinear constraints.
This involves solving a feasible point problem in which the objective function is omitted.
Otherwise, this option requests no additional iterations be performed.
**'Minimize'** : valueless
Default
If option 'Feasible Point' is selected, this option attempts to find a feasible point (if any) for the nonlinear constraints by omitting the objective function.
It can also be used to check whether the nonlinear constraints are feasible.
Otherwise, this option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` will be reversed.
**'Maximize'** : valueless
If option 'Feasible Point' is selected, this option attempts to find a feasible point (if any) for the nonlinear constraints by omitting the objective function.
It can also be used to check whether the nonlinear constraints are feasible.
Otherwise, this option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` will be reversed.
**'Feasible Point'** : valueless
If option 'Feasible Point' is selected, this option attempts to find a feasible point (if any) for the nonlinear constraints by omitting the objective function.
It can also be used to check whether the nonlinear constraints are feasible.
Otherwise, this option specifies the required direction of the optimization.
It applies to both linear and nonlinear terms (if any) in the objective function.
Note that if two problems are the same except that one minimizes :math:`f\left(x\right)` and the other maximizes :math:`{-f}\left(x\right)`, their solutions will be the same but the signs of the dual variables :math:`\pi_i` and the reduced gradients :math:`d_j` will be reversed.
**'Forward Difference Interval'** : float
Default :math:`\text{} = \sqrt{\text{‘Function Precision'}}`
This option defines an interval used to estimate derivatives by forward differences in the following circumstances:
(a) For verifying the objective and/or constraint gradients (see the description of the option 'Verify Level').
(#) For estimating unspecified elements of the objective gradient and/or the constraint Jacobian.
A derivative with respect to :math:`x_j` is estimated by perturbing that element of :math:`x` to the value :math:`x_j+r\left(1+\left\lvert x_j\right\rvert \right)` and then evaluating :math:`f\left(x\right)` and/or :math:`F\left(x\right)` (as appropriate) at the perturbed point.
The resulting gradient estimates should be accurate to :math:`\mathrm{O}\left(r\right)`, unless the functions are badly scaled.
Judicious alteration of :math:`r` may sometimes lead to greater accuracy.
See Gill `et al.` (1981) for a discussion of the accuracy in finite difference approximations.
If :math:`r\leq 0`, the default value is used.
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.8}`
This argument defines the `relative function precision` :math:`\epsilon_r`, which is intended to be a measure of the relative accuracy with which the nonlinear functions can be computed.
For example, if :math:`f\left(x\right)` (or :math:`F_i\left(x\right)`) is computed as :math:`1000.56789` for some relevant :math:`x` and the first :math:`6` significant digits are known to be correct, then the appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
Ideally the functions :math:`f\left(x\right)` or :math:`F_i\left(x\right)` should have magnitude of order :math:`1`.
If all functions are substantially `less` than :math:`1` in magnitude, :math:`\epsilon_r` should be the `absolute` precision.
For example, if :math:`f\left(x\right)` (or :math:`F_i\left(x\right)`) is computed as :math:`1.23456789\times 10^{-4}` for some relevant :math:`x` and the first :math:`6` significant digits are known to be correct, then the appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
In some cases the function values will be the result of extensive computation, possibly involving an iterative procedure that can provide few digits of precision at reasonable cost.
Specifying an appropriate value of :math:`r` may, therefore, lead to savings, by allowing the linesearch procedure to terminate when the difference between function values along the search direction becomes as small as the absolute error in the values.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
**'Hessian Frequency'** : int
Default :math:`\text{} = 99999999`
This option forces the approximate Hessian formed from :math:`i` BFGS updates to be reset to the identity matrix upon completion of a major iteration.
It is intended to be used in conjunction with option 'Hessian Full Memory'.
If :math:`i\leq 0`, the default value is used and effectively no resets occur.
**'Hessian Full Memory'** : valueless
Default when :math:`\bar{n} < 75`
These options specify the method for storing and updating the quasi-Newton approximation to the Hessian of the Lagrangian function.
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and BFGS quasi-Newton updates are applied explicitly.
This is most efficient when the total number of nonlinear variables is not too large (say, :math:`\bar{n} < 75`).
In this case, the storage requirement is fixed and you can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should only be specified when :math:`\bar{n}` is very large.
In this case a limited memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
Note that if :math:`\text{‘Hessian Frequency'} = 20` is used in conjunction with 'Hessian Full Memory', the effect will be similar to using 'Hessian Limited Memory' in conjunction with :math:`\text{‘Hessian Updates'} = 20`, except that the latter will retain the current diagonal during resets.
**'Hessian Limited Memory'** : valueless
Default when :math:`\bar{n}\geq 75`
These options specify the method for storing and updating the quasi-Newton approximation to the Hessian of the Lagrangian function.
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and BFGS quasi-Newton updates are applied explicitly.
This is most efficient when the total number of nonlinear variables is not too large (say, :math:`\bar{n} < 75`).
In this case, the storage requirement is fixed and you can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should only be specified when :math:`\bar{n}` is very large.
In this case a limited memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
Note that if :math:`\text{‘Hessian Frequency'} = 20` is used in conjunction with 'Hessian Full Memory', the effect will be similar to using 'Hessian Limited Memory' in conjunction with :math:`\text{‘Hessian Updates'} = 20`, except that the latter will retain the current diagonal during resets.
**'Hessian Updates'** : int
Default :math:`\text{} = 20` or :math:`99999999`
The default value of :math:`i` is :math:`20` when 'Hessian Limited Memory' is in effect and :math:`99999999` when 'Hessian Full Memory' is in effect, in which case no updates are performed.
If 'Hessian Limited Memory' is in effect, this option defines the maximum number of pairs of Hessian update vectors that are to be used to define the quasi-Newton approximate Hessian.
Once the limit of :math:`i` updates is reached, all but the diagonal elements of the accumulated updates are discarded and the process starts again.
Broadly speaking, the more updates that are stored, the better the quality of the approximate Hessian.
On the other hand, the more vectors that are stored, the greater the cost of each QP iteration.
The default value of :math:`i` is likely to give a robust algorithm without significant expense, but faster convergence may be obtained with far fewer updates (e.g., :math:`i = 5`).
If :math:`i < 0`, the default value is used.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = 10000`
The value of :math:`i` specifies the maximum number of minor iterations allowed (i.e., iterations of the simplex method or the QP algorithm), summed over all major iterations. (See also the description of the options 'Major Iteration Limit' and 'Minor Iteration Limit'.)
If :math:`i < 0`, the default value is used.
**'Linesearch Tolerance'** : float
Default :math:`\text{} = 0.9`
This option controls the accuracy with which a step length will be located along the direction of search at each iteration.
At the start of each linesearch a target directional derivative for the Lagrangian merit function is identified.
The value of :math:`r`, therefore, determines the accuracy to which this target value is approximated.
The default value :math:`r = 0.9` requests an inaccurate search and is appropriate for most problems, particularly those with any nonlinear constraints.
If the nonlinear functions are cheap to evaluate, a more accurate search may be appropriate; try :math:`r = 0.1,0.01` or :math:`0.001`.
The number of major iterations required to solve the problem might decrease.
If the nonlinear functions are expensive to evaluate, a less accurate search may be appropriate.
If :math:`\text{‘Derivative Level'} = 3`, try :math:`r = 0.99`. (The number of major iterations required to solve the problem might increase, but the total number of function evaluations may decrease enough to compensate.)
If :math:`\text{‘Derivative Level'} < 3`, a moderately accurate search may be appropriate; try :math:`r = 0.5`.
Each search will (typically) require only :math:`1-5` function values, but many function calls will then be needed to estimate the missing gradients for the next iteration.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'LU Density Tolerance'** : float
Default :math:`\text{} = 0.6`
If :math:`r_1 > 0`, :math:`r_1` defines the density tolerance used during the :math:`LU` factorization of the basis matrix.
Columns of :math:`L` and rows of :math:`U` are formed one at a time and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated.
The remaining matrix is then factorized using a dense :math:`LU` procedure.
Increasing the value of :math:`r_1` towards unity may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
If :math:`r_1\leq 0`, the default value is used.
If :math:`r_2 > 0`, :math:`r_2` defines the singularity tolerance used to guard against ill-conditioned basis matrices.
Whenever the basis is refactorized, the diagonal elements of :math:`U` are tested as follows.
If :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\times \mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable.
This is most likely to occur when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04ug-py2-py-parameters>`), or at the start of a major iteration.
If :math:`r_2\leq 0`, the default value is used.
In some cases, the Jacobian matrix may converge to values that make the basis exactly singular (e.g., a whole row of the Jacobian matrix could be zero at an optimal solution).
Before exact singularity occurs, the basis could become very ill-conditioned and the optimization could progress very slowly (if at all).
Setting :math:`r_2 = 0.00001` (say) may, therefore, help cause a judicious change of basis in such situations.
**'LU Singularity Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.67}`
If :math:`r_1 > 0`, :math:`r_1` defines the density tolerance used during the :math:`LU` factorization of the basis matrix.
Columns of :math:`L` and rows of :math:`U` are formed one at a time and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated.
The remaining matrix is then factorized using a dense :math:`LU` procedure.
Increasing the value of :math:`r_1` towards unity may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
If :math:`r_1\leq 0`, the default value is used.
If :math:`r_2 > 0`, :math:`r_2` defines the singularity tolerance used to guard against ill-conditioned basis matrices.
Whenever the basis is refactorized, the diagonal elements of :math:`U` are tested as follows.
If :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\times \mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable.
This is most likely to occur when :math:`\mathrm{start} = \texttt{'W'}` (see :ref:`Parameters <e04ug-py2-py-parameters>`), or at the start of a major iteration.
If :math:`r_2\leq 0`, the default value is used.
In some cases, the Jacobian matrix may converge to values that make the basis exactly singular (e.g., a whole row of the Jacobian matrix could be zero at an optimal solution).
Before exact singularity occurs, the basis could become very ill-conditioned and the optimization could progress very slowly (if at all).
Setting :math:`r_2 = 0.00001` (say) may, therefore, help cause a judicious change of basis in such situations.
**'LU Factor Tolerance'** : float
Default :math:`\text{} = 5.0` or :math:`100.0`
The default value of :math:`r_1` is :math:`5.0` if there are any nonlinear constraints and :math:`100.0` otherwise.
The default value of :math:`r_2` is :math:`5.0` if there are any nonlinear constraints and :math:`10.0` otherwise.
If :math:`r_1\geq 1` and :math:`r_2\geq 1`, the values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updating, respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}\text{,}
where the multipliers :math:`\mu` satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
Smaller values of :math:`r_i` favour stability, while larger values favour sparsity.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise.
For large and relatively dense problems, setting :math:`r_1 = 10.0` or :math:`5.0` (say) may give a marked improvement in sparsity without impairing stability to a serious degree.
Note that for problems involving band matrices, it may be necessary to reduce :math:`r_1` and/or :math:`r_2` in order to achieve stability.
If :math:`r_1 < 1` or :math:`r_2 < 1`, the appropriate default value is used.
**'LU Update Tolerance'** : float
Default :math:`\text{} = 5.0` or :math:`10.0`
The default value of :math:`r_1` is :math:`5.0` if there are any nonlinear constraints and :math:`100.0` otherwise.
The default value of :math:`r_2` is :math:`5.0` if there are any nonlinear constraints and :math:`10.0` otherwise.
If :math:`r_1\geq 1` and :math:`r_2\geq 1`, the values of :math:`r_1` and :math:`r_2` affect the stability and sparsity of the basis factorization :math:`B = LU`, during refactorization and updating, respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}\text{,}
where the multipliers :math:`\mu` satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
Smaller values of :math:`r_i` favour stability, while larger values favour sparsity.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise.
For large and relatively dense problems, setting :math:`r_1 = 10.0` or :math:`5.0` (say) may give a marked improvement in sparsity without impairing stability to a serious degree.
Note that for problems involving band matrices, it may be necessary to reduce :math:`r_1` and/or :math:`r_2` in order to achieve stability.
If :math:`r_1 < 1` or :math:`r_2 < 1`, the appropriate default value is used.
**'Major Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option specifies how accurately the nonlinear constraints should be satisfied.
The default value is appropriate when the linear and nonlinear constraints contain data to approximately that accuracy.
A larger value may be appropriate if some of the problem functions are known to be of low accuracy.
Let `rowerr` be defined as the maximum nonlinear constraint violation normalized by the size of the solution.
It is required to satisfy
.. math::
\textit{rowerr} = \mathrm{max}_i\frac{\textit{viol}_i}{{\left\lVert \left(x, s\right)\right\rVert }}\leq r\text{,}
where :math:`\textit{viol}_i` is the violation of the :math:`i`\ th nonlinear constraint.
If :math:`r\leq \epsilon`, the default value is used.
**'Major Iteration Limit'** : int
Default :math:`\text{} = 1000`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
It is intended to guard against an excessive number of linearizations of the nonlinear constraints.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the objective and constraint gradients will be checked if :math:`\text{‘Verify Level'} > 0` and the workspace needed to start solving the problem will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Major Optimality Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option specifies the final accuracy of the dual variables.
If ``nlp1_sparse_solve`` terminates with no exception or warning is raised, a primal and dual solution (:math:`x,s,\pi`) will have been computed such that
.. math::
\textit{maxgap} = \mathrm{max}_j\frac{\textit{gap}_j}{\left\lVert \pi \right\rVert }\leq r\text{,}
where :math:`\textit{gap}_j` is an estimate of the complementarity gap for the :math:`j`\ th variable and :math:`\left\lVert \pi \right\rVert` is a measure of the size of the QP dual variables (or Lagrange multipliers) given by
.. math::
\left\lVert \pi \right\rVert = \mathrm{max}\left(\frac{\sigma }{\sqrt{m}}, 1\right)\text{, where }\quad \sigma = \sum_{{i = 1}}^m\left\lvert \pi_i\right\rvert \text{.}
It is included to make the tests independent of a scale factor on the objective function.
Specifically, :math:`\textit{gap}_j` is computed from the final QP solution using the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j`, where :math:`g_j` is the :math:`j`\ th element of the objective gradient and :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`:
.. math::
\textit{gap}_j = \left\{\begin{array}{cc} d_j\mathrm{min}\left({x_j-l_j}, 1\right)&\text{if }d_j\geq 0\text{;}\\-d_j\mathrm{min}\left({u_j-x_j}, 1\right)&\text{if }d_j < 0\text{.}\end{array}\right.
If :math:`r\leq 0`, the default value is used.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option specifies the final accuracy of the dual variables.
If ``nlp1_sparse_solve`` terminates with no exception or warning is raised, a primal and dual solution (:math:`x,s,\pi`) will have been computed such that
.. math::
\textit{maxgap} = \mathrm{max}_j\frac{\textit{gap}_j}{\left\lVert \pi \right\rVert }\leq r\text{,}
where :math:`\textit{gap}_j` is an estimate of the complementarity gap for the :math:`j`\ th variable and :math:`\left\lVert \pi \right\rVert` is a measure of the size of the QP dual variables (or Lagrange multipliers) given by
.. math::
\left\lVert \pi \right\rVert = \mathrm{max}\left(\frac{\sigma }{\sqrt{m}}, 1\right)\text{, where }\quad \sigma = \sum_{{i = 1}}^m\left\lvert \pi_i\right\rvert \text{.}
It is included to make the tests independent of a scale factor on the objective function.
Specifically, :math:`\textit{gap}_j` is computed from the final QP solution using the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j`, where :math:`g_j` is the :math:`j`\ th element of the objective gradient and :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`:
.. math::
\textit{gap}_j = \left\{\begin{array}{cc} d_j\mathrm{min}\left({x_j-l_j}, 1\right)&\text{if }d_j\geq 0\text{;}\\-d_j\mathrm{min}\left({u_j-x_j}, 1\right)&\text{if }d_j < 0\text{.}\end{array}\right.
If :math:`r\leq 0`, the default value is used.
**'Major Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_sparse_solve``, as indicated below.
A detailed description of the printed output is given in `Major Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-majorprintout>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=====================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Major Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-majorprintout>`__) for each major iteration (no printout of the final solution).|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One long line of output (:math:`\text{} < 120` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one long line of output for each major iteration. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|The final solution, one long line of output for each major iteration, matrix statistics (initial status of rows and columns, number of elements, density, biggest and smallest elements, etc.), details of the scale factors resulting from the scaling procedure (if :math:`\text{‘Scale Option'} = 1` or :math:`2`), basis factorization statistics and details of the initial basis resulting from the Crash procedure (if :math:`\mathrm{start} = \texttt{'C'}`; see :ref:`Parameters <e04ug-py2-py-parameters>`).|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Print Level'** : int
The value of :math:`i` controls the amount of printout produced by the major iterations of ``nlp1_sparse_solve``, as indicated below.
A detailed description of the printed output is given in `Major Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-majorprintout>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=====================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Major Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-majorprintout>`__) for each major iteration (no printout of the final solution).|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One long line of output (:math:`\text{} < 120` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one long line of output for each major iteration. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|The final solution, one long line of output for each major iteration, matrix statistics (initial status of rows and columns, number of elements, density, biggest and smallest elements, etc.), details of the scale factors resulting from the scaling procedure (if :math:`\text{‘Scale Option'} = 1` or :math:`2`), basis factorization statistics and details of the initial basis resulting from the Crash procedure (if :math:`\mathrm{start} = \texttt{'C'}`; see :ref:`Parameters <e04ug-py2-py-parameters>`).|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Major Step Limit'** : float
Default :math:`\text{} = 2.0`
If :math:`r > 0,r` limits the change in :math:`x` during a linesearch.
It applies to all nonlinear problems once a 'feasible solution' or 'feasible subproblem' has been found.
A linesearch determines a step :math:`\alpha` in the interval :math:`0 < \alpha \leq \beta`, where :math:`\beta = 1` if there are any nonlinear constraints, or the step to the nearest upper or lower bound on :math:`x` if all the constraints are linear.
Normally, the first step attempted is :math:`\alpha_1 = \mathrm{min}\left(1, \beta \right)`.
In some cases, such as :math:`f\left(x\right) = ae^{{bx}}` or :math:`f\left(x\right) = ax^b`, even a moderate change in the elements of :math:`x` can lead to floating-point overflow.
The argument :math:`r` is, therefore, used to define a step limit :math:`\bar{\beta }` given by
.. math::
\bar{\beta } = \frac{{r\left(1+\left\lVert x\right\rVert_2\right)}}{{\left\lVert p\right\rVert_2}}\text{,}
where :math:`p` is the search direction and the first evaluation of :math:`f\left(x\right)` is made at the (potentially) smaller step length :math:`\alpha_1 = \mathrm{min}\left(1, \bar{\beta }, \beta \right)`.
Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at meaningless points.
The default value :math:`r = 2.0` should not affect progress on well-behaved functions, but values such as :math:`r = 0.1` or :math:`0.01` may be helpful when rapidly varying functions are present.
If a small value of :math:`r` is selected, a 'good' starting point may be required.
An important application is to the class of nonlinear least squares problems.
If :math:`r\leq 0`, the default value is used.
**'Minor Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option attempts to ensure that all variables eventually satisfy their upper and lower bounds to within the tolerance :math:`r`.
Since this includes slack variables, general linear constraints should also be satisfied to within :math:`r`.
Note that feasibility with respect to nonlinear constraints is judged by the value of option 'Major Feasibility Tolerance' and not by :math:`r`.
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
If :math:`\text{‘Scale Option'} \geq 1`, feasibility is defined in terms of the `scaled` problem (since it is more likely to be meaningful).
Nonlinear functions will only be evaluated at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every effort should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x_1, x_2\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both :math:`x_1` and :math:`x_2`.
If the value :math:`r = 10^{-6}` is used, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious; in general, you should attempt to keep :math:`x` as far away from singularities as possible.)
In reality, :math:`r` is used as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible and the function is, then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic). (See also the description of 'Elastic Weight'.)
If :math:`r\leq \epsilon`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option attempts to ensure that all variables eventually satisfy their upper and lower bounds to within the tolerance :math:`r`.
Since this includes slack variables, general linear constraints should also be satisfied to within :math:`r`.
Note that feasibility with respect to nonlinear constraints is judged by the value of option 'Major Feasibility Tolerance' and not by :math:`r`.
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
If :math:`\text{‘Scale Option'} \geq 1`, feasibility is defined in terms of the `scaled` problem (since it is more likely to be meaningful).
Nonlinear functions will only be evaluated at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every effort should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x_1, x_2\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both :math:`x_1` and :math:`x_2`.
If the value :math:`r = 10^{-6}` is used, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious; in general, you should attempt to keep :math:`x` as far away from singularities as possible.)
In reality, :math:`r` is used as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible and the function is, then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic). (See also the description of 'Elastic Weight'.)
If :math:`r\leq \epsilon`, the default value is used.
**'Minor Iteration Limit'** : int
Default :math:`\text{} = 500`
The value of :math:`i` specifies the maximum number of iterations allowed between successive linearizations of the nonlinear constraints.
A value in the range :math:`10\leq i\leq 50` prevents excessive effort being expended on early major iterations, but allows later QP subproblems to be solved to completion.
Note that an extra :math:`m` minor iterations are allowed if the first QP subproblem to be solved starts with the all-slack basis :math:`B = I`. (See the description of the option 'Crash Option'.)
In general, it is unsafe to specify values as small as :math:`i = 1` or :math:`2` (because even when an optimal solution has been reached, a few minor iterations may be needed for the corresponding QP subproblem to be recognized as optimal).
If :math:`i\leq 0`, the default value is used.
**'Minor Optimality Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
This option is used to judge optimality for each QP subproblem.
Let the QP reduced gradients be :math:`d_j = g_j-\pi^\mathrm{T}a_j`, where :math:`g_j` is the :math:`j`\ th element of the QP gradient, :math:`a_j` is the associated column of the QP constraint matrix and :math:`\pi` is the set of QP dual variables.
By construction, the reduced gradients for basic variables are always zero.
The QP subproblem will be declared optimal if the reduced gradients for nonbasic variables at their upper or lower bounds satisfy
.. math::
\frac{d_j}{\left\lVert \pi \right\rVert }\geq -r\quad \text{ or }\quad \frac{d_j}{\left\lVert \pi \right\rVert }\leq r
respectively, and if :math:`\frac{{\left\lvert d_j\right\rvert }}{\left\lVert \pi \right\rVert }\leq r` for superbasic variables.
Note that :math:`\left\lVert \pi \right\rVert` is a measure of the size of the dual variables.
It is included to make the tests independent of a scale factor on the objective function. (The value of :math:`\left\lVert \pi \right\rVert` actually used is defined in the description for option 'Major Optimality Tolerance'.)
If the objective is scaled down to be very `small`, the optimality test reduces to comparing :math:`d_j` against :math:`r`.
If :math:`r\leq 0`, the default value is used.
**'Minor Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the minor iterations of ``nlp1_sparse_solve`` (i.e., the iterations of the quadratic programming algorithm), as indicated below.
A detailed description of the printed output is given in `Minor Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-minorprintout>`__ (summary output at each minor iteration) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__ (monitoring information at each minor iteration). (See also the description of the option 'Major Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+=====================+=================================================================================================================================================================================================================+
|:math:`0` |No output. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 1`|One line of summary output (:math:`\text{} < 80` characters; see `Minor Iteration Printout <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#fc-minorprintout>`__) for each minor iteration.|
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+=====================+=======================================================================================================================================================================================================+
|:math:`0` |No output. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 1`|One long line of output (:math:`\text{} < 120` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#monitoring>`__) for each minor iteration.|
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Major Print Level'} \geq 5` or :math:`i\geq 0` and :math:`\text{‘Minor Print Level'} \geq 1`, then monitoring information is produced by ``nlp1_sparse_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Major Print Level'} < 5` and :math:`\text{‘Minor Print Level'} < 1`, then no monitoring information is produced.
**'Partial Price'** : int
Default :math:`\text{} = 1` or :math:`10`
The default value of :math:`i` is :math:`1` if there are any nonlinear constraints and :math:`10` otherwise.
This option is recommended for large problems that have significantly more variables than constraints (i.e., :math:`n≫m`).
It reduces the work required for each 'pricing' operation (i.e., when a nonbasic variable is selected to become superbasic).
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+=====================+=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`1` |All columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}` are searched. |
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 2`|Both :math:`A` and :math:`I` are partitioned to give :math:`i` roughly equal segments :math:`A_{\textit{j}},I_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,p` (modulo :math:`p`). If the previous pricing search was successful on :math:`A_j,I_j`, the next search begins on the segments :math:`A_{{j+1}},I_{{j+1}}`. If a reduced gradient is found that is larger than some dynamic tolerance, the variable with the largest such reduced gradient (of appropriate sign) is selected to enter the basis. If nothing is found, the search continues on the next segments :math:`A_{{j+2}},I_{{j+2}}` and so on.|
+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i\leq 0`, the default value is used.
**'Pivot Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.67}`
If :math:`r > 0`, :math:`r` is used during the solution of QP subproblems to prevent columns entering the basis if they would cause the basis to become almost singular.
When :math:`x` changes to :math:`x+\alpha p` for some specified search direction :math:`p`, a 'ratio test' is used to determine which element of :math:`x` reaches an upper or lower bound first.
The corresponding element of :math:`p` is called the `pivot element`.
Elements of :math:`p` are ignored (and, therefore, cannot be pivot elements) if they are smaller than :math:`r`.
It is common in practice for two (or more) variables to reach a bound at essentially the same time.
In such cases, the 'Minor Feasibility Tolerance' provides some freedom to maximize the pivot element and thereby improve numerical stability.
Excessively `small` values of 'Minor Feasibility Tolerance' should, therefore, not be specified.
To a lesser extent, the 'Expand Frequency' also provides some freedom to maximize the pivot element.
Excessively `large` values of 'Expand Frequency' should, therefore, not be specified.
If :math:`r\leq 0`, the default value is used.
**'Scale Option'** : int
Default :math:`\text{} = 1` or :math:`2`
The default value of :math:`i` is :math:`1` if there are any nonlinear constraints and :math:`2` otherwise.
This option enables you to scale the variables and constraints using an iterative procedure due to Fourer (1982), which attempts to compute row scales :math:`r_i` and column scales :math:`c_j` such that the scaled matrix coefficients :math:`\bar{a}_{{ij}} = a_{{ij}}\times \left(c_j/r_i\right)` are as close as possible to unity. (The lower and upper bounds on the variables and slacks for the scaled problem are redefined as :math:`\bar{l}_j = l_j/c_j` and :math:`\bar{u}_j = u_j/c_j` respectively, where :math:`c_j\equiv r_{{j-n}}` if :math:`j > n`.) The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=====================================================================================================================================================================================================================================+
|0 |No scaling is performed. This is recommended if it is known that the elements of :math:`x` and the constraint matrix :math:`A` (along with its Jacobian) never become large (say, :math:`\text{} > 1000`). |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |All linear constraints and variables are scaled. This may improve the overall efficiency of the function on some problems. |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |All constraints and variables are scaled. Also, an additional scaling is performed that takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If there are any nonlinear constraints present, the scale factors depend on the Jacobian at the first point that satisfies the linear constraints and the upper and lower bounds.
The setting :math:`i = 2` should, therefore, be used only if a 'good' starting point is available and the problem is not highly nonlinear.
If :math:`i < 0` or :math:`i > 2`, the default value is used.
**'Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
Note that this option does not apply when :math:`\text{‘Scale Option'} = 0`.
The value :math:`r` (:math:`0 < r < 1`) is used to control the number of scaling passes to be made through the constraint matrix :math:`A`.
At least :math:`3` (and at most :math:`10`) passes will be made.
More precisely, let :math:`a_p` denote the largest column ratio (i.e., :math:`\frac{{\text{‘biggest'}\text{ element}}}{{\text{‘smallest'}\text{ element}}}` in some sense) after the :math:`p`\ th scaling pass through :math:`A`.
The scaling procedure is terminated if :math:`a_p\geq a_{{p-1}}\times r` for some :math:`p\geq 3`.
Thus, increasing the value of :math:`r` from :math:`0.9` to :math:`0.99` (say) will probably increase the number of passes through :math:`A`.
If :math:`r\leq 0` or :math:`r\geq 1`, the default value is used.
**'Start Objective Check At Column'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, then it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
Similarly for columns of the Jacobian.
If the first :math:`30` variables occur nonlinearly in the constraints but the remaining variables are nonlinear only in the objective, then :math:`\mathrm{objfun}` must set the first :math:`30` elements of the array :math:`\mathrm{objgrd}` to zero, but these hardly need to be verified.
Again it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
If :math:`i_2\leq 0` or :math:`i_2 > n_1^{\prime }`, the default value is used.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{min}\left({n_1^{\prime }}, i_2\right)`, the default value is used.
If :math:`i_4\leq 0` or :math:`i_4 > n_1^{{\prime \prime }}`, the default value is used.
If :math:`i_3\leq 0` or :math:`i_3 > \mathrm{min}\left({n_1^{{\prime \prime }}}, i_4\right)`, the default value is used.
**'Stop Objective Check At Column'** : int
Default :math:`\text{} = n_1^{\prime }`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, then it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
Similarly for columns of the Jacobian.
If the first :math:`30` variables occur nonlinearly in the constraints but the remaining variables are nonlinear only in the objective, then :math:`\mathrm{objfun}` must set the first :math:`30` elements of the array :math:`\mathrm{objgrd}` to zero, but these hardly need to be verified.
Again it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
If :math:`i_2\leq 0` or :math:`i_2 > n_1^{\prime }`, the default value is used.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{min}\left({n_1^{\prime }}, i_2\right)`, the default value is used.
If :math:`i_4\leq 0` or :math:`i_4 > n_1^{{\prime \prime }}`, the default value is used.
If :math:`i_3\leq 0` or :math:`i_3 > \mathrm{min}\left({n_1^{{\prime \prime }}}, i_4\right)`, the default value is used.
**'Start Constraint Check At Column'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, then it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
Similarly for columns of the Jacobian.
If the first :math:`30` variables occur nonlinearly in the constraints but the remaining variables are nonlinear only in the objective, then :math:`\mathrm{objfun}` must set the first :math:`30` elements of the array :math:`\mathrm{objgrd}` to zero, but these hardly need to be verified.
Again it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
If :math:`i_2\leq 0` or :math:`i_2 > n_1^{\prime }`, the default value is used.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{min}\left({n_1^{\prime }}, i_2\right)`, the default value is used.
If :math:`i_4\leq 0` or :math:`i_4 > n_1^{{\prime \prime }}`, the default value is used.
If :math:`i_3\leq 0` or :math:`i_3 > \mathrm{min}\left({n_1^{{\prime \prime }}}, i_4\right)`, the default value is used.
**'Stop Constraint Check At Column'** : int
Default :math:`\text{} = n_1^{{\prime \prime }}`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}` and/or Jacobian elements computed by :math:`\mathrm{confun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, then it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
Similarly for columns of the Jacobian.
If the first :math:`30` variables occur nonlinearly in the constraints but the remaining variables are nonlinear only in the objective, then :math:`\mathrm{objfun}` must set the first :math:`30` elements of the array :math:`\mathrm{objgrd}` to zero, but these hardly need to be verified.
Again it is reasonable to specify :math:`\text{‘Start Objective Check At Column'} = 31`.
If :math:`i_2\leq 0` or :math:`i_2 > n_1^{\prime }`, the default value is used.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{min}\left({n_1^{\prime }}, i_2\right)`, the default value is used.
If :math:`i_4\leq 0` or :math:`i_4 > n_1^{{\prime \prime }}`, the default value is used.
If :math:`i_3\leq 0` or :math:`i_3 > \mathrm{min}\left({n_1^{{\prime \prime }}}, i_4\right)`, the default value is used.
**'Superbasics Limit'** : int
Default :math:`\text{} = \mathrm{min}\left(500, {\bar{n}+1}\right)`
Note that this option does not apply to linear problems.
It places a limit on the storage allocated for superbasic variables.
Ideally, the value of :math:`i` should be set slightly larger than the 'number of degrees of freedom' expected at the solution.
For nonlinear problems, the number of degrees of freedom is often called the 'number of independent variables'.
Normally, the value of :math:`i` need not be greater than :math:`\bar{n}+1`, but for many problems it may be considerably smaller. (This will save storage if :math:`\bar{n}` is very large.)
If :math:`i\leq 0`, the default value is used.
**'Unbounded Objective'** : float
Default :math:`\text{} = 10^{15}`
These options are intended to detect unboundedness in nonlinear problems.
During the linesearch, the objective function :math:`f` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert f\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, the iterations are terminated and the function returns with :math:`\mathrm{errno}` = 3.
If singularities are present, unboundedness in :math:`f\left(x\right)` may manifest itself by a floating-point overflow during the evaluation of :math:`f\left(x+\alpha p\right)`, before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
If :math:`r_1\leq 0` or :math:`r_2\leq 0`, the appropriate default value is used.
**'Unbounded Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd},10^{20}\right)`
These options are intended to detect unboundedness in nonlinear problems.
During the linesearch, the objective function :math:`f` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert f\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, the iterations are terminated and the function returns with :math:`\mathrm{errno}` = 3.
If singularities are present, unboundedness in :math:`f\left(x\right)` may manifest itself by a floating-point overflow during the evaluation of :math:`f\left(x+\alpha p\right)`, before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
If :math:`r_1\leq 0` or :math:`r_2\leq 0`, the appropriate default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
This option refers to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
Gradients are verified at the first point that satisfies the linear constraints and the upper and lower bounds.
Unspecified gradient elements are not checked and hence they result in no overhead.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring three calls to :math:`\mathrm{objfun}` and two calls to :math:`\mathrm{confun}`. Note that no checks are carried out if every column of the constraint gradients (Jacobian) contains a missing element. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Individual objective gradient elements will be checked using a reliable (but more expensive) test. If :math:`\text{‘Major Print Level'} > 0`, a key of the form OK or BAD? indicates whether or not each element appears to be correct. If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``nlp1_sparse_solve`` will also exit with an error indicator in argument :math:`\textit{errno}`.|
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Individual columns of the constraint gradients (Jacobian) will be checked using a reliable (but more expensive) test. If :math:`\text{‘Major Print Level'} > 0`, a key of the form OK or BAD? indicates whether or not each element appears to be correct. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Check both constraint and objective gradients (in that order) as described above for :math:`i = 2` and :math:`i = 1` respectively. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The value :math:`i = 3` should be used whenever a new function function is being developed.
The 'Start Objective Check At Column' and 'Stop Objective Check At Column' keywords may be used to limit the number of nonlinear variables to be checked.
If :math:`i < -1` or :math:`i > 3`, the default value is used.
**'Violation Limit'** : float
Default :math:`\text{} = 10.0`
This option defines an absolute limit on the magnitude of the maximum constraint violation after the linesearch.
Upon completion of the linesearch, the new iterate :math:`x_{{k+1}}` satisfies the condition
.. math::
v_i\left(x_{{k+1}}\right)\leq r\times \mathrm{max}\left(1, {v_i\left(x_0\right)}\right)\text{,}
where :math:`x_0` is the point at which the nonlinear constraints are first evaluated and :math:`v_i\left(x\right)` is the :math:`i`\ th nonlinear constraint violation :math:`v_i\left(x\right) = \mathrm{max}\left(0, {l_i-F_i\left(x\right)}, {F_i\left(x\right)-u_i}\right)`.
The effect of the violation limit is to restrict the iterates to lie in an `expanded` feasible region whose size depends on the magnitude of :math:`r`.
This makes it possible to keep the iterates within a region where the objective function is expected to be well-defined and bounded below (or above in the case of maximization).
If the objective function is bounded below (or above in the case of maximization) for all values of the variables, then :math:`r` may be any large positive value.
If :math:`r\leq 0`, the default value is used.
.. _e04ug-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{start} = \texttt{'C'}` or :math:`\texttt{'W'}`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq 1`.
(`errno` :math:`7`)
On entry, :math:`\textit{nnz} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \textit{nnz}\leq n\times \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ncnln}\leq \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{nonln} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{nonln}\leq n`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ncnln} = 0` and :math:`\mathrm{njnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncnln} = 0` then :math:`\mathrm{nonln} = 0`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{njnln} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncnln} > 0` then :math:`1\leq \mathrm{njnln}\leq n`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iobj}\geq -1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ncnln} < \mathrm{iobj}\leq \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\textit{nname} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nname} = 1` or :math:`\textit{nname} = n+\mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{leniz} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{leniz}\geq \mathrm{max}\left(500, {\mathrm{m}+n}\right)`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{lenz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lenz}\geq 500`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ha}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{ha}[j]\leq \mathrm{m}` for all :math:`j`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ka}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[0] = 1`.
(`errno` :math:`7`)
On entry, :math:`n+1 = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ka}[n] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnz}+1 = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[n] = \textit{nnz}+1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{ka}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{ka}[j]\geq 1` for all :math:`j`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ka}[j] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ka}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle`, for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ka}[j+1]-\mathrm{ka}[j]\leq \mathrm{m}` for all :math:`j`.
(`errno` :math:`7`)
On entry, duplicate element found in row :math:`\langle\mathit{\boldsymbol{value}}\rangle`, column :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \texttt{'C'}` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \texttt{'C'}` then :math:`\mathrm{istate}[i]\geq 0` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \texttt{'C'}` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \texttt{'C'}` then :math:`\mathrm{istate}[i]\leq 5` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \texttt{'W'}` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \texttt{'W'}` then :math:`\mathrm{istate}[i]\geq 0` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{start} = \texttt{'W'}` and :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{start} = \texttt{'W'}` then :math:`\mathrm{istate}[i]\leq 3` for all :math:`i`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[n+\left\lvert \mathrm{iobj}\right\rvert -1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bl}[n+\left\lvert \mathrm{iobj}\right\rvert -1]\leq {-\textit{bigbnd}}`.
(`errno` :math:`7`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[n+\left\lvert \mathrm{iobj}\right\rvert -1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{bu}[n+\left\lvert \mathrm{iobj}\right\rvert -1]\geq \textit{bigbnd}`.
(`errno` :math:`7`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`8`)
Function :math:`\mathrm{objfun}` appears to be giving incorrect gradients.
(`errno` :math:`9`)
Function :math:`\mathrm{confun}` appears to be giving incorrect gradients.
(`errno` :math:`11`)
Numerical error in trying to satisfy the linear constraints.
(`errno` :math:`12`)
Not enough integer workspace for the basis factors.
(`errno` :math:`13`)
Not enough real workspace for the basis factors.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`-1`)
Constraint and objective values could not be calculated.
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{mode}` negative in :math:`\mathrm{objfun}` or :math:`\mathrm{confun}`.
(`errno` :math:`1`)
Infeasible problem, nonlinear infeasibilities minimized.
(`errno` :math:`1`)
No feasible point for the nonlinear constraints.
(`errno` :math:`1`)
No feasible point for the linear constraints.
(`errno` :math:`2`)
The problem is unbounded (or badly scaled).
(`errno` :math:`3`)
Violation Limit exceeded. The problem may be unbounded.
(`errno` :math:`5`)
Feasible solution, but requested accuracy could not be achieved.
(`errno` :math:`10`)
Current point cannot be improved upon.
(`errno` :math:`14`)
The basis is singular after :math:`15` factorization attempts.
(`errno` :math:`15`)
Not enough integer workspace to start solving the problem.
(`errno` :math:`16`)
Not enough real workspace to start solving the problem.
**NagAlgorithmicMajorWarning**
(`errno` :math:`4`)
Major Iteration Limit exceeded.
(`errno` :math:`4`)
Minor Iteration Limit exceeded.
(`errno` :math:`4`)
Iteration Limit exceeded.
(`errno` :math:`6`)
The value of the option 'Superbasics Limit' is too small.
.. _e04ug-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``nlp1_sparse_solve`` is designed to solve a class of nonlinear programming problems that are assumed to be stated in the following general form:
.. math::
\mathrm{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\F\left(x\right)\\Gx\end{array}\right\}\leq u\text{,}
where :math:`x = \left(x_1,x_2,\ldots,x_n\right)^\mathrm{T}` is a set of variables, :math:`f\left(x\right)` is a smooth scalar objective function, :math:`l` and :math:`u` are constant lower and upper bounds, :math:`F\left(x\right)` is a vector of smooth nonlinear constraint functions :math:`\left\{F_i\left(x\right)\right\}` and :math:`G` is a `sparse` matrix.
The constraints involving :math:`F` and :math:`Gx` are called the `general constraints`.
Note that upper and lower bounds are specified for all variables and constraints.
This form allows full generality in specifying various types of constraint.
In particular, the :math:`j`\ th constraint can be defined as an `equality` by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
``nlp1_sparse_solve`` converts the upper and lower bounds on the :math:`m` elements of :math:`F` and :math:`Gx` to equalities by introducing a set of `slack variables` :math:`s`, where :math:`s = \left(s_1,s_2,\ldots,s_m\right)^\mathrm{T}`.
For example, the linear constraint :math:`5\leq 2x_1+3x_2\leq {+\infty }` is replaced by :math:`2x_1+3x_2-s_1 = 0`, together with the bounded slack :math:`5\leq s_1\leq {+\infty }`.
The problem defined by `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn1>`__ can, therefore, be re-written in the following equivalent form:
.. math::
\mathrm{minimize}_{{x \in R^n,s \in R^m}}f\left(x\right)\quad \text{ subject to }\quad \left\{\begin{array}{c}Gx\end{array}\right\}-s = 0\text{, }\quad l\leq \left\{\begin{array}{c}x\\s\end{array}\right\}\leq u\text{.}
Since the slack variables :math:`s` are subject to the same upper and lower bounds as the elements of :math:`F` and :math:`Gx`, the bounds on :math:`F` and :math:`Gx` can simply be thought of as bounds on the combined vector :math:`\left(x, s\right)`.
The elements of :math:`x` and :math:`s` are partitioned into `basic`, `nonbasic` and `superbasic variables` defined as follows:
- a basic variable (:math:`x_j` say) is the :math:`j`\ th variable associated with the :math:`j`\ th column of the basis matrix :math:`B`;
- a nonbasic variable is a variable that is temporarily fixed at its current value (usually its upper or lower bound);
- a superbasic variable is a nonbasic variable which is not at one of its bounds that is free to move in any desired direction (namely one that will improve the value of the objective function or reduce the sum of infeasibilities).
For example, in the simplex method (see Gill `et al.` (1981)) the elements of :math:`x` can be partitioned at each vertex into a set of :math:`m` basic variables (all non-negative) and a set of :math:`\left(n-m\right)` nonbasic variables (all zero).
This is equivalent to partitioning the columns of the constraint matrix as :math:`\begin{pmatrix}B&N\end{pmatrix}`, where :math:`B` contains the :math:`m` columns that correspond to the basic variables and :math:`N` contains the :math:`\left(n-m\right)` columns that correspond to the nonbasic variables.
Note that :math:`B` is square and nonsingular.
The option 'Maximize' may be used to specify an alternative problem in which :math:`f\left(x\right)` is maximized.
If the objective function is nonlinear and all the constraints are linear, :math:`F` is absent and the problem is said to be `linearly constrained`.
In general, the objective and constraint functions are `structured` in the sense that they are formed from sums of linear and nonlinear functions.
This structure can be exploited by the function during the solution process as follows.
Consider the following nonlinear optimization problem with four variables (:math:`u,v,z,w`):
.. math::
\mathrm{minimize}_{{u,v,z,w}}\quad \text{ }\quad \left(u+v+z\right)^2+3z+5w
subject to the constraints
.. math::
\begin{array}{c}u^2+v^2+z = 2\\u^4+v^4 +w = 4\\2u + 4v \geq 0\end{array}
and to the bounds
.. math::
\begin{array}{c}z\geq 0\\w\geq 0\text{.}\end{array}
This problem has several characteristics that can be exploited by the function:
- the objective function is nonlinear. It is the sum of a `nonlinear` function of the variables (:math:`u,v,z`) and a `linear` function of the variables (:math:`z,w`);
- the first two constraints are nonlinear. The third is linear;
- each nonlinear constraint function is the sum of a `nonlinear` function of the variables (:math:`u,v`) and a `linear` function of the variables (:math:`z,w`).
The nonlinear terms are defined by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` (see :ref:`Parameters <e04ug-py2-py-parameters>`), which involve only the appropriate subset of variables.
For the objective, we define the function :math:`f\left(u, v, z\right) = \left(u+v+z\right)^2` to include only the nonlinear part of the objective.
The three variables (:math:`u,v,z`) associated with this function are known as the `nonlinear objective variables`.
The number of them is given by :math:`\mathrm{nonln}` (see :ref:`Parameters <e04ug-py2-py-parameters>`) and they are the only variables needed in :math:`\mathrm{objfun}`.
The linear part :math:`3z+5w` of the objective is stored in row :math:`\mathrm{iobj}` (see :ref:`Parameters <e04ug-py2-py-parameters>`) of the (constraint) Jacobian matrix :math:`A` (see below).
Thus, if :math:`x^{\prime }` and :math:`y^{\prime }` denote the nonlinear and linear objective variables, respectively, the objective may be re-written in the form
.. math::
f\left(x^{\prime }\right)+c^\mathrm{T}x^{\prime }+d^\mathrm{T}y^{\prime }\text{,}
where :math:`f\left(x^{\prime }\right)` is the nonlinear part of the objective and :math:`c` and :math:`d` are constant vectors that form a row of :math:`A`.
In this example, :math:`x^{\prime } = \left(u, v, z\right)` and :math:`y^{\prime } = w`.
Similarly for the constraints, we define a vector function :math:`F\left(u, v\right)` to include just the nonlinear terms.
In this example, :math:`F_1\left(u, v\right) = u^2+v^2` and :math:`F_2\left(u, v\right) = u^4+v^4`, where the two variables (:math:`u,v`) are known as the `nonlinear Jacobian variables`.
The number of them is given by :math:`\mathrm{njnln}` (see :ref:`Parameters <e04ug-py2-py-parameters>`) and they are the only variables needed in :math:`\mathrm{confun}`.
Thus, if :math:`x^{{\prime \prime }}` and :math:`y^{{\prime \prime }}` denote the nonlinear and linear Jacobian variables, respectively, the constraint functions and the linear part of the objective have the form
.. math::
\begin{pmatrix}F\left(x^{{\prime \prime }}\right)+A_2y^{{\prime \prime }}\\A_3x^{{\prime \prime }}+A_4y^{{\prime \prime }}\end{pmatrix}\text{,}
where :math:`x^{{\prime \prime }} = \left(u, v\right)` and :math:`y^{{\prime \prime }} = \left(z, w\right)` in this example.
This ensures that the Jacobian is of the form
.. math::
A = \begin{pmatrix}J\left(x^{{\prime \prime }}\right)&A_2\\A_3&A_4\end{pmatrix}\text{,}
where :math:`J\left(x^{{\prime \prime }}\right) = \frac{{\partial F\left(x^{{\prime \prime }}\right)}}{{\partial x}}`.
Note that :math:`J\left(x^{{\prime \prime }}\right)` `always` appears in the `top left-hand corner` of :math:`A`.
The inequalities :math:`l_1\leq F\left(x^{{\prime \prime }}\right)+A_2y^{{\prime \prime }}\leq u_1` and :math:`l_2\leq A_3x^{{\prime \prime }}+A_4y^{{\prime \prime }}\leq u_2` implied by the constraint functions in `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn3>`__ are known as the `nonlinear` and `linear` constraints, respectively.
The nonlinear constraint vector :math:`F\left(x^{{\prime \prime }}\right)` in `(3) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn3>`__ and (optionally) its partial derivative matrix :math:`J\left(x^{{\prime \prime }}\right)` are set in :math:`\mathrm{confun}`.
The matrices :math:`A_2`, :math:`A_3` and :math:`A_4` contain any (constant) linear terms.
Along with the sparsity pattern of :math:`J\left(x^{{\prime \prime }}\right)` they are stored in the arrays :math:`\mathrm{a}`, :math:`\mathrm{ha}` and :math:`\mathrm{ka}` (see :ref:`Parameters <e04ug-py2-py-parameters>`).
In general, the vectors :math:`x^{\prime }` and :math:`x^{{\prime \prime }}` have different dimensions, but they `always overlap`, in the sense that the shorter vector is always the beginning of the other.
In the above example, the nonlinear Jacobian variables :math:`\left(u, v\right)` are an ordered subset of the nonlinear objective variables :math:`\left(u, v, z\right)`.
In other cases it could be the other way round (whichever is the most convenient), but the first way keeps :math:`J\left(x^{{\prime \prime }}\right)` as small as possible.
Note that the nonlinear objective function :math:`f\left(x^{\prime }\right)` may involve either a subset or superset of the variables appearing in the nonlinear constraint functions :math:`F\left(x^{{\prime \prime }}\right)`.
Thus, :math:`\mathrm{nonln}\leq \mathrm{njnln}` (or vice-versa).
Sometimes the objective and constraints really involve `disjoint sets of nonlinear variables`.
In such cases the variables should be ordered so that :math:`\mathrm{nonln} > \mathrm{njnln}` and :math:`x^{\prime } = \left({x^{{\prime \prime }}}, {x^{{\prime \prime \prime }}}\right)`, where the objective is nonlinear in just the last vector :math:`x^{{\prime \prime \prime }}`.
The first :math:`\mathrm{njnln}` elements of the gradient array :math:`\mathrm{objgrd}` should also be set to zero in :math:`\mathrm{objfun}`.
If all elements of the constraint Jacobian are known (i.e., the option :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any constant elements may be assigned their correct values in :math:`\mathrm{a}`, :math:`\mathrm{ha}` and :math:`\mathrm{ka}`.
The corresponding elements of the constraint Jacobian array :math:`\mathrm{fjac}` need not be reset in :math:`\mathrm{confun}`.
This includes values that are identically zero as constraint Jacobian elements are assumed to be zero unless specified otherwise.
It must be emphasized that, if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{fjac}` are `not` treated as constant; they are estimated by finite differences, at nontrivial expense.
If there are no nonlinear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn1>`__ and :math:`f\left(x\right)` is linear or quadratic, then it may be more efficient to use :meth:`qpconvex2_sparse_solve` to solve the resulting linear or quadratic programming problem, or one of :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve` if :math:`G` is a `dense` matrix.
If the problem is dense and does have nonlinear constraints then one of :meth:`nlp2_solve`, :meth:`nlp1_rcomm` or :meth:`lsq_gencon_deriv` (as appropriate) should be used instead.
You must supply an initial estimate of the solution to `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#eqn1>`__, together with versions of :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` that define :math:`f\left(x^{\prime }\right)` and :math:`F\left(x^{{\prime \prime }}\right)`, respectively, and as many first partial derivatives as possible.
Note that if there are any nonlinear constraints, then the `first` call to :math:`\mathrm{confun}` will precede the `first` call to :math:`\mathrm{objfun}`.
``nlp1_sparse_solve`` is based on the SNOPT package described in Gill `et al.` (2002), which in turn utilizes functions from the MINOS package (see Murtagh and Saunders (1995)).
It incorporates a Sequential Quadratic Programming (SQP) method that obtains search directions from a sequence of Quadratic Programming (QP) subproblems.
Each QP subproblem minimizes a quadratic model of a certain Lagrangian function subject to a linearization of the constraints.
An augmented Lagrangian merit function is reduced along each search direction to ensure convergence from any starting point.
Further details can be found in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ugf.html#algdetails>`__.
Throughout this document the symbol :math:`\epsilon` is used to represent the machine precision (see :meth:`machine.precision <naginterfaces.library.machine.precision>`).
.. _e04ug-py2-py-references:
**References**
Conn, A R, 1973, `Constrained optimization using a nondifferentiable penalty function`, SIAM J. Numer. Anal. (10), 760--779
Eldersveld, S K, 1991, `Large-scale sequential quadratic programming algorithms`, PhD Thesis, Department of Operations Research, Stanford University, Stanford
Fletcher, R, 1984, `An` :math:`l_1` `penalty method for nonlinear constraints`, Numerical Optimization 1984, (eds P T Boggs, R H Byrd and R B Schnabel), 26--40, SIAM Philadelphia
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E, Murray, W and Saunders, M A, 2002, `SNOPT: An SQP Algorithm for Large-scale Constrained Optimization` (12), 979--1006, SIAM J. Optim.
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1992, `Some theoretical properties of an augmented Lagrangian merit function`, Advances in Optimization and Parallel Computing, (ed P M Pardalos), 101--128, North Holland
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Murtagh, B A and Saunders, M A, 1995, `MINOS 5.4 users' guide`, Report SOL 83-20R, Department of Operations Research, Stanford University
Ortega, J M and Rheinboldt, W C, 1970, `Iterative Solution of Nonlinear Equations in Several Variables`, Academic Press
Powell, M J D, 1974, `Introduction to constrained optimization`, Numerical Methods for Constrained Optimization, (eds P E Gill and W Murray), 1--28, Academic Press
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp1_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp1_sparse_option_file(ioptns, comm, io_manager=None):
r"""
``nlp1_sparse_option_file`` may be used to supply options to :meth:`nlp1_sparse_solve` from an external file.
.. deprecated:: 28.3.0.0
``nlp1_sparse_option_file`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04uh-py2-py-doc:
For full information please refer to the NAG Library document for e04uh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uhf.html
.. _e04uh-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04uh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04uh-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``nlp1_sparse_option_file`` may be used to supply values for options to :meth:`nlp1_sparse_solve`. ``nlp1_sparse_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp1_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp1_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp1_sparse_solve <e04ug-py2-py-other_params>`.
.. _e04uh-py2-py-references:
**References**
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
"""
raise NotImplementedError
[docs]def nlp1_sparse_option_string(optstr, comm, io_manager=None):
r"""
``nlp1_sparse_option_string`` may be used to supply individual options to :meth:`nlp1_sparse_solve`.
.. deprecated:: 28.3.0.0
``nlp1_sparse_option_string`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04uj-py2-py-doc:
For full information please refer to the NAG Library document for e04uj
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ujf.html
.. _e04uj-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04uj-py2-py-notes>` and in :ref:`Other Parameters for nlp1_sparse_solve <e04ug-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04uj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04uj-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``nlp1_sparse_option_string`` may be used to supply values for options to :meth:`nlp1_sparse_solve`.
It is only necessary to call ``nlp1_sparse_option_string`` for those arguments whose values are to be different from their default values.
One call to ``nlp1_sparse_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp1_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp1_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp1_sparse_solve <e04ug-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_gencon_deriv_option_file(ioptns, comm, io_manager=None):
r"""
``lsq_gencon_deriv_option_file`` may be used to supply options to :meth:`lsq_gencon_deriv` from an external file.
.. _e04uq-py2-py-doc:
For full information please refer to the NAG Library document for e04uq
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uqf.html
.. _e04uq-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04uq-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04uq-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_gencon_deriv_option_file`` may be used to supply values for options to :meth:`lsq_gencon_deriv`. ``lsq_gencon_deriv_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
For ``lsq_gencon_deriv_option_file`` printing is turned off by default, but may be turned on at any time using the keyword 'List'.
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lsq_gencon_deriv` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lsq_gencon_deriv`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lsq_gencon_deriv <e04us-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_gencon_deriv_option_string(optstr, comm, io_manager=None):
r"""
``lsq_gencon_deriv_option_string`` may be used to supply individual options to :meth:`lsq_gencon_deriv`.
.. _e04ur-py2-py-doc:
For full information please refer to the NAG Library document for e04ur
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04urf.html
.. _e04ur-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04ur-py2-py-notes>` and in :ref:`Other Parameters for lsq_gencon_deriv <e04us-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ur-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ur-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_gencon_deriv_option_string`` may be used to supply values for options to :meth:`lsq_gencon_deriv`.
It is only necessary to call ``lsq_gencon_deriv_option_string`` for those arguments whose values are to be different from their default values.
One call to ``lsq_gencon_deriv_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lsq_gencon_deriv` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lsq_gencon_deriv`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lsq_gencon_deriv <e04us-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_gencon_deriv(bl, bu, y, objfun, x, comm, a=None, confun=None, istate=None, cjac=None, fjac=None, clamda=None, r=None, data=None, io_manager=None):
r"""
``lsq_gencon_deriv`` is designed to minimize an arbitrary smooth sum of squares function subject to constraints (which may include simple bounds on the variables, linear constraints and smooth nonlinear constraints) using a Sequential Quadratic Programming (SQP) method.
As many first derivatives as possible should be supplied by you; any unspecified derivatives are approximated by finite differences.
See the description of the option 'Derivative Level', in :ref:`Other Parameters <e04us-py2-py-other_params>`.
It is not intended for large sparse problems.
``lsq_gencon_deriv`` may also be used for unconstrained, bound-constrained and linearly constrained optimization.
Note: this function uses optional algorithmic parameters, see also: :meth:`lsq_gencon_deriv_option_file`, :meth:`lsq_gencon_deriv_option_string`, :meth:`nlp1_init`.
.. _e04us-py2-py-doc:
For full information please refer to the NAG Library document for e04us
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html
.. _e04us-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints.
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints.
**y** : float, array-like, shape :math:`\left(m\right)`
The coefficients of the constant vector :math:`y` of the objective function.
**objfun** : callable f = objfun(mode, needfi, x, fjac, nstate, data=None)
:math:`\mathrm{objfun}` must calculate either the :math:`i`\ th element of the vector :math:`f\left(x\right) = {\left({f_1\left(x\right)}, {f_2\left(x\right)}, \ldots, {f_m\left(x\right)}\right)}^\mathrm{T}` or all :math:`m` elements of :math:`f\left(x\right)` and (optionally) its Jacobian (:math:`\text{} = \frac{{\partial f}}{{\partial x}}`) for a specified :math:`n`-element vector :math:`x`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0` and :math:`\mathrm{needfi} = i`, where :math:`i > 0`
:math:`\mathrm{f}[i-1]`.
:math:`\mathrm{mode} = 0` and :math:`\mathrm{needfi} < 0`
:math:`\mathrm{f}`.
:math:`\mathrm{mode} = 1` and :math:`\mathrm{needfi} < 0`
All available elements of :math:`\mathrm{fjac}`.
:math:`\mathrm{mode} = 2` and :math:`\mathrm{needfi} < 0`
:math:`\mathrm{f}` and all available elements of :math:`\mathrm{fjac}`.
**needfi** : int
If :math:`\mathrm{needfi} = i > 0`, only the :math:`i`\ th element of :math:`f\left(x\right)` needs to be evaluated at :math:`x`; the remaining elements need not be set. This can result in significant computational savings when :math:`m≫n`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which :math:`f\left(x\right)` and/or all available elements of its Jacobian are to be evaluated.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`, to be modified in place
`On entry`: is set to a special value.
`On exit`: if :math:`\mathrm{mode} = 1` or :math:`2` and :math:`\mathrm{needfi} < 0`, the :math:`i`\ th row of :math:`\mathrm{fjac}` must contain the available elements of the vector :math:`\nabla f_i` given by
.. math::
\nabla f_i = \left(\frac{{\partial f_i}}{{\partial x_1}},\frac{{\partial f_i}}{{\partial x_2}},\ldots,\frac{{\partial f_i}}{{\partial x_n}}\right)^\mathrm{T}\text{,}
evaluated at the point :math:`x`. See also the argument :math:`\mathrm{nstate}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``lsq_gencon_deriv`` is calling :math:`\mathrm{objfun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**f** : float, array-like, shape :math:`\left(m\right)`
If :math:`\mathrm{mode} = 0` and :math:`\mathrm{needfi} = i > 0`, :math:`\mathrm{f}[i-1]` must contain the value of :math:`f_i` at :math:`x`.
If :math:`\mathrm{mode} = 0` or :math:`2` and :math:`\mathrm{needfi} < 0`, :math:`\mathrm{f}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**a** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` contains the :math:`\textit{i}`\ th row of the matrix :math:`A_L` of general linear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#eqn1>`__. That is, the :math:`\textit{i}`\ th row contains the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nclin}`.
**confun** : None or callable c = confun(mode, needc, x, cjac, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the vector :math:`c\left(x\right)` of nonlinear constraint functions and (optionally) its Jacobian (:math:`\text{} = \frac{{\partial c}}{{\partial x}}`) for a specified :math:`n`-element vector :math:`x`.
If there are no nonlinear constraints (i.e., :math:`\textit{ncnln} = 0`), :math:`\mathrm{confun}` will never be called by ``lsq_gencon_deriv`` and :math:`\mathrm{confun}` may be **None**. If there are nonlinear constraints, the first call to :math:`\mathrm{confun}` will occur before the first call to :math:`\mathrm{objfun}`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{confun}`. Only the following values need be assigned, for each value of :math:`i` such that :math:`\mathrm{needc}[i-1] > 0`:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{c}[i-1]`.
:math:`\mathrm{mode} = 1`
All available elements in the :math:`i`\ th row of :math:`\mathrm{cjac}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{c}[i-1]` and all available elements in the :math:`i`\ th row of :math:`\mathrm{cjac}`.
**needc** : int, ndarray, shape :math:`\left(\textit{ncnln}\right)`
The indices of the elements of :math:`\mathrm{c}` and/or :math:`\mathrm{cjac}` that must be evaluated by :math:`\mathrm{confun}`. If :math:`\mathrm{needc}[i-1] > 0`, the :math:`i`\ th element of :math:`\mathrm{c}` and/or the available elements of the :math:`i`\ th row of :math:`\mathrm{cjac}` (see argument :math:`\mathrm{mode}`) must be evaluated at :math:`x`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the constraint functions and/or all available elements of the constraint Jacobian are to be evaluated.
**cjac** : float, ndarray, shape :math:`\left(\max\left(1,\textit{ncnln}\right), n\right)`, to be modified in place
`On entry`: is set to a special value.
`On exit`: if :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 1` or :math:`2`, the :math:`i`\ th row of :math:`\mathrm{cjac}` must contain the available elements of the vector :math:`\nabla c_i` given by
.. math::
\nabla c_i = \left(\frac{{\partial c_i}}{{\partial x_1}},\frac{{\partial c_i}}{{\partial x_2}},\ldots,\frac{{\partial c_i}}{{\partial x_n}}\right)^\mathrm{T}\text{,}
where :math:`\frac{{\partial c_i}}{{\partial x_j}}` is the partial derivative of the :math:`i`\ th constraint with respect to the :math:`j`\ th variable, evaluated at the point :math:`x`. See also the argument :math:`\mathrm{nstate}`. The remaining rows of :math:`\mathrm{cjac}`, corresponding to non-positive elements of :math:`\mathrm{needc}`, are ignored.
If all elements of the constraint Jacobian are known (i.e., :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any constant elements may be assigned to :math:`\mathrm{cjac}` one time only at the start of the optimization.
An element of :math:`\mathrm{cjac}` that is not subsequently assigned in :math:`\mathrm{confun}` will retain its initial value throughout.
Constant elements may be loaded into :math:`\mathrm{cjac}` either before the call to ``lsq_gencon_deriv`` or during the first call to :math:`\mathrm{confun}` (signalled by the value :math:`\mathrm{nstate} = 1`).
The ability to preload constants is useful when many Jacobian elements are identically zero, in which case :math:`\mathrm{cjac}` may be initialized to zero and nonzero elements may be reset by :math:`\mathrm{confun}`.
Note that constant nonzero elements do affect the values of the constraints.
Thus, if :math:`\mathrm{cjac}[i-1,j-1]` is set to a constant value, it need not be reset in subsequent calls to :math:`\mathrm{confun}`, but the value :math:`\mathrm{cjac}[i-1,j-1]\times \mathrm{x}[j-1]` must nonetheless be added to :math:`\mathrm{c}[i-1]`.
For example, if :math:`\mathrm{cjac}[0,0] = 2` and :math:`\mathrm{cjac}[0,1] = -5`, the term :math:`2\times \mathrm{x}[0]-5\times \mathrm{x}[1]` must be included in the definition of :math:`\mathrm{c}[0]`.
It must be emphasized that, if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{cjac}` are not treated as constant; they are estimated by finite differences, at nontrivial expense.
If you do not supply a value for the option 'Difference Interval', an interval for each element of :math:`x` is computed automatically at the start of the optimization.
The automatic procedure can usually identify constant elements of :math:`\mathrm{cjac}`, which are then computed once only by finite differences.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``lsq_gencon_deriv`` is calling :math:`\mathrm{confun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**c** : float, array-like, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{c}[i-1]` must contain the value of the :math:`i`\ th constraint at :math:`x`. The remaining elements of :math:`\mathrm{c}`, corresponding to the non-positive elements of :math:`\mathrm{needc}`, are ignored.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, the elements of :math:`\mathrm{istate}` corresponding to the bounds and linear constraints define the initial working set for the procedure that finds a feasible point for the linear constraints and bounds.
The active set at the conclusion of this procedure and the elements of :math:`\mathrm{istate}` corresponding to nonlinear constraints then define the initial working set for the first QP subproblem.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, the next :math:`n_L` elements refer to the upper and lower bounds on :math:`A_Lx`, and the next :math:`n_N` elements refer to the upper and lower bounds on :math:`c\left(x\right)`.
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+===================================================================================================================================================+
|0 |The corresponding constraint is `not` in the initial QP working set. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |This inequality constraint should be in the working set at its lower bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |This inequality constraint should be in the working set at its upper bound. |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |This equality constraint should be in the initial working set. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be modified by the function.
If ``lsq_gencon_deriv`` has been called previously with the same values of :math:`\textit{n}`, :math:`\textit{nclin}` and :math:`\textit{ncnln}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**cjac** : None or float, array-like, shape :math:`\left(\textit{ncnln}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{ncnln} > 0`: :math:`n`; otherwise: :math:`1`.
In general, :math:`\mathrm{cjac}` need not be initialized before the call to ``lsq_gencon_deriv``. However, if :math:`\text{‘Derivative Level'} = 3`, you may optionally set the constant elements of :math:`\mathrm{cjac}` (see argument :math:`\mathrm{nstate}` in the description of :math:`\mathrm{confun}`). Such constant elements need not be re-assigned on subsequent calls to :math:`\mathrm{confun}`.
**fjac** : None or float, array-like, shape :math:`\left(m, n\right)`, optional
In general, :math:`\mathrm{fjac}` need not be initialized before the call to ``lsq_gencon_deriv``. However, if :math:`\text{‘Derivative Level'} = 3`, you may optionally set the constant elements of :math:`\mathrm{fjac}` (see argument :math:`\mathrm{nstate}` in the description of :math:`\mathrm{objfun}`). Such constant elements need not be re-assigned on subsequent calls to :math:`\mathrm{objfun}`.
**clamda** : None or float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{clamda}[\textit{j}-1]` must contain a multiplier estimate for each nonlinear constraint with a sign that matches the status of the constraint specified by the :math:`\mathrm{istate}` array, for :math:`\textit{j} = n+\textit{nclin}+1,\ldots,n+\textit{nclin}+\textit{ncnln}`.
The remaining elements need not be set.
Note that if the :math:`j`\ th constraint is defined as 'inactive' by the initial value of the :math:`\mathrm{istate}` array (i.e., :math:`\mathrm{istate}[j-1] = 0`), :math:`\mathrm{clamda}[j-1]` should be zero; if the :math:`j`\ th constraint is an inequality active at its lower bound (i.e., :math:`\mathrm{istate}[j-1] = 1`), :math:`\mathrm{clamda}[j-1]` should be non-negative; if the :math:`j`\ th constraint is an inequality active at its upper bound (i.e., :math:`\mathrm{istate}[j-1] = 2`, :math:`\mathrm{clamda}[j-1]` should be non-positive.
If necessary, the function will modify :math:`\mathrm{clamda}` to match these rules.
**r** : None or float, array-like, shape :math:`\left(n, n\right)`, optional
Need not be initialized if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{r}` must contain the upper triangular Cholesky factor :math:`R` of the initial approximation of the Hessian of the Lagrangian function, with the variables in the natural order.
Elements not in the upper triangular part of :math:`\mathrm{r}` are assumed to be zero and need not be assigned.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**itera** : int
The number of major iterations performed.
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
The status of the constraints in the QP working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================================================================+
|:math:`-2` |This constraint violates its lower bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |This constraint violates its upper bound by more than the appropriate feasibility tolerance (see the options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance'). This value can occur only when no feasible point can be found for a QP subproblem.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the QP working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the QP working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the QP working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the QP working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**c** : None or float, ndarray, shape :math:`\left(\textit{ncnln}\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{c}[\textit{i}-1]` contains the value of the :math:`\textit{i}`\ th nonlinear constraint function :math:`c_{\textit{i}}` at the final iterate, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
If :math:`\textit{ncnln} = 0`, :math:`\mathrm{c}` is returned as **None**.
**cjac** : None or float, ndarray, shape :math:`\left(\textit{ncnln}, :\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{cjac}` contains the Jacobian matrix of the nonlinear constraint functions at the final iterate, i.e., :math:`\mathrm{cjac}[\textit{i}-1,\textit{j}-1]` contains the partial derivative of the :math:`\textit{i}`\ th constraint function with respect to the :math:`\textit{j}`\ th variable, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`. (See the discussion of argument :math:`\mathrm{cjac}` under :math:`\mathrm{confun}`.)
If :math:`\textit{ncnln} = 0`, :math:`\mathrm{cjac}` is returned as **None**.
**f** : float, ndarray, shape :math:`\left(m\right)`
:math:`\mathrm{f}[\textit{i}-1]` contains the value of the :math:`\textit{i}`\ th function :math:`f_{\textit{i}}` at the final iterate, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
The Jacobian matrix of the functions :math:`f_1,f_2,\ldots,f_m` at the final iterate, i.e., :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the partial derivative of the :math:`\textit{i}`\ th function with respect to the :math:`\textit{j}`\ th variable, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`. (See also the discussion of argument :math:`\mathrm{fjac}` under :math:`\mathrm{objfun}`.)
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
The values of the QP multipliers from the last QP subproblem. :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1` and non-positive if :math:`\mathrm{istate}[j-1] = 2`.
**objf** : float
The value of the objective function at the final iterate.
**r** : float, ndarray, shape :math:`\left(n, n\right)`
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`Q^\mathrm{T}\tilde{H}Q`, an estimate of the transformed and reordered Hessian of the Lagrangian at :math:`x` (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__). If :math:`\text{‘Hessian'} = \texttt{'YES'}`, :math:`\mathrm{r}` contains the upper triangular Cholesky factor :math:`R` of :math:`H`, the approximate (untransformed) Hessian of the Lagrangian, with the variables in the natural order.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final estimate of the solution.
.. _e04us-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default values are computed
If the algorithm switches to central differences because the forward-difference approximation is not sufficiently accurate, the value of :math:`r` is used as the difference interval for every element of :math:`x`.
The switch to central differences is indicated by C at the end of each line of intermediate printout produced by the major iterations (see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__).
The use of finite differences is discussed further under the option 'Difference Interval'.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Cold Start'** : valueless
Default
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds, and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``lsq_gencon_deriv`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04us-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``lsq_gencon_deriv`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A 'Warm Start' will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_gencon_deriv`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option controls the specification of the initial working set in both the procedure for finding a feasible point for the linear constraints and bounds, and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``lsq_gencon_deriv`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{r}` as discussed in :ref:`Parameters <e04us-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``lsq_gencon_deriv`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A 'Warm Start' will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_gencon_deriv`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``lsq_gencon_deriv`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Level'** : int
Default :math:`\text{} = 3`
This argument indicates which derivatives are provided in functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
The possible choices for :math:`i` are the following.
.. rst-class:: nag-rules-none nag-align-left
+---------+---------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+===========================================================================================================================+
|3 |All elements of the objective Jacobian and the constraint Jacobian are provided by you. |
+---------+---------------------------------------------------------------------------------------------------------------------------+
|2 |All elements of the constraint Jacobian are provided, but some elements of the objective Jacobian are not specified by you.|
+---------+---------------------------------------------------------------------------------------------------------------------------+
|1 |All elements of the objective Jacobian are provided, but some elements of the constraint Jacobian are not specified by you.|
+---------+---------------------------------------------------------------------------------------------------------------------------+
|0 |Some elements of both the objective Jacobian and the constraint Jacobian are not specified by you. |
+---------+---------------------------------------------------------------------------------------------------------------------------+
The value :math:`i = 3` should be used whenever possible, since ``lsq_gencon_deriv`` is more reliable (and will usually be more efficient) when all derivatives are exact.
If :math:`i = 0` or :math:`2`, ``lsq_gencon_deriv`` will approximate unspecified elements of the objective Jacobian, using finite differences.
The computation of finite difference approximations usually increases the total run-time, since a call to :math:`\mathrm{objfun}` is required for each unspecified element.
Furthermore, less accuracy can be attained in the solution (see Module 8 of Gill `et al.` (1981), for a discussion of limiting accuracy).
If :math:`i = 0` or :math:`1`, ``lsq_gencon_deriv`` will approximate unspecified elements of the constraint Jacobian.
One call to :math:`\mathrm{confun}` is needed for each variable for which partial derivatives are not available.
For example, if the constraint Jacobian has the form
.. math::
\begin{pmatrix}\text{*}&\text{*}&\text{*}&\text{*}\\\text{*}&\text{?}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{?}&\text{*}\\\text{*}&\text{*}&\text{*}&\text{*}\end{pmatrix}
where ':math:`*`' indicates an element provided by you and '?' indicates an unspecified element, ``lsq_gencon_deriv`` will call :math:`\mathrm{confun}` twice: once to estimate the missing element in column :math:`2`, and again to estimate the two missing elements in column :math:`3`. (Since columns :math:`1` and :math:`4` are known, they require no calls to :math:`\mathrm{confun}`.)
At times, central differences are used rather than forward differences, in which case twice as many calls to :math:`\mathrm{objfun}` and :math:`\mathrm{confun}` are needed. (The switch to central differences is not under your control.)
If :math:`i < 0` or :math:`i > 3`, the default value is used.
**'Difference Interval'** : float
Default values are computed
This option defines an interval used to estimate derivatives by finite differences in the following circumstances:
(a) For verifying the objective and/or constraint gradients (see the description of the option 'Verify').
(#) For estimating unspecified elements of the objective and/or constraint Jacobian matrix.
In general, a derivative with respect to the :math:`j`\ th variable is approximated using the interval :math:`\delta_j`, where :math:`\delta_j = r\left(1+\left\lvert \hat{x}_j\right\rvert \right)`, with :math:`\hat{x}` the first point feasible with respect to the bounds and linear constraints.
If the functions are well scaled, the resulting derivative approximation should be accurate to :math:`\mathrm{O}\left(r\right)`.
See Gill `et al.` (1981) for a discussion of the accuracy in finite difference approximations.
If a difference interval is not specified, a finite difference interval will be computed automatically for each variable by a procedure that requires up to six calls of :math:`\mathrm{confun}` and :math:`\mathrm{objfun}` for each element.
This option is recommended if the function is badly scaled or you wish to have ``lsq_gencon_deriv`` determine constant elements in the objective and constraint gradients (see the descriptions of :math:`\mathrm{confun}` and :math:`\mathrm{objfun}` in :ref:`Parameters <e04us-py2-py-parameters>`).
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The scalar :math:`r` defines the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a constraint is considered satisfied if its violation does not exceed :math:`r`.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
Using this keyword sets both options 'Linear Feasibility Tolerance' and 'Nonlinear Feasibility Tolerance' to :math:`r`, if :math:`\epsilon \leq r < 1`. (Additional details are given under the descriptions of these options.)
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.9}`
This argument defines :math:`\epsilon_r`, which is intended to be a measure of the accuracy with which the problem functions :math:`F\left(x\right)` and :math:`c\left(x\right)` can be computed.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
The value of :math:`\epsilon_r` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`; i.e., :math:`\epsilon_r` acts as a relative precision when :math:`\left\lvert F\right\rvert` is large and as an absolute precision when :math:`\left\lvert F\right\rvert` is small.
For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
In contrast, if :math:`F\left(x\right)` is typically of order :math:`10^{-4}` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
However, when the accuracy of the computed function values is known to be significantly worse than full precision, the value of :math:`\epsilon_r` should be large enough so that ``lsq_gencon_deriv`` will not attempt to distinguish between function values that differ by less than the error inherent in the calculation.
**'Hessian'** : str
Default :math:`= \mathrm{NO}`
This option controls the contents of the upper triangular matrix :math:`R` (see :ref:`Parameters <e04us-py2-py-parameters>`). ``lsq_gencon_deriv`` works exclusively with the `transformed and reordered` Hessian :math:`H_Q`, and hence extra computation is required to form the Hessian itself.
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{r}` contains the Cholesky factor of the transformed and reordered Hessian.
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, the Cholesky factor of the approximate Hessian itself is formed and stored in :math:`\mathrm{r}`.
You should select :math:`\text{‘Hessian'} = \texttt{'YES'}` if a 'Warm Start' will be used for the next call to ``lsq_gencon_deriv``.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that is treated as a step to an unbounded solution.
If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'JTJ Initial Hessian'** : valueless
Default
This option controls the initial value of the upper triangular matrix :math:`R`.
If :math:`J` denotes the objective Jacobian matrix :math:`\nabla f\left(x\right)`, :math:`J^\mathrm{T}J` is often a good approximation to the objective Hessian matrix :math:`\nabla^2F\left(x\right)` (see also option 'Reset Frequency').
**'Unit Initial Hessian'** : valueless
This option controls the initial value of the upper triangular matrix :math:`R`.
If :math:`J` denotes the objective Jacobian matrix :math:`\nabla f\left(x\right)`, :math:`J^\mathrm{T}J` is often a good approximation to the objective Hessian matrix :math:`\nabla^2F\left(x\right)` (see also option 'Reset Frequency').
**'Line Search Tolerance'** : float
Default :math:`\text{} = 0.9`
The value :math:`r` (:math:`0\leq r < 1`) controls the accuracy with which the step :math:`\alpha` taken during each iteration approximates a minimum of the merit function along the search direction (the smaller the value of :math:`r`, the more accurate the linesearch).
The default value :math:`r = 0.9` requests an inaccurate search and is appropriate for most problems, particularly those with any nonlinear constraints.
If there are no nonlinear constraints, a more accurate search may be appropriate when it is desirable to reduce the number of major iterations -- for example, if the objective function is cheap to evaluate, or if a substantial number of derivatives are unspecified.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'Linear Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`, and similarly for a nonlinear constraint and :math:`r_2`.
If :math:`r_m < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``lsq_gencon_deriv``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``lsq_gencon_deriv`` (see also option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'Nonlinear Feasibility Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.33}` or :math:`\sqrt{\epsilon }`
The default value of :math:`r_2` is :math:`\epsilon^{0.33}` if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, and :math:`\sqrt{\epsilon }` otherwise.
The scalars :math:`r_1` and :math:`r_2` define the maximum acceptable `absolute` violations in linear and nonlinear constraints at a 'feasible' point; i.e., a linear constraint is considered satisfied if its violation does not exceed :math:`r_1`, and similarly for a nonlinear constraint and :math:`r_2`.
If :math:`r_m < \epsilon` or :math:`r_{\textit{m}}\geq 1`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
On entry to ``lsq_gencon_deriv``, an iterative procedure is executed in order to find a point that satisfies the linear constraints and bounds on the variables to within the tolerance :math:`r_1`.
All subsequent iterates will satisfy the linear constraints to within the same tolerance (unless :math:`r_1` is comparable to the finite difference interval).
For nonlinear constraints, the feasibility tolerance :math:`r_2` defines the largest constraint violation that is acceptable at an optimal point.
Since nonlinear constraints are generally not satisfied until the final iterate, the value of option 'Nonlinear Feasibility Tolerance' acts as a partial termination criterion for the iterative sequence generated by ``lsq_gencon_deriv`` (see also option 'Optimality Tolerance').
These tolerances should reflect the precision of the corresponding constraints.
For example, if the variables and the coefficients in the linear constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r_1` as :math:`10^{-6}`.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Major Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L\right)+10n_N}\right)`
The value of :math:`i` specifies the maximum number of major iterations allowed before termination.
Setting :math:`i = 0` and :math:`\text{‘Major Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Major Print Level'** : int
The value of :math:`i` controls the amount of printout produced by the major iterations of ``lsq_gencon_deriv``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization (see `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__).|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Print Level'** : int
Default for ``lsq_gencon_deriv`` :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the major iterations of ``lsq_gencon_deriv``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__ (summary output at each major iteration and the final solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__ (monitoring information at each major iteration). (See also the description of the option 'Minor Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__) for each major iteration (no printout of the final solution).|
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each major iteration. |
+----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__) for each major iteration (no printout of the final solution). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each major iteration, the objective function, the Euclidean norm of the nonlinear constraint violations, the values of the nonlinear constraints (the vector :math:`c`), the values of the linear constraints (the vector :math:`A_Lx`), and the current values of the variables (the vector :math:`x`). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each major iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization (see `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__) of the QP working set, and the diagonal elements of :math:`R`, the triangular factor of the transformed and reordered Hessian (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__).|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Minor Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {3\left(n+n_L+n_N\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations for finding a feasible point with respect to the bounds and linear constraints (if any).
The value of :math:`i` also specifies the maximum number of minor iterations for the optimality phase of each QP subproblem.
If :math:`i\leq 0`, the default value is used.
**'Minor Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by the minor iterations of ``lsq_gencon_deriv`` (i.e., the iterations of the quadratic programming algorithm), as indicated below.
A detailed description of the printed output is given in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__ (summary output at each minor iteration and the final QP solution) and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__ (monitoring information at each minor iteration). (See also the description of the option 'Major Print Level'.)
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+================================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final QP solution only. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__) for each minor iteration (no printout of the final QP solution).|
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final QP solution and one line of summary output for each minor iteration. |
+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=======================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#monitoring>`__) for each minor iteration (no printout of the final QP solution). |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each minor iteration, the current estimates of the QP multipliers, the current estimate of the QP search direction, the QP constraint values, and the status of each QP constraint. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each minor iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization (see `(5) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn5>`__) of the QP working set, and the diagonal elements of the Cholesky factor :math:`R` of the transformed Hessian (see `(6) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#eqn6>`__).|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Major Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Major Print Level'} \geq 5` or :math:`i\geq 0` and :math:`\text{‘Minor Print Level'} \geq 5`, monitoring information produced by ``lsq_gencon_deriv`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Major Print Level'} < 5` and :math:`\text{‘Minor Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon_R^{0.8}`
The argument :math:`r` (:math:`\epsilon_R\leq r < 1`) specifies the accuracy to which you wish the final iterate to approximate a solution of the problem.
Broadly speaking, :math:`r` indicates the number of correct figures desired in the objective function at the solution.
For example, if :math:`r` is :math:`10^{-6}` and ``lsq_gencon_deriv`` terminates successfully, the final value of :math:`F` should have approximately six correct figures.
If :math:`r < \epsilon_R` or :math:`r\geq 1`, the default value is used.
``lsq_gencon_deriv`` will terminate successfully if the iterative sequence of :math:`x` values is judged to have converged and the final point satisfies the first-order Kuhn--Tucker conditions (see `Overview <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04uff.html#ad-overview>`__).
The sequence of iterates is considered to have converged at :math:`x` if
.. math::
\alpha \left\lVert p\right\rVert \leq \sqrt{r}\left(1+\left\lVert x\right\rVert \right)\text{,}
where :math:`p` is the search direction and :math:`\alpha` the step length.
An iterate is considered to satisfy the first-order conditions for a minimum if
.. math::
\left\lVert Z^\mathrm{T}g_{\mathrm{FR}}\right\rVert \leq \sqrt{r}\left(1+\mathrm{max}\left({1+\left\lvert F\left(x\right)\right\rvert }, \left\lVert g_{\mathrm{FR}}\right\rVert \right)\right)
and
.. math::
\left\lvert \textit{res}_j\right\rvert \leq \textit{ftol}\quad \text{ for all }\quad j\text{,}
where :math:`Z^\mathrm{T}g_{\mathrm{FR}}` is the projected gradient, :math:`g_{\mathrm{FR}}` is the gradient of :math:`F\left(x\right)` with respect to the free variables, :math:`\textit{res}_j` is the violation of the :math:`j`\ th active nonlinear constraint, and :math:`\textit{ftol}` is the 'Nonlinear Feasibility Tolerance'.
**'Reset Frequency'** : int
Default :math:`\text{} = 2`
If :math:`i > 0`, this argument allows you to reset the approximate Hessian matrix to :math:`J^\mathrm{T}J` every :math:`i` iterations, where :math:`J` is the objective Jacobian matrix :math:`\nabla f\left(x\right)` (see also the description of the option 'JTJ Initial Hessian').
At any point where there are no nonlinear constraints active and the values of :math:`f` are small in magnitude compared to the norm of :math:`J`, :math:`J^\mathrm{T}J` will be a good approximation to the objective Hessian :math:`\nabla^2F\left(x\right)`.
Under these circumstances, frequent resetting can significantly improve the convergence rate of ``lsq_gencon_deriv``.
Resetting is suppressed at any iteration during which there are nonlinear constraints active.
If :math:`i\leq 0`, the default value is used.
**'Start Objective Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of Jacobian elements computed by functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
For example, if the first :math:`30` columns of the objective Jacobian appeared to be correct in an earlier run, so that only column :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the subfunctions, so that the corresponding Jacobian elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Objective Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of Jacobian elements computed by functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
For example, if the first :math:`30` columns of the objective Jacobian appeared to be correct in an earlier run, so that only column :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the subfunctions, so that the corresponding Jacobian elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Start Constraint Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of Jacobian elements computed by functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
For example, if the first :math:`30` columns of the objective Jacobian appeared to be correct in an earlier run, so that only column :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the subfunctions, so that the corresponding Jacobian elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Stop Constraint Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of Jacobian elements computed by functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
For example, if the first :math:`30` columns of the objective Jacobian appeared to be correct in an earlier run, so that only column :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the subfunctions, so that the corresponding Jacobian elements are constant, the above choice would also be appropriate.
If :math:`i_{{2\textit{m}-1}}\leq 0` or :math:`i_{{2\textit{m}-1}} > \mathrm{min}\left(n, i_{{2\textit{m}}}\right)`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
If :math:`i_{{2\textit{m}}}\leq 0` or :math:`i_{{2\textit{m}}} > n`, the default value is used, for :math:`\textit{m} = 1,2,\ldots,2`.
**'Step Limit'** : float
Default :math:`\text{} = 2.0`
If :math:`r > 0,r` specifies the maximum change in variables at the first step of the linesearch.
In some cases, such as :math:`F\left(x\right) = ae^{{bx}}` or :math:`F\left(x\right) = ax^b`, even a moderate change in the elements of :math:`x` can lead to floating-point overflow.
The argument :math:`r` is, therefore, used to encourage evaluation of the problem functions at meaningful points.
Given any major iterate :math:`x`, the first point :math:`\tilde{x}` at which :math:`F` and :math:`c` are evaluated during the linesearch is restricted so that
.. math::
\left\lVert \tilde{x}-x\right\rVert_2\leq r\left(1+\left\lVert x\right\rVert_2\right)\text{.}
The linesearch may go on and evaluate :math:`F` and :math:`c` at points further from :math:`x` if this will result in a lower value of the merit function (indicated by L at the end of each line of output produced by the major iterations; see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#fc-printedoutput>`__).
If L is printed for most of the iterations, :math:`r` should be set to a larger value.
Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at wild values.
The default value :math:`\text{‘Step Limit'} = 2.0` should not affect progress on well-behaved functions, but values such as :math:`0.1` or :math:`0.01` may be helpful when rapidly varying functions are present.
If a small value of 'Step Limit' is selected, a good starting point may be required.
An important application is to the class of nonlinear least squares problems.
If :math:`r\leq 0`, the default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are the following:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`i = -1`, no checking will be performed.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``lsq_gencon_deriv`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``lsq_gencon_deriv`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are the following:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`i = -1`, no checking will be performed.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``lsq_gencon_deriv`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``lsq_gencon_deriv`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Constraint Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are the following:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`i = -1`, no checking will be performed.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``lsq_gencon_deriv`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``lsq_gencon_deriv`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are the following:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`i = -1`, no checking will be performed.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``lsq_gencon_deriv`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``lsq_gencon_deriv`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
**'Verify Objective Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`. (Unspecified gradient elements are not checked.) The possible choices for :math:`i` are the following:
.. rst-class:: nag-rules-none nag-align-left
+----------------+---------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+================+=============================================================================================+
|:math:`-1` |No checks are performed. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------------+---------------------------------------------------------------------------------------------+
|:math:`{\geq 1}`|Individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------------+---------------------------------------------------------------------------------------------+
It is possible to specify :math:`\text{‘Verify Level'} = 0` to :math:`3` in several ways.
For example, the nonlinear objective gradient (if any) will be verified if either 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
Similarly, the objective and the constraint gradients will be verified if :math:`\text{‘Verify'} = \texttt{'YES'}` or :math:`\text{‘Verify Level'} = 3` or 'Verify' is specified.
If :math:`i = -1`, no checking will be performed.
If :math:`0\leq i\leq 3`, gradients will be verified at the first point that satisfies the linear constraints and bounds.
If :math:`i = 0`, only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}` and (if appropriate) one call to :math:`\mathrm{confun}`.
If :math:`1\leq i\leq 3`, a more reliable (but more expensive) check will be made on individual gradient elements, within the ranges specified by the 'Start Objective Check At Variable' and 'Stop Objective Check At Variable' keywords.
A result of the form OK or BAD? is printed by ``lsq_gencon_deriv`` to indicate whether or not each element appears to be correct.
If a gradient element is determined to be extremely poor (i.e., if it appears to have no significant digits of accuracy at all), then ``lsq_gencon_deriv`` will also exit with an error indicator in argument :math:`\textit{errno}`.
If :math:`10\leq i\leq 13`, the action is the same as for :math:`i-10`, except that it will take place at the user-specified initial value of :math:`x`.
If :math:`i < -1` or :math:`4\leq i\leq 9` or :math:`i > 13`, the default value is used.
We suggest that :math:`\text{‘Verify Level'} = 3` be used whenever a new function function is being developed.
.. _e04us-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`9`)
On entry, :math:`m = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`m\geq 1`.
(`errno` :math:`9`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`9`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`9`)
On entry, :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{ncnln}\geq 0`.
(`errno` :math:`9`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Optimal solution found, but requested accuracy not achieved.
(`errno` :math:`2`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
No feasible point for the nonlinear constraints.
(`errno` :math:`4`)
Too many major iterations.
(`errno` :math:`6`)
Current point cannot be improved upon.
(`errno` :math:`7`)
Large errors found in the derivatives.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{mode}` negative in :math:`\mathrm{objfun}` or :math:`\mathrm{confun}`.
.. _e04us-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_gencon_deriv`` is designed to solve the nonlinear least squares programming problem -- the minimization of a smooth nonlinear sum of squares function subject to a set of constraints on the variables.
The problem is assumed to be stated in the following form:
.. math::
\textit{minimize}_{{x \in R^n}}F\left(x\right) = \frac{1}{2}\sum_{{i = 1}}^m{\left(y_i-f_i\left(x\right)\right)}^2\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\A_Lx\\c\left(x\right)\end{array}\right\}\leq u\text{,}
where :math:`F\left(x\right)` (the `objective function`) is a nonlinear function which can be represented as the sum of squares of :math:`m` subfunctions :math:`\left(y_1-f_1\left(x\right)\right),\left(y_2-f_2\left(x\right)\right),\ldots,\left(y_m-f_m\left(x\right)\right)`, the :math:`y_i` are constant, :math:`A_L` is an :math:`n_L\times n` constant matrix, and :math:`c\left(x\right)` is an :math:`n_N` element vector of nonlinear constraint functions. (The matrix :math:`A_L` and the vector :math:`c\left(x\right)` may be empty.) The objective function and the constraint functions are assumed to be smooth, i.e., at least twice-continuously differentiable. (The method of ``lsq_gencon_deriv`` will usually solve `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#eqn1>`__ if any isolated discontinuities are away from the solution.)
Note that although the bounds on the variables could be included in the definition of the linear constraints, we prefer to distinguish between them for reasons of computational efficiency.
For the same reason, the linear constraints should **not** be included in the definition of the nonlinear constraints.
Upper and lower bounds are specified for all the variables and for all the constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
You must supply an initial estimate of the solution to `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04usf.html#eqn1>`__, together with functions that define :math:`f\left(x\right) = \left({f_1\left(x\right)}, {f_2\left(x\right)}, \ldots, {f_m\left(x\right)}\right)^\mathrm{T}`, :math:`c\left(x\right)` and as many first partial derivatives as possible; unspecified derivatives are approximated by finite differences.
The subfunctions are defined by the array :math:`\mathrm{y}` and :math:`\mathrm{objfun}`, and the nonlinear constraints are defined by :math:`\mathrm{confun}`.
On every call, these functions must return appropriate values of :math:`f\left(x\right)` and :math:`c\left(x\right)`.
You should also provide the available partial derivatives.
Any unspecified derivatives are approximated by finite differences for a discussion of the option 'Derivative Level'.
Note that if there `are` any nonlinear constraints, then the `first` call to :math:`\mathrm{confun}` will precede the `first` call to :math:`\mathrm{objfun}`.
For maximum reliability, it is preferable for you to provide all partial derivatives (see Module 8 of Gill `et al.` (1981) for a detailed discussion).
If all gradients cannot be provided, it is similarly advisable to provide as many as possible.
While developing :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`, the option 'Verify' should be used to check the calculation of any known gradients.
.. _e04us-py2-py-references:
**References**
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
See Also
--------
:meth:`naginterfaces.library.examples.opt.lsq_gencon_deriv_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_sparse_init():
r"""
``nlp2_sparse_init`` is used to initialize the function :meth:`nlp2_sparse_solve`.
.. deprecated:: 28.3.0.0
``nlp2_sparse_init`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vg-py2-py-doc:
For full information please refer to the NAG Library document for e04vg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vgf.html
**Returns**
**comm** : dict, communication object
Communication structure.
.. _e04vg-py2-py-notes:
**Notes**
``nlp2_sparse_init`` initializes the arrays :math:`\mathrm{comm}`\ ['cw'], :math:`\mathrm{comm}`\ ['iw'] and :math:`\mathrm{comm}`\ ['rw'] for the function :meth:`nlp2_sparse_solve`.
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp2_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_sparse_solve(start, objadd, objrow, prob, usrfun, iafun, javar, nea, a, igfun, jgvar, neg, xlow, xupp, xnames, flow, fupp, fnames, x, xstate, f, fstate, fmul, ns, comm, data=None, io_manager=None):
r"""
``nlp2_sparse_solve`` solves sparse linear and nonlinear programming problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`nlp2_sparse_option_file`, :meth:`nlp2_sparse_option_string`, :meth:`nlp2_sparse_option_integer_set`, :meth:`nlp2_sparse_option_double_set`, :meth:`nlp2_sparse_init`, :meth:`nlp2_sparse_jacobian`.
.. deprecated:: 28.3.0.0
``nlp2_sparse_solve`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vh-py2-py-doc:
For full information please refer to the NAG Library document for e04vh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html
.. _e04vh-py2-py-parameters:
**Parameters**
**start** : int
Indicates how a starting point is to be obtained.
:math:`\mathrm{start} = 0`
Requests that the Crash procedure be used, unless a Basis file is provided via options 'Old Basis File', 'Insert File' or 'Load File'.
:math:`\mathrm{start} = 1`
Is the same as :math:`\mathrm{start} = 0` but is more meaningful when a Basis file is given.
:math:`\mathrm{start} = 2`
Means that :math:`\mathrm{xstate}` and :math:`\mathrm{fstate}` define a valid starting point (probably from an earlier call, though not necessarily).
**objadd** : float
Is a constant that will be added to the objective row :math:`F_{\mathrm{obj}}` for printing purposes. Typically, :math:`\mathrm{objadd} = 0.0e+0`.
**objrow** : int
Says which row of :math:`F\left(x\right)` is to act as the objective function. If there is no such row, set :math:`\mathrm{objrow} = 0`. Then ``nlp2_sparse_solve`` will seek a feasible point such that :math:`l_F\leq F\left(x\right)\leq u_F` and :math:`l_x\leq x\leq u_x`.
**prob** : str, length 8
Is an :math:`8`-character name for the problem. :math:`\mathrm{prob}` is used in the printed solution and in some functions that output Basis files. A blank name may be used.
**usrfun** : callable (status, f, g) = usrfun(status, x, needf, f, needg, g, data=None)
:math:`\mathrm{usrfun}` must define the nonlinear portion :math:`f\left(x\right)` of the problem functions :math:`F\left(x\right) = f\left(x\right)+Ax`, along with its gradient elements :math:`G_{{ij}}\left(x\right) = \frac{{\partial f_i\left(x\right)}}{{\partial x_j}}`. (A dummy function is needed even if :math:`f\equiv 0` and all functions are linear.)
In general, :math:`\mathrm{usrfun}` should return all function and gradient values on every entry except perhaps the last.
This provides maximum reliability and corresponds to the default option setting, :math:`\text{‘Derivative Option'} = 1`.
The elements of :math:`G\left(x\right)` are stored in the array :math:`\mathrm{g}\left(1:\mathrm{leng}\right)` in the order specified by the input arrays :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}`.
In practice it is often convenient `not` to code gradients. ``nlp2_sparse_solve`` is able to estimate them by finite differences, using a call to :math:`\mathrm{usrfun}` for each variable :math:`x_j` for which some :math:`\frac{{\partial f_i\left(x\right)}}{{\partial x_j}}` needs to be estimated.
However, this reduces the reliability of the optimization algorithm, and it can be very expensive if there are many such variables :math:`x_j`.
As a compromise, ``nlp2_sparse_solve`` allows you to code `as many gradients as you like`.
This option is implemented as follows.
Just before :math:`\mathrm{usrfun}` is called, each element of the derivative array :math:`\mathrm{g}` is initialized to a specific value.
On exit, any element retaining that value must be estimated by finite differences.
Some rules of thumb follow:
(i) for maximum reliability, compute all gradients;
(#) if the gradients are expensive to compute, specify option 'Nonderivative Linesearch' and use the value of the input argument :math:`\mathrm{needg}` to avoid computing them on certain entries. (There is no need to compute gradients if :math:`\mathrm{needg} = 0` on entry to :math:`\mathrm{usrfun}`.);
(#) if not all gradients are known, you must specify :math:`\text{‘Derivative Option'} = 0`. You should still compute as many gradients as you can. (It often happens that some of them are constant or zero.);
(#) again, if the known gradients are expensive, don't compute them if :math:`\mathrm{needg} = 0` on entry to :math:`\mathrm{usrfun}`;
(#) use the input argument :math:`\mathrm{status}` to test for special actions on the first or last entries;
(#) while :math:`\mathrm{usrfun}` is being developed, use the option 'Verify Level' to check the computation of gradients that are supposedly known;
(#) :math:`\mathrm{usrfun}` is not called until the linear constraints and bounds on :math:`x` are satisfied. This helps confine :math:`x` to regions where the functions :math:`f_i\left(x\right)` are likely to be defined. However, be aware of the option 'Minor Feasibility Tolerance' if the functions have singularities on the constraint boundaries;
(#) set :math:`\mathrm{status} = -1` if some of the functions are undefined. The linesearch will shorten the step and try again;
(#) set :math:`\mathrm{status}\leq -2` if you want ``nlp2_sparse_solve`` to stop.
**Parameters**
**status** : int
Indicates the first and last calls to :math:`\mathrm{usrfun}`.
:math:`\mathrm{status} = 0`
There is nothing special about the current call to :math:`\mathrm{usrfun}`.
:math:`\mathrm{status} = 1`
``nlp2_sparse_solve`` is calling your function for the `first` time. You may wish to do something special such as read data from a file.
:math:`\mathrm{status}\geq 2`
``nlp2_sparse_solve`` is calling your function for the `last` time. This argument setting allows you to perform some additional computation on the final solution.
:math:`\mathrm{status} = 2`
The current :math:`\mathrm{x}` is `optimal`.
:math:`\mathrm{status} = 3`
The problem appears to be infeasible.
:math:`\mathrm{status} = 4`
The problem appears to be unbounded.
:math:`\mathrm{status} = 5`
An iterations limit was reached.
If the functions are expensive to evaluate, it may be desirable to do nothing on the last call.
**x** : float, ndarray, shape :math:`\left(n\right)`
The variables :math:`x` at which the problem functions are to be calculated. The array :math:`x` must not be altered.
**needf** : int
Indicates whether :math:`\mathrm{f}` must be assigned during this call of :math:`\mathrm{usrfun}`.
:math:`\mathrm{needf} = 0`
:math:`\mathrm{f}` is not required and is ignored.
:math:`\mathrm{needf} > 0`
The components of :math:`f\left(x\right)` corresponding to the nonlinear part of :math:`F\left(x\right)` must be calculated and assigned to :math:`\mathrm{f}`.
If :math:`F_i\left(x\right)` is linear and completely defined by the :math:`i`\ th row of :math:`A`, :math:`A_i^{\prime }`, the associated value :math:`f_i\left(x\right)` is ignored and need not be assigned.
However, if :math:`F_i\left(x\right)` has a nonlinear portion :math:`f_i\left(x\right)` that happens to be zero at :math:`x`, it is still necessary to set :math:`f_i\left(x\right) = 0`.
If the linear part :math:`A_i^{\prime }` of a nonlinear :math:`F_i\left(x\right)` is provided using the arrays :math:`\mathrm{iafun}`, :math:`\mathrm{javar}` and :math:`\mathrm{a}`, it must not be computed again as part of :math:`f_i\left(x\right)`.
To simplify the code, you may ignore the value of :math:`\mathrm{needf}` and compute :math:`f\left(x\right)` on every entry to :math:`\mathrm{usrfun}`.
:math:`\mathrm{needf}` may also be ignored with 'Derivative Linesearch' and :math:`\text{‘Derivative Option'} = 1`.
In this case, :math:`\mathrm{needf}` is always :math:`1`, and :math:`\mathrm{f}` must always be assigned.
**f** : float, ndarray, shape :math:`\left(\textit{nf}\right)`
Concerns the calculation of :math:`f\left(x\right)`.
**needg** : int
Indicates whether :math:`\mathrm{g}` must be assigned during this call of :math:`\mathrm{usrfun}`.
:math:`\mathrm{needg} = 0`
:math:`\mathrm{g}` is not required and is ignored.
:math:`\mathrm{needg} > 0`
The partial derivatives of :math:`f\left(x\right)` must be calculated and assigned to :math:`\mathrm{g}`. The value of :math:`\mathrm{g}[k-1]` should be :math:`\frac{{\partial f_i\left(x\right)}}{{\partial x_j}}`, where :math:`i = \mathrm{igfun}[k-1]`, :math:`j = \mathrm{jgvar}[k-1]` and :math:`k = 1,2,\ldots,\textit{leng}`.
**g** : float, ndarray, shape :math:`\left(\textit{leng}\right)`
Concerns the calculations of the derivatives of the function :math:`f\left(x\right)`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**status** : int
May be used to indicate that you are unable to evaluate :math:`f` or its gradients at the current :math:`x`. (For example, the problem functions may not be defined there.)
During the linesearch, :math:`f\left(x\right)` is evaluated at points :math:`x = x_k+\alpha p_k` for various step lengths :math:`\alpha`, where :math:`f\left(x_k\right)` has already been evaluated satisfactorily.
For any such :math:`x`, if you set :math:`\mathrm{status} = -1`, ``nlp2_sparse_solve`` will reduce :math:`\alpha` and evaluate :math:`f` again (closer to :math:`x_k`, where :math:`f\left(x_k\right)` is more likely to be defined).
If for some reason you wish to terminate the current problem, set :math:`\mathrm{status}\leq -2`.
**f** : float, array-like, shape :math:`\left(\textit{nf}\right)`
:math:`\mathrm{f}` contains the computed functions :math:`f\left(x\right)` (except perhaps if :math:`\mathrm{needf} = 0`).
**g** : float, array-like, shape :math:`\left(\textit{leng}\right)`
Contains the computed derivatives :math:`G\left(x\right)` (unless :math:`\mathrm{needg} = 0`).
These derivative elements must be stored in :math:`\mathrm{g}` in exactly the same positions as implied by the definitions of arrays :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}`.
There is no internal check for consistency (except indirectly via the option 'Verify Level'), so great care is essential.
**iafun** : int, array-like, shape :math:`\left(\textit{lena}\right)`
:math:`\mathrm{iafun}` and :math:`\mathrm{javar}` define the coordinates :math:`\left(i, j\right)` of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
In particular, :math:`\mathrm{nea}` triples :math:`\left(\mathrm{iafun}[k-1], \mathrm{javar}[k-1], \mathrm{a}[k-1]\right)` define the row and column indices :math:`i = \mathrm{iafun}[k-1]` and :math:`j = \mathrm{javar}[k-1]` of the element :math:`A_{{ij}} = \mathrm{a}[k-1]`.
The coordinates may define the elements of :math:`A` in any order.
**javar** : int, array-like, shape :math:`\left(\textit{lena}\right)`
:math:`\mathrm{iafun}` and :math:`\mathrm{javar}` define the coordinates :math:`\left(i, j\right)` of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
In particular, :math:`\mathrm{nea}` triples :math:`\left(\mathrm{iafun}[k-1], \mathrm{javar}[k-1], \mathrm{a}[k-1]\right)` define the row and column indices :math:`i = \mathrm{iafun}[k-1]` and :math:`j = \mathrm{javar}[k-1]` of the element :math:`A_{{ij}} = \mathrm{a}[k-1]`.
The coordinates may define the elements of :math:`A` in any order.
**nea** : int
Is the number of nonzero entries in :math:`A` such that :math:`F\left(x\right) = f\left(x\right)+Ax`.
**a** : float, array-like, shape :math:`\left(\textit{lena}\right)`
Defines the values :math:`A_{{ij}}` of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
In particular, :math:`\mathrm{nea}` triples :math:`\left(\mathrm{iafun}[k-1], \mathrm{javar}[k-1], \mathrm{a}[k-1]\right)` define the row and column indices :math:`i = \mathrm{iafun}[k-1]` and :math:`j = \mathrm{javar}[k-1]` of the element :math:`A_{{ij}} = \mathrm{a}[k-1]`.
The coordinates may define the elements of :math:`A` in any order.
**igfun** : int, array-like, shape :math:`\left(\textit{leng}\right)`
:math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}` define the coordinates :math:`\left(i, j\right)` of the nonzero elements of :math:`G`, the nonlinear part of the derivative :math:`J\left(x\right) = G\left(x\right)+A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`. :meth:`nlp2_sparse_jacobian` may be used to define these two arrays.
The coordinates can define the elements of :math:`G` in any order.
However, :math:`\mathrm{usrfun}` must define the actual elements of :math:`\mathrm{g}` in exactly the same order as defined by the coordinates :math:`\left(\mathrm{igfun}, \mathrm{jgvar}\right)`.
**jgvar** : int, array-like, shape :math:`\left(\textit{leng}\right)`
:math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}` define the coordinates :math:`\left(i, j\right)` of the nonzero elements of :math:`G`, the nonlinear part of the derivative :math:`J\left(x\right) = G\left(x\right)+A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`. :meth:`nlp2_sparse_jacobian` may be used to define these two arrays.
The coordinates can define the elements of :math:`G` in any order.
However, :math:`\mathrm{usrfun}` must define the actual elements of :math:`\mathrm{g}` in exactly the same order as defined by the coordinates :math:`\left(\mathrm{igfun}, \mathrm{jgvar}\right)`.
**neg** : int
The number of nonzero entries in :math:`G`.
**xlow** : float, array-like, shape :math:`\left(n\right)`
Contain the lower and upper bounds :math:`l_x` and :math:`u_x` on the variables :math:`x`.
To specify a nonexistent lower bound :math:`\left[l_x\right]_j = {-\infty }`, set :math:`\mathrm{xlow}[j-1]\leq {-\textit{bigbnd}}`, where :math:`\textit{bigbnd}` is the option 'Infinite Bound Size'.
To specify a nonexistent upper bound :math:`\left[u_x\right]_j = \infty`, set :math:`\mathrm{xupp}[j-1]\geq \textit{bigbnd}`.
To fix the :math:`j`\ th variable at :math:`x_j = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`, set :math:`\mathrm{xlow}[j-1] = \mathrm{xupp}[j-1] = \beta`.
**xupp** : float, array-like, shape :math:`\left(n\right)`
Contain the lower and upper bounds :math:`l_x` and :math:`u_x` on the variables :math:`x`.
To specify a nonexistent lower bound :math:`\left[l_x\right]_j = {-\infty }`, set :math:`\mathrm{xlow}[j-1]\leq {-\textit{bigbnd}}`, where :math:`\textit{bigbnd}` is the option 'Infinite Bound Size'.
To specify a nonexistent upper bound :math:`\left[u_x\right]_j = \infty`, set :math:`\mathrm{xupp}[j-1]\geq \textit{bigbnd}`.
To fix the :math:`j`\ th variable at :math:`x_j = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`, set :math:`\mathrm{xlow}[j-1] = \mathrm{xupp}[j-1] = \beta`.
**xnames** : str, length 8, array-like, shape :math:`\left(\textit{nxname}\right)`
The optional names for the variables.
If :math:`\textit{nxname} = 1`, :math:`\mathrm{xnames}` is not referenced and default names will be used for output.
If :math:`\textit{nxname} = n`, :math:`\mathrm{xnames}[j-1]` should contain the :math:`8`-character name of the :math:`j`\ th variable.
**flow** : float, array-like, shape :math:`\left(\textit{nf}\right)`
Contain the lower and upper bounds :math:`l_F` and :math:`u_F` on :math:`F\left(x\right)`.
To specify a nonexistent lower bound :math:`\left[l_F\right]_i = {-\infty }`, set :math:`\mathrm{flow}[i-1]\leq {-\textit{bigbnd}}`.
For a nonexistent upper bound :math:`\left[u_F\right]_i = \infty`, set :math:`\mathrm{fupp}[i-1]\geq \textit{bigbnd}`.
To make the :math:`i`\ th constraint an `equality` at :math:`F_i = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`, set :math:`\mathrm{flow}[i-1] = \mathrm{fupp}[i-1] = \beta`.
**fupp** : float, array-like, shape :math:`\left(\textit{nf}\right)`
Contain the lower and upper bounds :math:`l_F` and :math:`u_F` on :math:`F\left(x\right)`.
To specify a nonexistent lower bound :math:`\left[l_F\right]_i = {-\infty }`, set :math:`\mathrm{flow}[i-1]\leq {-\textit{bigbnd}}`.
For a nonexistent upper bound :math:`\left[u_F\right]_i = \infty`, set :math:`\mathrm{fupp}[i-1]\geq \textit{bigbnd}`.
To make the :math:`i`\ th constraint an `equality` at :math:`F_i = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`, set :math:`\mathrm{flow}[i-1] = \mathrm{fupp}[i-1] = \beta`.
**fnames** : str, length 8, array-like, shape :math:`\left(\textit{nfname}\right)`
The optional names for the problem functions.
If :math:`\textit{nfname} = 1`, :math:`\mathrm{fnames}` is not referenced and default names will be used for output.
If :math:`\textit{nfname} = \textit{nf}`, :math:`\mathrm{fnames}[i-1]` should contain the :math:`8`-character name of the :math:`i`\ th row of :math:`F`.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the variables :math:`x`. See the following description of :math:`\mathrm{xstate}`.
**xstate** : int, array-like, shape :math:`\left(n\right)`
The initial state for each variable :math:`x`.
If :math:`\mathrm{start} = 0` or :math:`1` and no basis information is provided (the options 'Old Basis File', 'Insert File' and 'Load File' are all set to :math:`0`; the default) :math:`\mathrm{x}` and :math:`\mathrm{xstate}` must be defined.
If nothing special is known about the problem, or if there is no wish to provide special information, you may set :math:`\mathrm{x}[j-1] = 0.0`, :math:`\mathrm{xstate}[j-1] = 0`, for all :math:`j = 1,2,\ldots,n`.
If you set :math:`\mathrm{x}[j-1] = \mathrm{xlow}[j-1]` set :math:`\mathrm{xstate}[j-1] = 4`; if you set :math:`\mathrm{x}[j-1] = \mathrm{xupp}[j-1]`, then set :math:`\mathrm{xstate}[j-1] = 5`.
In this case a Crash procedure is used to select an initial basis.
If :math:`\mathrm{start} = 0` or :math:`1` and basis information is provided (at least one of the options 'Old Basis File', 'Insert File' and 'Load File' is nonzero) :math:`\mathrm{x}` and :math:`\mathrm{xstate}` need not be set.
If :math:`\mathrm{start} = 2` (Warm Start), :math:`\mathrm{x}` and :math:`\mathrm{xstate}` must be set (probably from a previous call).
In this case :math:`\mathrm{xstate}[\textit{j}-1]` must be :math:`0`, :math:`1`, :math:`2` or :math:`3`, for :math:`\textit{j} = 1,2,\ldots,n`.
**f** : float, array-like, shape :math:`\left(\textit{nf}\right)`
An initial value for the problem functions :math:`F`. See the following description of :math:`\mathrm{fstate}`.
**fstate** : int, array-like, shape :math:`\left(\textit{nf}\right)`
The initial state for the problem functions :math:`F`.
If :math:`\mathrm{start} = 0` or :math:`1` and no basis information is provided (the options 'Old Basis File', 'Insert File' and 'Load File' are all set to :math:`0`; the default, :math:`\mathrm{f}` and :math:`\mathrm{fstate}` must be defined.
If nothing special is known about the problem, or if there is no wish to provide special information, you may set :math:`\mathrm{f}[i-1] = 0.0`, :math:`\mathrm{fstate}[i-1] = 0`, for all :math:`i = 1,2,\ldots,\textit{nf}`.
Less trivially, to say that the optimal value of function :math:`\mathrm{f}[i-1]` will probably be equal to one of its bounds, set :math:`\mathrm{f}[i-1] = \mathrm{flow}[i-1]` and :math:`\mathrm{fstate}[i-1] = 4` or :math:`\mathrm{f}[i-1] = \mathrm{fupp}[i-1]` and :math:`\mathrm{fstate}[i-1] = 5` as appropriate.
In this case a Crash procedure is used to select an initial basis.
If :math:`\mathrm{start} = 0` or :math:`1` and basis information is provided (at least one of the options 'Old Basis File', 'Insert File' and 'Load File' is nonzero), :math:`\mathrm{f}` and :math:`\mathrm{fstate}` need not be set.
If :math:`\mathrm{start} = 2` (Warm Start), :math:`\mathrm{f}` and :math:`\mathrm{fstate}` must be set (probably from a previous call).
In this case :math:`\mathrm{fstate}[\textit{i}-1]` must be :math:`0`, :math:`1`, :math:`2` or :math:`3`, for :math:`\textit{i} = 1,2,\ldots,\textit{nf}`.
**fmul** : float, array-like, shape :math:`\left(\textit{nf}\right)`
An estimate of :math:`\gamma`, the vector of Lagrange multipliers (shadow prices) for the constraints :math:`l_F\leq F\left(x\right)\leq u_F`. All :math:`\textit{nf}` components must be defined. If nothing is known about :math:`\gamma`, set :math:`\mathrm{fmul}[\textit{i}-1] = 0.0`, for :math:`\textit{i} = 1,2,\ldots,\textit{nf}`. For warm start use the values from a previous call.
**ns** : int
The number of superbasic variables. :math:`\mathrm{ns}` need not be specified for cold starts, but should retain its value from a previous call when warm start is used.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final values of the variable :math:`x`.
**xstate** : int, ndarray, shape :math:`\left(n\right)`
The final state of the variables.
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+---------------------------+-----------------------------------------------------------------+
|:math:`\mathrm{xstate}[j-1]`|State of variable :math:`j`|Usual value of :math:`\mathrm{x}[j-1]` |
+============================+===========================+=================================================================+
|0 |nonbasic |:math:`\mathrm{xlow}[j-1]` |
+----------------------------+---------------------------+-----------------------------------------------------------------+
|1 |nonbasic |:math:`\mathrm{xupp}[j-1]` |
+----------------------------+---------------------------+-----------------------------------------------------------------+
|2 |superbasic |Between :math:`\mathrm{xlow}[j-1]` and :math:`\mathrm{xupp}[j-1]`|
+----------------------------+---------------------------+-----------------------------------------------------------------+
|3 |basic |Between :math:`\mathrm{xlow}[j-1]` and :math:`\mathrm{xupp}[j-1]`|
+----------------------------+---------------------------+-----------------------------------------------------------------+
Basic and superbasic variables may be outside their bounds by as much as the option 'Minor Feasibility Tolerance'.
Note that if scaling is specified, the feasibility tolerance applies to the variables of the `scaled` problem.
In this case, the variables of the original problem may be as much as :math:`0.1` outside their bounds, but this is unlikely unless the problem is very badly scaled.
Check the value of Primal infeasibility output to the unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) associated with the option 'Print File'.
Very occasionally some nonbasic variables may be outside their bounds by as much as the option 'Minor Feasibility Tolerance', and there may be some nonbasics for which :math:`\mathrm{x}[j-1]` lies strictly between its bounds.
If :math:`\mathrm{ninf} > 0`, some basic and superbasic variables may be outside their bounds by an arbitrary amount (bounded by :math:`\mathrm{sinf}` if scaling was not used).
**xmul** : float, ndarray, shape :math:`\left(n\right)`
The vector of the dual variables (Lagrange multipliers) for the simple bounds :math:`l_x\leq x\leq u_x`.
**f** : float, ndarray, shape :math:`\left(\textit{nf}\right)`
The final values for the problem functions :math:`F` (the values :math:`F` at the final point :math:`\mathrm{x}`).
**fstate** : int, ndarray, shape :math:`\left(\textit{nf}\right)`
The final state of the variables. The elements of :math:`\mathrm{fstate}` have the following meaning:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+-----------------------------------------+-----------------------------------------------------------------+
|:math:`\mathrm{fstate}[i-1]`|State of the corresponding slack variable|Usual value of :math:`\mathrm{f}[i-1]` |
+============================+=========================================+=================================================================+
|0 |nonbasic |:math:`\mathrm{flow}[i-1]` |
+----------------------------+-----------------------------------------+-----------------------------------------------------------------+
|1 |nonbasic |:math:`\mathrm{fupp}[i-1]` |
+----------------------------+-----------------------------------------+-----------------------------------------------------------------+
|2 |superbasic |Between :math:`\mathrm{flow}[i-1]` and :math:`\mathrm{fupp}[i-1]`|
+----------------------------+-----------------------------------------+-----------------------------------------------------------------+
|3 |basic |Between :math:`\mathrm{flow}[i-1]` and :math:`\mathrm{fupp}[i-1]`|
+----------------------------+-----------------------------------------+-----------------------------------------------------------------+
Basic and superbasic slack variables may lead to the corresponding functions being outside their bounds by as much as the option 'Minor Feasibility Tolerance'.
Very occasionally some functions may be outside their bounds by as much as the option 'Minor Feasibility Tolerance', and there may be some nonbasics for which :math:`\mathrm{f}[i-1]` lies strictly between its bounds.
If :math:`\mathrm{ninf} > 0`, some basic and superbasic variables may be outside their bounds by an arbitrary amount (bounded by :math:`\mathrm{sinf}` if scaling was not used).
**fmul** : float, ndarray, shape :math:`\left(\textit{nf}\right)`
The vector of the dual variables (Lagrange multipliers) for the general constraints :math:`l_F\leq F\left(x\right)\leq u_F`
**ns** : int
The final number of superbasic variables.
**ninf** : int
Are the number and the sum of the infeasibilities of constraints that lie outside one of their bounds by more than the option 'Minor Feasibility Tolerance' `before the solution is unscaled`.
If any `linear` constraints are infeasible, :math:`x` minimizes the sum of the infeasibilities of the linear constraints subject to the upper and lower bounds being satisfied.
In this case :math:`\mathrm{ninf}` gives the number of variables and linear constraints lying outside their upper or lower bounds.
The nonlinear constraints are not evaluated.
Otherwise, :math:`x` minimizes the sum of infeasibilities of the `nonlinear` constraints subject to the linear constraints and upper and lower bounds being satisfied.
In this case :math:`\mathrm{ninf}` gives the number of components of :math:`F\left(x\right)` lying outside their bounds by more than the option 'Minor Feasibility Tolerance'.
Again this is `before the solution is unscaled`.
**sinf** : float
Are the number and the sum of the infeasibilities of constraints that lie outside one of their bounds by more than the option 'Minor Feasibility Tolerance' `before the solution is unscaled`.
If any `linear` constraints are infeasible, :math:`x` minimizes the sum of the infeasibilities of the linear constraints subject to the upper and lower bounds being satisfied.
In this case :math:`\mathrm{ninf}` gives the number of variables and linear constraints lying outside their upper or lower bounds.
The nonlinear constraints are not evaluated.
Otherwise, :math:`x` minimizes the sum of infeasibilities of the `nonlinear` constraints subject to the linear constraints and upper and lower bounds being satisfied.
In this case :math:`\mathrm{ninf}` gives the number of components of :math:`F\left(x\right)` lying outside their bounds by more than the option 'Minor Feasibility Tolerance'.
Again this is `before the solution is unscaled`.
.. _e04vh-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default :math:`\text{} = \epsilon_r^{\frac{1}{3}}`
When :math:`\text{‘Derivative Option'} = 0`, the central-difference interval :math:`r` is used near an optimal solution to obtain more accurate (but more expensive) estimates of gradients.
Twice as many function evaluations are required compared to forward differencing.
The interval used for the :math:`j`\ th variable is :math:`h_j = r\left(1+\left\lvert x_j\right\rvert \right)`.
The resulting derivative estimates should be accurate to :math:`\mathrm{O}\left(r^2\right)`, unless the functions are badly scaled.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Check Frequency'** : int
Default :math:`\text{} = 60`
Every :math:`i`\ th minor iteration after the most recent basis factorization, a numerical test is made to see if the current solution :math:`x` satisfies the general linear constraints (the linear constraints and the linearized nonlinear constraints, if any).
The constraints are of the form :math:`Ax-s = b`, where :math:`s` is the set of slack variables.
To perform the numerical test, the residual vector :math:`r = b-Ax+s` is computed.
If the largest component of :math:`r` is judged to be too large, the current basis is refactorized and the basic variables are recomputed to satisfy the general constraints more accurately.
If :math:`i\leq 0`, the value of :math:`i = 99999999` is used and effectively no checks are made.
:math:`\text{‘Check Frequency'} = 1` is useful for debugging purposes, but otherwise this option should not be needed.
**'Crash Option'** : int
Default :math:`\text{} = 3`
Except on restarts, an internal Crash procedure is used to select an initial basis from certain rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The 'Crash Option' :math:`i` determines which rows and columns of :math:`A` are eligible initially, and how many times the Crash procedure is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of :math:`A`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called twice (if there are nonlinear constraints). The first call looks for a triangular basis in linear rows, and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (retaining the current basis for linear rows).|
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called up to three times (if there are nonlinear constraints). The first two calls treat `linear equalities` and `linear inequalities` separately. As before, the last call treats nonlinear rows before the first major iteration. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound).
The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' :math:`r` allows the starting Crash procedure to ignore certain 'small' nonzeros in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros of :math:`a_{{ij}}` in the columns are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` should be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.1`
Except on restarts, an internal Crash procedure is used to select an initial basis from certain rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The 'Crash Option' :math:`i` determines which rows and columns of :math:`A` are eligible initially, and how many times the Crash procedure is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of :math:`A`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called twice (if there are nonlinear constraints). The first call looks for a triangular basis in linear rows, and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (retaining the current basis for linear rows).|
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called up to three times (if there are nonlinear constraints). The first two calls treat `linear equalities` and `linear inequalities` separately. As before, the last call treats nonlinear rows before the first major iteration. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound).
The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' :math:`r` allows the starting Crash procedure to ignore certain 'small' nonzeros in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros of :math:`a_{{ij}}` in the columns are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` should be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Option'** : int
Default :math:`\text{} = 1`
Option 'Derivative Option' specifies which nonlinear function gradients are known analytically and will be supplied to ``nlp2_sparse_solve`` by :math:`\mathrm{usrfun}`.
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------+
|:math:`i`|Meaning |
+=========+=====================================+
|:math:`0`|Some problem derivatives are unknown.|
+---------+-------------------------------------+
|:math:`1`|All problem derivatives are known. |
+---------+-------------------------------------+
The value :math:`i = 1` should be used whenever possible.
It is the most reliable and will usually be the most efficient.
If :math:`i = 0`, ``nlp2_sparse_solve`` will `estimate` the missing components of :math:`G\left(x\right)` using finite differences.
This may simplify the coding of :math:`\mathrm{usrfun}`.
However, it could increase the total run-time substantially (since a special call to :math:`\mathrm{usrfun}` is required for each column of the Jacobian that has a missing element), and there is less assurance that an acceptable solution will be located.
If the nonlinear variables are not well scaled, it may be necessary to specify a nonstandard option 'Difference Interval'.
For each column of the Jacobian, one call to :math:`\mathrm{usrfun}` is needed to estimate all missing elements in that column, if any.
At times, central differences are used rather than forward differences.
Twice as many calls to :math:`\mathrm{usrfun}` are needed. (This is not under your control.)
**'Derivative Linesearch'** : valueless
Default
At each major iteration a linesearch is used to improve the merit function. Option 'Derivative Linesearch' uses safeguarded cubic interpolation and requires both function and gradient values to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided, or option 'Nonderivative Linesearch' is specified, ``nlp2_sparse_solve`` employs a linesearch based upon safeguarded quadratic interpolation, which does not require gradient evaluations.
A nonderivative linesearch can be slightly less robust on difficult problems, and it is recommended that the default be used if the functions and derivatives can be computed at approximately the same cost.
If the gradients are very expensive relative to the functions, a nonderivative linesearch may give a significant decrease in computation time.
If 'Nonderivative Linesearch' is selected, ``nlp2_sparse_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{usrfun}` with :math:`\mathrm{needg} = 0`.
Once the linesearch is completed, the problem functions are called again with :math:`\mathrm{needf} = 0` and :math:`\mathrm{needg} = 0`.
If the potential saving provided by a nonderivative linesearch is to be realised, it is essential that :math:`\mathrm{usrfun}` be coded so that derivatives are not computed when :math:`\mathrm{needg} = 0`.
**'Nonderivative Linesearch'** : valueless
At each major iteration a linesearch is used to improve the merit function. Option 'Derivative Linesearch' uses safeguarded cubic interpolation and requires both function and gradient values to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided, or option 'Nonderivative Linesearch' is specified, ``nlp2_sparse_solve`` employs a linesearch based upon safeguarded quadratic interpolation, which does not require gradient evaluations.
A nonderivative linesearch can be slightly less robust on difficult problems, and it is recommended that the default be used if the functions and derivatives can be computed at approximately the same cost.
If the gradients are very expensive relative to the functions, a nonderivative linesearch may give a significant decrease in computation time.
If 'Nonderivative Linesearch' is selected, ``nlp2_sparse_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{usrfun}` with :math:`\mathrm{needg} = 0`.
Once the linesearch is completed, the problem functions are called again with :math:`\mathrm{needf} = 0` and :math:`\mathrm{needg} = 0`.
If the potential saving provided by a nonderivative linesearch is to be realised, it is essential that :math:`\mathrm{usrfun}` be coded so that derivatives are not computed when :math:`\mathrm{needg} = 0`.
**'Difference Interval'** : float
Default :math:`\text{} = \sqrt{\epsilon_r}`
This alters the interval :math:`r` used to estimate gradients by forward differences.
It does so in the following circumstances:
- in the interval ('cheap') phase of verifying the problem derivatives;
- for verifying the problem derivatives;
- for estimating missing derivatives.
In all cases, a derivative with respect to :math:`x_j` is estimated by perturbing that component of :math:`x` to the value :math:`x_j+r\left(1+\left\lvert x_j\right\rvert \right)`, and then evaluating :math:`F_{\mathrm{obj}}\left(x\right)` or :math:`f\left(x\right)` at the perturbed point.
The resulting gradient estimates should be accurate to :math:`\mathrm{O}\left(r\right)` unless the functions are badly scaled.
Judicious alteration of :math:`r` may sometimes lead to greater accuracy.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Dump File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i_1`.
If :math:`i_2 > 0`, the 'Load File', containing basis information, will be read.
The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Load File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i_1`.
If :math:`i_2 > 0`, the 'Load File', containing basis information, will be read.
The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Elastic Weight'** : float
Default :math:`\text{} = 10^4`
This keyword determines the initial weight :math:`\gamma` associated with the problem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#ad-treatment>`__).
At major iteration :math:`k`, if elastic mode has not yet started, a scale factor :math:`\sigma_k = 1+\left\lVert g\left(x_k\right)\right\rVert_\infty` is defined from the current objective gradient.
Elastic mode is then started if the QP subproblem is infeasible, or the QP dual variables are larger in magnitude than :math:`\sigma_k\text{}r`.
The QP is resolved in elastic mode with :math:`\gamma = \sigma_k\text{}r`.
Thereafter, major iterations continue in elastic mode until they converge to a point that is optimal for `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#ad-treatment>`__).
If the point is feasible for equation `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn1>`__ :math:`\left(v = w = 0\right)`, it is declared locally optimal.
Otherwise, :math:`\gamma` is increased by a factor of :math:`10` and major iterations continue.
If :math:`\gamma` has already reached a maximum allowable value, equation `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn1>`__ is declared locally infeasible.
**'Expand Frequency'** : int
Default :math:`\text{} = 10000`
This option is part of the anti-cycling procedure designed to make progress even on highly degenerate problems.
For linear models, the strategy is to force a positive step at every iteration, at the expense of violating the bounds on the variables by a small amount.
Suppose that the option 'Minor Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the tolerance actually used by ``nlp2_sparse_solve`` increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
For nonlinear models, the same procedure is used for iterations in which there is only one superbasic variable. (Cycling can occur only when the current solution is at a vertex of the feasible region.) Thus, zero steps are allowed if there is more than one superbasic variable, but otherwise positive steps are enforced.
Increasing :math:`i` helps reduce the number of slightly infeasible nonbasic variables (most of which are eliminated during a resetting procedure).
However, it also diminishes the freedom to choose a large pivot element (see option 'Pivot Tolerance').
**'Factorization Frequency'** : int
Default :math:`\text{} = 50`
At most :math:`i` basis changes will occur between factorizations of the basis matrix.
With linear programs, the basis factors are usually updated every iteration.
The default :math:`i` is reasonable for typical problems.
Higher values up to :math:`i = 100` (say) may be more efficient on well-scaled problems.
When the objective function is nonlinear, fewer basis updates will occur as an optimum is approached.
The number of iterations between basis factorizations will, therefore, increase.
During these iterations a test is made regularly (according to the option 'Check Frequency') to ensure that the general constraints are satisfied.
If necessary the basis will be refactorized before the limit of :math:`i` updates is reached.
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.8}`
The `relative function precision` :math:`\epsilon_r` is intended to be a measure of the relative accuracy with which the nonlinear functions can be computed.
For example, if :math:`f\left(x\right)` is computed as :math:`1000.56789` for some relevant :math:`x` and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-6`.
Ideally the functions :math:`f_i\left(x\right)` should have magnitude of order :math:`1`.
If all functions are substantially `less` than :math:`1` in magnitude, :math:`\epsilon_r` should be the `absolute` precision.
For example, if :math:`f\left(x\right) = 1.23456789e-4` at some point and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-10`.)
The default value of :math:`\epsilon_r` is appropriate for simple analytic functions.
In some cases the function values will be the result of extensive computation, possibly involving a costly iterative procedure that can provide few digits of precision.
Specifying an appropriate 'Function Precision' may lead to savings, by allowing the linesearch procedure to terminate when the difference between function values along the search direction becomes as small as the absolute error in the values.
**'Hessian Full Memory'** : valueless
Default if :math:`n_1\leq 75`
These options select the method for storing and updating the approximate Hessian. (``nlp2_sparse_solve`` uses a quasi-Newton approximation to the Hessian of the Lagrangian.
A BFGS update is applied after each major iteration.)
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and the BFGS updates are applied explicitly.
This option is most efficient when the number of variables :math:`n` is not too large (say, less than :math:`75`).
In this case, the storage requirement is fixed and one can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should be used on problems where :math:`n` is very large.
In this case a limited-memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
**'Hessian Limited Memory'** : valueless
Default if :math:`n_1 > 75`
These options select the method for storing and updating the approximate Hessian. (``nlp2_sparse_solve`` uses a quasi-Newton approximation to the Hessian of the Lagrangian.
A BFGS update is applied after each major iteration.)
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and the BFGS updates are applied explicitly.
This option is most efficient when the number of variables :math:`n` is not too large (say, less than :math:`75`).
In this case, the storage requirement is fixed and one can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should be used on problems where :math:`n` is very large.
In this case a limited-memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
**'Hessian Frequency'** : int
Default :math:`\text{} = 99999999`
If option 'Hessian Full Memory' is in effect and :math:`i` BFGS updates have already been carried out, the Hessian approximation is reset to the identity matrix. (For certain problems, occasional resets may improve convergence, but in general they should not be necessary.)
'Hessian Full Memory' and :math:`\text{‘Hessian Frequency'} = 10` have a similar effect to 'Hessian Limited Memory' and :math:`\text{‘Hessian Updates'} = 10` (except that the latter retains the current diagonal during resets).
**'Hessian Updates'** : int
Default :math:`\text{} = \text{‘Hessian Frequency'}` if 'Hessian Full Memory', :math:`10` otherwise
If option 'Hessian Limited Memory' is in effect and :math:`i` BFGS updates have already been carried out, all but the diagonal elements of the accumulated updates are discarded and the updating process starts again.
Broadly speaking, the more updates stored, the better the quality of the approximate Hessian.
However, the more vectors stored, the greater the cost of each QP iteration.
The default value is likely to give a robust algorithm without significant expense, but faster convergence can sometimes be obtained with significantly fewer updates (e.g., :math:`i = 5`).
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r\geq 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Iterations Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(10000, {10\mathrm{max}\left(n, \textit{nf}\right)}\right)`
The value of :math:`i` specifies the maximum number of minor iterations allowed (i.e., iterations of the simplex method or the QP algorithm), summed over all major iterations. (See also the description of the option 'Minor Iterations Limit'.)
**'Linesearch Tolerance'** : float
Default :math:`= 0.9`
This tolerance, :math:`r`, controls the accuracy with which a step length will be located along the direction of search each iteration.
At the start of each linesearch a target directional derivative for the merit function is identified.
This argument determines the accuracy to which this target value is approximated, and it must be a value in the range :math:`0.0\leq r\leq 1.0`.
The default value :math:`r = 0.9` requests just moderate accuracy in the linesearch.
If the nonlinear functions are cheap to evaluate, a more accurate search may be appropriate; try :math:`r = 0.1\text{, }0.01\text{ or }0.001`.
If the nonlinear functions are expensive to evaluate, a less accurate search may be appropriate. `If all gradients are known`, try :math:`r = 0.99`. (The number of major iterations might increase, but the total number of function evaluations may decrease enough to compensate.)
If not all gradients are known, a moderately accurate search remains appropriate.
Each search will require only :math:`1`--5 function values (typically), but many function calls will then be needed to estimate missing gradients for the next iteration.
**'LU Density Tolerance'** : float
Default :math:`\text{} = 0.6`
The density tolerance, :math:`r_1`, is used during :math:`LU` factorization of the basis matrix :math:`B`.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated, and the remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
The singularity tolerance, :math:`r_2`, helps guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows: if :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable. (This is most likely to occur after a restart.)
**'LU Singularity Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
The density tolerance, :math:`r_1`, is used during :math:`LU` factorization of the basis matrix :math:`B`.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated, and the remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
The singularity tolerance, :math:`r_2`, helps guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows: if :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable. (This is most likely to occur after a restart.)
**'LU Factor Tolerance'** : float
Default :math:`\text{} = 3.99`
The values of :math:`r_1` and :math:`r_2` affect the stability of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Update Tolerance'** : float
Default :math:`\text{} = 3.99`
The values of :math:`r_1` and :math:`r_2` affect the stability of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Partial Pivoting'** : valueless
Default
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_sparse_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'LU Complete Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_sparse_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'LU Rook Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_sparse_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'Major Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This tolerance, :math:`r`, specifies how accurately the nonlinear constraints should be satisfied.
The default value is appropriate when the linear and nonlinear constraints contain data to about that accuracy.
Let :math:`v_{\mathrm{max}}` be the maximum nonlinear constraint violation, normalized by the size of the solution, which is required to satisfy
.. math::
v_{\mathrm{max}} = \mathrm{max}_iv_i\text{ }/\text{ }\left\lVert x\right\rVert \leq r\text{,}
where :math:`v_{\textit{i}}` is the violation of the :math:`\textit{i}`\ th nonlinear constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nf}`.
In the major iteration log (see `Minor Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#minoriteration>`__), :math:`v_{\mathrm{max}}` appears as the quantity labelled 'Feasible'.
If some of the problem functions are known to be of low accuracy, a larger 'Major Feasibility Tolerance' may be appropriate.
**'Major Optimality Tolerance'** : float
Default :math:`\text{} = 2\mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This tolerance, :math:`r`, specifies the final accuracy of the dual variables.
On successful termination, ``nlp2_sparse_solve`` will have computed a solution :math:`\left(x, s, \pi \right)` such that
.. math::
c_{\mathrm{max}} = \mathrm{max}_jc_j\text{ }/\text{ }\left\lVert \pi \right\rVert \leq r\text{,}
where :math:`c_{\textit{j}}` is an estimate of the complementarity slackness for variable :math:`\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n+\textit{nf}`.
The values :math:`c_i` are computed from the final QP solution using the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j` (where :math:`g_j` is the :math:`j`\ th component of the objective gradient, :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`, and :math:`\pi` is the set of QP dual variables):
.. math::
c_j = \left\{\begin{array}{cc} d_j \mathrm{min}\left({x_j-l_j}, 1\right) & \text{if } d_j\geq 0 \text{;} \\ {-d_j} \mathrm{min}\left({u_j-x_j}, 1\right) & \text{if } d_j < 0 \text{.} \end{array}\right)
In the 'Print File', :math:`c_{\mathrm{max}}` appears as the quantity labelled 'Optimal'.
**'Major Iterations Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(1000, {3\mathrm{max}\left(n, \textit{nf}\right)}\right)`
This is the maximum number of major iterations allowed.
It is intended to guard against an excessive number of linearizations of the constraints.
If :math:`i = 0`, optimality and feasibility are checked.
**'Major Print Level'** : int
Default :math:`\text{} = 1`
This controls the amount of output to the options 'Print File' and 'Summary File' at each major iteration. :math:`\text{‘Major Print Level'} = 0` suppresses most output, except for error messages. :math:`\text{‘Major Print Level'} = 1` gives normal output for linear and nonlinear problems, and :math:`\text{‘Major Print Level'} = 11` gives additional details of the Jacobian factorization that commences each major iteration.
In general, the value being specified may be thought of as a binary number of the form
.. math::
\text{‘Major Print Level'}\text{ }JFDXbs
where each letter stands for a digit that is either :math:`0` or :math:`1` as follows:
.. rst-class:: nag-rules-none
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`s`|a single line that gives a summary of each major iteration. (This entry in :math:`JFDXbs` is not strictly binary since the summary line is printed whenever :math:`JFDXbs\geq 1`);|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`b`|basis statistics, i.e., information relating to the basis matrix whenever it is refactorized. (This output is always provided if :math:`JFDXbs\geq 10`); |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`X`|:math:`x_k`, the nonlinear variables involved in the objective function or the constraints. These appear under the heading 'Jacobian variables'; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`D`|:math:`\pi_k`, the dual variables for the nonlinear constraints. These appear under the heading 'Multiplier estimates'; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`F`|:math:`f\left(x_k\right)`, the values of the nonlinear constraint functions; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`J`|:math:`J\left(x_k\right)`, the Jacobian matrix. This appears under the heading ':math:`x` and Jacobian'. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
To obtain output of any items :math:`JFDXbs`, set the corresponding digit to :math:`1`, otherwise to :math:`0`.
If :math:`J = 1`, the Jacobian matrix will be output column-wise at the start of each major iteration.
Column :math:`j` will be preceded by the value of the corresponding variable :math:`x_j` and a key to indicate whether the variable is basic, superbasic or nonbasic. (Hence if :math:`J = 1`, there is no reason to specify :math:`X = 1` unless the objective contains more nonlinear variables than the Jacobian.) A typical line of output is
::
3 1.250000e+01 BS 1 1.00000e+00 4 2.00000e+00
which would mean that :math:`x_3` is basic at value :math:`12.5`, and the third column of the Jacobian has elements of :math:`1.0` and :math:`2.0` in rows :math:`1` and :math:`4`.
**'Major Step Limit'** : float
Default :math:`\text{} = 2.0`
This argument limits the change in :math:`x` during a linesearch.
It applies to all nonlinear problems, once a 'feasible solution' or 'feasible subproblem' has been found.
(1) A linesearch determines a step :math:`\alpha` over the range :math:`0 < \alpha \leq \beta`, where :math:`\beta` is :math:`1` if there are nonlinear constraints or is the step to the nearest upper or lower bound on :math:`x` if all the constraints are linear. Normally, the first step length tried is :math:`\alpha_1 = \mathrm{min}\left(1, \beta \right)`.
(#) In some cases, such as :math:`f\left(x\right) = ae^{{bx}}` or :math:`f\left(x\right) = ax^b`, even a moderate change in the components of :math:`x` can lead to floating-point overflow. The argument :math:`r` is, therefore, used to define a limit :math:`\bar{\beta } = r\left(1+\left\lVert x\right\rVert \right)/\left\lVert p\right\rVert` (where :math:`p` is the search direction), and the first evaluation of :math:`f\left(x\right)` is at the potentially smaller step length :math:`\alpha_1 = \mathrm{min}\left(1, \bar{\beta }, \beta \right)`.
(#) Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at meaningless points. The option 'Major Step Limit' provides an additional safeguard. The default value :math:`r = 2.0` should not affect progress on well behaved problems, but setting :math:`r = 0.1\text{ or }0.01` may be helpful when rapidly varying functions are present. A 'good' starting point may be required. An important application is to the class of nonlinear least squares problems.
(#) In cases where several local optima exist, specifying a small value for :math:`r` may help locate an optimum near the starting point.
**'Minimize'** : valueless
Default
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_sparse_solve``.
**'Maximize'** : valueless
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_sparse_solve``.
**'Feasible Point'** : valueless
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_sparse_solve``.
**'Minor Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
``nlp2_sparse_solve`` tries to ensure that all variables eventually satisfy their upper and lower bounds to within this tolerance, :math:`r`.
This includes slack variables.
Hence, general linear constraints should also be satisfied to within :math:`r`.
Feasibility with respect to nonlinear constraints is judged by the option 'Major Feasibility Tolerance' (not by :math:`r`).
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
If :math:`\mathrm{sinf}` is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Nonlinear functions will be evaluated only at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every attempt should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both variables.
If :math:`r = 1.0e-6`, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious.
In general, keep :math:`x` as far away from singularities as possible.)
If :math:`\text{‘Scale Option'}\geq 1`, feasibility is defined in terms of the `scaled` problem (since it is then more likely to be meaningful).
In reality, ``nlp2_sparse_solve`` uses :math:`r` as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible. ``nlp2_sparse_solve`` is then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic).
See the description of the option 'Elastic Weight'.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left\{10^{-6}, \sqrt{\epsilon }\right\}`
``nlp2_sparse_solve`` tries to ensure that all variables eventually satisfy their upper and lower bounds to within this tolerance, :math:`r`.
This includes slack variables.
Hence, general linear constraints should also be satisfied to within :math:`r`.
Feasibility with respect to nonlinear constraints is judged by the option 'Major Feasibility Tolerance' (not by :math:`r`).
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
If :math:`\mathrm{sinf}` is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Nonlinear functions will be evaluated only at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every attempt should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both variables.
If :math:`r = 1.0e-6`, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious.
In general, keep :math:`x` as far away from singularities as possible.)
If :math:`\text{‘Scale Option'}\geq 1`, feasibility is defined in terms of the `scaled` problem (since it is then more likely to be meaningful).
In reality, ``nlp2_sparse_solve`` uses :math:`r` as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible. ``nlp2_sparse_solve`` is then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic).
See the description of the option 'Elastic Weight'.
**'Minor Iterations Limit'** : int
Default :math:`\text{} = 500`
If the number of minor iterations for the optimality phase of the QP subproblem exceeds :math:`i`, then all nonbasic QP variables that have not yet moved are frozen at their current values and the reduced QP is solved to optimality.
Note that more than :math:`i` minor iterations may be necessary to solve the reduced QP to optimality.
These extra iterations are necessary to ensure that the terminated point gives a suitable direction for the linesearch.
In the major iteration log (see `Minor Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#minoriteration>`__) a t at the end of a line indicates that the corresponding QP was artificially terminated using the limit :math:`i`.
Compare with the option 'Iterations Limit', which defines an independent `absolute` limit on the `total` number of minor iterations (summed over all QP subproblems).
**'Minor Print Level'** : int
Default :math:`\text{} = 1`
This controls the amount of output to the 'Print File' and 'Summary File' during solution of the QP subproblems.
The value of :math:`i` has the following effect:
.. rst-class:: nag-rules-none nag-align-left
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+===============+=====================================================================================================================================================================================================================================================+
|:math:`0` |No minor iteration output except error messages. |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1` |A single line of output at each minor iteration (controlled by options 'Print Frequency' and 'Summary Frequency'. |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 10`|Basis factorization statistics generated during the periodic refactorization of the basis (see the option 'Factorization Frequency'). Statistics for the `first factorization` each major iteration are controlled by the option 'Major Print Level'.|
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'New Basis File'** : int
Default :math:`\text{} = 0`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved in the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first in the file associated with unit :math:`12` and then immediately in the file associated with unit :math:`11`.
If the run is interrupted at iteration :math:`2000` during the save in the file associated with unit :math:`12`, there will still be a usable basis in the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Backup Basis File'** : int
Default :math:`\text{} = 0`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved in the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first in the file associated with unit :math:`12` and then immediately in the file associated with unit :math:`11`.
If the run is interrupted at iteration :math:`2000` during the save in the file associated with unit :math:`12`, there will still be a usable basis in the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Save Frequency'** : int
Default :math:`\text{} = 100`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved in the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_sparse_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified.
The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first in the file associated with unit :math:`12` and then immediately in the file associated with unit :math:`11`.
If the run is interrupted at iteration :math:`2000` during the save in the file associated with unit :math:`12`, there will still be a usable basis in the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'New Superbasics Limit'** : int
Default :math:`\text{} = 99`
This option causes early termination of the QP subproblems if the number of free variables has increased significantly since the first feasible point.
If the number of new superbasics is greater than :math:`i`, the nonbasic variables that have not yet moved are frozen and the resulting smaller QP is solved to optimality.
In the major iteration log (see `Major Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#majoriteration>`__), a t at the end of a line indicates that the QP was terminated early in this way.
**'Nolist'** : valueless
Default
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Old Basis File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the basis maps information will be obtained from this file. The file will usually have been output previously as a 'New Basis File' or 'Backup Basis File'.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
The file will not be acceptable if the number of rows or columns in the problem has been altered.
**'Partial Price'** : int
Default :math:`\text{} = 1`
This argument is recommended for large problems that have significantly more variables than constraints.
It reduces the work required for each 'pricing' operation (where a nonbasic variable is selected to become superbasic).
When :math:`i = 1`, all columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}` are searched.
Otherwise, :math:`A` and :math:`I` are partitioned to give :math:`i` roughly equal segments :math:`A_{\textit{j}}` and :math:`I_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,i`.
If the previous pricing search was successful on :math:`A_{{j-1}}` and :math:`I_{{j-1}}`, the next search begins on the segments :math:`A_j` and :math:`I_j`. (All subscripts here are modulo :math:`i`.) If a reduced gradient is found that is larger than some dynamic tolerance, the variable with the largest such reduced gradient (of appropriate sign) is selected to become superbasic.
If nothing is found, the search continues on the next segments :math:`A_{{j+1}}` and :math:`I_{{j+1}}`, and so on.
For time-stage models having :math:`t` time periods, 'Partial Price' :math:`t` (or :math:`t/2` or :math:`t/3`) may be appropriate.
**'Pivot Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
During the solution of QP subproblems, the pivot tolerance is used to prevent columns entering the basis if they would cause the basis to become almost singular.
When :math:`x` changes to :math:`x+\alpha p` for some search direction :math:`p`, a 'ratio test' determines which component of :math:`x` reaches an upper or lower bound first.
The corresponding element of :math:`p` is called the pivot element.
Elements of :math:`p` are ignored (and, therefore, cannot be pivot elements) if they are smaller than the pivot tolerance :math:`r`.
It is common for two or more variables to reach a bound at essentially the same time.
In such cases, the 'Minor Feasibility Tolerance' (say, :math:`t`) provides some freedom to maximize the pivot element and thereby improve numerical stability.
Excessively small values of :math:`t` should, therefore, not be specified.
To a lesser extent, the 'Expand Frequency' (say, :math:`f`) also provides some freedom to maximize the pivot element.
Excessively `large` values of :math:`f` should, therefore, not be specified.
**'Print File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the following information is output to a file associated with unit :math:`i` during the solution of each problem:
- a listing of the options;
- some statistics about the problem;
- the amount of storage available for the :math:`LU` factorization of the basis matrix;
- notes about the initial basis resulting from a Crash procedure or a Basis file;
- the iteration log;
- basis factorization statistics;
- the exit :math:`\textit{errno}` condition and some statistics about the solution obtained;
- the printed solution, if requested.
These items are described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#fcomments>`__ and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#fcomments>`__.
Further brief output may be directed to the 'Summary File'.
**'Print Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i > 0`, one line of the iteration log will be printed every :math:`i`\ th iteration.
A value such as :math:`i = 10` is suggested for those interested only in the final solution.
If :math:`i\leq 0`, the value of :math:`i = 99999999` is used and effectively no checks are made.
**'Proximal Point Method'** : int
Default :math:`\text{} = 1`
:math:`i = 1\text{ or }2` specifies minimization of :math:`\left\lVert x-x_0\right\rVert_1` or :math:`\frac{1}{2}\left\lVert x-x_0\right\rVert_2^2` when the starting point :math:`x_0` is changed to satisfy the linear constraints (where :math:`x_0` refers to nonlinear variables).
**'Punch File'** : int
Default :math:`= 0`
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to the file. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0` the 'Insert File' containing basis information will be read from unit :math:`i_2`. The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'Insert File'** : int
Default :math:`= 0`
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to the file. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0` the 'Insert File' containing basis information will be read from unit :math:`i_2`. The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'Scale Option'** : int
Default :math:`\text{} = 0`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Print'** : valueless
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Solution File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the final solution will be output to file :math:`i` (whether optimal or not).
All numbers are printed in ``1pe16.6`` format.
To see more significant digits in the printed solution, it will sometimes be useful to make :math:`i` refer to 'Print File'.
**'Summary File'** : int
Default :math:`\text{} = 0`
If :math:`i_1 > 0`, a brief log will be output to the file associated with unit :math:`i_1`, including one line of information every :math:`i_2`\ th iteration.
In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `The Summary File <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#monitoring-summaryfile>`__.
**'Summary Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i_1 > 0`, a brief log will be output to the file associated with unit :math:`i_1`, including one line of information every :math:`i_2`\ th iteration.
In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `The Summary File <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#monitoring-summaryfile>`__.
**'Superbasics Limit'** : int
Default :math:`\text{} = n_1`
This option places a limit on the storage allocated for superbasic variables.
Ideally, :math:`i` should be set slightly larger than the 'number of degrees of freedom' expected at an optimal solution.
For nonlinear problems, the number of degrees of freedom is often called the 'number of independent variables'.
Normally, :math:`i` need not be greater than :math:`n+1`, where :math:`n_1` is the number of nonlinear variables.
For many problems, :math:`i` may be considerably smaller than :math:`n`.
This will save storage if :math:`n` is very large.
**'Suppress Parameters'** : valueless
Normally ``nlp2_sparse_solve`` prints the options file as it is being read, and then prints a complete list of the available keywords and their final values.
The option 'Suppress Parameters' tells ``nlp2_sparse_solve`` not to print the full list.
**'System Information No'** : valueless
Default
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#monitoring>`__.
**'System Information Yes'** : valueless
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#monitoring>`__.
**'Timing Level'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, some timing information will be output to the Print file, if :math:`\text{‘Print File'} > 0`.
**'Unbounded Objective'** : float
Default :math:`\text{} = 1.0e+15`
These arguments are intended to detect unboundedness in nonlinear problems.
During a linesearch, :math:`F_{\mathrm{obj}}` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert F_{\mathrm{obj}}\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, iterations are terminated with the exit message :math:`\mathrm{errno}` = 5.
If singularities are present, unboundedness in :math:`F_{\mathrm{obj}}\left(x\right)` may be manifested by a floating-point overflow (during the evaluation of :math:`F_{\mathrm{obj}}\left(x+\alpha p\right)`), before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
**'Unbounded Step Size'** : float
Default :math:`\text{} = \textit{infbnd}`
These arguments are intended to detect unboundedness in nonlinear problems.
During a linesearch, :math:`F_{\mathrm{obj}}` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert F_{\mathrm{obj}}\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, iterations are terminated with the exit message :math:`\mathrm{errno}` = 5.
If singularities are present, unboundedness in :math:`F_{\mathrm{obj}}\left(x\right)` may be manifested by a floating-point overflow (during the evaluation of :math:`F_{\mathrm{obj}}\left(x+\alpha p\right)`), before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
**'Verify Level'** : int
Default :math:`\text{} = 0`
This option refers to finite difference checks on the derivatives computed by the user-supplied functions.
Derivatives are checked at the first point that satisfies all bounds and linear constraints.
.. rst-class:: nag-rules-none nag-align-left
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+=============================================================================================================================================================+
|:math:`0` |Only a 'cheap' test will be performed, requiring two calls to :math:`\mathrm{usrfun}`. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Individual gradients will be checked (with a more reliable test). A key of the form OK or Bad? indicates whether or not each component appears to be correct.|
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Individual columns of the problem Jacobian will be checked. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Options 2 and 1 will both occur (in that order). |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1`|Derivative checking is disabled. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
:math:`\text{‘Verify Level'} = 3` should be specified whenever a new :math:`\mathrm{usrfun}` is being developed.
**'Violation Limit'** : float
Default :math:`\text{} = 1.0e+6`
This keyword defines an absolute limit on the magnitude of the maximum constraint violation, :math:`r`, after the linesearch.
On completion of the linesearch, the new iterate :math:`x_{{k+1}}` satisfies the condition
.. math::
v_i\left(x_{{k+1}}\right)\leq r\text{ }\mathrm{max}\left(1, {v_i\left(x_0\right)}\right)\text{,}
where :math:`x_0` is the point at which the nonlinear constraints are first evaluated and :math:`v_i\left(x\right)` is the :math:`i`\ th nonlinear constraint violation :math:`v_i\left(x\right) = \mathrm{max}\left(0, {l_i-f_i\left(x\right)}, {f_i\left(x\right)-u_i}\right)`.
The effect of this violation limit is to restrict the iterates to lie in an `expanded` feasible region whose size depends on the magnitude of :math:`r`.
This makes it possible to keep the iterates within a region where the objective is expected to be well-defined and bounded below.
If the objective is bounded below for all values of the variables, :math:`r` may be any large positive value.
.. _e04vh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
On entry, one but not both of :math:`\textit{nxname}` and :math:`\textit{nfname}` is equal to :math:`1`. :math:`\textit{nxname} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nfname} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{objrow} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{objrow}\leq \textit{nf}`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{xlow}` and :math:`\mathrm{xupp}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{xlow} = \mathrm{xupp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{xlow} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xupp} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{xlow}` and :math:`\mathrm{xupp}` for variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{xlow} = \mathrm{xupp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{xlow} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xupp} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{flow}` and :math:`\mathrm{fupp}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{flow} = \mathrm{fupp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{flow} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{fupp} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{flow}` and :math:`\mathrm{fupp}` for variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{flow} = \mathrm{fupp} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{infbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{flow} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{fupp} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
Basis file dimensions do not match this problem.
(`errno` :math:`2`)
Array element :math:`\mathrm{igfun}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` is out of range :math:`1` to :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`, or array element :math:`\mathrm{jgvar}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` is out of range :math:`1` to :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`\textit{lena} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lena}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\textit{leng} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{leng}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nf}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{start} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{start} = 0`, :math:`1` or :math:`2`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{nea} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nea}\geq 0`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{neg} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{neg}\geq 0`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{nea} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nea}\leq n\times \textit{nf}`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{neg} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{neg}\leq n\times \textit{nf}`.
(`errno` :math:`2`)
On entry, :math:`\textit{nxname} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nxname} = 1` or :math:`\textit{n}`.
(`errno` :math:`2`)
On entry, :math:`\textit{nfname} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nfname} = 1` or :math:`\textit{nf}`.
(`errno` :math:`8`)
User-supplied function computes incorrect objective derivatives.
(`errno` :math:`8`)
User-supplied function computes incorrect constraint derivatives.
(`errno` :math:`11`)
Internal error: memory allocation failed when attempting to allocate workspace sizes :math:`\langle\mathit{\boldsymbol{value}}\rangle`, :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle`. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`12`)
Internal memory allocation was insufficient. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`13`)
An error has occurred in the basis package. Check that arrays :math:`\mathrm{iafun}`, :math:`\mathrm{javar}`, :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}` contain values in the appropriate ranges and do not define duplicate elements of :math:`\mathrm{a}` or :math:`\mathrm{g}`. Set the option 'Print File' and examine the output carefully for further information.
(`errno` :math:`14`)
An unexpected error has occurred. Set the option 'Print File' and examine the output carefully for further information.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The requested accuracy could not be achieved.
(`errno` :math:`4`)
The linear constraints appear to be infeasible.
(`errno` :math:`4`)
The problem appears to be infeasible. The linear equality constraints could not be satisfied.
(`errno` :math:`4`)
The problem appears to be infeasible. Nonlinear infeasibilites have been minimized.
(`errno` :math:`4`)
The problem appears to be infeasible. Infeasibilites have been minimized.
(`errno` :math:`5`)
The problem appears to be unbounded. The objective function is unbounded.
(`errno` :math:`5`)
The problem appears to be unbounded. The constraint violation limit has been reached.
(`errno` :math:`10`)
User-supplied function requested termination.
**NagAlgorithmicMajorWarning**
(`errno` :math:`6`)
Iteration limit reached.
(`errno` :math:`6`)
Major iteration limit reached.
(`errno` :math:`6`)
The value of the option 'Superbasics Limit' is too small.
(`errno` :math:`7`)
Numerical difficulties have been encountered and no further progress can be made.
(`errno` :math:`9`)
User-supplied function is undefined at the first feasible point.
(`errno` :math:`9`)
User-supplied function is undefined at the initial point.
(`errno` :math:`9`)
Unable to proceed into undefined region of user-supplied function.
.. _e04vh-py2-py-notes:
**Notes**
``nlp2_sparse_solve`` is designed to minimize a linear or nonlinear function subject to bounds on the variables and sparse linear or nonlinear constraints.
It is suitable for large-scale linear and quadratic programming and for linearly constrained optimization, as well as for general nonlinear programs of the form
.. math::
\mathrm{minimize}_x\text{}f_0\left(x\right)\quad \text{ subject to }l\leq \begin{pmatrix}x\\f\left(x\right)\\A_Lx\end{pmatrix}\leq u\text{,}
where :math:`x` is an :math:`n`-vector of variables, :math:`l` and :math:`u` are constant lower and upper bounds, :math:`f_0\left(x\right)` is a smooth scalar objective function, :math:`A_L` is a sparse matrix, and :math:`f\left(x\right)` is a vector of smooth nonlinear constraint functions :math:`\left\{f_i\left(x\right)\right\}`.
The option 'Maximize' specifies that :math:`f_0\left(x\right)` should be maximized instead of minimized.
Ideally, the first derivatives (gradients) of :math:`f_0\left(x\right)` and :math:`f_i\left(x\right)` should be known and coded by you.
If only some of the gradients are known, ``nlp2_sparse_solve`` estimates the missing ones by finite differences.
If :math:`f_0\left(x\right)` is linear and :math:`f\left(x\right)` is absent, `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn1>`__ is a linear program (LP) and ``nlp2_sparse_solve`` applies the primal simplex method (see Dantzig (1963)).
Sparse basis factors are maintained by LUSOL (see Gill `et al.` (1987)) as in MINOS (see Murtagh and Saunders (1995)).
If only the objective is nonlinear, the problem is linearly constrained (LC) and tends to solve more easily than the general case with nonlinear constraints (NC).
For both nonlinear cases, ``nlp2_sparse_solve`` applies a sparse Sequential Quadratic Programming (SQP) method (see Gill `et al.` (2002)), using limited-memory quasi-Newton approximations to the Hessian of the Lagrangian.
The merit function for step-length control is an augmented Lagrangian, as in the dense SQP solver :meth:`nlp2_solve` (see Gill `et al.` (1986) and Gill `et al.` (1992)).
``nlp2_sparse_solve`` is suitable for nonlinear problems with thousands of constraints and variables, and is most efficient if only some of the variables enter nonlinearly, or there are relatively few degrees of freedom at a solution (i.e., many constraints are active).
However, there is no limit on the number of degrees of freedom.
``nlp2_sparse_solve`` allows linear and nonlinear constraints and variables to be entered in an `arbitrary order`, and uses one function to define all the nonlinear functions.
The optimization problem is assumed to be in the form
.. math::
\mathrm{minimize}_xF_{\mathrm{obj}}\left(x\right)\quad \text{ subject to }l_x\leq x\leq u_x\text{, }\quad l_F\leq F\left(x\right)\leq u_F\text{,}
where the upper and lower bounds are constant, :math:`F\left(x\right)` is a vector of smooth linear and nonlinear constraint functions :math:`\left\{F_i\left(x\right)\right\}`, and :math:`F_{\mathrm{obj}}\left(x\right)` is one of the components of :math:`F` to be minimized, as specified by the input argument :math:`\mathrm{objrow}`. ``nlp2_sparse_solve`` reorders the variables and constraints so that the problem is in the form `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vhf.html#eqn1>`__.
Upper and lower bounds are specified for all variables and functions.
The :math:`j`\ th constraint may be defined as an equality by setting :math:`l_j = u_j`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` should be set to special values that are treated as :math:`-\infty` or :math:`+\infty`.
Free variables and free constraints ('free rows') have both bounds infinite.
In general, the components of :math:`F` are `structured` in the sense that they are formed from sums of linear and nonlinear functions of just some of the variables.
This structure can be exploited by ``nlp2_sparse_solve``.
In many cases, the vector :math:`F\left(x\right)` is a sum of linear and nonlinear functions. ``nlp2_sparse_solve`` allows these terms to be specified separately, so that the linear part is defined just once by the input arguments :math:`\mathrm{iafun}`, :math:`\mathrm{javar}` and :math:`\mathrm{a}`.
Only the nonlinear part is recomputed at each :math:`x`.
Suppose that each component of :math:`F\left(x\right)` is of the form
.. math::
F_i\left(x\right) = f_i\left(x\right)+\sum_{{j = 1}}^nA_{{ij}}x_j\text{,}
where :math:`f_i\left(x\right)` is a nonlinear function (possibly zero) and the elements :math:`A_{{ij}}` are constant.
The :math:`nf\times n` Jacobian of :math:`F\left(x\right)` is the sum of two sparse matrices of the same size: :math:`F^{\prime }\left(x\right) = G\left(x\right)+A`, where :math:`G\left(x\right) = f^{\prime }\left(x\right)` and :math:`A` is the matrix with elements :math:`\left\{A_{{ij}}\right\}`.
The two matrices must be `non-overlapping` in the sense that each element of the Jacobian :math:`F^{\prime }\left(x\right) = G\left(x\right)+A` comes from :math:`G\left(x\right)` or :math:`A`, but `not both`.
The element cannot be split between :math:`G\left(x\right)` and :math:`A`.
For example, the function
.. math::
F\left(x\right) = \begin{pmatrix} 3x_1 + e^{x_2} x_4 + x_2^2 + 4x_4 - x_3 + x_5 \\ x_2 + x_3^2 + \sin\left(x_4\right) - 3x_5 \\ x_1 - x_3 \end{pmatrix}
can be written as
.. math::
F\left(x\right) = f\left(x\right)+Ax = \begin{pmatrix} e^{x_2} x_4 + x_2^2 + 4x_4 \\ x_3^2 + \sin\left(x_4\right) \\0\end{pmatrix}+\begin{pmatrix} 3x_1 - x_3 + x_5 \\ x_2 - 3x_5 \\ x_1 - x_3 \end{pmatrix}\text{,}
in which case
.. math::
F^{\prime }\left(x\right) = \begin{pmatrix}3& e^{x_2} x_4 + 2x_2 &-1& e^{x_2} + 4 & 1\\0&1& 2x_3 & \cos\left(x_4\right) &-3\\1&0&-1&0& 0\end{pmatrix}
can be written as :math:`F^{\prime }\left(x\right) = f^{\prime }\left(x\right)+A = G\left(x\right)+A`, where
.. math::
G\left(x\right) = \begin{pmatrix}0& e^{x_2} x_4 + 2x_2 &0& e^{x_2} + 4 &0\\0&0& 2x_3 & \cos\left(x_4\right) &0\\0&0&0&0&0\end{pmatrix}\text{, }\quad A = \begin{pmatrix}3&0&-1&0& 1\\0&1& 0&0&-3\\1&0&-1&0& 0\end{pmatrix}\text{.}
**Note:** the element :math:`e^{x_2}+4` of :math:`F^{\prime }\left(x\right)` appears in :math:`G\left(x\right)` and is not split between :math:`G\left(x\right)` and :math:`A` although it contains a linear term.
The nonzero elements of :math:`A` and :math:`G` are provided to ``nlp2_sparse_solve`` in coordinate form.
The elements of :math:`A` are entered as triples :math:`\left(i, j, A_{{ij}}\right)` in the arrays :math:`\mathrm{iafun}`, :math:`\mathrm{javar}` and :math:`\mathrm{a}`.
The sparsity pattern :math:`G` is entered as pairs :math:`\left(i, j\right)` in the arrays :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}`.
The corresponding entries :math:`G_{{ij}}` (any that are known) are assigned to appropriate array elements :math:`\mathrm{g}\left(k\right)` in :math:`\mathrm{usrfun}`.
The elements of :math:`A` and :math:`G` may be stored in any order.
Duplicate entries are ignored. :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}` may be defined automatically by function :meth:`nlp2_sparse_jacobian` when :math:`\text{‘Derivative Option'} = 0` is specified and :math:`\mathrm{usrfun}` does not provide any gradients.
Throughout this document the symbol :math:`\epsilon` is used to represent the machine precision (see :meth:`machine.precision <naginterfaces.library.machine.precision>`).
``nlp2_sparse_solve`` is based on SNOPTA, which is part of the SNOPT package described in Gill `et al.` (2005b).
.. _e04vh-py2-py-references:
**References**
Dantzig, G B, 1963, `Linear Programming and Extensions`, Princeton University Press
Eldersveld, S K, 1991, `Large-scale sequential quadratic programming algorithms`, PhD Thesis, Department of Operations Research, Stanford University, Stanford
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E, Murray, W and Saunders, M A, 2002, `SNOPT: An SQP Algorithm for Large-scale Constrained Optimization` (12), 979--1006, SIAM J. Optim.
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SQOPT 7: a Fortran package for large-scale linear and quadratic programming`, Report NA 05-1, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sqdoc7.pdf
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SNOPT 7.1: a Fortran package for large-scale linear nonlinear programming`, Report NA 05-2, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sndoc7.pdf
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1987, `Maintaining` :math:`LU` `factors of a general sparse matrix`, Linear Algebra and its Applics. (88/89), 239--270
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1992, `Some theoretical properties of an augmented Lagrangian merit function`, Advances in Optimization and Parallel Computing, (ed P M Pardalos), 101--128, North Holland
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
Murtagh, B A and Saunders, M A, 1978, `Large-scale linearly constrained optimization`, 14, 41--72, Math. Programming
Murtagh, B A and Saunders, M A, 1982, `A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints`, Math. Program. Stud. (16), 84--118
Murtagh, B A and Saunders, M A, 1995, `MINOS 5.4 users' guide`, Report SOL 83-20R, Department of Operations Research, Stanford University
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp2_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_sparse_jacobian(nf, usrfun, lena, leng, x, xlow, xupp, comm, data=None, io_manager=None):
r"""
``nlp2_sparse_jacobian`` may be used before :meth:`nlp2_sparse_solve` to determine the sparsity pattern for the Jacobian.
.. deprecated:: 28.3.0.0
``nlp2_sparse_jacobian`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vj-py2-py-doc:
For full information please refer to the NAG Library document for e04vj
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vjf.html
.. _e04vj-py2-py-parameters:
**Parameters**
**nf** : int
:math:`\textit{nf}`, the number of problem functions in :math:`F\left(x\right)`, including the objective function (if any) and the linear and nonlinear constraints. Simple upper and lower bounds on :math:`x` can be defined using the arguments :math:`\mathrm{xlow}` and :math:`\mathrm{xupp}` and should not be included in :math:`F`.
**usrfun** : callable (status, f, g) = usrfun(status, x, needf, f, needg, g, data=None)
:math:`\mathrm{usrfun}` must define the problem functions :math:`F\left(x\right)`.
This function is passed to ``nlp2_sparse_jacobian`` as the external argument :math:`\mathrm{usrfun}`.
**Parameters**
**status** : int
Indicates the first call to :math:`\mathrm{usrfun}`.
:math:`\mathrm{status} = 0`
There is nothing special about the current call to :math:`\mathrm{usrfun}`.
:math:`\mathrm{status} = 1`
``nlp2_sparse_jacobian`` is calling your function for the `first` time. Some data may need to be input or computed and saved.
**x** : float, ndarray, shape :math:`\left(n\right)`
The variables :math:`x` at which the problem functions are to be calculated. The array :math:`x` must not be altered.
**needf** : int
Indicates if :math:`\mathrm{f}` must be assigned during the call to :math:`\mathrm{usrfun}` (see :math:`\mathrm{f}`).
**f** : float, ndarray, shape :math:`\left(\textit{nf}\right)`
This will be set by ``nlp2_sparse_jacobian``.
**needg** : int
``nlp2_sparse_jacobian`` will call :math:`\mathrm{usrfun}` with :math:`\mathrm{needg} = 0` to indicate that :math:`\mathrm{g}` is not required.
**g** : float, ndarray, shape :math:`\left(\textit{leng}\right)`
Concerns the calculations of the derivatives of the function :math:`f\left(x\right)`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**status** : int
May be used to indicate that you are unable to evaluate :math:`F` at the current :math:`x`. (For example, the problem functions may not be defined there).
``nlp2_sparse_jacobian`` evaluates :math:`F\left(x\right)` at random perturbation of the initial point :math:`x`, say :math:`x_p`.
If the functions cannot be evaluated at :math:`x_p`, you can set :math:`\mathrm{status} = -1`, ``nlp2_sparse_jacobian`` will use another random perturbation.
If for some reason you wish to terminate the current problem, set :math:`\mathrm{status}\leq -2`.
**f** : float, array-like, shape :math:`\left(\textit{nf}\right)`
The computed :math:`F\left(x\right)` according to the setting of :math:`\mathrm{needf}`.
If :math:`\mathrm{needf} = 0`, :math:`\mathrm{f}` is not required and is ignored.
If :math:`\mathrm{needf} > 0`, the components of :math:`F\left(x\right)` must be calculated and assigned to :math:`\mathrm{f}`. ``nlp2_sparse_jacobian`` will always call :math:`\mathrm{usrfun}` with :math:`\mathrm{needf} > 0`.
To simplify the code, you may ignore the value of :math:`\mathrm{needf}` and compute :math:`F\left(x\right)` on every entry to :math:`\mathrm{usrfun}`.
**g** : float, array-like, shape :math:`\left(\textit{leng}\right)`
``nlp2_sparse_jacobian`` will always call :math:`\mathrm{usrfun}` with :math:`\mathrm{needg} = 0`: :math:`\mathrm{g}` is not required to be set on exit but must be declared correctly.
**lena** : int
:math:`\mathrm{lena}` should be an `overestimate` of the number of elements in the linear part of the Jacobian.
**leng** : int
:math:`\mathrm{leng}` should be an `overestimate` of the number of elements in the nonlinear part of the Jacobian.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the variables :math:`x`. The contents of :math:`x` will be used by ``nlp2_sparse_jacobian`` in the call of :math:`\mathrm{usrfun}`, and so each element of :math:`\mathrm{x}` should be within the bounds given by :math:`\mathrm{xlow}` and :math:`\mathrm{xupp}`.
**xlow** : float, array-like, shape :math:`\left(n\right)`
Contain the lower and upper bounds :math:`l_x` and :math:`u_x` on the variables :math:`x`.
To specify a nonexistent lower bound :math:`\left[l_x\right]_j = {-\infty }`, set :math:`\mathrm{xlow}[j-1]\leq {-\textit{bigbnd}}`, where :math:`\textit{bigbnd}` is the option 'Infinite Bound Size'.
To specify a nonexistent upper bound :math:`\mathrm{xupp}[j-1]\geq \textit{bigbnd}`.
To fix the :math:`j`\ th variable (say, :math:`x_j = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`), set :math:`\mathrm{xlow}[j-1] = \mathrm{xupp}[j-1] = \beta`.
**xupp** : float, array-like, shape :math:`\left(n\right)`
Contain the lower and upper bounds :math:`l_x` and :math:`u_x` on the variables :math:`x`.
To specify a nonexistent lower bound :math:`\left[l_x\right]_j = {-\infty }`, set :math:`\mathrm{xlow}[j-1]\leq {-\textit{bigbnd}}`, where :math:`\textit{bigbnd}` is the option 'Infinite Bound Size'.
To specify a nonexistent upper bound :math:`\mathrm{xupp}[j-1]\geq \textit{bigbnd}`.
To fix the :math:`j`\ th variable (say, :math:`x_j = \beta`, where :math:`\left\lvert \beta \right\rvert < \textit{bigbnd}`), set :math:`\mathrm{xlow}[j-1] = \mathrm{xupp}[j-1] = \beta`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**iafun** : int, ndarray, shape :math:`\left(\mathrm{lena}\right)`
Define the :math:`i` coordinates of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
**javar** : int, ndarray, shape :math:`\left(\mathrm{lena}\right)`
Define the :math:`j` coordinates of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
**nea** : int
Is the number of nonzero entries in :math:`A` such that :math:`F\left(x\right) = f\left(x\right)+Ax`.
**a** : float, ndarray, shape :math:`\left(\mathrm{lena}\right)`
Define the coordinates :math:`\left(i, j\right)` and values :math:`A_{{ij}}` of the nonzero elements of the linear part :math:`A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
In particular, :math:`\mathrm{nea}` triples :math:`\left({\mathrm{iafun}[k-1]}, {\mathrm{javar}[k-1]}, {\mathrm{a}[k-1]}\right)` define the row and column indices :math:`i = \mathrm{iafun}[k-1]` and :math:`j = \mathrm{javar}[k-1]` of the element :math:`A_{{ij}} = \mathrm{a}[k-1]`.
**igfun** : int, ndarray, shape :math:`\left(\mathrm{leng}\right)`
Define the coordinates :math:`\left(i, j\right)` of the nonzero elements of :math:`G`, the nonlinear part of the derivatives :math:`J\left(x\right) = G\left(x\right)+A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
**jgvar** : int, ndarray, shape :math:`\left(\mathrm{leng}\right)`
Define the coordinates :math:`\left(i, j\right)` of the nonzero elements of :math:`G`, the nonlinear part of the derivatives :math:`J\left(x\right) = G\left(x\right)+A` of the function :math:`F\left(x\right) = f\left(x\right)+Ax`.
**neg** : int
The number of nonzero entries in :math:`G`.
.. _e04vj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
On entry, :math:`\mathrm{lena} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lena}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{leng} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{leng}\geq 1`.
(`errno` :math:`3`)
User-supplied function :math:`\mathrm{usrfun}` indicates that functions are undefined near given point :math:`\mathrm{x}`.
(`errno` :math:`5`)
Either :math:`\mathrm{lena}` or :math:`\mathrm{leng}` is too small. Increase both of them and corresponding array sizes. :math:`\mathrm{lena} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{leng} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
Cannot estimate Jacobian structure at given point :math:`\mathrm{x}`.
(`errno` :math:`7`)
Internal error: memory allocation failed when attempting to allocate workspace sizes :math:`\langle\mathit{\boldsymbol{value}}\rangle`, :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle`. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`8`)
Internal memory allocation was insufficient. Please contact `NAG <https://www.nag.com>`__.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`4`)
User-supplied function :math:`\mathrm{usrfun}` requested termination.
.. _e04vj-py2-py-notes:
**Notes**
When using :meth:`nlp2_sparse_solve`, if you set the option :math:`\text{‘Derivative Option'} = 0` and :math:`\mathrm{usrfun}` provides none of the derivatives, you may need to call ``nlp2_sparse_jacobian`` to determine the input arrays :math:`\mathrm{iafun}`, :math:`\mathrm{javar}`, :math:`\mathrm{a}`, :math:`\mathrm{igfun}` and :math:`\mathrm{jgvar}`.
These arrays define the pattern of nonzeros in the Jacobian matrix.
``nlp2_sparse_jacobian`` determines the sparsity pattern for the Jacobian and identifies the constant elements automatically.
To do so, ``nlp2_sparse_jacobian`` approximates the problem functions, :math:`F\left(x\right)`, at three random perturbations of the given initial point :math:`x`.
If an element of the approximate Jacobian is the same at all three points, then it is taken to be constant.
If it is zero, it is taken to be identically zero.
Since the random points are not chosen close together, the heuristic will correctly classify the Jacobian elements in the vast majority of cases.
In general, ``nlp2_sparse_jacobian`` finds that the Jacobian can be permuted to the form:
.. math::
\begin{pmatrix}G\left(x\right)&A_3\\A_2&A_4\end{pmatrix}\text{,}
where :math:`A_2`, :math:`A_3` and :math:`A_4` are constant.
Note that :math:`G\left(x\right)` might contain elements that are also constant, but ``nlp2_sparse_jacobian`` must classify them as nonlinear.
This is because :meth:`nlp2_sparse_solve` 'removes' linear variables from the calculation of :math:`F` by setting them to zero before calling :math:`\mathrm{usrfun}`.
A knowledgeable user would be able to move such elements from :math:`F\left(x\right)` in :math:`\mathrm{usrfun}` and enter them as part of :math:`\mathrm{iafun}`, :math:`\mathrm{javar}` and :math:`\mathrm{a}` for :meth:`nlp2_sparse_solve`.
.. _e04vj-py2-py-references:
**References**
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_file(ispecs, comm, io_manager=None):
r"""
``nlp2_sparse_option_file`` may be used to supply options to :meth:`nlp2_sparse_solve` from an external file.
The initialization function :meth:`nlp2_sparse_init` **must** have been called before calling ``nlp2_sparse_option_file``.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_file`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vk-py2-py-doc:
For full information please refer to the NAG Library document for e04vk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vkf.html
.. _e04vk-py2-py-parameters:
**Parameters**
**ispecs** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the option file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04vk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
At least one line of the options file is invalid.
(`errno` :math:`2`)
Could not read options file on unit :math:`\mathrm{ispecs} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vk-py2-py-notes:
**Notes**
``nlp2_sparse_option_file`` may be used to supply values for options to :meth:`nlp2_sparse_solve`. ``nlp2_sparse_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword.
- a phrase that qualifies the keyword.
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`16` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Option settings are preserved following a call to :meth:`nlp2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_string(string, comm, io_manager=None):
r"""
``nlp2_sparse_option_string`` may be used to supply individual options to :meth:`nlp2_sparse_solve`.
The initialization function :meth:`nlp2_sparse_init` **must** have been called before calling ``nlp2_sparse_option_string``.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_string`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vl-py2-py-doc:
For full information please refer to the NAG Library document for e04vl
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vlf.html
.. _e04vl-py2-py-parameters:
**Parameters**
**string** : str
A single valid option string (see :ref:`Notes <e04vl-py2-py-notes>` in ``nlp2_sparse_option_string`` and :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04vl-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vl-py2-py-notes:
**Notes**
``nlp2_sparse_option_string`` may be used to supply values for options to :meth:`nlp2_sparse_solve`.
It is only necessary to call ``nlp2_sparse_option_string`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_sparse_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_integer_set(string, ivalue, comm, io_manager=None):
r"""
``nlp2_sparse_option_integer_set`` may be used to supply individual `int` options to :meth:`nlp2_sparse_solve`.
The initialization function :meth:`nlp2_sparse_init` **must** have been called before calling ``nlp2_sparse_option_integer_set``.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_integer_set`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vm-py2-py-doc:
For full information please refer to the NAG Library document for e04vm
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vmf.html
.. _e04vm-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`).
**ivalue** : int
An `int` value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04vm-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ivalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vm-py2-py-notes:
**Notes**
``nlp2_sparse_option_integer_set`` may be used to supply values for `int` options to :meth:`nlp2_sparse_solve`.
It is only necessary to call ``nlp2_sparse_option_integer_set`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_sparse_option_integer_set`` sets one argument value.
Each `int` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{ivalue}`.
Option settings are preserved following a call to :meth:`nlp2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_double_set(string, rvalue, comm, io_manager=None):
r"""
``nlp2_sparse_option_double_set`` may be used to supply individual `float` options to :meth:`nlp2_sparse_solve`.
The initialization function :meth:`nlp2_sparse_init` **must** have been called before calling ``nlp2_sparse_option_double_set``.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_double_set`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vn-py2-py-doc:
For full information please refer to the NAG Library document for e04vn
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vnf.html
.. _e04vn-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`).
**rvalue** : float
The value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04vn-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{rvalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vn-py2-py-notes:
**Notes**
``nlp2_sparse_option_double_set`` may be used to supply values for `float` options to :meth:`nlp2_sparse_solve`.
It is only necessary to call ``nlp2_sparse_option_double_set`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_sparse_option_double_set`` sets one argument value.
Each `float` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{rvalue}`.
Option settings are preserved following a call to :meth:`nlp2_sparse_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_sparse_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_integer_get(string, comm, io_manager=None):
r"""
``nlp2_sparse_option_integer_get`` is used to get the value of an `int` option. ``nlp2_sparse_option_integer_get`` can be used before or after calling :meth:`nlp2_sparse_solve`.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_integer_get`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vr-py2-py-doc:
For full information please refer to the NAG Library document for e04vr
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vrf.html
.. _e04vr-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**ivalue** : int
The `int` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04vr-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option string is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vr-py2-py-notes:
**Notes**
``nlp2_sparse_option_integer_get`` obtains the current value of an integer option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_sparse_option_double_get(string, comm, io_manager=None):
r"""
``nlp2_sparse_option_double_get`` is used to get the value of a `float` option. ``nlp2_sparse_option_double_get`` can be used before or after calling :meth:`nlp2_sparse_solve`.
.. deprecated:: 28.3.0.0
``nlp2_sparse_option_double_get`` is deprecated.
Please use :meth:`handle_solve_ssqp` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04vs-py2-py-doc:
For full information please refer to the NAG Library document for e04vs
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04vsf.html
.. _e04vs-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_sparse_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**rvalue** : float
The `float` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04vs-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_sparse_init` has not been called.
(`errno` :math:`2`)
The supplied option string is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04vs-py2-py-notes:
**Notes**
``nlp2_sparse_option_double_get`` obtains the current value of a `float` option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_sparse_solve <e04vh-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp1_init(rname):
r"""
``nlp1_init`` is used to initialize functions :meth:`uncon_conjgrd_comp`, :meth:`lp_solve`, :meth:`lsq_lincon_solve`, :meth:`qp_dense_solve`, :meth:`qpconvex1_sparse_solve`, :meth:`nlp1_solve`, :meth:`nlp1_rcomm`, :meth:`nlp1_sparse_solve` and :meth:`lsq_gencon_deriv`.
.. _e04wb-py2-py-doc:
For full information please refer to the NAG Library document for e04wb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wbf.html
.. _e04wb-py2-py-parameters:
**Parameters**
**rname** : str
The name of the function to be initialized.
**Returns**
**comm** : dict, communication object
Communication structure.
.. _e04wb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{rname} = \langle\mathit{\boldsymbol{value}}\rangle` was not a valid function name.
.. _e04wb-py2-py-notes:
**Notes**
``nlp1_init`` initializes some or all of the arrays :math:`\mathrm{comm}`\ ['cwsav'], :math:`\mathrm{comm}`\ ['lwsav'], :math:`\mathrm{comm}`\ ['iwsav'] and :math:`\mathrm{comm}`\ ['rwsav'] for the function and any associated option setting functions.
See Also
--------
:meth:`naginterfaces.library.examples.mip.ilp_dense_ex.main`
:meth:`naginterfaces.library.examples.opt.lsq_gencon_deriv_ex.main`
:meth:`naginterfaces.library.examples.opt.nlp1_rcomm_ex.main`
:meth:`naginterfaces.library.examples.opt.nlp1_solve_ex.main`
:meth:`naginterfaces.library.examples.opt.nlp1_sparse_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_init(io_manager=None):
r"""
``nlp2_init`` is used to initialize the function :meth:`nlp2_solve`.
.. _e04wc-py2-py-doc:
For full information please refer to the NAG Library document for e04wc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wcf.html
.. _e04wc-py2-py-parameters:
**Parameters**
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**comm** : dict, communication object
Communication structure.
.. _e04wc-py2-py-notes:
**Notes**
``nlp2_init`` initializes the arrays :math:`\mathrm{comm}`\ ['iw'] and :math:`\mathrm{comm}`\ ['rw'] for the function :meth:`nlp2_solve`.
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp2_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_solve(a, bl, bu, objfun, istate, ccon, cjac, clamda, h, x, comm, confun=None, data=None, io_manager=None, spiked_sorder='C'):
r"""
``nlp2_solve`` is designed to minimize an arbitrary smooth function subject to constraints (which may include simple bounds on the variables, linear constraints and smooth nonlinear constraints) using a Sequential Quadratic Programming (SQP) method.
As many first derivatives as possible should be supplied by you; any unspecified derivatives are approximated by finite differences.
It is not intended for large sparse problems.
``nlp2_solve`` may also be used for unconstrained, bound-constrained and linearly constrained optimization.
``nlp2_solve`` uses **forward** **communication** for evaluating the objective function, the nonlinear constraint functions, and any of their derivatives.
The initialization function :meth:`nlp2_init` **must** have been called before to calling ``nlp2_solve``.
Note: this function uses optional algorithmic parameters, see also: :meth:`nlp2_option_file`, :meth:`nlp2_option_string`, :meth:`nlp2_option_integer_set`, :meth:`nlp2_option_double_set`, :meth:`nlp2_init`.
.. _e04wd-py2-py-doc:
For full information please refer to the NAG Library document for e04wd
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html
.. _e04wd-py2-py-parameters:
**Parameters**
**a** : float, array-like, shape :math:`\left(\textit{nclin}, :\right)`
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` contains the :math:`\textit{i}`\ th row of the matrix :math:`A_L` of general linear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__. That is, the :math:`\textit{i}`\ th row contains the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nclin}`.
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**objfun** : callable (mode, objf, grad) = objfun(mode, x, grad, nstate, data=None)
:math:`\mathrm{objfun}` must calculate the objective function :math:`F\left(x\right)` and (optionally) its gradient :math:`g\left(x\right) = \frac{{\partial F}}{{\partial x}}` for a specified :math:`n`-vector :math:`x`.
**Parameters**
**mode** : int
Is set by ``nlp2_solve`` to indicate which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{objf}`.
:math:`\mathrm{mode} = 1`
All available elements of :math:`\mathrm{grad}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{objf}` and all available elements of :math:`\mathrm{grad}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the objective function and/or all available elements of its gradient are to be evaluated.
**grad** : float, ndarray, shape :math:`\left(n\right)`
The elements of :math:`\mathrm{grad}` are set to special values.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``nlp2_solve`` is calling :math:`\mathrm{objfun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**mode** : int
May be used to indicate that you are unable or unwilling to evaluate the objective function at the current :math:`x`.
During the linesearch, the function is evaluated at points of the form :math:`x = x_k+\alpha p_k` after they have already been evaluated satisfactorily at :math:`x_k`.
For any such :math:`x`, if you set :math:`\mathrm{mode} = -1`, ``nlp2_solve`` will reduce :math:`\alpha` and evaluate the functions again (closer to :math:`x_k`, where they are more likely to be defined).
If for some reason you wish to terminate the current problem, set :math:`\mathrm{mode} < -1`.
**objf** : float
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{objf}` must be set to the value of the objective function at :math:`x`.
**grad** : float, array-like, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 1` or :math:`2`, :math:`\mathrm{grad}` must return the available elements of the gradient evaluated at :math:`x`, i.e., :math:`\mathrm{grad}[i-1]` contains the partial derivative :math:`\frac{{\partial F}}{{\partial x_i}}`.
**istate** : int, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
Is an integer array that need not be initialized if ``nlp2_solve`` is called with the 'Cold Start' option (the default).
If option 'Warm Start' has been chosen, every element of :math:`\mathrm{istate}` must be set.
If ``nlp2_solve`` has just been called on a problem with the same dimensions, :math:`\mathrm{istate}` already contains valid values.
Otherwise, :math:`\mathrm{istate}[j-1]` should indicate whether either of the constraints :math:`r_j\left(x\right)\geq l_j` or :math:`r_j\left(x\right)\leq u_j` is expected to be active at a solution of `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__.
The ordering of :math:`\mathrm{istate}` is the same as for :math:`\mathrm{bl}`, :math:`\mathrm{bu}` and :math:`r\left(x\right)`, i.e., the first :math:`\textit{n}` components of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, the next :math:`\textit{nclin}` refer to the bounds on :math:`A_Lx`, and the last :math:`\textit{ncnln}` refer to the bounds on :math:`c\left(x\right)`.
Possible values of :math:`\mathrm{istate}[i-1]` follow:
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0`|Neither :math:`r_j\left(x\right)\geq l_j` nor :math:`r_j\left(x\right)\leq u_j` is expected to be active. |
+---------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|:math:`r_j\left(x\right)\geq l_j` is expected to be active. |
+---------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|:math:`r_j\left(x\right)\leq u_j` is expected to be active. |
+---------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`|This may be used if :math:`l_j = u_j`. Normally an equality constraint :math:`r_j\left(x\right) = l_j = u_j` is active at a solution.|
+---------+-------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`1`, :math:`2` or :math:`3` all have the same effect when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.
If necessary, ``nlp2_solve`` will override your specification of :math:`\mathrm{istate}`, so that a poor choice will not cause the algorithm to fail.
**ccon** : float, array-like, shape :math:`\left(\max\left(1,\textit{ncnln}\right)\right)`
:math:`\mathrm{ccon}` need not be initialized if the (default) option 'Cold Start' is used.
For a 'Warm Start', and if :math:`\textit{ncnln} > 0`, :math:`\mathrm{ccon}` contains values of the nonlinear constraint functions :math:`c_{\textit{i}}`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`, calculated in a previous call to ``nlp2_solve``.
**cjac** : float, array-like, shape :math:`\left(\textit{ncnln}, :\right)`
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{ncnln} > 0`: :math:`n`; otherwise: :math:`1`.
In general, :math:`\mathrm{cjac}` need not be initialized before the call to ``nlp2_solve``. However, if :math:`\text{‘Derivative Level'} = 2` or :math:`3`, any constant elements of :math:`\mathrm{cjac}` may be initialized. Such elements need not be reassigned on subsequent calls to :math:`\mathrm{confun}`.
**clamda** : float, array-like, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{clamda}[\textit{j}-1]` must contain a multiplier estimate for each nonlinear constraint, with a sign that matches the status of the constraint specified by the :math:`\mathrm{istate}` array, for :math:`\textit{j} = n+\textit{nclin}+1,\ldots,n+\textit{nclin}+\textit{ncnln}`.
The remaining elements need not be set.
If the :math:`j`\ th constraint is defined as 'inactive' by the initial value of the :math:`\mathrm{istate}` array (i.e., :math:`\mathrm{istate}[j-1] = 0`), :math:`\mathrm{clamda}[j-1]` should be zero; if the :math:`j`\ th constraint is an inequality active at its lower bound (i.e., :math:`\mathrm{istate}[j-1] = 1`), :math:`\mathrm{clamda}[j-1]` should be non-negative; if the :math:`j`\ th constraint is an inequality active at its upper bound (i.e., :math:`\mathrm{istate}[j-1] = 2`), :math:`\mathrm{clamda}[j-1]` should be non-positive.
If necessary, the function will modify :math:`\mathrm{clamda}` to match these rules.
**h** : float, array-like, shape :math:`\left(n, n\right)`
:math:`\mathrm{h}` need not be initialized if the (default) option 'Cold Start' is used, and will be set to the identity.
If the option 'Warm Start' has been chosen, :math:`\mathrm{h}` provides the initial approximation of the Hessian of the Lagrangian, i.e., :math:`\mathrm{h}[i-1,j-1]≈\frac{{\partial^2\mathcal{L}\left(x, \lambda \right)}}{{\partial x_i\partial x_j}}`, where :math:`\mathcal{L}\left(x, \lambda \right) = F\left(x\right)- {c\left(x\right)}^\mathrm{T}\lambda` and :math:`\lambda` is an estimate of the Lagrange multipliers. :math:`\mathrm{h}` must be a positive definite matrix.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**confun** : None or callable (mode, ccon, cjac) = confun(mode, needc, x, cjac, nstate, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the vector :math:`c\left(x\right)` of nonlinear constraint functions and (optionally) its Jacobian, :math:`\frac{{\partial c}}{{\partial x}}`, for a specified :math:`n`-vector :math:`x`.
If there are no nonlinear constraints (i.e., :math:`\textit{ncnln} = 0`), ``nlp2_solve`` will never call :math:`\mathrm{confun}`, so it may be **None**. If there are nonlinear constraints, the first call to :math:`\mathrm{confun}` will occur before the first call to :math:`\mathrm{objfun}`.
If all constraint gradients (Jacobian elements) are known (i.e., :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any `constant` elements may be assigned to :math:`\mathrm{cjac}` once only at the start of the optimization.
An element of :math:`\mathrm{cjac}` that is not subsequently assigned in :math:`\mathrm{confun}` will retain its initial value throughout.
Constant elements may be loaded in :math:`\mathrm{cjac}` during the first call to :math:`\mathrm{confun}` (signalled by the value of :math:`\mathrm{nstate} = 1`).
The ability to preload constants is useful when many Jacobian elements are identically zero, in which case :math:`\mathrm{cjac}` may be initialized to zero and nonzero elements may be reset by :math:`\mathrm{confun}`.
It must be emphasized that, if :math:`\text{‘Derivative Level'} < 2`, unassigned elements of :math:`\mathrm{cjac}` are `not` treated as constant; they are estimated by finite differences, at nontrivial expense.
**Parameters**
**mode** : int
Is set by ``nlp2_solve`` to indicate which values must be assigned during each call of :math:`\mathrm{confun}`. Only the following values need be assigned, for each value of :math:`i` such that :math:`\mathrm{needc}[i-1] > 0`:
:math:`\mathrm{mode} = 0`
The components of :math:`\mathrm{ccon}` corresponding to positive values in :math:`\mathrm{needc}` must be set. Other components and the array :math:`\mathrm{cjac}` are ignored.
:math:`\mathrm{mode} = 1`
The known components of the rows of :math:`\mathrm{cjac}` corresponding to positive values in :math:`\mathrm{needc}` must be set. Other rows of :math:`\mathrm{cjac}` and the array :math:`\mathrm{ccon}` will be ignored.
:math:`\mathrm{mode} = 2`
Only the elements of :math:`\mathrm{ccon}` corresponding to positive values of :math:`\mathrm{needc}` need to be set (and similarly for the known components of the rows of :math:`\mathrm{cjac}`).
**needc** : int, ndarray, shape :math:`\left(\textit{ncnln}\right)`
The indices of the elements of :math:`\mathrm{ccon}` and/or :math:`\mathrm{cjac}` that must be evaluated by :math:`\mathrm{confun}`. If :math:`\mathrm{needc}[i-1] > 0`, the :math:`i`\ th element of :math:`\mathrm{ccon}` and/or the available elements of the :math:`i`\ th row of :math:`\mathrm{cjac}` (see argument :math:`\mathrm{mode}`) must be evaluated at :math:`x`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the constraint functions and/or the available elements of the constraint Jacobian are to be evaluated.
**cjac** : float, ndarray, shape :math:`\left(\max\left(1,\textit{ncnln}\right), n\right)`
The elements of :math:`\mathrm{cjac}` are set to special values that enable ``nlp2_solve`` to detect whether they are reset by :math:`\mathrm{confun}`.
**nstate** : int
If :math:`\mathrm{nstate} = 1`, then ``nlp2_solve`` is calling :math:`\mathrm{confun}` for the first time. This argument setting allows you to save computation time if certain data must be read or calculated only once.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**mode** : int
May be used to indicate that you are unable or unwilling to evaluate the constraint functions at the current :math:`x`.
During the linesearch, the constraint functions are evaluated at points of the form :math:`x = x_k+\alpha p_k` after they have already been evaluated satisfactorily at :math:`x_k`.
At any such :math:`\alpha`, if you set :math:`\mathrm{mode} = -1`, ``nlp2_solve`` will evaluate the functions at some point closer to :math:`x_k` (where they are more likely to be defined).
If for some reason you wish to terminate the current problem, set :math:`\mathrm{mode} < -1`.
**ccon** : float, array-like, shape :math:`\left(\max\left(1,\textit{ncnln}\right)\right)`
If :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{ccon}[i-1]` must contain the value of the :math:`i`\ th constraint at :math:`x`. The remaining elements of :math:`\mathrm{ccon}`, corresponding to the non-positive elements of :math:`\mathrm{needc}`, are ignored.
**cjac** : float, array-like, shape :math:`\left(\max\left(1,\textit{ncnln}\right), n\right)`
If :math:`\mathrm{needc}[i-1] > 0` and :math:`\mathrm{mode} = 1` or :math:`2`, the :math:`i`\ th row of :math:`\mathrm{cjac}` must contain the available elements of the vector :math:`\nabla c_i` given by
.. math::
\nabla c_i = \left(\frac{{\partial c_i}}{{\partial x_1}},\frac{{\partial c_i}}{{\partial x_2}},\ldots,\frac{{\partial c_i}}{{\partial x_n}}\right)^\mathrm{T}\text{,}
where :math:`\frac{{\partial c_i}}{{\partial x_j}}` is the partial derivative of the :math:`i`\ th constraint with respect to the :math:`j`\ th variable, evaluated at the point :math:`x`. See also the argument :math:`\mathrm{nstate}`. The remaining rows of :math:`\mathrm{cjac}`, corresponding to non-positive elements of :math:`\mathrm{needc}`, are ignored.
If all elements of the constraint Jacobian are known (i.e., :math:`\text{‘Derivative Level'} = 2` or :math:`3`), any constant elements may be assigned to :math:`\mathrm{cjac}` one time only at the start of the optimization.
An element of :math:`\mathrm{cjac}` that is not subsequently assigned in :math:`\mathrm{confun}` will retain its initial value throughout.
Constant elements may be loaded into :math:`\mathrm{cjac}` during the first call to :math:`\mathrm{confun}` (signalled by the value :math:`\mathrm{nstate} = 1`).
The ability to preload constants is useful when many Jacobian elements are identically zero, in which case :math:`\mathrm{cjac}` may be initialized to zero and nonzero elements may be reset by :math:`\mathrm{confun}`.
Note that constant nonzero elements do affect the values of the constraints.
Thus, if :math:`\mathrm{cjac}[i-1,j-1]` is set to a constant value, it need not be reset in subsequent calls to :math:`\mathrm{confun}`, but the value :math:`\mathrm{cjac}[i-1,j-1]\times \mathrm{x}[j-1]` must nonetheless be added to :math:`\mathrm{ccon}[i-1]`.
For example, if :math:`\mathrm{cjac}[0,0] = 2` and :math:`\mathrm{cjac}[0,1] = -5`, then the term :math:`2\times \mathrm{x}[0]-5\times \mathrm{x}[1]` must be included in the definition of :math:`\mathrm{ccon}[0]`.
It must be emphasized that, if :math:`\text{‘Derivative Level'} = 0` or :math:`1`, unassigned elements of :math:`\mathrm{cjac}` are not treated as constant; they are estimated by finite differences, at nontrivial expense.
If you do not supply a value for the option 'Difference Interval', an interval for each element of :math:`x` is computed automatically at the start of the optimization.
The automatic procedure can usually identify constant elements of :math:`\mathrm{cjac}`, which are then computed once only by finite differences.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**spiked_sorder** : str, optional
If :math:`\mathrm{a}` and :math:`\mathrm{h}` are spiked (i.e., have unit extent in all but one dimension, or have size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with them in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
Two-dimensional arrays returned from callback functions in this routine must then use the same storage order.
**Returns**
**majits** : int
The number of major iterations performed.
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
Describes the status of the constraints :math:`l\leq r\left(x\right)\leq u`. For the :math:`j`\ th lower or upper bound, :math:`j = 1,2,\ldots,n+\textit{nclin}+\textit{ncnln}`, the possible values of :math:`\mathrm{istate}[j-1]` are as follows (see Figure [label omitted]). :math:`\delta` is the appropriate feasibility tolerance.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-2` |(Region 1) The lower bound is violated by more than :math:`\delta`. |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |(Region 5) The upper bound is violated by more than :math:`\delta`. |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`{0}`|(Region 3) Both bounds are satisfied by more than :math:`\delta`. |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`{1}`|(Region 2) The lower bound is active (to within :math:`\delta`). |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`{2}`|(Region 4) The upper bound is active (to within :math:`\delta`). |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
|:math:`{3}`|(:math:`\text{Region 2} = \text{Region 4}`) The bounds are equal and the equality constraint is satisfied (to within :math:`\delta`).|
+-----------+-------------------------------------------------------------------------------------------------------------------------------------+
These values of :math:`\mathrm{istate}` are labelled in the printed solution according to Table [label omitted].
+----------------------------+----------+---------+---------+---------+----------+-----------------+
|Region |:math:`1` |:math:`2`|:math:`3`|:math:`4`|:math:`5` |:math:`2\equiv 4`|
+----------------------------+----------+---------+---------+---------+----------+-----------------+
|:math:`\mathrm{istate}[j-1]`|:math:`-2`|:math:`1`|:math:`0`|:math:`2`|:math:`-1`|:math:`3` |
+----------------------------+----------+---------+---------+---------+----------+-----------------+
|Printed solution |-- |LL |FR |UL |++ |EQ |
+----------------------------+----------+---------+---------+---------+----------+-----------------+
**ccon** : float, ndarray, shape :math:`\left(\max\left(1,\textit{ncnln}\right)\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{ccon}[\textit{i}-1]` contains the value of the :math:`\textit{i}`\ th nonlinear constraint function :math:`c_{\textit{i}}` at the final iterate, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`.
If :math:`\textit{ncnln} = 0`, the array :math:`\mathrm{ccon}` is not referenced.
**cjac** : float, ndarray, shape :math:`\left(\textit{ncnln}, :\right)`
If :math:`\textit{ncnln} > 0`, :math:`\mathrm{cjac}` contains the Jacobian matrix of the nonlinear constraint functions at the final iterate, i.e., :math:`\mathrm{cjac}[\textit{i}-1,\textit{j}-1]` contains the partial derivative of the :math:`\textit{i}`\ th constraint function with respect to the :math:`\textit{j}`\ th variable, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,\textit{ncnln}`. (See the discussion of argument :math:`\mathrm{cjac}` under :math:`\mathrm{confun}`.)
If :math:`\textit{ncnln} = 0`, the array :math:`\mathrm{cjac}` is not referenced.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}+\textit{ncnln}\right)`
The values of the QP multipliers from the last QP subproblem. :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1` and non-positive if :math:`\mathrm{istate}[j-1] = 2`.
**objf** : float
The value of the objective function at the final iterate.
**grad** : float, ndarray, shape :math:`\left(n\right)`
The gradient of the objective function (or its finite difference approximation) at the final iterate.
**h** : float, ndarray, shape :math:`\left(n, n\right)`
Contains the Hessian of the Lagrangian at the final estimate :math:`x`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final estimate of the solution.
.. _e04wd-py2-py-other_params:
**Other Parameters**
**'Central Difference Interval'** : float
Default :math:`\text{} = \epsilon_r^{\frac{1}{3}}`
When :math:`\text{‘Derivative Level'} < 3`, the central-difference interval :math:`r` is used near an optimal solution to obtain more accurate (but more expensive) estimates of gradients.
Twice as many function evaluations are required compared to forward differencing.
The interval used for the :math:`j`\ th variable is :math:`h_j = r\left(1+\left\lvert x_j\right\rvert \right)`.
The resulting derivative estimates should be accurate to :math:`\mathrm{O}\left(r^2\right)`, unless the functions are badly scaled.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Check Frequency'** : int
Default :math:`\text{} = 60`
Every :math:`i`\ th minor iteration after the most recent basis factorization, a numerical test is made to see if the current solution :math:`x` satisfies the general linear constraints (the linear constraints and the linearized nonlinear constraints, if any).
The constraints are of the form :math:`Ax-s = b`, where :math:`s` is the set of slack variables.
To perform the numerical test, the residual vector :math:`r = b-Ax+s` is computed.
If the largest component of :math:`r` is judged to be too large, the current basis is refactorized and the basic variables are recomputed to satisfy the general constraints more accurately.
If :math:`i\leq 0`, the value of :math:`i = 99999999` is used and effectively no checks are made.
:math:`\text{‘Check Frequency'} = 1` is useful for debugging purposes, but otherwise this option should not be needed.
**'Cold Start'** : valueless
Default
This option controls the specification of the initial working set in the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp2_solve`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{h}` as discussed in :ref:`Parameters <e04wd-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp2_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp2_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option controls the specification of the initial working set in the procedure for finding a feasible point for the linear constraints and bounds and in the first QP subproblem thereafter.
With a 'Cold Start', the first working set is chosen by ``nlp2_solve`` based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must set the :math:`\mathrm{istate}` array and define :math:`\mathrm{clamda}` and :math:`\mathrm{h}` as discussed in :ref:`Parameters <e04wd-py2-py-parameters>`. :math:`\mathrm{istate}` values associated with bounds and linear constraints determine the initial working set of the procedure to find a feasible point with respect to the bounds and linear constraints. :math:`\mathrm{istate}` values associated with nonlinear constraints determine the initial working set of the first QP subproblem after such a feasible point has been found. ``nlp2_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``nlp2_solve`` is called repeatedly to solve related problems.
**'Crash Option'** : int
Default :math:`\text{} = 3`
If a 'Cold Start' is specified, an internal Crash procedure is used to select an initial basis from certain rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The option 'Crash Option' :math:`i` determines which rows and columns of :math:`A` are eligible initially, and how many times the Crash procedure is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of :math:`A`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called twice (if there are nonlinear constraints). The first call looks for a triangular basis in linear rows, and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (retaining the current basis for linear rows).|
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called up to three times (if there are nonlinear constraints). The first two calls treat `linear equalities` and `linear inequalities` separately. As before, the last call treats nonlinear rows before the first major iteration. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound).
The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' :math:`r` allows the starting Crash procedure to ignore certain 'small' nonzeros in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros of :math:`a_{{ij}}` in the columns are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` must be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.1`
If a 'Cold Start' is specified, an internal Crash procedure is used to select an initial basis from certain rows and columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`.
The option 'Crash Option' :math:`i` determines which rows and columns of :math:`A` are eligible initially, and how many times the Crash procedure is called.
Columns of :math:`{-I}` are used to pad the basis where necessary.
.. rst-class:: nag-rules-none nag-align-left
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=========================================================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`0`|The initial basis contains only slack variables: :math:`B = I`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1`|The Crash procedure is called once, looking for a triangular basis in all rows and columns of :math:`A`. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2`|The Crash procedure is called twice (if there are nonlinear constraints). The first call looks for a triangular basis in linear rows, and the iteration proceeds with simplex iterations until the linear constraints are satisfied. The Jacobian is then evaluated for the first major iteration and the Crash procedure is called again to find a triangular basis in the nonlinear rows (retaining the current basis for linear rows).|
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3`|The Crash procedure is called up to three times (if there are nonlinear constraints). The first two calls treat `linear equalities` and `linear inequalities` separately. As before, the last call treats nonlinear rows before the first major iteration. |
+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`i\geq 1`, certain slacks on inequality rows are selected for the basis first. (If :math:`i\geq 2`, numerical values are used to exclude slacks that are close to a bound).
The Crash procedure then makes several passes through the columns of :math:`A`, searching for a basis matrix that is essentially triangular.
A column is assigned to 'pivot' on a particular row if the column contains a suitably large element in a row that has not yet been assigned. (The pivot elements ultimately form the diagonals of the triangular basis.) For remaining unassigned rows, slack variables are inserted to complete the basis.
The 'Crash Tolerance' :math:`r` allows the starting Crash procedure to ignore certain 'small' nonzeros in each column of :math:`A`.
If :math:`a_{\mathrm{max}}` is the largest element in column :math:`j`, other nonzeros of :math:`a_{{ij}}` in the columns are ignored if :math:`\left\lvert a_{{ij}}\right\rvert \leq a_{\mathrm{max}}\times r`. (To be meaningful, :math:`r` must be in the range :math:`0\leq r < 1`.)
When :math:`r > 0.0`, the basis obtained by the Crash procedure may not be strictly triangular, but it is likely to be nonsingular and almost triangular.
The intention is to obtain a starting basis containing more columns of :math:`A` and fewer (arbitrary) slacks.
A feasible solution may be reached sooner on some problems.
For example, suppose the first :math:`m` columns of :math:`A` form the matrix shown under 'LU Factor Tolerance'; i.e., a tridiagonal matrix with entries :math:`-1`, :math:`4`, :math:`-1`.
To help the Crash procedure choose all :math:`m` columns for the initial basis, we would specify a 'Crash Tolerance' of :math:`r` for some value of :math:`r > 0.5`.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Derivative Level'** : int
Default :math:`\text{} = 3`
Option 'Derivative Level' specifies which nonlinear function gradients are known analytically and will be supplied to ``nlp2_solve`` by functions :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`.
.. rst-class:: nag-rules-none nag-align-left
+---------+-------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+=======================================================================================================+
|:math:`3`|All objective and constraint gradients are known. |
+---------+-------------------------------------------------------------------------------------------------------+
|:math:`2`|All constraint gradients are known, but some or all components of the objective gradient are unknown. |
+---------+-------------------------------------------------------------------------------------------------------+
|:math:`1`|The objective gradient is known, but some or all of the constraint gradients are unknown. |
+---------+-------------------------------------------------------------------------------------------------------+
|:math:`0`|Some components of the objective gradient are unknown and some of the constraint gradients are unknown.|
+---------+-------------------------------------------------------------------------------------------------------+
The value :math:`i = 3` should be used whenever possible.
It is the most reliable and will usually be the most efficient.
If :math:`i = 0` or :math:`2`, ``nlp2_solve`` will `estimate` the missing components of the objective gradient, using finite differences.
This may simplify the coding of :math:`\mathrm{objfun}`.
However, it could increase the total run-time substantially (since a special call to :math:`\mathrm{objfun}` is required for each missing element), and there is less assurance that an acceptable solution will be located.
If the nonlinear variables are not well scaled, it may be necessary to specify a non-default option 'Difference Interval'.
If :math:`i = 0` or :math:`1`, ``nlp2_solve`` will estimate missing elements of the Jacobian.
For each column of the Jacobian, one call to :math:`\mathrm{confun}` is needed to estimate all missing elements in that column, if any.
At times, central differences are used rather than forward differences. (This is not under your control.)
**'Derivative Linesearch'** : valueless
Default
At each major iteration a linesearch is used to improve the merit function. Option 'Derivative Linesearch' uses safeguarded cubic interpolation and requires both function and gradient values to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided, or option 'Nonderivative Linesearch' is specified, ``nlp2_solve`` employs a linesearch based upon safeguarded quadratic interpolation, which does not require gradient evaluations.
A nonderivative linesearch can be slightly less robust on difficult problems, and it is recommended that the default be used if the functions and derivatives can be computed at approximately the same cost.
If the gradients are very expensive relative to the functions, a nonderivative linesearch may give a significant decrease in computation time.
If 'Nonderivative Linesearch' is selected, ``nlp2_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{objfun}` with :math:`\mathrm{mode} = 0`.
If the potential saving provided by a nonderivative linesearch is to be realised, it is essential that :math:`\mathrm{objfun}` be coded so that derivatives are not computed when :math:`\mathrm{mode} = 0`.
**'Nonderivative Linesearch'** : valueless
At each major iteration a linesearch is used to improve the merit function. Option 'Derivative Linesearch' uses safeguarded cubic interpolation and requires both function and gradient values to compute estimates of the step :math:`\alpha_k`.
If some analytic derivatives are not provided, or option 'Nonderivative Linesearch' is specified, ``nlp2_solve`` employs a linesearch based upon safeguarded quadratic interpolation, which does not require gradient evaluations.
A nonderivative linesearch can be slightly less robust on difficult problems, and it is recommended that the default be used if the functions and derivatives can be computed at approximately the same cost.
If the gradients are very expensive relative to the functions, a nonderivative linesearch may give a significant decrease in computation time.
If 'Nonderivative Linesearch' is selected, ``nlp2_solve`` signals the evaluation of the linesearch by calling :math:`\mathrm{objfun}` with :math:`\mathrm{mode} = 0`.
If the potential saving provided by a nonderivative linesearch is to be realised, it is essential that :math:`\mathrm{objfun}` be coded so that derivatives are not computed when :math:`\mathrm{mode} = 0`.
**'Difference Interval'** : float
Default :math:`\text{} = \sqrt{\epsilon_r}`
This alters the interval :math:`r` used to estimate gradients by forward differences.
It does so in the following circumstances:
- in the interval ('cheap') phase of verifying the problem derivatives;
- for verifying the problem derivatives;
- for estimating missing derivatives.
In all cases, a derivative with respect to :math:`x_j` is estimated by perturbing that component of :math:`x` to the value :math:`x_j+r\left(1+\left\lvert x_j\right\rvert \right)`, and then evaluating :math:`F\left(x\right)` or :math:`c\left(x\right)` at the perturbed point.
The resulting gradient estimates should be accurate to :math:`\mathrm{O}\left(r\right)` unless the functions are badly scaled.
Judicious alteration of :math:`r` may sometimes lead to greater accuracy.
If you supply a value for this option, a small value between :math:`0.0` and :math:`1.0` is appropriate.
**'Dump File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i_1`.
If :math:`i_2 > 0`, the 'Load File', containing basis information, will be read.
The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Load File'** : int
Default :math:`\text{} = 0`
Options 'Dump File' and 'Load File' are similar to options 'Punch File' and 'Insert File', but they record solution information in a manner that is more direct and more easily modified.
A full description of information recorded in options 'Dump File' and 'Load File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the last solution obtained will be output to the file with unit number :math:`i_1`.
If :math:`i_2 > 0`, the 'Load File', containing basis information, will be read.
The file will usually have been output previously as a 'Dump File'.
The file will not be accessed if options 'Old Basis File' or 'Insert File' are specified.
**'Elastic Weight'** : float
Default :math:`\text{} = 10^4`
This keyword determines the initial weight :math:`\gamma` associated with the problem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#ad-treatment>`__).
At major iteration :math:`k`, if elastic mode has not yet started, a scale factor :math:`\sigma_k = 1+\left\lVert g\left(x_k\right)\right\rVert_\infty` is defined from the current objective gradient.
Elastic mode is then started if the QP subproblem is infeasible, or the QP dual variables are larger in magnitude than :math:`\sigma_k\text{}r`.
The QP is resolved in elastic mode with :math:`\gamma = \sigma_k\text{}r`.
Thereafter, major iterations continue in elastic mode until they converge to a point that is optimal for `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#ad-treatment>`__).
If the point is feasible for equation `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__ :math:`\left(v = w = 0\right)`, it is declared locally optimal.
Otherwise, :math:`\gamma` is increased by a factor of :math:`10` and major iterations continue.
If :math:`\gamma` has already reached a maximum allowable value, equation `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__ is declared locally infeasible.
**'Expand Frequency'** : int
Default :math:`\text{} = 10000`
This option is part of the anti-cycling procedure designed to make progress even on highly degenerate problems.
For linear models, the strategy is to force a positive step at every iteration, at the expense of violating the bounds on the variables by a small amount.
Suppose that the option 'Minor Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the tolerance actually used by ``nlp2_solve`` increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
For nonlinear models, the same procedure is used for iterations in which there is only one superbasic variable. (Cycling can occur only when the current solution is at a vertex of the feasible region.) Thus, zero steps are allowed if there is more than one superbasic variable, but otherwise positive steps are enforced.
Increasing :math:`i` helps reduce the number of slightly infeasible nonbasic variables (most of which are eliminated during a resetting procedure).
However, it also diminishes the freedom to choose a large pivot element (see option 'Pivot Tolerance').
**'Factorization Frequency'** : int
Default :math:`\text{} = 50`
At most :math:`i` basis changes will occur between factorizations of the basis matrix.
With linear programs, the basis factors are usually updated every iteration.
The default :math:`i` is reasonable for typical problems.
Higher values up to :math:`i = 100` (say) may be more efficient on well-scaled problems.
When the objective function is nonlinear, fewer basis updates will occur as an optimum is approached.
The number of iterations between basis factorizations will, therefore, increase.
During these iterations a test is made regularly (according to the option 'Check Frequency') to ensure that the general constraints are satisfied.
If necessary the basis will be refactorized before the limit of :math:`i` updates is reached.
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.8}`
The `relative function precision` :math:`\epsilon_r` is intended to be a measure of the relative accuracy with which the functions can be computed.
For example, if :math:`F\left(x\right)` is computed as :math:`1000.56789` for some relevant :math:`x` and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-6`.
(Ideally the functions :math:`F\left(x\right)` or :math:`c_i\left(x\right)` should have magnitude of order :math:`1`.
If all functions are substantially `less` than :math:`1` in magnitude, :math:`\epsilon_r` should be the `absolute` precision.
For example, if :math:`F\left(x\right) = 1.23456789e-4` at some point and if the first :math:`6` significant digits are known to be correct, the appropriate value for :math:`\epsilon_r` would be :math:`1.0e-10`.)
The default value of :math:`\epsilon_r` is appropriate for simple analytic functions.
In some cases the function values will be the result of extensive computation, possibly involving a costly iterative procedure that can provide few digits of precision.
Specifying an appropriate 'Function Precision' may lead to savings, by allowing the linesearch procedure to terminate when the difference between function values along the search direction becomes as small as the absolute error in the values.
**'Hessian Full Memory'** : valueless
Default if :math:`n\leq 75`
These options select the method for storing and updating the approximate Hessian. (``nlp2_solve`` uses a quasi-Newton approximation to the Hessian of the Lagrangian.
A BFGS update is applied after each major iteration.)
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and the BFGS updates are applied explicitly.
This option is most efficient when the number of variables :math:`n` is not too large (say, less than :math:`75`).
In this case, the storage requirement is fixed and one can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should be used on problems where :math:`n` is very large.
In this case a limited-memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
**'Hessian Limited Memory'** : valueless
Default if :math:`n > 75`
These options select the method for storing and updating the approximate Hessian. (``nlp2_solve`` uses a quasi-Newton approximation to the Hessian of the Lagrangian.
A BFGS update is applied after each major iteration.)
If 'Hessian Full Memory' is specified, the approximate Hessian is treated as a dense matrix and the BFGS updates are applied explicitly.
This option is most efficient when the number of variables :math:`n` is not too large (say, less than :math:`75`).
In this case, the storage requirement is fixed and one can expect :math:`1`-step Q-superlinear convergence to the solution.
'Hessian Limited Memory' should be used on problems where :math:`n` is very large.
In this case a limited-memory procedure is used to update a diagonal Hessian approximation :math:`H_r` a limited number of times. (Updates are accumulated as a list of vector pairs.
They are discarded at regular intervals after :math:`H_r` has been reset to their diagonal.)
**'Hessian Frequency'** : int
Default :math:`\text{} = 99999999`
If option 'Hessian Full Memory' is in effect and :math:`i` BFGS updates have already been carried out, the Hessian approximation is reset to the identity matrix. (For certain problems, occasional resets may improve convergence, but in general they should not be necessary.)
'Hessian Full Memory' and :math:`\text{‘Hessian Frequency'} = 10` have a similar effect to 'Hessian Limited Memory' and :math:`\text{‘Hessian Updates'} = 10` (except that the latter retains the current diagonal during resets).
**'Hessian Updates'** : int
Default :math:`\text{} = \text{‘Hessian Frequency'}` if 'Hessian Full Memory', :math:`10` otherwise
If option 'Hessian Limited Memory' is in effect and :math:`i` BFGS updates have already been carried out, all but the diagonal elements of the accumulated updates are discarded and the updating process starts again.
Broadly speaking, the more updates stored, the better the quality of the approximate Hessian.
However, the more vectors stored, the greater the cost of each QP iteration.
The default value is likely to give a robust algorithm without significant expense, but faster convergence can sometimes be obtained with significantly fewer updates (e.g., :math:`i = 5`).
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Iterations Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(10000, {10\mathrm{max}\left(n, {n_L+n_N}\right)}\right)`
The value of :math:`i` specifies the maximum number of minor iterations allowed (i.e., iterations of the simplex method or the QP algorithm), summed over all major iterations. (See also the description of the option 'Minor Iterations Limit'.)
**'Linesearch Tolerance'** : float
Default :math:`= 0.9`
This tolerance, :math:`r`, controls the accuracy with which a step length will be located along the direction of search each iteration.
At the start of each linesearch a target directional derivative for the merit function is identified.
This argument determines the accuracy to which this target value is approximated, and it must be a value in the range :math:`0.0\leq r\leq 1.0`.
The default value :math:`r = 0.9` requests just moderate accuracy in the linesearch.
If the nonlinear functions are cheap to evaluate, a more accurate search may be appropriate; try :math:`r = 0.1\text{, }0.01\text{ or }0.001`.
If the nonlinear functions are expensive to evaluate, a less accurate search may be appropriate. `If all gradients are known`, try :math:`r = 0.99`. (The number of major iterations might increase, but the total number of function evaluations may decrease enough to compensate.)
If not all gradients are known, a moderately accurate search remains appropriate.
Each search will require only :math:`1`--5 function values (typically), but many function calls will then be needed to estimate missing gradients for the next iteration.
**'Nolist'** : valueless
Default
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'LU Density Tolerance'** : float
Default :math:`\text{} = 0.6`
The density tolerance, :math:`r_1`, is used during :math:`LU` factorization of the basis matrix :math:`B`.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated, and the remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
The singularity tolerance, :math:`r_2`, helps guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows: if :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable. (This is most likely to occur after a restart.)
**'LU Singularity Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
The density tolerance, :math:`r_1`, is used during :math:`LU` factorization of the basis matrix :math:`B`.
Columns of :math:`L` and rows of :math:`U` are formed one at a time, and the remaining rows and columns of the basis are altered appropriately.
At any stage, if the density of the remaining matrix exceeds :math:`r_1`, the Markowitz strategy for choosing pivots is terminated, and the remaining matrix is factored by a dense :math:`LU` procedure.
Raising the density tolerance towards :math:`1.0` may give slightly sparser :math:`LU` factors, with a slight increase in factorization time.
The singularity tolerance, :math:`r_2`, helps guard against ill-conditioned basis matrices.
After :math:`B` is refactorized, the diagonal elements of :math:`U` are tested as follows: if :math:`\left\lvert u_{{jj}}\right\rvert \leq r_2` or :math:`\left\lvert u_{{jj}}\right\rvert < r_2\mathrm{max}_i\left\lvert u_{{ij}}\right\rvert`, the :math:`j`\ th column of the basis is replaced by the corresponding slack variable. (This is most likely to occur after a restart.)
**'LU Factor Tolerance'** : float
Default :math:`\text{} = 1.10`
The values of :math:`r_1` and :math:`r_2` affect the stability of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Update Tolerance'** : float
Default :math:`\text{} = 1.10`
The values of :math:`r_1` and :math:`r_2` affect the stability of the basis factorization :math:`B = LU`, during refactorization and updates respectively.
The lower triangular matrix :math:`L` is a product of matrices of the form
.. math::
\begin{pmatrix}1& \\\mu &1\end{pmatrix}
where the multipliers :math:`\mu` will satisfy :math:`\left\lvert \mu \right\rvert \leq r_i`.
The default values of :math:`r_1` and :math:`r_2` usually strike a good compromise between stability and sparsity.
They must satisfy :math:`r_1`, :math:`r_2\geq 1.0`.
For large and relatively dense problems, :math:`r_1 = 10.0` or :math:`5.0` (say) may give a useful improvement in stability without impairing sparsity to a serious degree.
For certain very regular structures (e.g., band matrices) it may be necessary to reduce :math:`r_1\text{ and/or }r_2` in order to achieve stability.
For example, if the columns of :math:`A` include a sub-matrix of the form
.. math::
\begin{pmatrix}4&-1\\-1&4&-1\\&-1&4&-1\\&&\ldots &\ldots &\ldots \\&&&-1&4&-1\\&&&&-1&4\end{pmatrix}\text{,}
one should set both :math:`r_1` and :math:`r_2` to values in the range :math:`1.0\leq r_i < 4.0`.
**'LU Partial Pivoting'** : valueless
Default
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'LU Complete Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'LU Rook Pivoting'** : valueless
The :math:`LU` factorization implements a Markowitz-type search for pivots that locally minimize the fill-in subject to a threshold pivoting stability criterion.
The default option is to use threshhold partial pivoting.
The options 'LU Rook Pivoting' and 'LU Complete Pivoting' are more expensive than partial pivoting but are more stable and better at revealing rank, as long as 'LU Factor Tolerance' is not too large (say :math:`< 2.0`).
When numerical difficulties are encountered, ``nlp2_solve`` automatically reduces the :math:`LU` tolerance towards :math:`1.0` and switches (if necessary) to rook or complete pivoting, before reverting to the default or specified options at the next refactorization (with 'System Information Yes', relevant messages are output to the 'Print File').
**'Major Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This tolerance, :math:`r`, specifies how accurately the nonlinear constraints should be satisfied.
The default value is appropriate when the linear and nonlinear constraints contain data to about that accuracy.
Let :math:`v_{\mathrm{max}}` be the maximum nonlinear constraint violation, normalized by the size of the solution, which is required to satisfy
.. math::
v_{\mathrm{max/}} = \mathrm{max}_i\textit{v}_i\text{}/\text{}\left\lVert x\right\rVert \leq r\text{,}
where :math:`\textit{v}_i` is the violation of the :math:`i`\ th nonlinear constraint :math:`\left(i = 1:n_L\right)`.
In the major iteration log (see `Minor Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#minoriteration>`__, :math:`v_{\mathrm{max}}` appears as the quantity labelled 'Feasible'.
If some of the problem functions are known to be of low accuracy, a larger 'Major Feasibility Tolerance' may be appropriate.
**'Major Optimality Tolerance'** : float
Default :math:`\text{} = 2\mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This tolerance, :math:`r`, specifies the final accuracy of the dual variables.
On successful termination, ``nlp2_solve`` will have computed a solution :math:`\left(x, s, \pi \right)` such that
.. math::
c_{\mathrm{max}} = \mathrm{max}_jc_j\text{}/\text{}\left\lVert \pi \right\rVert \leq r\text{,}
where :math:`c_j` is an estimate of the complementarity slackness for variable :math:`j` where :math:`j = 1:n+n_L+n_N`.
The values :math:`c_i` are computed from the final QP solution using the reduced gradients :math:`d_j = g_j-\pi^\mathrm{T}a_j` (where :math:`g_j` is the :math:`j`\ th component of the objective gradient, :math:`a_j` is the associated column of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}`, and :math:`\pi` is the set of QP dual variables):
.. math::
c_j = \left\{\begin{array}{cc} d_j \mathrm{min}\left({x_j-l_j}, 1\right) & \text{if } d_j\geq 0 \text{;} \\ {-d_j} \mathrm{min}\left({u_j-x_j}, 1\right) & \text{if } d_j < 0 \text{.} \end{array}\right)
In the 'Print File', :math:`c_{\mathrm{max}}` appears as the quantity labelled 'Optimal'.
**'Major Iterations Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(1000, {3\mathrm{max}\left(n, {n_L+n_N}\right)}\right)`
This is the maximum number of major iterations allowed.
It is intended to guard against an excessive number of linearizations of the constraints.
If :math:`i = 0`, optimality and feasibility are checked.
**'Major Print Level'** : int
Default :math:`\text{} = 000001`
This controls the amount of output to the options 'Print File' and 'Summary File' at each major iteration. :math:`\text{‘Major Print Level'} = 0` suppresses most output, except for error messages. :math:`\text{‘Major Print Level'} = 1` gives normal output for linear and nonlinear problems, and :math:`\text{‘Major Print Level'} = 11` gives additional details of the Jacobian factorization that commences each major iteration.
In general, the value being specified may be thought of as a binary number of the form
.. math::
\text{‘Major Print Level'}\text{ }JFDXbs
where each letter stands for a digit that is either :math:`0` or :math:`1` as follows:
.. rst-class:: nag-rules-none
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`s`|a single line that gives a summary of each major iteration. (This entry in :math:`JFDXbs` is not strictly binary since the summary line is printed whenever :math:`JFDXbs\geq 1`);|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`b`|basis statistics, i.e., information relating to the basis matrix whenever it is refactorized. (This output is always provided if :math:`JFDXbs\geq 10`); |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`X`|:math:`x_k`, the nonlinear variables involved in the objective function or the constraints. These appear under the heading 'Jacobian variables'; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`D`|:math:`\pi_k`, the dual variables for the nonlinear constraints. These appear under the heading 'Multiplier estimates'; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`F`|:math:`c\left(x_k\right)`, the values of the nonlinear constraint functions; |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`J`|:math:`J\left(x_k\right)`, the Jacobian matrix. This appears under the heading ':math:`x` and Jacobian'. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
To obtain output of any items :math:`JFDXbs`, set the corresponding digit to :math:`1`, otherwise to :math:`0`.
If :math:`J = 1`, the Jacobian matrix will be output column-wise at the start of each major iteration.
Column :math:`j` will be preceded by the value of the corresponding variable :math:`x_j` and a key to indicate whether the variable is basic, superbasic or nonbasic. (Hence if :math:`J = 1`, there is no reason to specify :math:`X = 1` unless the objective contains more nonlinear variables than the Jacobian.) A typical line of output is
::
3 1.250000e+01 BS 1 1.00000e+00 4 2.00000e+00
which would mean that :math:`x_3` is basic at value :math:`12.5`, and the third column of the Jacobian has elements of :math:`1.0` and :math:`2.0` in rows :math:`1` and :math:`4`.
**'Major Step Limit'** : float
Default :math:`\text{} = 2.0`
This argument limits the change in :math:`x` during a linesearch.
It applies to all nonlinear problems, once a 'feasible solution' or 'feasible subproblem' has been found.
A linesearch determines a step :math:`\alpha` over the range :math:`0 < \alpha \leq \beta`, where :math:`\beta` is :math:`1` if there are nonlinear constraints, or is the step to the nearest upper or lower bound on :math:`x` if all the constraints are linear.
Normally, the first step length tried is :math:`\alpha_1 = \mathrm{min}\left(1, \beta \right)`.
(1) In some cases, such as :math:`f\left(x\right) = ae^{{bx}}` or :math:`f\left(x\right) = ax^b`, even a moderate change in the components of :math:`x` can lead to floating-point overflow. The argument :math:`r` is, therefore, used to define a limit :math:`\bar{\beta } = r\left(1+\left\lVert x\right\rVert \right)/\left\lVert p\right\rVert` (where :math:`p` is the search direction), and the first evaluation of :math:`f\left(x\right)` is at the potentially smaller step length :math:`\alpha_1 = \mathrm{min}\left(1, \bar{\beta }, \beta \right)`.
(#) Wherever possible, upper and lower bounds on :math:`x` should be used to prevent evaluation of nonlinear functions at meaningless points. The option 'Major Step Limit' provides an additional safeguard. The default value :math:`r = 2.0` should not affect progress on well behaved problems, but setting :math:`r = 0.1\text{ or }0.01` may be helpful when rapidly varying functions are present. A 'good' starting point may be required. An important application is to the class of nonlinear least squares problems.
(#) In cases where several local optima exist, specifying a small value for :math:`r` may help locate an optimum near the starting point.
**'Minimize'** : valueless
Default
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_solve``.
**'Maximize'** : valueless
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_solve``.
**'Feasible Point'** : valueless
The keywords 'Minimize' and 'Maximize' specify the required direction of optimization.
It applies to both linear and nonlinear terms in the objective.
The keyword 'Feasible Point' means 'Ignore the objective function, while finding a feasible point for the linear and nonlinear constraints'.
It can be used to check that the nonlinear constraints are feasible without altering the call to ``nlp2_solve``.
**'Minor Feasibility Tolerance'** : valueless
``nlp2_solve`` tries to ensure that all variables eventually satisfy their upper and lower bounds to within this tolerance, :math:`r`.
This includes slack variables.
Hence, general linear constraints should also be satisfied to within :math:`r`.
Feasibility with respect to nonlinear constraints is judged by the option 'Major Feasibility Tolerance' (not by :math:`r`).
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
If the corresponding sum of infeasibilities is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Nonlinear functions will be evaluated only at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every attempt should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both variables.
If :math:`r = 1.0e-6`, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious.
In general, keep :math:`x` as far away from singularities as possible.)
If :math:`\text{‘Scale Option'}\geq 1`, feasibility is defined in terms of the `scaled` problem (since it is then more likely to be meaningful).
In reality, ``nlp2_solve`` uses :math:`r` as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible. ``nlp2_solve`` is then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic).
See the description of the option 'Elastic Weight'.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \mathrm{max}\left\{10^{-6}, \sqrt{\epsilon }\right\}`
``nlp2_solve`` tries to ensure that all variables eventually satisfy their upper and lower bounds to within this tolerance, :math:`r`.
This includes slack variables.
Hence, general linear constraints should also be satisfied to within :math:`r`.
Feasibility with respect to nonlinear constraints is judged by the option 'Major Feasibility Tolerance' (not by :math:`r`).
If the bounds and linear constraints cannot be satisfied to within :math:`r`, the problem is declared `infeasible`.
If the corresponding sum of infeasibilities is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Nonlinear functions will be evaluated only at points that satisfy the bounds and linear constraints.
If there are regions where a function is undefined, every attempt should be made to eliminate these regions from the problem.
For example, if :math:`f\left(x\right) = \sqrt{x_1}+\log\left(x_2\right)`, it is essential to place lower bounds on both variables.
If :math:`r = 1.0e-6`, the bounds :math:`x_1\geq 10^{-5}` and :math:`x_2\geq 10^{-4}` might be appropriate. (The log singularity is more serious.
In general, keep :math:`x` as far away from singularities as possible.)
If :math:`\text{‘Scale Option'}\geq 1`, feasibility is defined in terms of the `scaled` problem (since it is then more likely to be meaningful).
In reality, ``nlp2_solve`` uses :math:`r` as a feasibility tolerance for satisfying the bounds on :math:`x` and :math:`s` in each QP subproblem.
If the sum of infeasibilities cannot be reduced to zero, the QP subproblem is declared infeasible. ``nlp2_solve`` is then in `elastic mode` thereafter (with only the linearized nonlinear constraints defined to be elastic).
See the description of the option 'Elastic Weight'.
**'Minor Iterations Limit'** : int
Default :math:`\text{} = 500`
If the number of minor iterations for the optimality phase of the QP subproblem exceeds :math:`i`, then all nonbasic QP variables that have not yet moved are frozen at their current values and the reduced QP is solved to optimality.
Note that more than :math:`i` minor iterations may be necessary to solve the reduced QP to optimality.
These extra iterations are necessary to ensure that the terminated point gives a suitable direction for the linesearch.
In the major iteration log (see `Minor Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#minoriteration>`__) a t at the end of a line indicates that the corresponding QP was artificially terminated using the limit :math:`i`.
Compare with the option 'Iterations Limit', which defines an independent `absolute` limit on the `total` number of minor iterations (summed over all QP subproblems).
**'Minor Print Level'** : int
Default :math:`\text{} = 1`
This controls the amount of output to the 'Print File' and the 'Summary File' during solution of the QP subproblems.
The value of :math:`i` has the following effect:
.. rst-class:: nag-rules-none nag-align-left
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===============+=====================================================================================================================================================================================================================================================+
|:math:`0` |No minor iteration output except error messages. |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 1` |A single line of output at each minor iteration (controlled by options 'Print Frequency' and 'Summary Frequency'. |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\geq 10`|Basis factorization statistics generated during the periodic refactorization of the basis (see the option 'Factorization Frequency'). Statistics for the `first factorization` each major iteration are controlled by the option 'Major Print Level'.|
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'New Basis File'** : int
Default :math:`\text{} = 0`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified. The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first on the file associated with unit :math:`12` and then immediately on the file associated with unit :math:`11`. If the run is interrupted at iteration :math:`2000` during the save on the file associated with unit :math:`12`, there will still be a usable basis on the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Backup Basis File'** : int
Default :math:`\text{} = 0`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified. The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first on the file associated with unit :math:`12` and then immediately on the file associated with unit :math:`11`. If the run is interrupted at iteration :math:`2000` during the save on the file associated with unit :math:`12`, there will still be a usable basis on the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'Save Frequency'** : int
Default :math:`\text{} = 100`
'New Basis File' and 'Backup Basis File' are sometimes referred to as basis maps.
They contain the most compact representation of the state of each variable.
They are intended for restarting the solution of a problem at a point that was reached by an earlier run.
For nontrivial problems, it is advisable to save basis maps at the end of a run, in order to restart the run if necessary.
If :math:`i_1 > 0`, a basis map will be saved on the file associated with unit :math:`i_1` every :math:`i_3`\ th iteration.
The first record of the file will contain the word ``PROCEEDING`` if the run is still in progress.
A basis map will also be saved at the end of a run, with some other word indicating the final solution status.
Use of :math:`i_2 > 0` is intended as a safeguard against losing the results of a long run.
Suppose that a 'New Basis File' is being saved every :math:`100` ('Save Frequency') iterations, and that ``nlp2_solve`` is about to save such a basis at iteration :math:`2000`.
It is conceivable that the run may be interrupted during the next few milliseconds (in the middle of the save).
In this case the Basis file will be corrupted and the run will have been essentially wasted.
To eliminate this risk, both a 'New Basis File' and a 'Backup Basis File' may be specified. The following would be suitable for the above example:
::
Backup Basis File 11
New Basis File 12
The current basis will then be saved every :math:`100` iterations, first on the file associated with unit :math:`12` and then immediately on the file associated with unit :math:`11`. If the run is interrupted at iteration :math:`2000` during the save on the file associated with unit :math:`12`, there will still be a usable basis on the file associated with unit :math:`11` (corresponding to iteration :math:`1900`).
Note that a new basis will be saved in 'New Basis File' at the end of a run if it terminates normally, but it will not be saved in 'Backup Basis File'.
In the above example, if an optimum solution is found at iteration :math:`2050` (or if the iteration limit is :math:`2050`), the final basis in the file associated with unit :math:`12` will correspond to iteration :math:`2050`, but the last basis saved in the file associated with unit :math:`11` will be the one for iteration :math:`2000`.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
**'New Superbasics Limit'** : int
Default :math:`\text{} = 99`
This option causes early termination of the QP subproblems if the number of free variables has increased significantly since the first feasible point.
If the number of new superbasics is greater than :math:`i`, the nonbasic variables that have not yet moved are frozen and the resulting smaller QP is solved to optimality.
In the major iteration log (see `Major Iteration Log <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#majoriteration>`__), a t at the end of a line indicates that the QP was terminated early in this way.
**'Old Basis File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the basis maps information will be obtained from this file. The file will usually have been output previously as a 'New Basis File' or 'Backup Basis File'.
A full description of information recorded in 'New Basis File' and 'Backup Basis File' is given in Gill `et al.` (2005a).
The file will not be acceptable if the number of rows or columns in the problem has been altered.
**'Partial Price'** : int
Default :math:`\text{} = 1`
This argument is recommended for large problems that have significantly more variables than constraints.
It reduces the work required for each 'pricing' operation (where a nonbasic variable is selected to become superbasic).
When :math:`i = 1`, all columns of the constraint matrix :math:`\begin{pmatrix}A&-I\end{pmatrix}` are searched.
Otherwise, :math:`A` and :math:`I` are partitioned to give :math:`i` roughly equal segments :math:`A_{\textit{j}}` and :math:`I_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,i`.
If the previous pricing search was successful on :math:`A_{{j-1}}` and :math:`I_{{j-1}}`, the next search begins on the segments :math:`A_j` and :math:`I_j`. (All subscripts here are modulo :math:`i`.) If a reduced gradient is found that is larger than some dynamic tolerance, the variable with the largest such reduced gradient (of appropriate sign) is selected to become superbasic.
If nothing is found, the search continues on the next segments :math:`A_{{j+1}}` and :math:`I_{{j+1}}`, and so on.
For time-stage models having :math:`t` time periods, 'Partial Price' :math:`t` (or :math:`t/2` or :math:`t/3`) may be appropriate.
**'Pivot Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{2}{3}}`
During the solution of QP subproblems, the pivot tolerance is used to prevent columns entering the basis if they would cause the basis to become almost singular.
When :math:`x` changes to :math:`x+\alpha p` for some search direction :math:`p`, a 'ratio test' determines which component of :math:`x` reaches an upper or lower bound first.
The corresponding element of :math:`p` is called the pivot element.
Elements of :math:`p` are ignored (and, therefore, cannot be pivot elements) if they are smaller than the pivot tolerance :math:`r`.
It is common for two or more variables to reach a bound at essentially the same time.
In such cases, the 'Minor Feasibility Tolerance' (say, :math:`t`) provides some freedom to maximize the pivot element and thereby improve numerical stability.
Excessively small values of :math:`t` should, therefore, not be specified.
To a lesser extent, the 'Expand Frequency' (say, :math:`f`) also provides some freedom to maximize the pivot element.
Excessively `large` values of :math:`f` should, therefore, not be specified.
**'Print File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the following information is output to a file associated with unit :math:`i` during the solution of each problem:
- a listing of the options;
- some statistics about the problem;
- the amount of storage available for the :math:`LU` factorization of the basis matrix;
- notes about the initial basis resulting from a Crash procedure or a Basis file;
- the iteration log;
- basis factorization statistics;
- the exit :math:`\textit{errno}` condition and some statistics about the solution obtained;
- the printed solution, if requested.
These items are described in `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#fcomments>`__ and `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#fcomments>`__.
Further brief output may be directed to the 'Summary File'.
**'Print Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i > 0`, one line of the iteration log will be printed every :math:`i`\ th iteration.
A value such as :math:`i = 10` is suggested for those interested only in the final solution.
If :math:`i\leq 0`, the value of :math:`i = 99999999` is used and effectively no checks are made.
**'Proximal Point Method'** : int
Default :math:`\text{} = 1`
:math:`i = 1\text{ or }2` specifies minimization of :math:`\left\lVert x-x_0\right\rVert_1` or :math:`\frac{1}{2}\left\lVert x-x_0\right\rVert_2^2` when the starting point :math:`x_0` is changed to satisfy the linear constraints (where :math:`x_0` refers to nonlinear variables).
**'Punch File'** : int
Default :math:`= 0`
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to the file. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0` the 'Insert File' containing basis information will be read from unit :math:`i_2`. The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'Insert File'** : int
Default :math:`= 0`
The 'Punch File' from a previous run may be used as an 'Insert File' for a later run on the same problem.
A full description of information recorded in 'Insert File' and 'Punch File' is given in Gill `et al.` (2005a).
If :math:`i_1 > 0`, the final solution obtained will be output to the file. For linear programs, this format is compatible with various commercial systems.
If :math:`i_2 > 0` the 'Insert File' containing basis information will be read from unit :math:`i_2`. The file will usually have been output previously as a 'Punch File'.
The file will not be accessed if 'Old Basis File' is specified.
**'QPSolver Cholesky'** : valueless
Default
Specifies the active-set algorithm used to solve subproblem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#ad-treatment>`__). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of the subproblem in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__.
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``nlp2_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'QPSolver CG'** : valueless
Specifies the active-set algorithm used to solve subproblem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#ad-treatment>`__). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of the subproblem in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__.
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``nlp2_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'QPSolver QN'** : valueless
Specifies the active-set algorithm used to solve subproblem `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__ (see `Treatment of Constraint Infeasibilities <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#ad-treatment>`__). 'QPSolver Cholesky' holds the full Cholesky factor :math:`R` of the reduced Hessian :math:`Z^\mathrm{T}HZ`.
As the QP iterations proceed, the dimension of :math:`R` changes with the number of superbasic variables.
If the number of superbasic variables needs to increase beyond the value of 'Reduced Hessian Dimension', the reduced Hessian cannot be stored and the solver switches to 'QPSolver CG'.
The Cholesky solver is reactivated if the number of superbasics stabilizes at a value less than 'Reduced Hessian Dimension'.
'QPSolver QN' solves the QP using a quasi-Newton method.
In this case, :math:`R` is the factor of a quasi-Newton approximate Hessian.
'QPSolver CG' uses an active-set method similar to 'QPSolver QN', but uses the conjugate-gradient method to solve all systems involving the reduced Hessian.
The Cholesky QP solver is the most robust, but may require a significant amount of computation if there are many superbasics.
The quasi-Newton QP solver does not require computation of the exact :math:`R` at the start of the subproblem in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn11>`__.
It may be appropriate when the number of superbasics is large but relatively few iterations are needed to reach a solution (e.g., if ``nlp2_solve`` is called with a Warm Start).
The conjugate-gradient QP solver is appropriate for problems with many degrees of freedom (say, more than :math:`2000` superbasics).
**'Reduced Hessian Dimension'** : int
Default :math:`= \mathrm{min}\left(2000, n\right)`
This specifies that an :math:`i\times i` triangular matrix :math:`R` (to define the reduced Hessian according to :math:`R^\mathrm{T}R = Z^\mathrm{T}HZ`) is to be available for use by the Cholesky QP solver.
**'Scale Option'** : int
Default :math:`\text{} = 0`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Tolerance'** : float
Default :math:`\text{} = 0.9`
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Scale Print'** : valueless
Three scale options are available as follows:
.. rst-class:: nag-rules-none nag-align-left
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i`|Meaning |
+=========+================================================================================================================================================================================================================================================================================================================================================================+
|0 |No scaling. This is recommended if it is known that :math:`x` and the constraint matrix never have very large elements (say, larger than :math:`100`). |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraints and variables are scaled by an iterative procedure that attempts to make the matrix coefficients as close as possible to :math:`1.0` (see Fourer (1982)). This will sometimes improve the performance of the solution procedures. |
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraints and variables are scaled by the iterative procedure. Also, a certain additional scaling is performed that may be helpful if the right-hand side :math:`b` or the solution :math:`x` is large. This takes into account columns of :math:`\begin{pmatrix}A&-I\end{pmatrix}` that are fixed or have positive lower bounds or negative upper bounds.|
+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Option 'Scale Tolerance' affects how many passes might be needed through the constraint matrix.
On each pass, the scaling procedure computes the ratio of the largest and smallest nonzero coefficients in each column:
.. math::
\rho_j = \mathrm{max}_j\left\lvert a_{{ij}}\right\rvert /\mathrm{min}_i\left\lvert a_{{ij}}\right\rvert \quad \text{ }\quad \left(a_{{ij}}\neq 0\right)\text{.}
If :math:`\mathrm{max}_j\rho_j` is less than :math:`r` times its previous value, another scaling pass is performed to adjust the row and column scales.
Raising :math:`r` from :math:`0.9` to :math:`0.99` (say) usually increases the number of scaling passes through :math:`A`.
At most :math:`10` passes are made.
The value of :math:`r` should lie in the range :math:`0 < r < 1`.
'Scale Print' causes the row scales :math:`r\left(i\right)` and column scales :math:`c\left(j\right)` to be printed to 'Print File', if 'System Information Yes' has been specified.
The scaled matrix coefficients are :math:`\bar{a}_{{ij}} = a_{{ij}}c\left(j\right)/r\left(i\right)`, and the scaled bounds on the variables and slacks are :math:`\bar{l}_j = l_j/c\left(j\right)`, :math:`\bar{u}_j = u_j/c\left(j\right)`, where :math:`c\left(j\right) = r\left(j-n\right)` if :math:`j > n`.
**'Solution File'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, the final solution will be output to file :math:`i` (whether optimal or not).
All numbers are printed in ``1pe16.6`` format.
To see more significant digits in the printed solution, it will sometimes be useful to make :math:`i` refer to 'Print File'.
**'Start Objective Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to contol the verification of gradient elements computed by function :math:`\mathrm{objfun}` and/or Jacobian elements computed by function :math:`\mathrm{confun}`.
For eample, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
**'Stop Objective Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to contol the verification of gradient elements computed by function :math:`\mathrm{objfun}` and/or Jacobian elements computed by function :math:`\mathrm{confun}`.
For eample, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
**'Start Constraint Check At Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to contol the verification of gradient elements computed by function :math:`\mathrm{objfun}` and/or Jacobian elements computed by function :math:`\mathrm{confun}`.
For eample, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
**'Stop Constraint Check At Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to contol the verification of gradient elements computed by function :math:`\mathrm{objfun}` and/or Jacobian elements computed by function :math:`\mathrm{confun}`.
For eample, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check At Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
**'Summary File'** : int
Default :math:`\text{} = 0`
If :math:`i_1 > 0`, a brief log will be output to the file associated with unit :math:`i_1`, including one line of information every :math:`i_2`\ th iteration.
In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `The Summary File <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#monitoring-summaryfile>`__.
**'Summary Frequency'** : int
Default :math:`\text{} = 100`
If :math:`i_1 > 0`, a brief log will be output to the file associated with unit :math:`i_1`, including one line of information every :math:`i_2`\ th iteration.
In an interactive environment, it is useful to direct this output to the terminal, to allow a run to be monitored online. (If something looks wrong, the run can be manually terminated.) Further details are given in `The Summary File <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#monitoring-summaryfile>`__.
**'Superbasics Limit'** : int
Default :math:`\text{} = n`
This option places a limit on the storage allocated for superbasic variables.
Ideally, :math:`i` should be set slightly larger than the 'number of degrees of freedom' expected at an optimal solution.
For nonlinear problems, the number of degrees of freedom is often called the 'number of independent variables'.
Normally, :math:`i` need not be greater than :math:`n_N+1`, where :math:`n_N` is the number of nonlinear variables.
For many problems, :math:`i` may be considerably smaller than :math:`n_N`.
This will save storage if :math:`n_N` is very large.
**'Suppress Parameters'** : valueless
Normally ``nlp2_solve`` prints the options file as it is being read, and then prints a complete list of the available keywords and their final values.
The option 'Suppress Parameters' tells ``nlp2_solve`` not to print the full list.
**'System Information No'** : valueless
Default
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#monitoring>`__.
**'System Information Yes'** : valueless
This option prints additional information on the progress of major and minor iterations, and Crash statistics.
See `Monitoring Information <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#monitoring>`__.
**'Timing Level'** : int
Default :math:`\text{} = 0`
If :math:`i > 0`, some timing information will be output to the Print file, if :math:`\text{‘Print File'} > 0`.
**'Unbounded Objective'** : float
Default :math:`\text{} = 1.0e+15`
These arguments are intended to detect unboundedness in nonlinear problems.
During a linesearch, :math:`F` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert F\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, iterations are terminated with the exit message :math:`\mathrm{errno}` = 5.
If singularities are present, unboundedness in :math:`F\left(x\right)` may be manifested by a floating-point overflow (during the evaluation of :math:`F\left(x+\alpha p\right)`), before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
**'Unbounded Step Size'** : float
Default :math:`\text{} = \textit{bigbnd}`
These arguments are intended to detect unboundedness in nonlinear problems.
During a linesearch, :math:`F` is evaluated at points of the form :math:`x+\alpha p`, where :math:`x` and :math:`p` are fixed and :math:`\alpha` varies.
If :math:`\left\lvert F\right\rvert` exceeds :math:`r_1` or :math:`\alpha` exceeds :math:`r_2`, iterations are terminated with the exit message :math:`\mathrm{errno}` = 5.
If singularities are present, unboundedness in :math:`F\left(x\right)` may be manifested by a floating-point overflow (during the evaluation of :math:`F\left(x+\alpha p\right)`), before the test against :math:`r_1` can be made.
Unboundedness in :math:`x` is best avoided by placing finite upper and lower bounds on the variables.
**'Verify Level'** : int
Default :math:`\text{} = 0`
This option refers to finite difference checks on the derivatives computed by the user-supplied functions.
Derivatives are checked at the first point that satisfies all bounds and linear constraints.
.. rst-class:: nag-rules-none nag-align-left
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+=============================================================================================================================================================+
|:math:`0` |Only a 'cheap' test will be performed, requiring two calls to :math:`\mathrm{confun}`. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Individual gradients will be checked (with a more reliable test). A key of the form OK or Bad? indicates whether or not each component appears to be correct.|
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Individual columns of the problem Jacobian will be checked. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Options 2 and 1 will both occur (in that order). |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1`|Derivative checking is disabled. |
+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
:math:`\text{‘Verify Level'} = 3` should be specified whenever a new function function is being developed.
Missing derivatives are not checked, so they result in no overhead.
**'Violation Limit'** : float
Default :math:`\text{} = 1.0e+6`
This keyword defines an absolute limit on the magnitude of the maximum constraint violation, :math:`r`, after the linesearch.
On completion of the linesearch, the new iterate :math:`x_{{k+1}}` satisfies the condition
.. math::
v_i\left(x_{{k+1}}\right)\leq r\text{ }\mathrm{max}\left(1, {v_i\left(x_0\right)}\right)\text{,}
where :math:`x_0` is the point at which the nonlinear constraints are first evaluated and :math:`v_i\left(x\right)` is the :math:`i`\ th nonlinear constraint violation :math:`v_i\left(x\right) = \mathrm{max}\left(0, {l_i-c_i\left(x\right)}, {c_i\left(x\right)-u_i}\right)`.
The effect of this violation limit is to restrict the iterates to lie in an `expanded` feasible region whose size depends on the magnitude of :math:`r`.
This makes it possible to keep the iterates within a region where the objective is expected to be well defined and bounded below.
If the obective is bounded below for all values of the variables, :math:`r` may be any large positive value.
.. _e04wd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{bl}` and :math:`\mathrm{bu}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{bl} = \mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds :math:`\mathrm{bl}` and :math:`\mathrm{bu}` for :math:`\langle\mathit{\boldsymbol{value}}\rangle`:math:`\langle\mathit{\boldsymbol{value}}\rangle` are equal and infinite. :math:`\mathrm{bl} = \mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, bounds for :math:`\langle\mathit{\boldsymbol{value}}\rangle`:math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent. :math:`\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
Basis file dimensions do not match this problem.
(`errno` :math:`2`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
On entry, :math:`\textit{ncnln} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{ncnln}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`8`)
User-supplied function computes incorrect objective derivatives.
(`errno` :math:`8`)
User-supplied function computes incorrect constraint derivatives.
(`errno` :math:`11`)
Internal error: memory allocation failed when attempting to allocate workspace sizes :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\langle\mathit{\boldsymbol{value}}\rangle`. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`12`)
Internal memory allocation was insufficient. Please contact `NAG <https://www.nag.com>`__.
(`errno` :math:`13`)
An error has occurred in the basis package, perhaps indicating incorrect setup of arrays. Set the option 'Print File' and examine the output carefully for further information.
(`errno` :math:`14`)
An unexpected error has occurred. Set the option 'Print File' and examine the output carefully for further information.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The requested accuracy could not be achieved.
(`errno` :math:`4`)
The linear constraints appear to be infeasible.
(`errno` :math:`4`)
The problem appears to be infeasible. The linear equality constraints could not be satisfied.
(`errno` :math:`4`)
The problem appears to be infeasible. Nonlinear infeasibilites have been minimized.
(`errno` :math:`4`)
The problem appears to be infeasible. Infeasibilites have been minimized.
(`errno` :math:`5`)
The problem appears to be unbounded. The objective function is unbounded.
(`errno` :math:`5`)
The problem appears to be unbounded. The constraint violation limit has been reached.
(`errno` :math:`7`)
Numerical difficulties have been encountered and no further progress can be made.
(`errno` :math:`10`)
User-supplied constraint function requested termination.
(`errno` :math:`10`)
User-supplied objective function requested termination.
**NagAlgorithmicMajorWarning**
(`errno` :math:`6`)
Iteration limit reached.
(`errno` :math:`6`)
Major iteration limit reached.
(`errno` :math:`6`)
The value of the option 'Superbasics Limit' is too small.
(`errno` :math:`9`)
User-supplied function is undefined at the first feasible point.
(`errno` :math:`9`)
User-supplied function is undefined at the initial point.
(`errno` :math:`9`)
Unable to proceed into undefined region of user-supplied function.
.. _e04wd-py2-py-notes:
**Notes**
``nlp2_solve`` is designed to solve nonlinear programming problems -- the minimization of a smooth nonlinear function subject to a set of constraints on the variables. ``nlp2_solve`` is suitable for small dense problems.
The problem is assumed to be stated in the following form:
.. math::
\textit{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad l\leq \begin{pmatrix}x\\A_Lx\\c\left(x\right)\end{pmatrix}\leq u\text{,}
where :math:`F\left(x\right)` (the `objective function`) is a nonlinear scalar function, :math:`A_L` is an :math:`n_L\times n` constant matrix, and :math:`c\left(x\right)` is an :math:`n_N`-vector of nonlinear constraint functions. (The matrix :math:`A_L` and the vector :math:`c\left(x\right)` may be empty.) The objective function and the constraint functions are assumed to be smooth, here meaning at least twice-continuously differentiable. (The method of ``nlp2_solve`` will usually solve `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__ if there are only isolated discontinuities away from the solution.) We also write :math:`r\left(x\right)` for the vector of combined functions:
.. math::
r\left(x\right) = \begin{pmatrix}x&A_Lx&c\left(x\right)\end{pmatrix}^\mathrm{T}\text{.}
Note that although the bounds on the variables could be included in the definition of the linear constraints, we prefer to distinguish between them for reasons of computational efficiency.
For the same reason, the linear constraints should **not** be included in the definition of the nonlinear constraints.
Upper and lower bounds are specified for all the variables and for all the constraints.
An `equality` constraint on :math:`r_i` can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
Figure [label omitted] illustrates the feasible region for the :math:`j`\ th pair of constraints :math:`l_j\leq r_j\left(x\right)\leq u_j`.
The quantity of :math:`\delta` is the 'Feasibility Tolerance', which can be set by you (see :ref:`Other Parameters <e04wd-py2-py-other_params>`).
The constraints :math:`l_j\leq r_j\leq u_j` are considered 'satisfied' if :math:`r_j` lies in Regions 2, 3 or 4, and 'inactive' if :math:`r_j` lies in Region 3.
The constraint :math:`r_j\geq l_j` is considered 'active' in Region 2, and 'violated' in Region 1.
Similarly, :math:`r_j\leq u_j` is active in Region 4, and violated in Region 5.
For equality constraints (:math:`l_j = u_j`), Regions 2 and 4 are the same and Region 3 is empty.
[figure omitted]
If there are no nonlinear constraints in `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__ and :math:`F` is linear or quadratic, then it will generally be more efficient to use one of :meth:`lp_solve`, :meth:`lsq_lincon_solve` or :meth:`qp_dense_solve`.
If the problem is large and sparse and does have nonlinear constraints, then :meth:`handle_solve_ssqp` should be used, since ``nlp2_solve`` treats all matrices as dense.
You must supply an initial estimate of the solution to `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#eqn1>`__, together with functions that define :math:`F\left(x\right)` and :math:`c\left(x\right)` with as many first partial derivatives as possible; unspecified derivatives are approximated by finite differences; see :ref:`Other Parameters <e04wd-py2-py-other_params>` for a discussion of the option 'Derivative Level'.
The objective function is defined by :math:`\mathrm{objfun}`, and the nonlinear constraints are defined by :math:`\mathrm{confun}`.
Note that if there `are` any nonlinear constraints then the `first` call to :math:`\mathrm{confun}` will precede the `first` call to :math:`\mathrm{objfun}`.
For maximum reliability, it is preferable for you to provide all partial derivatives (see Module 8 of Gill `et al.` (1981), for a detailed discussion).
If all gradients cannot be provided, it is similarly advisable to provide as many as possible.
While developing :math:`\mathrm{objfun}` and :math:`\mathrm{confun}`, the option 'Verify Level' should be used to check the calculation of any known gradients.
The method used by ``nlp2_solve`` is based on NPOPT, which is part of the SNOPT package described in Gill `et al.` (2005b), and the algorithm it uses is described in detail in `Algorithmic Details <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wdf.html#algdetails>`__.
.. _e04wd-py2-py-references:
**References**
Eldersveld, S K, 1991, `Large-scale sequential quadratic programming algorithms`, PhD Thesis, Department of Operations Research, Stanford University, Stanford
Fourer, R, 1982, `Solving staircase linear programs by the simplex method`, Math. Programming (23), 274--313
Gill, P E, Murray, W and Saunders, M A, 2002, `SNOPT: An SQP Algorithm for Large-scale Constrained Optimization` (12), 979--1006, SIAM J. Optim.
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SQOPT 7: a Fortran package for large-scale linear and quadratic programming`, Report NA 05-1, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sqdoc7.pdf
Gill, P E, Murray, W and Saunders, M A, 2005, `Users' guide for SNOPT 7.1: a Fortran package for large-scale linear nonlinear programming`, Report NA 05-2, Department of Mathematics, University of California, San Diego, https://www.ccom.ucsd.edu/~peg/papers/sndoc7.pdf
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming`, Report SOL 86-2, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1992, `Some theoretical properties of an augmented Lagrangian merit function`, Advances in Optimization and Parallel Computing, (ed P M Pardalos), 101--128, North Holland
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
See Also
--------
:meth:`naginterfaces.library.examples.opt.nlp2_solve_ex.main`
"""
raise NotImplementedError
[docs]def nlp2_option_file(ispecs, comm, io_manager=None):
r"""
``nlp2_option_file`` may be used to supply options to :meth:`nlp2_solve` from an external file.
The initialization function :meth:`nlp2_init` **must** have been called before calling ``nlp2_option_file``.
.. _e04we-py2-py-doc:
For full information please refer to the NAG Library document for e04we
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wef.html
.. _e04we-py2-py-parameters:
**Parameters**
**ispecs** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the option file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04we-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
At least one line of the options file is invalid.
(`errno` :math:`2`)
Could not read options file on unit :math:`\mathrm{ispecs} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04we-py2-py-notes:
**Notes**
``nlp2_option_file`` may be used to supply values for options to :meth:`nlp2_solve`. ``nlp2_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Option settings are preserved following a call to :meth:`nlp2_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
.. _e04we-py2-py-references:
**References**
Hock, W and Schittkowski, K, 1981, `Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems` (187), Springer--Verlag
"""
raise NotImplementedError
[docs]def nlp2_option_string(string, comm, io_manager=None):
r"""
``nlp2_option_string`` may be used to supply individual options to :meth:`nlp2_solve`.
The initialization function :meth:`nlp2_init` **must** have been called before calling ``nlp2_option_string``.
.. _e04wf-py2-py-doc:
For full information please refer to the NAG Library document for e04wf
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wff.html
.. _e04wf-py2-py-parameters:
**Parameters**
**string** : str
A single valid option string (see :ref:`Notes <e04wf-py2-py-notes>` in ``nlp2_option_string`` and :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04wf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04wf-py2-py-notes:
**Notes**
``nlp2_option_string`` may be used to supply values for options to :meth:`nlp2_solve`.
It is only necessary to call ``nlp2_option_string`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`nlp2_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_option_integer_set(string, ivalue, comm, io_manager=None):
r"""
``nlp2_option_integer_set`` may be used to supply individual `int` options to :meth:`nlp2_solve`.
The initialization function :meth:`nlp2_init` **must** have been called before calling ``nlp2_option_integer_set``.
.. _e04wg-py2-py-doc:
For full information please refer to the NAG Library document for e04wg
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wgf.html
.. _e04wg-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`).
**ivalue** : int
An `int` value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04wg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{ivalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04wg-py2-py-notes:
**Notes**
``nlp2_option_integer_set`` may be used to supply values for `int` options to :meth:`nlp2_solve`.
It is only necessary to call ``nlp2_option_integer_set`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_option_integer_set`` sets one argument value.
Each `int` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{ivalue}`.
Option settings are preserved following a call to :meth:`nlp2_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_option_double_set(string, rvalue, comm, io_manager=None):
r"""
``nlp2_option_double_set`` may be used to supply individual `float` options to :meth:`nlp2_solve`.
The initialization function :meth:`nlp2_init` **must** have been called before calling ``nlp2_option_double_set``.
.. _e04wh-py2-py-doc:
For full information please refer to the NAG Library document for e04wh
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04whf.html
.. _e04wh-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`).
**rvalue** : float
The value associated with the keyword in :math:`\mathrm{string}`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04wh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
The supplied option is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{rvalue} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04wh-py2-py-notes:
**Notes**
``nlp2_option_double_set`` may be used to supply values for `float` options to :meth:`nlp2_solve`.
It is only necessary to call ``nlp2_option_double_set`` for those arguments whose values are to be different from their default values.
One call to ``nlp2_option_double_set`` sets one argument value.
Each `float` option is defined by a single character string in :math:`\mathrm{string}` and the corresponding value in :math:`\mathrm{rvalue}`.
Option settings are preserved following a call to :meth:`nlp2_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`nlp2_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_option_integer_get(string, comm, io_manager=None):
r"""
``nlp2_option_integer_get`` is used to get the value of an `int` option. ``nlp2_option_integer_get`` can be used before or after calling :meth:`nlp2_solve`.
.. _e04wk-py2-py-doc:
For full information please refer to the NAG Library document for e04wk
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wkf.html
.. _e04wk-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of an `int` option (as described in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**ivalue** : int
The `int` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04wk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
The supplied option string is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04wk-py2-py-notes:
**Notes**
``nlp2_option_integer_get`` obtains the current value of an `int` option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def nlp2_option_double_get(string, comm, io_manager=None):
r"""
``nlp2_option_double_get`` is used to get the value of a `float` option. ``nlp2_option_double_get`` can be used before or after calling :meth:`nlp2_solve`.
.. _e04wl-py2-py-doc:
For full information please refer to the NAG Library document for e04wl
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04wlf.html
.. _e04wl-py2-py-parameters:
**Parameters**
**string** : str
A single valid keyword of a `float` option (as described in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp2_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**rvalue** : float
The `float` value associated with the keyword in :math:`\mathrm{string}`.
.. _e04wl-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
The initialization function :meth:`nlp2_init` has not been called.
(`errno` :math:`2`)
The supplied option string is invalid. Check that the keywords are neither ambiguous nor misspelt. The option string is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04wl-py2-py-notes:
**Notes**
``nlp2_option_double_get`` obtains the current value of a `float` option.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for nlp2_solve <e04wd-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def estimate_deriv(msglvl, epsrf, x, mode, objfun, hforw, data=None, io_manager=None):
r"""
``estimate_deriv`` computes an approximation to the gradient vector and/or the Hessian matrix for use in conjunction with, or following the use of an optimization function (such as :meth:`nlp1_rcomm`).
.. _e04xa-py2-py-doc:
For full information please refer to the NAG Library document for e04xa
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04xaf.html
.. _e04xa-py2-py-parameters:
**Parameters**
**msglvl** : int
Must indicate the amount of intermediate output desired (see `Further Comments <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04xaf.html#fc-printedoutput>`__ for a description of the printed output). All output is written on the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`).
.. rst-class:: nag-rules-none nag-align-left
+-----+---------------------------------------------------------------------------------------------------+
|Value|Definition |
+=====+===================================================================================================+
|0 |No printout |
+-----+---------------------------------------------------------------------------------------------------+
|1 |A summary is printed out for each variable plus any warning messages. |
+-----+---------------------------------------------------------------------------------------------------+
|Other|Values other than :math:`0` and :math:`1` should normally be used **only** at the direction of NAG.|
+-----+---------------------------------------------------------------------------------------------------+
**epsrf** : float
Must define :math:`e_R`, which is intended to be a measure of the accuracy with which the problem function :math:`F` can be computed. The value of :math:`e_R` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`, i.e., acts as a relative precision when :math:`\left\lvert F\right\rvert` is large, and as an absolute precision when :math:`\left\lvert F\right\rvert` is small. For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`e_R` would be :math:`1.0e-6`.
A discussion of :math:`\mathrm{epsrf}` is given in Module 8 of Gill `et al.` (1981).
If :math:`\mathrm{epsrf}` is either too small or too large on entry a warning will be printed if :math:`\mathrm{msglvl} = 1`, the argument :math:`\mathrm{iwarn}` set to the appropriate value on exit and ``estimate_deriv`` will use a default value of :math:`e_M^{0.9}`, where :math:`e_M` is the machine precision.
If :math:`\mathrm{epsrf}\leq 0.0` on entry, then ``estimate_deriv`` will use the default value internally.
The default value will be appropriate for most simple functions that are computed with full accuracy.
**x** : float, array-like, shape :math:`\left(n\right)`
The point :math:`x` at which the derivatives are to be computed.
**mode** : int
Indicates which derivatives are required.
:math:`\mathrm{mode} = 0`
The gradient and Hessian diagonal values having supplied the objective function via :math:`\mathrm{objfun}`.
:math:`\mathrm{mode} = 1`
The Hessian matrix having supplied both the objective function and gradients via :math:`\mathrm{objfun}`.
:math:`\mathrm{mode} = 2`
The gradient values and Hessian matrix having supplied the objective function via :math:`\mathrm{objfun}`.
**objfun** : callable (objf, objgrd) = objfun(mode, x, nstate, data=None)
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{objfun}` must calculate the objective function; otherwise, if :math:`\mathrm{mode} = 1`, :math:`\mathrm{objfun}` must calculate the objective function and the gradients.
**Parameters**
**mode** : int
:math:`\mathrm{mode}` indicates which argument values within :math:`\mathrm{objfun}` need to be set.
To :math:`\mathrm{objfun}`, :math:`\mathrm{mode}` is always set to the value that you set it to before the call to ``estimate_deriv``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the objective function (and gradients if :math:`\mathrm{mode} = 1`) is to be evaluated.
**nstate** : int
Will be set to :math:`1` on the first call of :math:`\mathrm{objfun}` by ``estimate_deriv``, and is :math:`0` for all subsequent calls. Thus, if you wish, :math:`\mathrm{nstate}` may be tested within :math:`\mathrm{objfun}` in order to perform certain calculations once only. For example you may read data.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**objf** : float
Must be set to the value of the objective function.
**objgrd** : float, array-like, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 1`, :math:`\mathrm{objgrd}[j-1]` must contain the value of the first derivative with respect to :math:`x`.
If :math:`\mathrm{mode}\neq 1`, :math:`\mathrm{objgrd}` need not be set.
**hforw** : float, array-like, shape :math:`\left(n\right)`
The initial trial interval for computing the appropriate partial derivative to the :math:`j`\ th variable.
If :math:`\mathrm{hforw}[j-1]\leq 0.0`, the initial trial interval is computed by ``estimate_deriv`` (see :ref:`Notes <e04xa-py2-py-notes>`).
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**mode** : int
Is changed **only** if you set :math:`\mathrm{mode}` negative in :math:`\mathrm{objfun}`, i.e., you have requested termination of ``estimate_deriv``.
**hforw** : float, ndarray, shape :math:`\left(n\right)`
:math:`\mathrm{hforw}[j-1]` is the best interval found for computing a forward-difference approximation to the appropriate partial derivative for the :math:`j`\ th variable.
**objf** : float
The value of the objective function evaluated at the input vector in :math:`\mathrm{x}`.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 0` or :math:`2`, :math:`\mathrm{objgrd}[j-1]` contains the best estimate of the first partial derivative for the :math:`j`\ th variable.
If :math:`\mathrm{mode} = 1`, :math:`\mathrm{objgrd}[j-1]` contains the first partial derivative for the :math:`j`\ th variable evaluated at the input vector in :math:`\mathrm{x}`.
**hcntrl** : float, ndarray, shape :math:`\left(n\right)`
:math:`\mathrm{hcntrl}[j-1]` is the best interval found for computing a central-difference approximation to the appropriate partial derivative for the :math:`j`\ th variable.
**h** : float, ndarray, shape :math:`\left(n, :\right)`
If :math:`\mathrm{mode} = 0`, the estimated Hessian diagonal elements are contained in the first column of this array.
If :math:`\mathrm{mode} = 1` or :math:`2`, the estimated Hessian matrix is contained in the leading :math:`n\times n` part of this array.
**iwarn** : int
:math:`\mathrm{iwarn} = 0` on successful exit.
If the value of :math:`\mathrm{epsrf}` on entry is too small or too large then :math:`\mathrm{iwarn}` is set to :math:`1` or :math:`2` respectively on exit and the default value for :math:`\mathrm{epsrf}` is used within ``estimate_deriv``.
If :math:`\mathrm{msglvl} > 0` then warnings will be printed if :math:`\mathrm{epsrf}` is too small or too large.
**info** : int, ndarray, shape :math:`\left(n\right)`
:math:`\mathrm{info}[j-1]` represents diagnostic information on variable :math:`j` as follows:
:math:`\mathrm{info}[j-1] = 1`
The appropriate function appears to be constant. :math:`\mathrm{hforw}[i-1]` is set to the initial trial interval value (see :ref:`Notes <e04xa-py2-py-notes>`) corresponding to a well-scaled problem and Error est. in the printed output is set to zero. This value occurs when the estimated relative condition error in the first derivative approximation is unacceptably large for every value of the finite difference interval. If this happens when the function is not constant the initial interval may be too small; in this case, it may be worthwhile to rerun ``estimate_deriv`` with larger initial trial interval values supplied in :math:`\mathrm{hforw}` (see :ref:`Notes <e04xa-py2-py-notes>`). This error may also occur if the function evaluation includes an inordinately large constant term or if :math:`\mathrm{epsrf}` is too large.
:math:`\mathrm{info}[j-1] = 2`
The appropriate function appears to be linear or odd. :math:`\mathrm{hforw}[i-1]` is set to the smallest interval with acceptable bounds on the relative condition error in the forward - and backward-difference estimates. In this case, the estimated relative condition error in the second derivative approximation remained large for every trial interval, but the estimated error in the first derivative approximation was acceptable for at least one interval. If the function is not linear or odd the relative condition error in the second derivative may be decreasing very slowly, it may be worthwhile to rerun ``estimate_deriv`` with larger initial trial interval values supplied in :math:`\mathrm{hforw}` (see :ref:`Notes <e04xa-py2-py-notes>`).
:math:`\mathrm{info}[j-1] = 3`
The second derivative of the appropriate function appears to be so large that it cannot be reliably estimated (i.e., near a singularity). :math:`\mathrm{hforw}[i-1]` is set to the smallest trial interval.
This value occurs when the relative condition error estimate in the second derivative remained very small for every trial interval.
If the second derivative is not large the relative condition error in the second derivative may be increasing very slowly.
It may be worthwhile to rerun ``estimate_deriv`` with smaller initial trial interval values supplied in :math:`\mathrm{hforw}` (see :ref:`Notes <e04xa-py2-py-notes>`).
This error may also occur when the given value of :math:`\mathrm{epsrf}` is not a good estimate of a bound on the absolute error in the appropriate function (i.e., :math:`\mathrm{epsrf}` is too small).
:math:`\mathrm{info}[j-1] = 4`
The algorithm terminated with an apparently acceptable estimate of the second derivative. However the forward-difference estimates of the appropriate first derivatives (computed with the final estimate of the 'optimal' forward-difference interval) and the central difference estimates (computed with the interval used to compute the final estimate of the second derivative) do not agree to half a decimal place. The usual reason that the forward - and central-difference estimates fail to agree is that the first derivative is small.
If the first derivative is not small, it may be helpful to execute the procedure at a different point.
.. _e04xa-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{mode} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{mode}\leq 2`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
One or more variables have a nonzero :math:`\mathrm{info}` value.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{mode}` negative in :math:`\mathrm{objfun}`.
.. _e04xa-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``estimate_deriv`` is similar to routine FDCALC described in Gill `et al.` (1983a).
It should be noted that this function aims to compute sufficiently accurate estimates of the derivatives for use with an optimization algorithm.
If you require more accurate estimates you should refer to submodule :mod:`~naginterfaces.library.numdiff`.
``estimate_deriv`` computes finite difference approximations to the gradient vector and the Hessian matrix for a given function.
The simplest approximation involves the forward-difference formula, in which the derivative :math:`f^{\prime }\left(x\right)` of a univariate function :math:`f\left(x\right)` is approximated by the quantity
.. math::
\rho_F\left(f, h\right) = \frac{{f\left(x+h\right)-f\left(x\right)}}{h}
for some interval :math:`h > 0`, where the subscript 'F' denotes 'forward-difference' (see Gill `et al.` (1983b)).
To summarise the procedure used by ``estimate_deriv`` (for the case when the objective function is available and you require estimates of gradient values and Hessian matrix diagonal values, i.e., :math:`\mathrm{mode} = 0`) consider a univariate function :math:`f` at the point :math:`x`. (In order to obtain the gradient of a multivariate function :math:`F\left(x\right)`, where :math:`x` is an :math:`n`-vector, the procedure is applied to each component of :math:`x`, keeping the other components fixed.) Roughly speaking, the method is based on the fact that the bound on the relative truncation error in the forward-difference approximation tends to be an increasing function of :math:`h`, while the relative condition error bound is generally a decreasing function of :math:`h`, hence changes in :math:`h` will tend to have opposite effects on these errors (see Gill `et al.` (1983b)).
The 'best' interval :math:`h` is given by
.. math::
h_F = 2\sqrt{\frac{{\left(1+\left\lvert f\left(x\right)\right\rvert \right)e_R}}{\left\lvert \Phi \right\rvert }}
where :math:`\Phi` is an estimate of :math:`f^{{\prime \prime }}\left(x\right)`, and :math:`e_R` is an estimate of the relative error associated with computing the function (see Module 8 of Gill `et al.` (1981)).
Given an interval :math:`h`, :math:`\Phi` is defined by the second-order approximation
.. math::
\Phi = \frac{{f\left(x+h\right)-2f\left(x\right)+f\left(x-h\right)}}{h^2}\text{.}
The decision as to whether a given value of :math:`\Phi` is acceptable involves :math:`\hat{c}\left(\Phi \right)`, the following bound on the relative condition error in :math:`\Phi`:
.. math::
\hat{c}\left(\Phi \right) = \frac{{4e_R\left(1+\left\lvert f\right\rvert \right)}}{{h^2\left\lvert \Phi \right\rvert }}
(When :math:`\Phi` is zero, :math:`\hat{c}\left(\Phi \right)` is taken as an arbitrary large number.)
The procedure selects the interval :math:`h_{\phi }` (to be used in computing :math:`\Phi`) from a sequence of trial intervals :math:`\left(h_k\right)`.
The initial trial interval is taken as :math:`10\bar{h}`, where
.. math::
\bar{h} = 2\left(1+\left\lvert x\right\rvert \right)\sqrt{e_R}
unless you specify the initial value to be used.
The value of :math:`\hat{c}\left(\Phi \right)` for a trial value :math:`h_k` is defined as 'acceptable' if it lies in the interval :math:`\left[0.001, 0.1\right]`.
In this case :math:`h_{\phi }` is taken as :math:`h_k`, and the current value of :math:`\Phi` is used to compute :math:`h_F` from `(1) <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04xaf.html#eqn1>`__.
If :math:`\hat{c}\left(\Phi \right)` is unacceptable, the next trial interval is chosen so that the relative condition error bound will either decrease or increase, as required.
If the bound on the relative condition error is too large, a larger interval is used as the next trial value in an attempt to reduce the condition error bound.
On the other hand, if the relative condition error bound is too small, :math:`h_k` is reduced.
The procedure will fail to produce an acceptable value of :math:`\hat{c}\left(\Phi \right)` in two situations.
Firstly, if :math:`f^{{\prime \prime }}\left(x\right)` is extremely small, then :math:`\hat{c}\left(\Phi \right)` may never become small, even for a very large value of the interval.
Alternatively, :math:`\hat{c}\left(\Phi \right)` may never exceed :math:`0.001`, even for a very small value of the interval.
This usually implies that :math:`f^{{\prime \prime }}\left(x\right)` is extremely large, and occurs most often near a singularity.
As a check on the validity of the estimated first derivative, the procedure provides a comparison of the forward-difference approximation computed with :math:`h_F` (as above) and the central-difference approximation computed with :math:`h_{\phi }`.
Using the central-difference formula the first derivative can be approximated by
.. math::
\rho_c\left(f, h\right) = \frac{{f\left(x+h\right)-f\left(x-h\right)}}{{2h}}
where :math:`h > 0`.
If the values :math:`h_F` and :math:`h_{\phi }` do not display some agreement, neither can be considered reliable.
When both function and gradients are available and you require the Hessian matrix (i.e., :math:`\mathrm{mode} = 1`) ``estimate_deriv`` follows a similar procedure to the case above with the exception that the gradient function :math:`g\left(x\right)` is substituted for the objective function and so the forward-difference interval for the first derivative of :math:`g\left(x\right)` with respect to variable :math:`x_j` is computed.
The :math:`j`\ th column of the approximate Hessian matrix is then defined as in Module 2 of Gill `et al.` (1981), by
.. math::
\frac{{g\left(x+h_je_j\right)-g\left(x\right)}}{h_j}
where :math:`h_j` is the best forward-difference interval associated with the :math:`j`\ th component of :math:`g` and :math:`e_j` is the vector with unity in the :math:`j`\ th position and zeros elsewhere.
When only the objective function is available and you require the gradients and Hessian matrix (i.e., :math:`\mathrm{mode} = 2`) ``estimate_deriv`` again follows the same procedure as the case for :math:`\mathrm{mode} = 0` except that this time the value of :math:`\hat{c}\left(\Phi \right)` for a trial value :math:`h_k` is defined as acceptable if it lies in the interval :math:`\left[0.0001, 0.01\right]` and the initial trial interval is taken as
.. math::
\bar{h} = 2\left(1+\left\lvert x\right\rvert \right)\sqrt[4]{e_R}\text{.}
The approximate Hessian matrix :math:`G` is then defined as in Module 2 of Gill `et al.` (1981), by
.. math::
G_{{ij}}\left(x\right) = \frac{1}{{h_ih_j}}\left(f\left(x+h_ie_i+h_je_j\right)-f\left(x+h_ie_i\right)-f\left(x+h_je_j\right)+f\left(x\right)\right)\text{.}
.. _e04xa-py2-py-references:
**References**
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1983, `Documentation for FDCALC and FDCORE`, Technical Report SOL, 83--6, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1983, `Computing forward-difference intervals for numerical optimization`, SIAM J. Sci. Statist. Comput. (4), 310--321
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
"""
raise NotImplementedError
[docs]def lsq_check_deriv(m, lsqfun, x, data=None, spiked_sorder='C'):
r"""
``lsq_check_deriv`` checks that a user-supplied function for evaluating a vector of functions and the matrix of their first derivatives produces derivative values which are consistent with the function values calculated.
.. _e04ya-py2-py-doc:
For full information please refer to the NAG Library document for e04ya
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04yaf.html
.. _e04ya-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (The minimization functions mentioned in :ref:`Notes <e04ya-py2-py-notes>` give you the option of resetting an argument to terminate immediately. ``lsq_check_deriv`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**m** : int
The numbers :math:`m` of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the point at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you reset :math:`\mathrm{iflag}` to some negative number in :math:`\mathrm{lsqfun}` and return control to ``lsq_check_deriv``, the function will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must be set to the coordinates of a suitable point at which to check the derivatives calculated by :math:`\mathrm{lsqfun}`. 'Obvious' settings, such as :math:`0` or :math:`1`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors can go undetected. For a similar reason, it is preferable that no two elements of :math:`\mathrm{x}` should have the same value.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{lsqfun}`, :math:`\mathrm{fvec}[\textit{i}-1]` contains the value of :math:`f_{\textit{i}}` at the point supplied by you in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{lsqfun}`, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{lsqfun}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
.. _e04ya-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`2`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsqfun}`.
.. _e04ya-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
Functions for minimizing a sum of squares of :math:`m` nonlinear functions (or 'residuals'), :math:`f_{\textit{i}}\left(x_1, x_2, \ldots, x_n\right)`, for :math:`\textit{i} = 1,2,\ldots,m` and :math:`m\geq n`, may require you to supply a function to evaluate the :math:`f_i` and their first derivatives. ``lsq_check_deriv`` checks the derivatives calculated by such functions, e.g., functions of the form required for :meth:`lsq_uncon_quasi_deriv_comp`, :meth:`lsq_uncon_mod_deriv_comp` and :meth:`lsq_uncon_mod_deriv2_comp`.
As well as the function to be checked (:math:`\mathrm{lsqfun}`), you must supply a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the check will be made. ``lsq_check_deriv`` is essentially identical to CHKLSJ in the NPL Algorithms Library.
``lsq_check_deriv`` first calls :math:`\mathrm{lsqfun}` to evaluate the :math:`f_i\left(x\right)` and their first derivatives, and uses these to calculate the sum of squares :math:`F\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2`, and its first derivatives :math:`g_j = \left. \frac{{\partial F}}{{\partial x_j}}\right|_x`, for :math:`j = 1,2,\ldots,n`.
The components of :math:`g` along two orthogonal directions (defined by unit vectors :math:`p_1` and :math:`p_2`, say) are then calculated; these will be :math:`g^\mathrm{T}p_1` and :math:`g^\mathrm{T}p_2` respectively.
The same components are also estimated by finite differences, giving quantities
.. math::
v_k = \frac{{F\left(x+hp_k\right)-F\left(x\right)}}{h}\text{, }\quad k = 1,2
where :math:`h` is a small positive scalar.
If the relative difference between :math:`v_1` and :math:`g^\mathrm{T}p_1` or between :math:`v_2` and :math:`g^\mathrm{T}p_2` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def lsq_check_hessian(m, lsqfun, lsqhes, x, lb, data=None, spiked_sorder='C'):
r"""
``lsq_check_hessian`` checks that a user-supplied function for evaluating the second derivative term of the Hessian matrix of a sum of squares is consistent with a user-supplied function for calculating the corresponding first derivatives.
.. _e04yb-py2-py-doc:
For full information please refer to the NAG Library document for e04yb
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ybf.html
.. _e04yb-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (:meth:`lsq_uncon_mod_deriv2_comp` gives you the option of resetting arguments of :math:`\mathrm{lsqfun}` to cause the minimization process to terminate immediately. ``lsq_check_hessian`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**m** : int
The numbers :math:`m` of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you reset :math:`\mathrm{iflag}` to some negative number in :math:`\mathrm{lsqfun}` and return control to ``lsq_check_hessian``, the function will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**lsqhes** : callable (iflag, b) = lsqhes(iflag, fvec, xc, lb, data=None)
:math:`\mathrm{lsqhes}` must calculate the elements of the symmetric matrix
.. math::
B\left(x\right) = \sum_{{i = 1}}^mf_i\left(x\right)G_i\left(x\right)\text{,}
at any point :math:`x`, where :math:`G_i\left(x\right)` is the Hessian matrix of :math:`f_i\left(x\right)`. (As with :math:`\mathrm{lsqfun}`, an argument can be set to cause immediate termination.)
**Parameters**
**iflag** : int
Is set to a non-negative number.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The value of the residual :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`, so that the values of the :math:`f_{\textit{i}}` can be used in the calculation of the elements of :math:`\mathrm{b}`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the elements of :math:`\mathrm{b}` are to be evaluated.
**lb** : int
Gives the length of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{lsqhes}` resets :math:`\mathrm{iflag}` to some negative number, ``lsq_check_hessian`` will terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**b** : float, array-like, shape :math:`\left(\mathrm{lb}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number :math:`\mathrm{b}` must contain the lower triangle of the matrix :math:`B\left(x\right)`, evaluated at the point in :math:`\mathrm{xc}`, stored by rows. (The upper triangle is not needed because the matrix is symmetric.) More precisely, :math:`\mathrm{b}[\textit{j}\left(\textit{j}-1\right)/2+\textit{k}-1]` must contain :math:`\sum_{{\textit{i} = 1}}^mf_{\textit{i}}\frac{{\partial^2f_{\textit{i}}}}{{\partial x_{\textit{j}}\partial x_{\textit{k}}}}` evaluated at the point :math:`x`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must be set to the coordinates of a suitable point at which to check the :math:`b_{{jk}}` calculated by :math:`\mathrm{lsqhes}`. 'Obvious' settings, such as :math:`0` or :math:`1`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. For a similar reason, it is preferable that no two elements of :math:`\mathrm{x}` should have the same value.
**lb** : int
The dimension of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{lsqfun}`, :math:`\mathrm{fvec}[\textit{i}-1]` contains the value of :math:`f_{\textit{i}}` at the point supplied by you in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{lsqfun}`, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{lsqfun}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**b** : float, ndarray, shape :math:`\left(\mathrm{lb}\right)`
Unless you set :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqhes}`, :math:`\mathrm{b}[\textit{j}\times \left(\textit{j}-1\right)/2+\textit{k}-1]` contains the value of :math:`b_{{\textit{j}\textit{k}}}` at the point given in :math:`\mathrm{x}` as calculated by :math:`\mathrm{lsqhes}`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
.. _e04yb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lb} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\left(n+1\right)\times n/2 = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lb}\geq \left(n+1\right)\times n/2`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}` or :math:`\mathrm{lsqhes}`.
(`errno` :math:`2`)
It is very likely that you have made an error in setting up :math:`\mathrm{b}` in :math:`\mathrm{lsqhes}`.
.. _e04yb-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
Functions for minimizing a sum of squares of :math:`m` nonlinear functions (or 'residuals'), :math:`f_{\textit{i}}\left(x_1, x_2, \ldots, x_n\right)`, for :math:`\textit{i} = 1,2,\ldots,m` and :math:`m\geq n`, may require you to supply a function to evaluate the quantities
.. math::
b_{{jk}} = \sum_{{i = 1}}^mf_i\frac{{\partial^2f_i}}{{\partial x_j\partial x_k}}
for :math:`j = 1,2,\ldots,n` and :math:`k = 1,2,\ldots,j`. ``lsq_check_hessian`` is designed to check the :math:`b_{{jk}}` calculated by such functions.
As well as the function to be checked (:math:`\mathrm{lsqhes}`), you must supply a function (:math:`\mathrm{lsqfun}`) to evaluate the :math:`f_i` and their first derivatives, and a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the checks will be made.
Note that ``lsq_check_hessian`` checks functions of the form required by :meth:`lsq_uncon_mod_deriv2_comp`. ``lsq_check_hessian`` is essentially identical to CHKLSH in the NPL Algorithms Library.
``lsq_check_hessian`` first calls functions :math:`\mathrm{lsqfun}` and :math:`\mathrm{lsqhes}` to evaluate the first derivatives and the :math:`b_{{jk}}` at :math:`x`.
Let :math:`J` denote the :math:`m\times n` matrix of first derivatives of the residuals.
The Hessian matrix of the sum of squares,
.. math::
G = J^\mathrm{T}J+B\text{,}
is calculated and projected onto two orthogonal vectors :math:`y` and :math:`z` to give the scalars :math:`y^\mathrm{T}Gy` and :math:`z^\mathrm{T}Gz` respectively.
The same projections of the Hessian matrix are also estimated by finite differences, giving
.. math::
\begin{array}{c}p = \left(y^\mathrm{T}g\left(x+hy\right)-y^\mathrm{T}g\left(x\right)\right)/h\quad \text{ and}\\q = \left(z^\mathrm{T}g\left(x+hz\right)-z^\mathrm{T}g\left(x\right)\right)/h\end{array}
respectively, where :math:`g\left(\right)` denotes the gradient vector of the sum of squares at the point in brackets and :math:`h` is a small positive scalar.
If the relative difference between :math:`p` and :math:`y^\mathrm{T}Gy` or between :math:`q` and :math:`z^\mathrm{T}Gz` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def lsq_uncon_covariance(job, m, fsumsq, s, v):
r"""
``lsq_uncon_covariance`` returns estimates of elements of the variance-covariance matrix of the estimated regression coefficients for a nonlinear least squares problem.
The estimates are derived from the Jacobian of the function :math:`f\left(x\right)` at the solution.
This function may be used following any one of the nonlinear least squares functions :meth:`lsq_uncon_mod_func_comp`, :meth:`lsq_uncon_quasi_deriv_comp`, :meth:`lsq_uncon_mod_deriv_comp` or :meth:`lsq_uncon_mod_deriv2_comp`.
.. _e04yc-py2-py-doc:
For full information please refer to the NAG Library document for e04yc
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04ycf.html
.. _e04yc-py2-py-parameters:
**Parameters**
**job** : int
Which elements of :math:`C` are returned as follows:
:math:`\mathrm{job} = -1`
The :math:`n\times n` symmetric matrix :math:`C` is returned.
:math:`\mathrm{job} = 0`
The diagonal elements of :math:`C` are returned.
:math:`\mathrm{job} > 0`
The elements of column :math:`\mathrm{job}` of :math:`C` are returned.
**m** : int
The number :math:`m` of observations (residuals :math:`f_i\left(x\right)`).
**fsumsq** : float
The sum of squares of the residuals, :math:`F\left(\bar{x}\right)`, at the solution :math:`\bar{x}`, as returned by the nonlinear least squares function.
**s** : float, array-like, shape :math:`\left(n\right)`
The :math:`n` singular values of the Jacobian as returned by the nonlinear least squares function. See :ref:`Notes <e04yc-py2-py-notes>` for information on supplying :math:`\mathrm{s}` following one of the easy-to-use functions.
**v** : float, array-like, shape :math:`\left(n, n\right)`
The :math:`n\times n` right-hand orthogonal matrix (the right singular vectors) of :math:`J` as returned by the nonlinear least squares function. See :ref:`Notes <e04yc-py2-py-notes>` for information on supplying :math:`\mathrm{v}` following one of the easy-to-use functions.
**Returns**
**v** : float, ndarray, shape :math:`\left(n, n\right)`
If :math:`\mathrm{job}\geq 0`, :math:`\mathrm{v}` is unchanged.
If :math:`\mathrm{job} = -1`, the leading :math:`n\times n` part of :math:`\mathrm{v}` is overwritten by the :math:`n\times n` matrix :math:`C`.
When ``lsq_uncon_covariance`` is called with :math:`\mathrm{job} = -1` following an easy-to-use function this means that :math:`C` is returned, column by column, in the :math:`n^2` elements of :math:`\textit{w}` given by :math:`{\textit{w}}[\textit{NV}-1],{\textit{w}}[\textit{NV}],\ldots,{\textit{w}}[\textit{NV}+n^2-2]`. (See :ref:`Notes <e04yc-py2-py-notes>` for the definition of :math:`\textit{NV}`.)
**cj** : float, ndarray, shape :math:`\left(n\right)`
If :math:`\mathrm{job} = 0`, :math:`\mathrm{cj}` returns the :math:`n` diagonal elements of :math:`C`.
If :math:`\mathrm{job} = j > 0`, :math:`\mathrm{cj}` returns the :math:`n` elements of the :math:`j`\ th column of :math:`C`.
If :math:`\mathrm{job} = -1`, :math:`\mathrm{cj}` is not referenced.
.. _e04yc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{job} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{job}\geq -1`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{job} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{job}\leq n`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{fsumsq} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{fsumsq}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{job} = -1`, :math:`\textit{ldv} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constaint: if :math:`\mathrm{job} = -1`, :math:`\textit{ldv}\geq n`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
The singular values are all zero, so that at the solution the Jacobian matrix has rank :math:`0`.
(`errno` :math:`i > 2`)
At the solution the Jacobian matrix contains linear, or near linear, dependencies amongst its columns. The required elements of :math:`C` have still been computed based upon :math:`J` having an assumed rank :math:`\textit{errno}-2 = \langle\mathit{\boldsymbol{value}}\rangle`. The rank is computed by regarding as zero singular values :math:`SV\left(j\right)` that are not larger than :math:`10\times \epsilon \times SV\left(1\right)`, where :math:`\epsilon` is the machine precision (see :meth:`machine.precision <naginterfaces.library.machine.precision>`).
.. _e04yc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_uncon_covariance`` is intended for use when the nonlinear least squares function, :math:`F\left(x\right) = f^\mathrm{T}\left(x\right)f\left(x\right)`, represents the goodness-of-fit of a nonlinear model to observed data.
The function assumes that the Hessian of :math:`F\left(x\right)`, at the solution, can be adequately approximated by :math:`2J^\mathrm{T}J`, where :math:`J` is the Jacobian of :math:`f\left(x\right)` at the solution.
The estimated variance-covariance matrix :math:`C` is then given by
.. math::
C = \sigma^2\left(J^\mathrm{T}J\right)^{-1}\text{, }\quad J^\mathrm{T}J\quad \text{ nonsingular,}
where :math:`\sigma^2` is the estimated variance of the residual at the solution, :math:`\bar{x}`, given by
.. math::
\sigma^2 = \frac{{F\left(\bar{x}\right)}}{{m-n}}\text{,}
:math:`m` being the number of observations and :math:`n` the number of variables.
The diagonal elements of :math:`C` are estimates of the variances of the estimated regression coefficients.
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html>`__, Bard (1974) and Wolberg (1967) for further information on the use of :math:`C`.
When :math:`J^\mathrm{T}J` is singular then :math:`C` is taken to be
.. math::
C = \sigma^2\left(J^\mathrm{T}J\right)^†\text{,}
where :math:`\left(J^\mathrm{T}J\right)^†` is the pseudo-inverse of :math:`J^\mathrm{T}J`, and
.. math::
\sigma^2 = \frac{{F\left(\bar{x}\right)}}{{m-k}}\text{, }\quad k = \text{rank }\left(J\right)
but in this case the argument :math:`\textit{errno}` is returned as nonzero as a warning to you that :math:`J` has linear dependencies in its columns.
The assumed rank of :math:`J` can be obtained from :math:`\textit{errno}`.
The function can be used to find either the diagonal elements of :math:`C`, or the elements of the :math:`j`\ th column of :math:`C`, or the whole of :math:`C`.
.. _e04yc-py2-py-references:
**References**
Bard, Y, 1974, `Nonlinear Parameter Estimation`, Academic Press
Wolberg, J R, 1967, `Prediction Analysis`, Van Nostrand
"""
raise NotImplementedError
[docs]def handle_opt_set(handle, optstr):
r"""
``handle_opt_set`` is an option setting function for all solvers from the NAG optimization modelling suite.
It can set a single option or reset all of them to their default.
.. _e04zm-py2-py-doc:
For full information please refer to the NAG Library document for e04zm
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04zmf.html
.. _e04zm-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**optstr** : str
A string identifying the option and its value to be set.
Defaults
Resets all options to their default values.
:math:`\textit{Option} = \textit{optval}`
See the documentation of the particular solver for details of valid values for :math:`\textit{option}` and :math:`\textit{optval}`. The equals sign (:math:`=`) delimiter must be used to separate the :math:`\textit{option}` from its :math:`\textit{optval}` value.
:math:`\textit{Option} = \mathbf{Default}`
Resets the given option back to its default value.
:math:`\mathrm{optstr}` is case insensitive.
Each token in the :math:`\textit{option}` and :math:`\textit{optval}` component must be separated by at least one space.
.. _e04zm-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The options cannot be modified while solver is running.
(`errno` :math:`11`)
On entry, the :math:`\textit{option}` supplied in :math:`\mathrm{optstr}` was not recognized: :math:`\mathrm{optstr} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`12`)
On entry, the expected delimiter ':math:`=`' was not found in :math:`\mathrm{optstr}`: :math:`\mathrm{optstr} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`13`)
On entry, could not convert the specified :math:`\textit{optval}` to an integer: :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`13`)
On entry, could not convert the specified :math:`\textit{optval}` to a real: :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`15`)
On entry, the :math:`\textit{optval}` supplied for the integer option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`15`)
On entry, the :math:`\textit{optval}` supplied for the integer option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval} > \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`15`)
On entry, the :math:`\textit{optval}` supplied for the integer option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval}\leq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`15`)
On entry, the :math:`\textit{optval}` supplied for the integer option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval} < \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`16`)
On entry, the :math:`\textit{optval}` supplied for the real option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`16`)
On entry, the :math:`\textit{optval}` supplied for the real option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval} > \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`16`)
On entry, the :math:`\textit{optval}` supplied for the real option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval}\leq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`16`)
On entry, the :math:`\textit{optval}` supplied for the real option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{optval} < \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`17`)
On entry, the :math:`\textit{optval}` supplied for the character option is not valid.
:math:`\textit{option} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\textit{optval} = \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04zm-py2-py-notes:
**Notes**
``handle_opt_set`` can only be called on handles which have been correctly initialized (e.g., by :meth:`handle_init`) and not during the call to the solver.
It has two purposes: to reset all options to their default values; or to set a single option to a user-supplied value.
Options and their values are, in general, presented as a character string, :math:`\mathrm{optstr}`, of the form ':math:`\textit{option} = \textit{optval}`'; alphabetic characters can be supplied in either upper or lower case.
Both :math:`\textit{option}` and :math:`\textit{optval}` may consist of one or more tokens separated by white space.
The tokens that comprise :math:`\textit{optval}` will normally be either an integer, real or character value as defined in the description of the specific option.
In addition all options can take an :math:`\textit{optval}` DEFAULT which resets the option to its default value.
Information relating to available option names and their corresponding valid values is given in the documentation of the particular solver.
See also `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
See Also
--------
:meth:`naginterfaces.library.examples.glopt.handle_solve_mcs_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_milp_ex.main`
:meth:`naginterfaces.library.examples.mip.handle_solve_minlp_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_ipopt_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
(and others)
"""
raise NotImplementedError
[docs]def handle_opt_get(handle, optstr):
r"""
``handle_opt_get`` is an option getting function for all solvers from the NAG optimization modelling suite.
It is used to query the value of options.
.. _e04zn-py2-py-doc:
For full information please refer to the NAG Library document for e04zn
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04znf.html
.. _e04zn-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**optstr** : str
A string identifying the option whose current value is required. See the documentation of the particular solver for information on valid options.
**Returns**
**optvalue** : dict
The option-value ``dict``, with the following keys:
``'value'`` : float, int or str
The value of the requested option.
``'annotation'`` : None or str
Possible additional information about the option value.
.. _e04zn-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`11`)
On entry, the :math:`\textit{option}` supplied in :math:`\mathrm{optstr}` was not recognized: :math:`\mathrm{optstr} = \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`41`)
On entry, :math:`\mathrm{optstr}` indicates a character option, but :math:`\textit{cvalue}` is too short to hold the stored value. The returned value will be truncated.
.. _e04zn-py2-py-notes:
**Notes**
``handle_opt_get`` is used to query the current values of options.
It can be especially useful to retrieve the options left for automatic choice by the solver.
Information relating to available option names is given in the documentation of the particular solver.
See also `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
[docs]def handle_opt_set_file(handle, infile, io_manager=None):
r"""
``handle_opt_set_file`` is an option setting function for all solvers from the NAG optimization modelling suite.
It can set one or more options from an external file.
.. _e04zp-py2-py-doc:
For full information please refer to the NAG Library document for e04zp
https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04zpf.html
.. _e04zp-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and **must not** be changed between calls to the NAG optimization modelling suite.
**infile** : int
The file identifier associated with the argument data file. **Note:** that the file needs to be opened in read mode.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04zp-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
The options cannot be modified while solver is running.
(`errno` :math:`3`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{infile}\geq 0`.
(`errno` :math:`4`)
End-of-file or read error detected before BEGIN was found.
(`errno` :math:`4`)
BEGIN found, but end-of-file found before END.
(`errno` :math:`11`)
The :math:`\textit{option}` provided on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not recognized.
(`errno` :math:`12`)
The expected delimiter ':math:`=`' was not found on line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`13`)
The :math:`\textit{optval}` provided on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` could not be converted to the expected numerical type.
(`errno` :math:`15`)
The :math:`\textit{optval}` provided on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` for the integer option is out of bounds.
(`errno` :math:`16`)
The :math:`\textit{optval}` provided on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` for the real option is out of bounds.
(`errno` :math:`17`)
The :math:`\textit{optval}` provided on line :math:`\langle\mathit{\boldsymbol{value}}\rangle` for the character option is not valid.
.. _e04zp-py2-py-notes:
**Notes**
``handle_opt_set_file`` may be used to supply values for options to the solver from an external file.
It can only be called on handles which have been correctly initialized (e.g., by :meth:`handle_init`) and before the call to the solver. ``handle_opt_set_file`` looks in the file for a specific section containing the optional parameters.
The section must start with a line ``Begin`` and must finish with a line ``End``.
Anything outside the section is ignored.
If there is more than one section like this, only the first one is processed.
Any line within the section is either blank or a comment which is ignored or defines a single option as if it had been set by :meth:`handle_opt_set`.
The implied data type (character, integer or real) of each value to be set **must** match that expected by the corresponding option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
A comment begins with an asterisk (*) and all subsequent characters to the end of the line are ignored.
Comments can also be placed after the option.
The file is case insensitive.
Note that the options printed by the solver or by :meth:`handle_print` are in the compatible format.
An example of a valid options file is:
::
Begin * Example options file
Print Level = 3
Monitoring Level = 5 * output all details
Monitoring File = 42 * to this file
End
See `the E04 Introduction <https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
"""
raise NotImplementedError
