# -*- coding: utf-8 -*-
r"""
Module Summary
--------------
Interfaces for the NAG Mark 29.3 `opt` Chapter.
``opt`` - Minimizing or Maximizing a Function
This module provides functions for solving various mathematical optimization problems by solvers based on local stopping criteria.
The main classes of problems covered in this module are:
**Linear Programming (LP)** -- dense and sparse;
**Quadratic Programming (QP)** -- convex and nonconvex, dense and sparse;
**Quadratically Constrained Quadratic Programming (QCQP)** -- convex and nonconvex;
**Nonlinear Programming (NLP)** -- dense and sparse, based on active-set SQP methods or interior point methods (IPM);
**Second-order Cone Programming (SOCP)**;
**Semidefinite Programming (SDP)** -- both linear matrix inequalities (LMI) and bilinear matrix inequalities (BMI);
**Derivative-free Optimization (DFO)**;
**Least Squares (LSQ)** -- linear and nonlinear, constrained and unconstrained;
**General Nonlinear Data Fitting (NLDF)** -- nonlinear loss functions with regularization, constrained and unconstrained.
For a full overview of the functionality offered in this module, see `the functionality index <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#index>`__ or the Module Contents (submodule ``opt``).
See also other modules in the NAG Library relevant to optimization:
submodule :mod:`~naginterfaces.library.glopt` contains functions to solve **global optimization** problems;
submodule :mod:`~naginterfaces.library.mip` addresses problems arising in **operational research** and focuses on **Mixed Integer Programming (MIP)**;
submodule :mod:`~naginterfaces.library.lapacklin` and submodule :mod:`~naginterfaces.library.lapackeig` include functions for linear algebra and in particular unconstrained linear least squares;
submodule :mod:`~naginterfaces.library.fit` focuses on curve and surface fitting, in which linear data fitting in :math:`l_1` or :math:`l_{\infty }` norm might be of interest;
submodule :mod:`~naginterfaces.library.correg` offers several regression (data fitting) functions, including linear, nonlinear and quantile regression, LARS, LASSO and others.
See Also
--------
``naginterfaces.library.examples.opt`` :
This subpackage contains examples for the ``opt`` module.
See also the :ref:`library_opt_ex` subsection.
Functionality Index
-------------------
**Linear programming (LP)**
dense
active-set method/primal simplex
alternative 1: :meth:`lp_solve`
alternative 2: :meth:`lsq_lincon_solve`
sparse
interior point method (IPM): :meth:`handle_solve_lp_ipm`
active-set method/primal simplex
recommended (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#variant>`__): :meth:`qpconvex2_sparse_solve`
alternative: :meth:`qpconvex1_sparse_solve`
**Quadratic programming (QP)**
dense
active-set method for (possibly nonconvex) QP problem: :meth:`qp_dense_solve`
active-set method for convex QP problem: :meth:`lsq_lincon_solve`
sparse
active-set method sparse convex QP problem
recommended (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#variant>`__): :meth:`qpconvex2_sparse_solve`
alternative: :meth:`qpconvex1_sparse_solve`
interior point method (IPM) for (possibly nonconvex) QP problems: :meth:`handle_solve_ipopt`
**Second-order Cone Programming (SOCP)**
dense or sparse
interior point method: :meth:`handle_solve_socp_ipm`
**Semidefinite programming (SDP)**
generalized augmented Lagrangian method for SDP and SDP with bilinear matrix inequalities (BMI-SDP): :meth:`handle_solve_pennon`
**Nonlinear programming (NLP)**
dense
active-set sequential quadratic programming (SQP)
direct communication
recommended (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#variant>`__): :meth:`nlp1_solve`
alternative: :meth:`nlp2_solve`
reverse communication: :meth:`nlp1_rcomm`
sparse
active-set sequential quadratic programming (SQP): :meth:`handle_solve_ssqp`
interior point method (IPM): :meth:`handle_solve_ipopt`
active-set sequential quadratic programming (SQP)
alternative: :meth:`nlp2_sparse_solve`
alternative: :meth:`nlp1_sparse_solve`
**Nonlinear programming (NLP) -- derivative-free optimization (DFO)**
model-based method for bound-constrained optimization: :meth:`bounds_bobyqa_func`
Nelder--Mead simplex method for unconstrained optimization: :meth:`uncon_simplex`
**Nonlinear programming (NLP) -- special cases**
unidimensional optimization (one-dimensional) with bound constraints
method based on quadratic interpolation, no derivatives: :meth:`one_var_func`
method based on cubic interpolation: :meth:`one_var_deriv`
unconstrained
preconditioned conjugate gradient method: :meth:`uncon_conjgrd_comp`
bound-constrained
first order active-set method (nonlinear conjugate gradient): :meth:`handle_solve_bounds_foas`
quasi-Newton algorithm, no derivatives: :meth:`bounds_quasi_func_easy`
quasi-Newton algorithm, first derivatives: :meth:`bounds_quasi_deriv_easy`
modified Newton algorithm, first derivatives: :meth:`bounds_mod_deriv_comp`
modified Newton algorithm, first derivatives, easy-to-use: :meth:`bounds_mod_deriv_easy`
modified Newton algorithm, first and second derivatives: :meth:`bounds_mod_deriv2_comp`
modified Newton algorithm, first and second derivatives, easy-to-use: :meth:`bounds_mod_deriv2_easy`
**Linear least squares, linear regression, data fitting**
constrained
bound-constrained least squares problem: :meth:`bnd_lin_lsq`
linearly-constrained active-set method: :meth:`lsq_lincon_solve`
**Data fitting**
general loss functions (for sum of squares, see nonlinear least squares): :meth:`handle_solve_nldf`
**Nonlinear least squares, data fitting**
unconstrained
combined Gauss--Newton and modified Newton algorithm
no derivatives: :meth:`lsq_uncon_mod_func_comp`
no derivatives, easy-to-use: :meth:`lsq_uncon_mod_func_easy`
first derivatives: :meth:`lsq_uncon_mod_deriv_comp`
first derivatives, easy-to-use: :meth:`lsq_uncon_mod_deriv_easy`
first and second derivatives: :meth:`lsq_uncon_mod_deriv2_comp`
first and second derivatives, easy-to-use: :meth:`lsq_uncon_mod_deriv2_easy`
combined Gauss--Newton and quasi-Newton algorithm
first derivatives: :meth:`lsq_uncon_quasi_deriv_comp`
first derivatives, easy-to-use: :meth:`lsq_uncon_quasi_deriv_easy`
covariance matrix for nonlinear least squares problem (unconstrained): :meth:`lsq_uncon_covariance`
bound constrained
model-based derivative-free algorithm
direct communication: :meth:`handle_solve_dfls`
reverse communication: :meth:`handle_solve_dfls_rcomm`
trust region algorithm
first derivatives, optionally second derivatives: :meth:`handle_solve_bxnl`
generic, including nonlinearly constrained
nonlinear constraints active-set sequential quadratic programming (SQP): :meth:`lsq_gencon_deriv`
**NAG optimization modelling suite**
initialization of a handle for the suite
initialization as an empty problem: :meth:`handle_init`
read a problem from a file to a handle: :meth:`handle_read_file`
problem definition
define a linear objective function: :meth:`handle_set_linobj`
define a linear or a quadratic objective function: :meth:`handle_set_quadobj`
define nonlinear residual functions: :meth:`handle_set_nlnls`
define a nonlinear objective function: :meth:`handle_set_nlnobj`
define a second-order cone: :meth:`handle_set_group`
define bounds of variables: :meth:`handle_set_simplebounds`
define a block of linear constraints: :meth:`handle_set_linconstr`
define a block of nonlinear constraints: :meth:`handle_set_nlnconstr`
define a structure of Hessian of the objective, constraints or the Lagrangian: :meth:`handle_set_nlnhess`
add one or more linear matrix inequality constraints: :meth:`handle_set_linmatineq`
define bilinear matrix terms: :meth:`handle_set_quadmatineq`
factor of quadratic coefficient matrix: :meth:`handle_set_qconstr_fac`
full quadratic coefficient matrix: :meth:`handle_set_qconstr`
set variable properties (e.g., integrality): :meth:`handle_set_property`
problem editing
define new variables: :meth:`handle_add_vars`
disable (temporarily remove) components of the model: :meth:`handle_disable`
enable (bring back) previously disabled components of the model: :meth:`handle_enable`
modify a single coefficient in a linear constraint: :meth:`handle_set_linconstr_coeff`
modify a single coefficient in the linear objective function: :meth:`handle_set_linobj_coeff`
modify bounds of an existing constraint or variable: :meth:`handle_set_bound`
solvers
interior point method (IPM) for linear programming (LP): :meth:`handle_solve_lp_ipm`
first order active-set method (nonlinear conjugate gradient): :meth:`handle_solve_bounds_foas`
active-set sequential quadratic programming method (SQP) for nonlinear programming (NLP): :meth:`handle_solve_ssqp`
interior point method (IPM) for nonlinear programming (NLP): :meth:`handle_solve_ipopt`
generalized augmented Lagrangian method for SDP and SDP with bilinear matrix inequalities (BMI-SDP): :meth:`handle_solve_pennon`
interior point method (IPM) for Second-order Cone programming (SOCP): :meth:`handle_solve_socp_ipm`
constrained nonlinear data fitting (NLDF): :meth:`handle_solve_nldf`
derivative-free optimisation (DFO) for nonlinear least squares problems
direct communication: :meth:`handle_solve_dfls`
reverse communication: :meth:`handle_solve_dfls_rcomm`
trust region optimisation for nonlinear least squares problems (BXNL): :meth:`handle_solve_bxnl`
model-based method for bound-constrained optimization
direct communication: :meth:`handle_solve_dfno`
reverse communication: :meth:`handle_solve_dfno_rcomm`
deallocation
destroy the problem handle: :meth:`handle_free`
service routines
print information about a problem handle: :meth:`handle_print`
set/get information in a problem handle: :meth:`handle_set_get_real`
set/get integer information in a problem handle: :meth:`handle_set_get_integer`
supply option values from a character string: :meth:`handle_opt_set`
get the setting of option: :meth:`handle_opt_get`
supply option values from external file: :meth:`handle_opt_set_file`
**Service functions**
input and output (I/O)
read MPS data file defining LP, QP, MILP or MIQP problem: :meth:`miqp_mps_read`
write MPS data file defining LP, QP, MILP or MIQP problem: :meth:`miqp_mps_write`
read sparse SPDA data files for linear SDP problems: :meth:`sdp_read_sdpa`
read MPS data file defining LP or QP problem (deprecated): :meth:`qpconvex1_sparse_mps`
read a problem from a file to a handle: :meth:`handle_read_file`
derivative check and approximation
check user's function for calculating first derivatives of function: :meth:`check_deriv`
check user's function for calculating second derivatives of function: :meth:`check_deriv2`
check user's function for calculating Jacobian of first derivatives: :meth:`lsq_check_deriv`
check user's function for calculating Hessian of a sum of squares: :meth:`lsq_check_hessian`
estimate (using numerical differentiation) gradient and/or Hessian of a function: :meth:`estimate_deriv`
determine the pattern of nonzeros in the Jacobian matrix for :meth:`nlp2_sparse_solve`: :meth:`nlp2_sparse_jacobian`
covariance matrix for nonlinear least squares problem (unconstrained): :meth:`lsq_uncon_covariance`
option setting functions
NAG optimization modelling suite
supply option values from a character string: :meth:`handle_opt_set`
get the setting of option: :meth:`handle_opt_get`
supply option values from external file: :meth:`handle_opt_set_file`
:meth:`uncon_conjgrd_comp`
supply option values from external file: :meth:`uncon_conjgrd_option_file`
supply option values from a character string: :meth:`uncon_conjgrd_option_string`
:meth:`lp_solve`
supply option values from external file: :meth:`lp_option_file`
supply option values from a character string: :meth:`lp_option_string`
:meth:`lsq_lincon_solve`
supply option values from external file: :meth:`lsq_lincon_option_file`
supply option values from a character string: :meth:`lsq_lincon_option_string`
:meth:`qp_dense_solve`
supply option values from external file: :meth:`qp_dense_option_file`
supply option values from a character string: :meth:`qp_dense_option_string`
:meth:`qpconvex1_sparse_solve`
supply option values from external file: :meth:`qpconvex1_sparse_option_file`
supply option values from a character string: :meth:`qpconvex1_sparse_option_string`
:meth:`qpconvex2_sparse_solve`
initialization function: :meth:`qpconvex2_sparse_init`
supply option values from external file: :meth:`qpconvex2_sparse_option_file`
set a single option from a character string: :meth:`qpconvex2_sparse_option_string`
set a single option from an integer argument: :meth:`qpconvex2_sparse_option_integer_set`
set a single option from a real argument: :meth:`qpconvex2_sparse_option_double_set`
get the setting of an integer valued option: :meth:`qpconvex2_sparse_option_integer_get`
get the setting of a real valued option: :meth:`qpconvex2_sparse_option_double_get`
:meth:`nlp1_solve` and :meth:`nlp1_rcomm`
initialization function for :meth:`nlp1_solve` and :meth:`nlp1_rcomm`: :meth:`nlp1_init`
supply option values from external file: :meth:`nlp1_option_file`
supply option values from a character string: :meth:`nlp1_option_string`
:meth:`nlp1_sparse_solve`
supply option values from external file: :meth:`nlp1_sparse_option_file`
supply option values from a character string: :meth:`nlp1_sparse_option_string`
:meth:`lsq_gencon_deriv`
supply option values from external file: :meth:`lsq_gencon_deriv_option_file`
supply option values from a character string: :meth:`lsq_gencon_deriv_option_string`
:meth:`nlp2_sparse_solve`
initialization function: :meth:`nlp2_sparse_init`
supply option values from external file: :meth:`nlp2_sparse_option_file`
set a single option from a character string: :meth:`nlp2_sparse_option_string`
set a single option from an integer argument: :meth:`nlp2_sparse_option_integer_set`
set a single option from a real argument: :meth:`nlp2_sparse_option_double_set`
get the setting of an integer valued option: :meth:`nlp2_sparse_option_integer_get`
get the setting of a real valued option: :meth:`nlp2_sparse_option_double_get`
:meth:`nlp2_solve`
initialization function: :meth:`nlp2_init`
supply option values from external file: :meth:`nlp2_option_file`
set a single option from a character string: :meth:`nlp2_option_string`
set a single option from an integer argument: :meth:`nlp2_option_integer_set`
set a single option from a real argument: :meth:`nlp2_option_double_set`
get the setting of an integer valued option: :meth:`nlp2_option_integer_get`
get the setting of a real valued option: :meth:`nlp2_option_double_get`
For full information please refer to the NAG Library document
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html
"""
# NAG Copyright 2017-2023.
[docs]def one_var_func(funct, e1, e2, a, b, maxcal, data=None):
r"""
``one_var_func`` searches for a minimum, in a given finite interval, of a continuous function of a single variable, using function values only.
The method (based on quadratic interpolation) is intended for functions which have a continuous first derivative (although it will usually work if the derivative has occasional discontinuities).
.. _e04ab-py2-py-doc:
For full information please refer to the NAG Library document for e04ab
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04abf.html
.. _e04ab-py2-py-parameters:
**Parameters**
**funct** : callable fc = funct(xc, data=None)
You must supply this function to calculate the value of the function :math:`F\left(x\right)` at any point :math:`x` in :math:`\left[a, b\right]`.
It should be tested separately before being used in conjunction with ``one_var_func``.
**Parameters**
**xc** : float
The point :math:`x` at which the value of :math:`F` is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Must be set to the value of the function :math:`F` at the current point :math:`x`.
**e1** : float
The relative accuracy to which the position of a minimum is required. (Note that, since :math:`\mathrm{e1}` is a relative tolerance, the scaling of :math:`x` is automatically taken into account.)
:math:`\mathrm{e1}` should be no smaller than :math:`2\epsilon`, and preferably not much less than :math:`\sqrt{\epsilon }`, where :math:`\epsilon` is the machine precision.
**e2** : float
The absolute accuracy to which the position of a minimum is required. :math:`\mathrm{e2}` should be no smaller than :math:`2\epsilon`.
**a** : float
The lower bound :math:`a` of the interval containing a minimum.
**b** : float
The upper bound :math:`b` of the interval containing a minimum.
**maxcal** : int
The maximum number of calls of :math:`F\left(x\right)` to be allowed.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**e1** : float
If you set :math:`\mathrm{e1}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e1}` will be reset to the default value :math:`\sqrt{\epsilon }` before starting the minimization process.
**e2** : float
If you set :math:`\mathrm{e2}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e2}` will be reset to the default value :math:`\sqrt{\epsilon }`.
**a** : float
An improved lower bound on the position of the minimum.
**b** : float
An improved upper bound on the position of the minimum.
**maxcal** : int
The total number of times that :math:`\mathrm{funct}` was actually called.
**x** : float
The estimated position of the minimum.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
.. _e04ab-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 3`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{a} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{e2} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{b} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{a}+\mathrm{e2} < \mathrm{b}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
The maximum number of function calls, :math:`\langle\mathit{\boldsymbol{value}}\rangle`, have been performed. This may have happened simply because :math:`\mathrm{maxcal}` was set too small for the particular problem, or may be due to a mistake in the user-supplied function :math:`\mathrm{funct}`. If no mistake can be found in :math:`\mathrm{funct}`, restart ``one_var_func`` (preferably with the values of :math:`\mathrm{a}` and :math:`\mathrm{b}` given on exit from the previous call to ``one_var_func``).
.. _e04ab-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``one_var_func`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right)\quad \text{ subject to }\quad a\leq x\leq b\text{.}
It normally computes a sequence of :math:`x` values which tend in the limit to a minimum of :math:`F\left(x\right)` subject to the given bounds.
It also progressively reduces the interval :math:`\left[a, b\right]` in which the minimum is known to lie.
It uses the safeguarded quadratic-interpolation method described in Gill and Murray (1973).
You must supply a :math:`\mathrm{funct}` to evaluate :math:`F\left(x\right)`.
The arguments :math:`\mathrm{e1}` and :math:`\mathrm{e2}` together specify the accuracy
.. math::
\textit{Tol}\left(x\right) = \mathrm{e1}\times \left\lvert x\right\rvert +\mathrm{e2}
to which the position of the minimum is required.
Note that :math:`\mathrm{funct}` is never called at any point which is closer than :math:`\textit{Tol}\left(x\right)` to a previous point.
If the original interval :math:`\left[a, b\right]` contains more than one minimum, ``one_var_func`` will normally find one of the minima.
.. _e04ab-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
"""
raise NotImplementedError
[docs]def one_var_deriv(funct, e1, e2, a, b, maxcal, data=None):
r"""
``one_var_deriv`` searches for a minimum, in a given finite interval, of a continuous function of a single variable, using function and first derivative values.
The method (based on cubic interpolation) is intended for functions which have a continuous first derivative (although it will usually work if the derivative has occasional discontinuities).
.. _e04bb-py2-py-doc:
For full information please refer to the NAG Library document for e04bb
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04bbf.html
.. _e04bb-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(xc, data=None)
You must supply this function to calculate the values of :math:`F\left(x\right)` and :math:`\frac{{dF}}{{dx}}` at any point :math:`x` in :math:`\left[a, b\right]`.
It should be tested separately before being used in conjunction with ``one_var_deriv``.
**Parameters**
**xc** : float
The point :math:`x` at which the values of :math:`F` and :math:`\frac{{dF}}{{dx}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Must be set to the value of the function :math:`F` at the current point :math:`x`.
**gc** : float
Must be set to the value of the first derivative :math:`\frac{{dF}}{{dx}}` at the current point :math:`x`.
**e1** : float
The relative accuracy to which the position of a minimum is required. (Note that, since :math:`\mathrm{e1}` is a relative tolerance, the scaling of :math:`x` is automatically taken into account.)
:math:`\mathrm{e1}` should be no smaller than :math:`2\epsilon`, and preferably not much less than :math:`\sqrt{\epsilon }`, where :math:`\epsilon` is the machine precision.
**e2** : float
The absolute accuracy to which the position of a minimum is required. :math:`\mathrm{e2}` should be no smaller than :math:`2\epsilon`.
**a** : float
The lower bound :math:`a` of the interval containing a minimum.
**b** : float
The upper bound :math:`b` of the interval containing a minimum.
**maxcal** : int
The maximum number of calls of :math:`\mathrm{funct}` to be allowed.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**e1** : float
If you set :math:`\mathrm{e1}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e1}` will be reset to the default value :math:`\sqrt{\epsilon }` before starting the minimization process.
**e2** : float
If you set :math:`\mathrm{e2}` to :math:`0.0` (or to any value less than :math:`\epsilon`), :math:`\mathrm{e2}` will be reset to the default value :math:`\sqrt{\epsilon }`.
**a** : float
An improved lower bound on the position of the minimum.
**b** : float
An improved upper bound on the position of the minimum.
**maxcal** : int
The total number of times that :math:`\mathrm{funct}` was actually called.
**x** : float
The estimated position of the minimum.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float
The value of the first derivative at the final point in :math:`\mathrm{x}`.
.. _e04bb-py2-py-errors:
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
On entry, :math:`\mathrm{a}+\mathrm{e2} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{b} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{a}+\mathrm{e2} < \mathrm{b}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 2`.
(`errno` :math:`2`)
The maximum number of function calls, :math:`\langle\mathit{\boldsymbol{value}}\rangle`, have been performed. This may have happened simply because :math:`\mathrm{maxcal}` was set too small for the particular problem, or may be due to a mistake in the user-supplied function :math:`\mathrm{funct}`. If no mistake can be found in :math:`\mathrm{funct}`, restart ``one_var_deriv`` (preferably with the values of :math:`\mathrm{a}` and :math:`\mathrm{b}` given on exit from the previous call to ``one_var_deriv``).
.. _e04bb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``one_var_deriv`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right)\quad \text{ subject to }\quad a\leq x\leq b
when the first derivative :math:`\frac{{dF}}{{dx}}` can be calculated.
The function normally computes a sequence of :math:`x` values which tend in the limit to a minimum of :math:`F\left(x\right)` subject to the given bounds.
It also progressively reduces the interval :math:`\left[a, b\right]` in which the minimum is known to lie.
It uses the safeguarded cubic-interpolation method described in Gill and Murray (1973).
You must supply a :math:`\mathrm{funct}` to evaluate :math:`F\left(x\right)` and :math:`\frac{{dF}}{{dx}}`.
The arguments :math:`\mathrm{e1}` and :math:`\mathrm{e2}` together specify the accuracy
.. math::
\textit{Tol}\left(x\right) = \mathrm{e1}\times \left\lvert x\right\rvert +\mathrm{e2}
to which the position of the minimum is required.
Note that :math:`\mathrm{funct}` is never called at a point which is closer than :math:`\textit{Tol}\left(x\right)` to a previous point.
If the original interval :math:`\left[a, b\right]` contains more than one minimum, ``one_var_deriv`` will normally find one of the minima.
.. _e04bb-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
"""
raise NotImplementedError
[docs]def uncon_simplex(x, tolf, tolx, funct, maxcal, monit=None, data=None):
r"""
``uncon_simplex`` minimizes a general function :math:`F\left(\mathbf{x}\right)` of :math:`n` independent variables :math:`\mathbf{x} = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` by the Nelder and Mead simplex method (see Nelder and Mead (1965)).
Derivatives of the function need not be supplied.
.. _e04cb-py2-py-doc:
For full information please refer to the NAG Library document for e04cb
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html
.. _e04cb-py2-py-parameters:
**Parameters**
**x** : float, array-like, shape :math:`\left(n\right)`
A guess at the position of the minimum. Note that the problem should be scaled so that the values of the :math:`\mathrm{x}[i-1]` are of order unity.
**tolf** : float
The error tolerable in the function values, in the following sense. If :math:`f_{\textit{i}}`, for :math:`\textit{i} = 1,2,\ldots,n+1`, are the individual function values at the vertices of the current simplex, and if :math:`f_m` is the mean of these values, then you can request that ``uncon_simplex`` should terminate if
.. math::
\sqrt{\frac{1}{{n+1}}\sum_{{i = 1}}^{{n+1}}\left(f_i-f_m\right)^2} < \mathrm{tolf}\text{.}
You may specify :math:`\mathrm{tolf} = 0` if you wish to use only the termination criterion `(2) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn2>`__ on the spatial values: see the description of :math:`\mathrm{tolx}`.
**tolx** : float
The error tolerable in the spatial values, in the following sense. If :math:`LV` denotes the 'linearized' volume of the current simplex, and if :math:`{LV}_{\mathrm{init}}` denotes the 'linearized' volume of the initial simplex, then you can request that ``uncon_simplex`` should terminate if
.. math::
\frac{{LV}}{{{LV}_{\mathrm{init}}}} < \mathrm{tolx}\text{.}
You may specify :math:`\mathrm{tolx} = 0` if you wish to use only the termination criterion `(1) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn1>`__ on function values: see the description of :math:`\mathrm{tolf}`.
**funct** : callable fc = funct(xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F` at a specified point.
It should be tested separately before being used in conjunction with ``uncon_simplex``.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point at which the function value is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`\mathbf{x}`.
**maxcal** : int
The maximum number of function evaluations to be allowed.
**monit** : None or callable monit(fmin, fmax, sim, ncall, serror, vratio, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` may be used to monitor the optimization process.
It is invoked once every iteration.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
**Parameters**
**fmin** : float
The smallest function value in the current simplex.
**fmax** : float
The largest function value in the current simplex.
**sim** : float, ndarray, shape :math:`\left(n+1, n\right)`
The :math:`n+1` position vectors of the current simplex.
**ncall** : int
The number of times that :math:`\mathrm{funct}` has been called so far.
**serror** : float
The current value of the standard deviation in function values used in termination test `(1) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn1>`__.
**vratio** : float
The current value of the linearized volume ratio used in termination test `(2) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn2>`__.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\mathbf{x}` corresponding to the function value in :math:`\mathrm{f}`.
**f** : float
The lowest function value found.
.. _e04cb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolx} = 0.0` and :math:`\mathrm{tolf} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolx} = 0.0` then :math:`\mathrm{tolf}` is greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolf} = 0.0` and :math:`\mathrm{tolx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolf} = 0.0` then :math:`\mathrm{tolx}` is greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{tolf} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{tolx} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{tolf}\neq 0.0` and :math:`\mathrm{tolx}\neq 0.0` then both should be greater than or equal to the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`2`)
:math:`\mathrm{maxcal}` function evaluations have been completed without any other termination test passing. Check the coding of :math:`\mathrm{funct}` before increasing the value of :math:`\mathrm{maxcal}`.
.. _e04cb-py2-py-notes:
**Notes**
``uncon_simplex`` finds an approximation to a minimum of a function :math:`F` of :math:`n` variables.
You must supply a function to calculate the value of :math:`F` for any set of values of the variables.
The method is iterative.
A simplex of :math:`n+1` points is set up in the :math:`n`-dimensional space of the variables (for example, in :math:`2` dimensions the simplex is a triangle) under the assumption that the problem has been scaled so that the values of the independent variables at the minimum are of order unity.
The starting point you have provided is the first vertex of the simplex, the remaining :math:`n` vertices are generated by ``uncon_simplex``.
The vertex of the simplex with the largest function value is reflected in the centre of gravity of the remaining vertices and the function value at this new point is compared with the remaining function values.
Depending on the outcome of this test the new point is accepted or rejected, a further expansion move may be made, or a contraction may be carried out.
See Nelder and Mead (1965) and Parkinson and Hutchinson (1972) for more details.
When no further progress can be made the sides of the simplex are reduced in length and the method is repeated.
The method can be slow, but computational bottlenecks have been reduced following Singer and Singer (2004).
However, ``uncon_simplex`` is robust, and, therefore, very useful for functions that are subject to inaccuracies.
There are the following options for successful termination of the method: based only on the function values at the vertices of the current simplex (see `(1) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn1>`__); based only on a volume ratio between the current simplex and the initial one (see `(2) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04cbf.html#eqn2>`__); or based on which one of the previous two tests passes first.
The volume test may be useful if :math:`F` is discontinuous, while the function-value test should be sufficient on its own if :math:`F` is continuous.
.. _e04cb-py2-py-references:
**References**
Nelder, J A and Mead, R, 1965, `A simplex method for function minimization`, Comput. J. (7), 308--313
Parkinson, J M and Hutchinson, D, 1972, `An investigation into the efficiency of variants of the simplex method`, Numerical Methods for Nonlinear Optimization, (ed F A Lootsma), Academic Press
Singer, S and Singer, S, 2004, `Efficient implementation of the Nelder--Mead search algorithm`, Appl. Num. Anal. Comp. Math. (1(3)), 524--534
See Also
--------
:meth:`naginterfaces.library.examples.opt.uncon_simplex_ex.main`
"""
raise NotImplementedError
[docs]def uncon_conjgrd_comp(objfun, x, comm, data=None, io_manager=None):
r"""
``uncon_conjgrd_comp`` minimizes an unconstrained nonlinear function of several variables using a pre-conditioned, limited memory quasi-Newton conjugate gradient method.
First derivatives (or an 'acceptable' finite difference approximation to them) are required.
It is intended for use on large scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`uncon_conjgrd_option_file`, :meth:`uncon_conjgrd_option_string`, :meth:`nlp1_init`.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_comp`` is deprecated.
Please use :meth:`handle_solve_bounds_foas` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04dg-py2-py-doc:
For full information please refer to the NAG Library document for e04dg
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04dgf.html
.. _e04dg-py2-py-parameters:
**Parameters**
**objfun** : callable (objf, objgrd) = objfun(mode, x, nstate, data=None)
:math:`\mathrm{objfun}` must calculate the objective function :math:`F\left(x\right)` and possibly its gradient as well for a specified :math:`n`-element vector :math:`x`.
**Parameters**
**mode** : int
Indicates which values must be assigned during each call of :math:`\mathrm{objfun}`. Only the following values need be assigned:
:math:`\mathrm{mode} = 0`
:math:`\mathrm{objf}`.
:math:`\mathrm{mode} = 2`
:math:`\mathrm{objf}` and :math:`\mathrm{objgrd}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`x`, the vector of variables at which the objective function and its gradient are to be evaluated.
**nstate** : int
Will be :math:`1` on the first call of :math:`\mathrm{objfun}` by ``uncon_conjgrd_comp``, and :math:`0` for all subsequent calls. Thus, you may wish to test, :math:`\mathrm{nstate}` within :math:`\mathrm{objfun}` in order to perform certain calculations once only. For example, you may read data or initialize global variables when :math:`\mathrm{nstate} = 1`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**objf** : float
The value of the objective function at :math:`x`.
**objgrd** : float, array-like, shape :math:`\left(n\right)`
If :math:`\mathrm{mode} = 2`, :math:`\mathrm{objgrd}[\textit{i}-1]` must contain the value of :math:`\frac{{\partial F}}{{\partial x_{\textit{i}}}}` evaluated at :math:`x`, for :math:`\textit{i} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**itera** : int
The total number of iterations performed.
**objf** : float
The value of the objective function at the final iterate.
**objgrd** : float, ndarray, shape :math:`\left(n\right)`
The gradient of the objective function at the final iterate (or its finite difference approximation).
**x** : float, ndarray, shape :math:`\left(n\right)`
The final estimate of the solution.
.. _e04dg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Estimated Optimal Function Value'** : float
This value of :math:`r` specifies the user-supplied guess of the optimum objective function value :math:`F_{\mathrm{est}}`.
This value is used to calculate an initial step length :math:`\alpha_0` (see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04dgf.html#algdetails>`__).
If the value of :math:`r` is not specified (the default), then this has the effect of setting :math:`\alpha_0` to unity.
It should be noted that for badly scaled functions a unit step along the steepest descent direction will often compute the objective function at very large values of :math:`x`.
**'Function Precision'** : float
Default :math:`\text{} = \epsilon^{0.9}`
The argument defines :math:`\epsilon_r`, which is intended to be a measure of the accuracy with which the problem function :math:`F\left(x\right)` can be computed.
If :math:`r < \epsilon` or :math:`r\geq 1`, the default value is used.
The value of :math:`\epsilon_r` should reflect the relative precision of :math:`1+\left\lvert F\left(x\right)\right\rvert`; i.e., :math:`\epsilon_r` acts as a relative precision when :math:`\left\lvert F\right\rvert` is large, and as an absolute precision when :math:`\left\lvert F\right\rvert` is small.
For example, if :math:`F\left(x\right)` is typically of order :math:`1000` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-6}`.
In contrast, if :math:`F\left(x\right)` is typically of order :math:`10^{-4}` and the first six significant digits are known to be correct, an appropriate value for :math:`\epsilon_r` would be :math:`10^{-10}`.
The choice of :math:`\epsilon_r` can be quite complicated for badly scaled problems; see Module 8 of Gill `et al.` (1981) for a discussion of scaling techniques.
The default value is appropriate for most simple functions that are computed with full accuracy.
However when the accuracy of the computed function values is known to be significantly worse than full precision, the value of :math:`\epsilon_r` should be large enough so that no attempt will be made to distinguish between function values that differ by less than the error inherent in the calculation.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5n}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
If :math:`i < 0`, the default value is used.
Problems whose Hessian matrices at the solution contain sets of clustered eigenvalues are likely to be minimized in significantly fewer than :math:`n` iterations.
Problems without this property may require anything between :math:`n` and :math:`5n` iterations, with approximately :math:`2n` iterations being a common figure for moderately difficult problems.
**'Linesearch Tolerance'** : float
Default :math:`\text{} = 0.9`
The value :math:`r` controls the accuracy with which the step :math:`\alpha` taken during each iteration approximates a minimum of the function along the search direction (the smaller the value of :math:`r`, the more accurate the linesearch).
The default value :math:`r = 0.9` requests an inaccurate search, and is appropriate for most problems.
A more accurate search may be appropriate when it is desirable to reduce the number of iterations -- for example, if the objective function is cheap to evaluate.
If :math:`r < 0` or :math:`r\geq 1`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default for ``uncon_conjgrd_comp`` :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Maximum Step Length'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, the maximum allowable step length for the linesearch is taken as :math:`\mathrm{min}\left(\frac{1}{{\texttt{machine.real_safe}\left(\right)}}, \frac{r}{{\left\lVert p_k\right\rVert }}\right)`.
If :math:`r\leq 0`, the default value is used.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon_R^{0.8}`
The argument :math:`r` specifies the accuracy to which you wish the final iterate to approximate a solution of the problem.
Broadly speaking, :math:`r` indicates the number of correct figures desired in the objective function at the solution.
For example, if :math:`r` is :math:`10^{-6}` and termination occurs with no exception or warning is raised (see :ref:`Parameters <e04dg-py2-py-parameters>`), then the final point satisfies the termination criteria, where :math:`\tau_F` represents 'Optimality Tolerance'.
If :math:`r < \epsilon_r` or :math:`r\geq 1`, the default value is used.
If 'Optimality Tolerance' is chosen below a certain threshold, it will automatically be reset to another value.
**'Print Level'** : int
The value :math:`i` controls the amount of printout produced by ``uncon_conjgrd_comp``, as indicated below.
A detailed description of the printout is given in `Description of Printed Output <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04dgf.html#fcomments1>`__ (summary output at each iteration and the final solution).
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+==========+==========================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Description of Printed Output <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04dgf.html#fcomments1>`__) for each iteration (no printout of the final solution).|
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10`|The final solution and one line of summary output for each iteration. |
+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**'Start Objective Check at Variable'** : int
Default :math:`\text{} = 1`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check at Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{max}\left(1, \mathrm{min}\left(n, i_2\right)\right)`, the default value is used.
If :math:`i_2\leq 0` or :math:`i_2 > n`, the default value is used.
**'Stop Objective Check at Variable'** : int
Default :math:`\text{} = n`
These keywords take effect only if :math:`\text{‘Verify Level'} > 0`.
They may be used to control the verification of gradient elements computed by :math:`\mathrm{objfun}`.
For example, if the first :math:`30` elements of the objective gradient appeared to be correct in an earlier run, so that only element :math:`31` remains questionable, it is reasonable to specify :math:`\text{‘Start Objective Check at Variable'} = 31`.
If the first :math:`30` variables appear linearly in the objective, so that the corresponding gradient elements are constant, the above choice would also be appropriate.
If :math:`i_1\leq 0` or :math:`i_1 > \mathrm{max}\left(1, \mathrm{min}\left(n, i_2\right)\right)`, the default value is used.
If :math:`i_2\leq 0` or :math:`i_2 > n`, the default value is used.
**'Verify Level'** : int
Default :math:`\text{} = 0`
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
**'Verify Objective Gradients'** : valueless
These keywords refer to finite difference checks on the gradient elements computed by :math:`\mathrm{objfun}`.
Gradients are verified at the user-supplied initial estimate of the solution.
The possible choices for :math:`i` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Meaning |
+==========+==============================================================================================================================+
|:math:`-1`|No checks are performed. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Only a 'cheap' test will be performed, requiring one call to :math:`\mathrm{objfun}`. |
+----------+------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |In addition to the 'cheap' test, individual gradient elements will also be checked using a reliable (but more expensive) test.|
+----------+------------------------------------------------------------------------------------------------------------------------------+
For example, the objective gradient will be verified if 'Verify', :math:`\text{‘Verify'} = \texttt{'YES'}`, 'Verify Gradients', 'Verify Objective Gradients' or :math:`\text{‘Verify Level'} = 1` is specified.
.. _e04dg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`8`)
Gradient at the starting point is too small.
(`errno` :math:`9`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`6`)
Current point cannot be improved upon.
(`errno` :math:`7`)
Large errors found in the derivatives.
**NagAlgorithmicMajorWarning**
(`errno` :math:`3`)
Too many iterations.
(`errno` :math:`4`)
Computed upper bound on step length is too small.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination.
.. _e04dg-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``uncon_conjgrd_comp`` is designed to solve unconstrained minimization problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad {-\infty }\leq x\leq \infty \text{,}
where :math:`x` is an :math:`n`-element vector.
You must supply an initial estimate of the solution.
For maximum reliability, it is preferable to provide all first partial derivatives.
If all of the derivatives cannot be provided, you are recommended to obtain approximate values (using finite differences) by calling :meth:`estimate_deriv` from within :math:`\mathrm{objfun}`.
The method used by ``uncon_conjgrd_comp`` is described in `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04dgf.html#algdetails>`__.
.. _e04dg-py2-py-references:
**References**
Gill, P E and Murray, W, 1979, `Conjugate-gradient methods for large-scale nonlinear optimization`, Technical Report SOL 79-15, Department of Operations Research, Stanford University
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
"""
raise NotImplementedError
[docs]def uncon_conjgrd_option_file(ioptns, comm, io_manager=None):
r"""
``uncon_conjgrd_option_file`` may be used to supply options to :meth:`uncon_conjgrd_comp` from an external file.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_option_file`` is deprecated.
There is no suggested replacement for this routine.
.. _e04dj-py2-py-doc:
For full information please refer to the NAG Library document for e04dj
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04djf.html
.. _e04dj-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04dj-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04dj-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``uncon_conjgrd_option_file`` may be used to supply values for options to :meth:`uncon_conjgrd_comp`. ``uncon_conjgrd_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`uncon_conjgrd_comp` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`uncon_conjgrd_comp`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def uncon_conjgrd_option_string(optstr, comm, io_manager=None):
r"""
``uncon_conjgrd_option_string`` may be used to supply individual options to :meth:`uncon_conjgrd_comp`.
.. deprecated:: 27.0.0.0
``uncon_conjgrd_option_string`` is deprecated.
There is no suggested replacement for this routine.
.. _e04dk-py2-py-doc:
For full information please refer to the NAG Library document for e04dk
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04dkf.html
.. _e04dk-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04dk-py2-py-notes>` and in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04dk-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04dk-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``uncon_conjgrd_option_string`` may be used to supply values for options to :meth:`uncon_conjgrd_comp`.
It is only necessary to call ``uncon_conjgrd_option_string`` for those arguments whose values are to be different from their default values.
One call to ``uncon_conjgrd_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`uncon_conjgrd_comp` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`uncon_conjgrd_comp`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for uncon_conjgrd_comp <e04dg-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_func_comp(m, lsqfun, x, lsqmon=None, iprint=1, maxcal=None, eta=None, xtol=0.0, stepmx=100000.0, data=None):
r"""
``lsq_uncon_mod_func_comp`` is a comprehensive algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
No derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04fc-py2-py-doc:
For full information please refer to the NAG Library document for e04fc
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fcf.html
.. _e04fc-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` at any point :math:`x`. (However, if you do not wish to calculate the residuals at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_func_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Has a non-negative value.
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{lsqfun}` resets :math:`\mathrm{iflag}` to some negative number, ``lsq_uncon_mod_func_comp`` will terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current approximation to the Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**igrade** : int
``lsq_uncon_mod_func_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_func_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far. (However, for intermediate calls of :math:`\mathrm{lsqmon}`, :math:`\mathrm{nf}` is calculated on the assumption that the latest linear search has been successful. If this is not the case, the :math:`n` evaluations allowed for approximating the Jacobian at the new point will not in fact have been made. :math:`\mathrm{nf}` will be accurate at the final call of :math:`\mathrm{lsqmon}`.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_func_comp``.
If :math:`\mathrm{iprint} = 0`, :math:`\mathrm{lsqmon}` is just called at the final point.
If :math:`\mathrm{iprint} < 0`, :math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`400\times n`.
The limit you set on the number of times that :math:`\mathrm{lsqfun}` may be called by ``lsq_uncon_mod_func_comp``. There will be an error exit (see :ref:`Exceptions <e04fc-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_func_comp`` involves a linear minimization, i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`.
Specifies how accurately the linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`). Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_mod_func_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made each iteration. On balance it is usually more efficient to perform a low accuracy minimization.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fcf.html#accuracy>`__.)
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.) ``lsq_uncon_mod_func_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2\text{,}
where :math:`k` is the iteration number. Thus, if the problem has more than one solution, ``lsq_uncon_mod_func_comp`` is most likely to find the one nearest to the starting point. On difficult problems, a realistic choice can prevent the sequence :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`. However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The estimate of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the estimated Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the estimated Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_func_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}`).
.. _e04fc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`4`)
Failure in computing SVD of estimated Jacobian matrix.
.. _e04fc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_uncon_mod_func_comp`` is essentially identical to the function LSQNDN in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply :math:`\mathrm{lsqfun}` to calculate the values of the :math:`f_i\left(x\right)` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is an approximation to the Gauss--Newton direction; otherwise additional function evaluations are made so as to enable :math:`p^{\left(k\right)}` to be a more accurate approximation to the Newton direction.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04fc-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
See Also
--------
:meth:`naginterfaces.library.examples.opt.lsq_uncon_mod_func_comp_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfls(handle, objfun, x, nres, monit=None, data=None, io_manager=None):
r"""
``handle_solve_dfls`` is a forward communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFLS) for small to medium-scale nonlinear least squares problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04ff-py2-py-doc:
For full information please refer to the NAG Library document for e04ff
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fff.html
.. _e04ff-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfls``. It **must not** be changed between calls to the NAG optimization modelling suite.
**objfun** : callable (rx, inform) = objfun(x, nres, inform, data=None)
:math:`\mathrm{objfun}` must evaluate the value of the nonlinear residuals :math:`r_i\left(x\right)` at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`m_r`, the number of residuals in the problem, as set during the initialization of the handle by :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residuals :math:`r_i\left(x\right)` at :math:`x`.
**inform** : int
May be used to indicate that the requested objective value could not be computed. Specifically, it can be set to a negative value:
:math:`\mathrm{inform} = -1`
The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 17.
:math:`\mathrm{inform} = -2`
The solver will cleanly exit with :math:`\mathrm{errno}` = 20 and return the best available point as well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`m_r`, the number of residuals in the problem. It must be unchanged from the value set during the definition of the objective structure by :meth:`handle_set_nlnls`.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'DFO Monitor Frequency' (default value :math:`0`, :math:`\mathrm{monit}` is never called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+------------+--------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)` (sum of the squared residuals).|
+------------+--------------------------------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+------------+--------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+--------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04ff-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFLS Small Residuals Tol'** : float
Default :math:`= \epsilon^{0.75}`
This option defines the tolerance on the value of the residuals.
Namely, the solver declares convergence if
.. math::
f\left(x\right) = \sum_{{i = 1}}^{m_r}{r_i\left(x\right)}^2 < \text{‘DFLS Small Residuals Tol'}\text{.}
Constraint: :math:`\text{‘DFLS Small Residuals Tol'} > \epsilon^2`.
**'DFO Initial Interp Points'** : str
Default :math:`= \texttt{'Coordinate'}`
Determines how the initial interpolation points are chosen.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}`, the interpolation points are chosen along the coordinate directions around the initial point.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Random'}`, the initial interpolation points are chosen along random orthogonal directions around the initial point.
Set 'DFO Random Seed' to a positive value to fix the random seed and get reproducible results.
Constraint: :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}` or :math:`\texttt{'Random'}`.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function if :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Initial Points'** : int
Default :math:`= {0}`
The initial number of interpolation points in :math:`Y_0` `(9) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fff.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Initial Points'} = 0`, the number of points is chosen to be equal to the total number of interpolation points set by 'DFO Number Interp Points'.
If this parameter is chosen to be lower than the maximum set by 'DFO Number Interp Points', the solver will progressively increase the number of interpolation points until it reaches that value.
In this release, it is only possible to grow the interpolation set if 'DFO Number Interp Points' is set to the default value.
Constraint: :math:`\text{‘DFO Number Initial Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`0\leq \text{‘DFO Number Initial Points'}\leq \text{‘DFO Number Interp Points'}`.
If :math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`, 'DFO Number Interp Points' must be set to the default value.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fff.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model or build the underdetermined models when the interpolation set has not fully grown (:math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`).
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Version'** : str
Default :math:`= {\texttt{'Latest'}}`
At Mark 27, the underlying algorithm of ``handle_solve_dfls`` underwent significant changes.
This option allows you to continue using the Mark 26 version if it is set to ''26''.
By default (recommended), the latest version of the code is called.
Constraint: :math:`\text{‘DFO Version'} = \texttt{'Latest'}` or :math:`\texttt{'26'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04ff-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is greater than the maximum.
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`6`)
Initial number of interpolation points :math:`\textit{ninit} = \langle\mathit{\boldsymbol{value}}\rangle`, total number of interpolation points :math:`\textit{npts} = \langle\mathit{\boldsymbol{value}}\rangle`, number of variables :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: growing interpolation set is only supported for linear models (:math:`\textit{npts} = \textit{nvar}+1`).
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`8`)
Maximization is not possible for a nonlinear least squares problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points were not provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
A rescue procedure has been called in order to correct damage from rounding errors when computing an update to a quadratic approximation of :math:`F`, but no further progress could be made. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`20`)
User requested termination after a call to the objective function.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04ff-py2-py-notes:
**Notes**
``handle_solve_dfls`` is aimed at minimizing a sum of squares objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & \sum_{{i = 1}}^{m_r} {r_i\left(x\right)}^2 \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here the :math:`r_i\left(x\right)` are smooth nonlinear functions called residuals and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
Typically, in a calibration or data fitting context, the residuals will be defined as the difference between the data points and a nonlinear model (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optclasses>`__).
``handle_solve_dfls`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnls` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfls`` always assumes that the Jacobian of the residuals is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnls` will have no effect.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fff.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04ff-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFLS (Derivative-free Least Squares).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfls`` is the same as the one used by :meth:`handle_solve_dfls_rcomm`. ``handle_solve_dfls`` serves as a forward communication interface to the derivative-free solver for nonlinear least squares problems.
.. _e04ff-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Cartis, C and Roberts, L, 2017, `A Derivative-Free Gauss-Newton Method`
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
Zhang, H, Conn, A R and Scheinberg, K, 2010, `A Derivative-Free Algorithm for Least-Squares Minimization`, SIAM J. Optim. (20(6)), 3555--3576
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_dfls_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfls_rcomm(handle, irevcm, x, rx, io_manager=None):
r"""
``handle_solve_dfls_rcomm`` is a reverse communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFLS) for small to medium-scale nonlinear least squares problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04fg-py2-py-doc:
For full information please refer to the NAG Library document for e04fg
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fgf.html
.. _e04fg-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfls_rcomm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**irevcm** : int
Does not need to be set on the first call of ``handle_solve_dfls_rcomm``. On subsequent calls, :math:`\mathrm{irevcm}` must be set to a positive integer if all the required function evaluations have been correctly provided in :math:`\mathrm{rx}`. Otherwise, if a problem occurred during a monitoring step or while providing objective values, it is possible to set it to a negative value:
:math:`\mathrm{irevcm} = -1`
If function evaluations were required, the solver will attempt a rescue procedure and request an alternative point. If no function were required (monitoring step), the solver will stop with :math:`\mathrm{errno}` = 20.
:math:`\mathrm{irevcm} \leq -2`
The solver will cleanly exit and return the best available point as a well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
The first column contains :math:`x_0`, the initial estimates of the variables :math:`x`.
**rx** : float, array-like, shape :math:`\left(\textit{nres}, \textit{maxeval}\right)`
Does not need to be set on the first call to ``handle_solve_dfls_rcomm``.
If :math:`\mathrm{irevcm} = 1` after the last call of ``handle_solve_dfls_rcomm``, the first :math:`\mathrm{neval}` columns must contain the residuals of the requested points.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**irevcm** : int
Indicates what action is to be performed before the next call to ``handle_solve_dfls_rcomm``.
:math:`\mathrm{irevcm} = 0`
Final exit of the solver.
:math:`\mathrm{irevcm} = 1`
:math:`\mathrm{neval}` objective evaluations are required.
:math:`\mathrm{irevcm} = 2`
Monitoring step, no evaluation is required, :math:`\mathrm{x}` and :math:`\mathrm{rx}` contain the best evaluation of the objective yet.
**neval** : int
Indicates the number of objective evaluations required for the next call of ``handle_solve_dfls_rcomm`` in :math:`\mathrm{rx}`. The coordinates of the points to evaluate are provided in the first :math:`\mathrm{neval}` columns of :math:`\mathrm{x}`.
**x** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the best computed estimate of the solution.
If :math:`\mathrm{irevcm} = 1`, the first :math:`\mathrm{neval}` columns contain the coordinates of the points to evaluate.
**rx** : float, ndarray, shape :math:`\left(\textit{nres}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the residuals of the best computed point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+------------+--------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)` (sum of the squared residuals).|
+------------+--------------------------------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+------------+--------------------------------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+------------+--------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+--------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04fg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFLS Small Residuals Tol'** : float
Default :math:`= \epsilon^{0.75}`
This option defines the tolerance on the value of the residuals.
Namely, the solver declares convergence if
.. math::
f\left(x\right) = \sum_{{i = 1}}^{m_r}{r_i\left(x\right)}^2 < \text{‘DFLS Small Residuals Tol'}\text{.}
Constraint: :math:`\text{‘DFLS Small Residuals Tol'} > \epsilon^2`.
**'DFO Initial Interp Points'** : str
Default :math:`= \texttt{'Coordinate'}`
Determines how the initial interpolation points are chosen.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}`, the interpolation points are chosen along the coordinate directions around the initial point.
If :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Random'}`, the initial interpolation points are chosen along random orthogonal directions around the initial point.
Set 'DFO Random Seed' to a positive value to fix the random seed and get reproducible results.
Constraint: :math:`\text{‘DFO Initial Interp Points'} = \texttt{'Coordinate'}` or :math:`\texttt{'Random'}`.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, the solver will stop at the end of every :math:`i`\ th step for monitoring purposes. ``handle_solve_dfls_rcomm`` needs to be called again to continue the optimization.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Initial Points'** : int
Default :math:`= {0}`
The initial number of interpolation points in :math:`Y_0` `(1) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fgf.html#eqn1>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Initial Points'} = 0`, the number of points is chosen to be equal to the total number of interpolation points set by 'DFO Number Interp Points'.
If this parameter is chosen to be lower than the maximum set by 'DFO Number Interp Points', the solver will progressively increase the number of interpolation points until it reaches that value.
In this release, it is only possible to grow the interpolation set if 'DFO Number Interp Points' is set to the default value.
Constraint: :math:`\text{‘DFO Number Initial Points'} \geq 0`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`0\leq \text{‘DFO Number Initial Points'}\leq \text{‘DFO Number Interp Points'}`.
If :math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`, 'DFO Number Interp Points' must be set to the default value.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(1) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fgf.html#eqn1>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model or build the underdetermined models when the interpolation set has not fully grown (:math:`\text{‘DFO Number Initial Points'} < \text{‘DFO Number Interp Points'}`).
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04fg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{maxeval} > 0`.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the first call of the function, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is greater than the maximum.
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`6`)
Initial number of interpolation points :math:`\textit{ninit} = \langle\mathit{\boldsymbol{value}}\rangle`, total number of interpolation points :math:`\textit{npts} = \langle\mathit{\boldsymbol{value}}\rangle`, number of variables :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: growing interpolation set is only supported for linear models (:math:`\textit{npts} = \textit{nvar}+1`).
Use 'DFO Number Interp Points' and 'DFO Number Initial Points' to control the number of interpolation points.
(`errno` :math:`8`)
Maximization is not possible for a nonlinear least squares problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points were not provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`20`)
User requested termination during an objective evaluation step.
.. _e04fg-py2-py-notes:
**Notes**
``handle_solve_dfls_rcomm`` uses reverse communication for function evaluations and monitoring steps.
Every time the solver requires an evaluation of the objective function, it pauses its progress, exits and waits for the function to be called again with the objective value provided in the argument :math:`\mathrm{rx}`.
``handle_solve_dfls_rcomm`` is aimed at minimizing a sum of squares objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & \sum_{{i = 1}}^{m_r} {r_i\left(x\right)}^2 \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here the :math:`r_i\left(x\right)` are smooth nonlinear functions called residuals and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
Typically, in a calibration or data fitting context, the residuals will be defined as the difference between the data points and a nonlinear model (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optclasses>`__).
``handle_solve_dfls_rcomm`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnls` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfls_rcomm`` always assumes that the Jacobian of the residuals is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnls` will have no effect.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fgf.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04fg-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFLS (Derivative-free Least Squares).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfls_rcomm`` is the same as the one used by :meth:`handle_solve_dfls`. ``handle_solve_dfls_rcomm`` serves as a reverse communication interface to the derivative-free solver for nonlinear least squares problems.
.. _e04fg-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Cartis, C and Roberts, L, 2017, `A Derivative-Free Gauss-Newton Method`
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
Zhang, H, Conn, A R and Scheinberg, K, 2010, `A Derivative-Free Algorithm for Least-Squares Minimization`, SIAM J. Optim. (20(6)), 3555--3576
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_func_easy(m, lsfun1, x, data=None):
r"""
``lsq_uncon_mod_func_easy`` is an easy-to-use algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
No derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04fy-py2-py-doc:
For full information please refer to the NAG Library document for e04fy
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fyf.html
.. _e04fy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun1** : callable fvec = lsfun1(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_func_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04fy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`400\times n` calls to :math:`\mathrm{lsfun1}`.
(`errno` :math:`4`)
Failure in computing SVD of estimated Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04fy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_func_easy`` is essentially identical to the function LSNDN1 in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}F\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate functions :math:`f_i\left(x\right)` at any point :math:`x`.
From a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04fy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_quasi_deriv_comp(m, selct, lsqfun, eta, xtol, x, lsqmon=None, iprint=1, maxcal=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_quasi_deriv_comp`` is a comprehensive quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. deprecated:: 28.3.0.0
``lsq_uncon_quasi_deriv_comp`` is deprecated.
Please use :meth:`handle_solve_bxnl` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04gb-py2-py-doc:
For full information please refer to the NAG Library document for e04gb
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gbf.html
.. _e04gb-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**selct** : int
:math:`\mathrm{selct}` enables you to specify whether the linear minimizations (i.e., minimizations of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`) are to be performed by a function which just requires the evaluation of the :math:`f_i\left(x\right)` (:math:`\mathrm{selct} = 1`), or by a function which also requires the first derivatives of the :math:`f_i\left(x\right)` (:math:`\mathrm{selct} = 2`).
It will often be possible to evaluate the first derivatives of the residuals in about the same amount of computer time that is required for the evaluation of the residuals themselves -- if this is so, then ``lsq_uncon_quasi_deriv_comp`` should be called with :math:`\mathrm{selct} = 2`.
However, if the evaluation of the derivatives takes more than about :math:`4` times as long as the evaluation of the residuals, :math:`\mathrm{selct} = 1` will usually be preferable.
If in doubt, use :math:`\mathrm{selct} = 2` as it is slightly more robust.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_quasi_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Will be set to :math:`0`, :math:`1` or :math:`2`.
:math:`\mathrm{iflag} = 0`
Indicates that only the residuals need to be evaluated
:math:`\mathrm{iflag} = 1`
Indicates that only the Jacobian matrix needs to be evaluated
:math:`\mathrm{iflag} = 2`
Indicates that both the residuals and the Jacobian matrix must be calculated.
If :math:`\mathrm{selct} = 2`, :math:`\mathrm{lsqfun}` will always be called with :math:`\mathrm{iflag}` set to :math:`2`.
**m** : int
:math:`m`, the number of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it is wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_quasi_deriv_comp``. ``lsq_uncon_quasi_deriv_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag} = 1` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[i-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag} = 0` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**eta** : float
Every iteration of ``lsq_uncon_quasi_deriv_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_quasi_deriv_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made every iteration.
On balance it is usually more efficient to perform a low accuracy minimization.
`Suggested value`:
:math:`\mathrm{eta} = 0.9` if :math:`n > 1` and :math:`\mathrm{selct} = 2`,
:math:`\mathrm{eta} = 0.5` if :math:`n > 1` and :math:`\mathrm{selct} = 1`,
:math:`\mathrm{eta} = 0.0` if :math:`n = 1`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gbf.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gbf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_quasi_deriv_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**igrade** : int
``lsq_uncon_quasi_deriv_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_quasi_deriv_comp``.
**nf** : int
The number of evaluations of the residuals. (If :math:`\mathrm{selct} = 2`, :math:`\mathrm{nf}` is also the number of evaluations of the Jacobian matrix.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_quasi_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`\mathrm{selct} = 1`: :math:`{ 75\times n }`; otherwise: :math:`{ 50\times n }`.
Enables you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_quasi_deriv_comp``. There will be an error exit (see :ref:`Exceptions <e04gb-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.)
``lsq_uncon_quasi_deriv_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``lsq_uncon_quasi_deriv_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_quasi_deriv_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., the number of calls of :math:`\mathrm{lsqfun}`). If :math:`\mathrm{selct} = 2`, :math:`\mathrm{nf}` is also the number of times that the Jacobian matrix has been evaluated.
.. _e04gb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{selct} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{selct} = 1` or :math:`2`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
**NagAlgorithmicMajorWarning**
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
.. _e04gb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_uncon_quasi_deriv_comp`` is essentially identical to the function LSQFDQ in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise the second derivatives of the :math:`f_i\left(x\right)` are taken into account using a quasi-Newton updating scheme.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04gb-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv_comp(m, lsqfun, xtol, x, lsqmon=None, iprint=1, maxcal=None, eta=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv_comp`` is a comprehensive modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04gd-py2-py-doc:
For full information please refer to the NAG Library document for e04gd
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gdf.html
.. _e04gd-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`1` or :math:`2`.
:math:`\mathrm{iflag} = 1`
Indicates that only the Jacobian matrix needs to be evaluated
:math:`\mathrm{iflag} = 2`
Indicates that both the residuals and the Jacobian matrix must be calculated
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_mod_deriv_comp``. ``lsq_uncon_mod_deriv_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag} = 1` on entry, or :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[i-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gdf.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gdf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_mod_deriv_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem. (If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called at the initial point before the singular values have been calculated. So the elements of :math:`\mathrm{s}` are set to zero for the first call of :math:`\mathrm{lsqmon}`.)
**igrade** : int
``lsq_uncon_mod_deriv_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far with :math:`\mathrm{iflag} = 2`. (In addition to these calls monitored by :math:`\mathrm{nf}`, :math:`\mathrm{lsqfun}` is called not more than :math:`\textit{n}` times per iteration with :math:`\mathrm{iflag}` set to :math:`1`.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
The frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
Enables you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_mod_deriv_comp``. There will be an error exit (see :ref:`Exceptions <e04gd-py2-py-errors>`) after :math:`\mathrm{maxcal}` evaluations of the residuals (i.e., calls of :math:`\mathrm{lsqfun}` with :math:`\mathrm{iflag}` set to :math:`2`). It should be borne in mind that, in addition to the calls of :math:`\mathrm{lsqfun}` which are limited directly by :math:`\mathrm{maxcal}`, there will be calls of :math:`\mathrm{lsqfun}` (with :math:`\mathrm{iflag}` set to :math:`1`) to evaluate only first derivatives.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_deriv_comp`` involves a linear minimization, i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`. :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations, they will tend to increase the number of calls of :math:`\mathrm{lsqfun}` (with :math:`\mathrm{iflag}` set to :math:`2`) needed for each linear minimization.
On balance it is usually efficient to perform a low accuracy linear minimization.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.) ``lsq_uncon_mod_deriv_comp`` will ensure that, for each iteration,
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2
where :math:`k` is the iteration number. Thus, if the problem has more than one solution, ``lsq_uncon_mod_deriv_comp`` is most likely to find the one nearest to the starting point. On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`. However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv_comp``.
**nf** : int
The number of times that the residuals have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}` with :math:`\mathrm{iflag}` set to :math:`2`).
.. _e04gd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` evaluations of the residuals.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
.. _e04gd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv_comp`` is essentially identical to the function LSQFDN in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise finite difference estimates of the second derivatives of the :math:`f_i\left(x\right)` are taken into account.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04gd-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def handle_solve_bxnl(handle, lsqfun, lsqgrd, x, nres, lsqhes=None, lsqhprd=None, monit=None, data=None, io_manager=None, spiked_sorder='C'):
r"""
``handle_solve_bxnl`` is a bound-constrained nonlinear least squares trust region solver (BXNL) from the NAG optimization modelling suite aimed for small to medium-scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04gg-py2-py-doc:
For full information please refer to the NAG Library document for e04gg
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html
.. _e04gg-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_bxnl``. It **must not** be changed between calls to the NAG optimization modelling suite.
**lsqfun** : callable (rx, inform) = lsqfun(x, nres, inform, data=None)
:math:`\mathrm{lsqfun}` must evaluate the value of the nonlinear residuals, :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right),i = 1,\ldots,n_{\mathrm{res}}`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residual vector, :math:`r\left(x\right)`, evaluated at :math:`x`.
**inform** : int
May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqgrd** : callable inform = lsqgrd(x, nres, rdx, inform, data=None)
:math:`\mathrm{lsqgrd}` evaluates the residual gradients, :math:`\nabla r_i\left(x\right)`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residual gradients, :math:`\nabla r_i\left(x\right)`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**rdx** : float, ndarray, shape :math:`\left(\textit{nnzrd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the vector containing the nonzero residual gradients evaluated at :math:`x`,
.. math::
\nabla r\left(x\right) = \left[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\ldots,\nabla r_{n_{\textit{res}}}\left(x\right)\right]\text{,}
where
.. math::
\nabla r_i\left(x\right) = \left[\frac{{\partial r_i\left(x\right)}}{{\partial x_1}},\ldots,\frac{{\partial r_i\left(x\right)}}{{\partial x_{n_{\textit{var}}}}}\right]^\mathrm{T}\text{.}
The elements must be stored in the same order as the defined sparsity pattern provided in the call to :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**lsqhes** : None or callable inform = lsqhes(x, lamda, hx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{lsqhes}` evaluates the residual Hessians, :math:`\nabla^2r_i\left(x\right)`, at a specified point :math:`x`.
By default, the option :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'NO'}` and :math:`\mathrm{lsqhes}` is never called. :math:`\mathrm{lsqhes}` may be **None**.
This function will only be called if the option :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` and if the model (see `Models <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#models>`__) requires second order information.
Under these circumstances, if you do not provide a valid :math:`\mathrm{lsqhes}` the solver will terminate with either :math:`\mathrm{errno}` = 6 or :math:`\mathrm{errno}` = 21.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**lamda** : float, ndarray, shape :math:`\left(\textit{nres}\right)`
:math:`\lambda`, the vector containing the (weighted) residuals at :math:`x`, :math:`\lambda_i = w_ir_i\left(x\right)`. See `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and `Residual Weights <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#residualweights>`__.
**hx** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{nvar}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: a dense square (symmetric) matrix containing the weighted sum of residual Hessians,
.. math::
H\left(x\right) = \sum_{{i = 1}}^{n_{\textit{res}}}\lambda_i\nabla^2r_i\left(x\right)\text{,}
where
.. math::
\nabla^2r_i\left(x\right) = \begin{pmatrix} \frac{\partial^2}{{\partial_{x_1}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_1}\partial_{x_2}}} r_i \left(x\right) &\ldots & \frac{\partial^2}{{\partial_{x_1}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \\ \frac{\partial^2}{{\partial_{x_2}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_2}\partial_{x_2}}} r_i \left(x\right) &…& \frac{\partial^2}{{\partial_{x_2}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \\⋮&⋮&⋱&⋮\\ \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_1}}} r_i \left(x\right) & \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_2}}} r_i \left(x\right) &…& \frac{\partial^2}{{\partial_{x_{n_{\mathrm{var}}}}\partial_{x_{n_{\mathrm{var}}}}}} r_i \left(x\right) \end{pmatrix}\text{,}
is also a dense square (symmetric) matrix containing the :math:`i`\ th residual Hessian evaluated at the point :math:`x`. All matrix elements must be provided: both upper and lower triangular parts.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqhprd** : None or callable (hxy, inform) = lsqhprd(x, y, hxy, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{lsqhprd}` evaluates the residual Hessians, :math:`\nabla^2r_i\left(x\right)`, at a specified point, :math:`x`, and performs matrix-vector products with a given vector, :math:`y`, returning the dense matrix :math:`\left[\nabla^2r_1\left(x\right)y,\nabla^2r_2\left(x\right)y,\ldots,\nabla^2r_{n_{\textit{res}}}\left(x\right)y\right]`.
If you do not supply this function, it may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**y** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`y`, the vector used to perform the required matrix-vector products.
**hxy** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{nres}\right)`
The elements should only be assigned and not referenced.
**inform** : int
The first call to :math:`\mathrm{lsqhprd}` will have a non-zero value and can be used to optimize your code in order to avoid recalculations of common quantities when evaluating the Hessians. For all other instances :math:`\mathrm{inform}` will have a value of zero. This notification argument may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**hxy** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{nres}\right)`
A dense matrix of size :math:`n_{\textit{var}}\times n_{\textit{res}}` containing the following matrix-vector products,
.. math::
H\left(x,y\right) = \left[\nabla^2r_1\left(x\right)y,\nabla^2r_2\left(x\right)y,\ldots,\nabla^2r_{n_{\textit{res}}}\left(x\right)y\right]\text{.}
**inform** : int
May be used to indicate that one or more elements of the residual Hessian could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and if the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25. The value of :math:`\mathrm{inform}` returned on the first call is ignored.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'Bxnl Monitor Frequency' (the default value is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**spiked_sorder** : str, optional
If :math:`\mathrm{hxy}` in :math:`\mathrm{lsqhprd}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value, :math:`f\left(x\right)`. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Norm of the projected gradient at the current iterate, see **PG STEP** in `Bound Constraints <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ggf.html#subprob_bcon>`__ and `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ggf.html#stoppingcriteria>`__.|
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Norm of the scaled projected gradient at the current iterate, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ggf.html#stoppingcriteria>`__ |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Norm of the step between the current and previous iterate. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Convergence tests result. A scalar value between :math:`0-7` indicates whether a convergence test has passed. Specifically, :math:`1` indicates small objective test passed, :math:`2` indicates small (scaled) gradient test passed, :math:`4` indicates small step test passed. In the case where two or more tests passed, they are accumulated. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Norm of the current iterate :math:`x`. If regularization is requested, then this value was used in the regularization and it might differ from :math:`\left\lVert x\right\rVert` if :math:`x` has fixed or disabled elements. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6`--:math:`99`|Reserved for future use. |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of calls to the objective function :math:`\mathrm{lsqfun}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of calls to the objective gradient function :math:`\mathrm{lsqgrd}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`3` |Total number of calls to the objective Hessian function :math:`\mathrm{lsqhes}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`4` |Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of calls to the objective function :math:`\mathrm{lsqfun}` required by linesearch steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`6` |Number of calls to the objective gradient function :math:`\mathrm{lsqgrd}` required by linesearch steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`7` |Number of calls to the objective function :math:`\mathrm{lsqfun}` required by projected gradient steps. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`8` |Number of calls to the objective gradient function :math:`\mathrm{lsqgrd}` required by projected gradient steps.|
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`9` |Number of inner iterations performed, see option :math:`\text{‘Bxnl Model'} = \texttt{'TENSOR-NEWTON'}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`10` |Number of linesearch iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`11` |Number of projected gradient iterations performed. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`12` |Total number of calls to the objective auxiliary Hessian function :math:`\mathrm{lsqhprd}`. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
|:math:`13`--:math:`99`|Reserved for future use. |
+----------------------+----------------------------------------------------------------------------------------------------------------+
.. _e04gg-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Bxnl Basereg Pow'** : float
Default :math:`= 2.0`
This argument defines the regularization power :math:`p` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and for the tensor Newton subproblem (when :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
Some values are restricted depending on the type of regularization specified, see 'Bxnl Basereg Type' for more details.
Constraint: :math:`\text{‘Bxnl Basereg Pow'} > 0`.
**'Bxnl Basereg Term'** : float
Default :math:`= 0.01`
This argument defines the regularization term :math:`\sigma` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and for the tensor Newton subproblem (when :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
Constraint: :math:`\text{‘Bxnl Basereg Term'} > 0`.
**'Bxnl Basereg Type'** : str
Default :math:`= \texttt{'NONE'}`
This argument specifies the method used to incorporate the regularizer into `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:lsq>`__ and optionally into the tensor Newton subproblem (when :math:`\text{‘Bxnl Model'} = \texttt{'Tensor-Newton'}` and :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`).
The option :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-NVAR-DOF'}` reformulates the original problem by expanding it with :math:`n_{\textit{var}}` degrees of freedom that is subsequently solved.
For the case :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-1-DOF'}` the residual vector is extended with a new term of the form :math:`\frac{\sigma }{p}\left\lVert x\right\rVert_2^p`; for this method a value of :math:`p = 3` is recommended.
If :math:`\text{‘Bxnl Basereg Type'} = \texttt{'EXPAND-NVAR-DOF'}` then the regularization power term :math:`p` must be :math:`2.0`, that is :math:`\text{‘Bxnl Basereg Pow'} = \texttt{'2.0'}`.
For further details see `Subproblems <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#subprob>`__.
Constraint: :math:`\text{‘Bxnl Basereg Type'} = \texttt{'NONE'}`, :math:`\texttt{'EXPAND-NVAR-DOF'}` or :math:`\texttt{'EXPAND-1-DOF'}`.
**'Bxnl Save Covariance Matrix'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates to the solver to store the covariance matrix into the handle.
If :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'YES'}` then the lower triangle part of the covariance matrix is stored in packed column order (see `the F07 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f07/f07intro.html#recomm_32>`__) into the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'COVARIANCE MATRIX'}` with :math:`{\textit{lrarr}} = \left(n_{\textit{var}}\times \left(n_{\textit{var}}+1\right)\right)/2`.
In the special case where :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'VARIANCE'}`, only the diagonal elements of the covariance matrix are stored in the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'VARIANCE'}` with :math:`{\textit{lrarr}} = n_{\textit{var}}`.
Similarly, if :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'HESSIAN'}` then the lower triangle part of the matrix :math:`H\left(x\right) = \nabla r\left(x\right)\left. \nabla r\left(x\right)\right.^\mathrm{T} = \left. J\left(x\right)\right.^\mathrm{T}J\left(x\right)` is stored in packed column order into the handle and can be retrieved via :meth:`handle_set_get_real` using :math:`{\textit{cmdstr}} = \texttt{'HESSIAN MATRIX'}` with :math:`{\textit{lrarr}} = \left(n_{\textit{var}}\times \left(n_{\textit{var}}+1\right)\right)/2`.
**Limitations:** If the number of enabled residuals is not greater than the number of enabled variables, or the pseudo-inverse of :math:`H\left(x\right)` could not be calculated, then the covariance matrix (variance vector) is not stored in the handle and will not be available.
For more information on how the covariance matrix is estimated, see :meth:`lsq_uncon_covariance`.
Constraint: :math:`\text{‘Bxnl Save Covariance Matrix'} = \texttt{'NO'}`, :math:`\texttt{'YES'}`, :math:`\texttt{'VARIANCE'}` or :math:`\texttt{'HESSIAN'}`.
**'Bxnl Stop Abs Tol Fun'** : float
Default :math:`\text{} = 2.2\epsilon^{\frac{1}{3}}`
This argument specifies the relative tolerance for the error test, specifically, it sets the value of :math:`\epsilon_{\textit{abs}}^f` of equation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stopcrit1>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Abs Tol Fun' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Abs Tol Fun'} > 0`.
**'Bxnl Stop Abs Tol Grd'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of :math:`\epsilon_{\textit{abs}}^g` of equation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Abs Tol Grd' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Abs Tol Grd'} > 0`.
**'Bxnl Stop Rel Tol Fun'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the error test, specifically, it sets the value of :math:`\epsilon_{\textit{rel}}^f` of equation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stopcrit1>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Rel Tol Fun' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Rel Tol Fun'} > 0`.
**'Bxnl Stop Rel Tol Grd'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{2}}`
This argument specifies the relative tolerance for the gradient test, specifically, it sets the value of :math:`\epsilon_{\textit{rel}}^g` of equation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Rel Tol Grd' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Constraint: :math:`\text{‘Bxnl Stop Rel Tol Grd'} > 0`.
**'Bxnl Stop Step Tol'** : float
Default :math:`\text{} = 2\epsilon`
Specifies the stopping tolerance for the step length test, specifically, it sets the value for :math:`\epsilon_{\textit{step}}` of equation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stopcrit3>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#stoppingcriteria>`__.
Setting 'Bxnl Stop Step Tol' to a large value may cause the solver to stop prematurely with a suboptimal solution.
Under certain circumstances, e.g., when in doubt of the quality of the first - or second-order derivatives, in the event of the solver exiting with a successful step length test, it is recommended to verify that either the error or the gradient norm is acceptably small.
Constraint: :math:`\text{‘Bxnl Stop Step Tol'} > 0`.
**'Bxnl Reg Order'** : str
Default :math:`\text{} = \texttt{'AUTO'}`
This argument specifies the order of the regularization :math:`p` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:reg>`__ used when :math:`\text{‘Bxnl Glob Method'} = \texttt{'Reg'}`.
Some values for :math:`p` are restricted depending on the method chosen in 'Bxnl Nlls Method', see `Regularization <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#subprob_reg>`__ for more details.
Constraint: :math:`\text{‘Bxnl Reg Order'} = \texttt{'AUTO'}`, :math:`\texttt{'QUADRATIC'}` or :math:`\texttt{'CUBIC'}`.
**'Bxnl Glob Method'** : str
Default :math:`= \texttt{'TR'}`
This argument specifies the globalization method used to estimate the next step :math:`s_k`.
It also determines the class of subproblem to solve.
The trust region subproblem finds the step by minimizing the specified model withing a given radius.
On the other hand, when :math:`\text{‘Bxnl Glob Method'} = REG`, the problem is reformulated by adding an aditional regularization term and minimized in order to find the next step :math:`s_k`.
See `Subproblems <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#subprob>`__ for more details.
Constraint: :math:`\text{‘Bxnl Glob Method'} = \texttt{'TR'}` or :math:`\texttt{'REG'}`.
**'Bxnl Nlls Method'** : str
Default :math:`= \texttt{'GALAHAD'}`
This argument defines the method used to estimate the next step :math:`s_k` in :math:`x_{{k+1}} = x_k+s_k`.
It only applies to :math:`\text{‘Bxnl Model'} = \texttt{'GAUSS-NEWTON'}`, :math:`\texttt{'QUASI-NEWTON'}` or :math:`\texttt{'HYBRID'}`.
When the globalization technique chosen is trust region (:math:`\text{‘Bxnl Glob Method'} = TR`) the methods for 'Bxnl Nlls Method' available are Powell's dogleg method, a generalized eigenvalue method (AINT) Adachi `et al.` (2015), a variant of Moré--Sorensen's method, and GALAHAD's DTRS method.
Otherwise, when the globalization method chosen is via regularization (:math:`\text{‘Bxnl Glob Method'} = REG`) the methods available are comprised by a linear system solver and GALAHAD's DRQS method.
See `Subproblems <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#subprob>`__ for more details.
Constraint: :math:`\text{‘Bxnl Nlls Method'} = \texttt{'POWELL-DOGLEG'}`, :math:`\texttt{'AINT'}`, :math:`\texttt{'MORE-SORENSEN'}`, :math:`\texttt{'LINEAR SOLVER'}` or :math:`\texttt{'GALAHAD'}`.
**'Bxnl Model'** : str
Default :math:`= \texttt{'HYBRID'}`
This argument specifies which model is used to approximate the objective function and estimate the next point that reduces the error.
This is one of the most important options and should be chosen according to the problem characteristics.
The models are briefly described in `Models <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#models>`__.
Constraint: :math:`\text{‘Bxnl Model'} = \texttt{'GAUSS-NEWTON'}`, :math:`\texttt{'QUASI-NEWTON'}`, :math:`\texttt{'HYBRID'}` or :math:`\texttt{'TENSOR-NEWTON'}`.
**'Bxnl Tn Method'** : str
Default :math:`= \texttt{'MIN-1-VAR'}`
This argument specifies how to solve the subproblem and find the next step :math:`s_k` for the tensor Newton model, :math:`\text{‘Bxnl Model'} = \texttt{'TENSOR-NEWTON'}`.
The subproblems are solved using a range of regularization schemes.
See `Tensor Newton subproblem <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#tensor_newton>`__.
Constraint: :math:`\text{‘Bxnl Tn Method'} = \texttt{'IMPLICIT'}`, :math:`\texttt{'MIN-1-VAR'}`, :math:`\texttt{'MIN-NVAR'}`, :math:`\texttt{'ADD-1-VAR'}` or :math:`\texttt{'ADD-NVAR'}`.
**'Bxnl Use Second Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates whether the weighted sum of residual Hessians are available through the call-back :math:`\mathrm{lsqhes}`.
If :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'NO'}` and the specified model in 'Bxnl Model' requires user-suppied second derivatives, then the solver will terminate with :math:`\mathrm{errno}` = 6.
Constraint: :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Bxnl Use Weights'** : str
Default :math:`= \texttt{'NO'}`
This argument indicates whether to use a weighted nonlinear least square model.
If :math:`\text{‘Bxnl Use Weights'} = \texttt{'YES'}` then the weights :math:`w_i > 0,i = 1,\ldots,n_{\textit{res}}` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#eq:lsq11>`__ must be supplied by you via :meth:`handle_set_get_real`.
If weights are to be used, then all :math:`n_{\textit{res}}` elements must be provided, see `Residual Weights <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#residualweights>`__.
If the solver is expecting to use weights but they are not provided or have non-positive values, then the solver will terminate with :math:`\mathrm{errno}` = 11.
Constraint: :math:`\text{‘Bxnl Use Second Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Bxnl Iteration Limit'** : int
Default :math:`= 1000`
This argument specifies the maximum amount of major iterations the solver is alloted.
If this limit is reached, then the solver will terminate with :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘Bxnl Iteration Limit'} \geq 1`.
**'Bxnl Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘Bxnl Monitor Frequency'} > 0`, the user-supplied function :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘Bxnl Monitor Frequency'} \geq 0`.
**'Bxnl Print Header'** : int
Default :math:`= 30`
This argument defines, in number of iterations, the frequency with which to print the iteration log header.
Constraint: :math:`\text{‘Bxnl Print Header'} \geq 1`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as for 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timing.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If at the end of an iteration this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04gg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`3`)
Unsupported option combinations.
(`errno` :math:`3`)
Unsupported model and method chosen.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the definition of the objective in the :math:`\mathrm{handle}`, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
There are no decision variables. :math:`\textit{nvar}` must be greater than zero.
(`errno` :math:`6`)
Exact second derivatives needed for tensor model.
(`errno` :math:`11`)
Data for residual weights not found or is invalid.
(`errno` :math:`21`)
The current starting point is unusable.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`18`)
Numerical difficulties encountered and solver was terminated.
(`errno` :math:`19`)
Iteration limit reached while solving a subproblem.
(`errno` :math:`19`)
Line Search failed.
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user-supplied function and recovery failed.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04gg-py2-py-notes:
**Notes**
``handle_solve_bxnl`` computes a solution :math:`x` to the nonlinear least squares problem
.. math::
\begin{array}{ll} \textit{minimize}_{{x \in ℝ^{n_{\textit{var}}}}} & f\left(x\right) = \frac{1}{2} \sum_{{i = 1}}^{n_{\textit{res}}} {\left[w_ir_i\left(x\right)\right]}^2 + \frac{\sigma }{p} \left\lVert x\right\rVert_2^p \\\text{subject to}& l_x \leq x \leq u_x \text{,} \end{array}
where :math:`r_i\left(x\right),i = 1,\ldots,n_{\textit{res}}`, are smooth nonlinear functions called residuals, :math:`w_i,i = 1,\ldots,n_{\textit{res}}` are weights (by default they are all defined to :math:`1`, see `Residual Weights <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#residualweights>`__ on how to change them), and the rightmost element represents the regularization term with argument :math:`\sigma \geq 0` and power :math:`p > 0` (by default the regularization term is not used, see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#algdetails>`__ on how to enable it).
The constraint elements :math:`l_x` and :math:`u_x` are :math:`n_{\textit{var}}`-dimensional vectors defining the bounds on the variables.
Typically in a calibration or data fitting context, the residuals will be defined as the difference between the observed values :math:`y_i` at :math:`t_i` and the values provided by a nonlinear model :math:`\phi \left(t; x\right)`, i.e., :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right)`.
If these residuals (errors) follow a Gaussian distribution, then the values of the optimal parameter vector :math:`x^*` are the maximum likelihood estimates.
For a description of the various algorithms implemented for solving this problem see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#algdetails>`__.
It is also recommended that you read `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optclasses>`__.
``handle_solve_bxnl`` serves as a solver for problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
A nonlinear least square residual objective can be added by calling :meth:`handle_set_nlnls` and, optionally, (simple) box constraints can be defined by calling :meth:`handle_set_simplebounds`.
It should be noted that ``handle_solve_bxnl`` internally works with a dense representation of the residual Jacobian even if a sparse structure was defined in the call to :meth:`handle_set_nlnls`.
Once the problem is fully described, the handle may be passed to the solver ``handle_solve_bxnl``.
When the handle is not needed anymore, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
For more information refer to the NAG optimization modelling suite in `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__.
The algorithm is based on the trust region framework and its behaviour can be modified by various options (see :ref:`Other Parameters <e04gg-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
Several options might have significant impact on the performance of the solver.
Even though the defaults were chosen to suit the majority of anticipated problems, it is recommended that you experiment with the option settings to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ggf.html#algdetails>`__ and :ref:`Other Parameters <e04gg-py2-py-other_params>` for further details.
.. _e04gg-py2-py-references:
**References**
Adachi, S, Iwata, S, Nakatsukasa, Y, and Takeda, A, 2015, `Solving the trust region subproblem by a generalized eigenvalue problem`, Technical report, METR 2015-14., Mathematical Engineering, The University of Tokyo, https://www.keisu.t.u-tokyo.ac.jp/data/2015/METR15-14.pdf
Conn, A R, Gould, N I M and Toint, Ph L, 2000, `Trust Region Methods`, SIAM, Philadephia
Gould, N I M, Orban, D, and Toint, Ph L, 2003, `GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization`, ACM Transactions on Mathematical Software (TOMS) (29(4)), 353--372
Gould, N I M, Rees, T, and Scott, J A, 2017, `A higher order method for solving nonlinear least-squares problems`, Technical report, RAL-P-1027-010, RAL Library. STFC Rutherford Appleton Laboratory, http://www.numerical.rl.ac.uk/people/rees/pdf/RAL-P-2017-010.pdf
Kanzow, C, Yamashita, N, and Fukushima, M, 2004, `Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints`, Journal of Computational and Applied Mathematics (174), 375--397
Lanczos, C, 1956, `Applied Analysis`, 272--280, Prentice Hall, Englewood Cliffs, NJ, USA
Nielsen, H B, 1999, `Damping parameter in Marquadt’s Method`, Technical report TR IMM-REP-1999-05., Department of Mathematical Modelling, Technical University of Denmark, http://www2.imm.dtu.dk/documents/ftp/tr99/tr05_99.pdf
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_disable_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, confun=None, congrd=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_nldf`` is a solver from the NAG optimization modelling suite for general nonlinear data-fitting problems with constraints.
Various loss and regularization functions are supported.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04gn-py2-py-doc:
For full information please refer to the NAG Library document for e04gn
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html
.. _e04gn-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_nldf``. It **must not** be changed between calls to the NAG optimization modelling suite.
**lsqfun** : callable (rx, inform) = lsqfun(x, nres, inform, data=None)
:math:`\mathrm{lsqfun}` must evaluate the value of the nonlinear residuals, :math:`r_i\left(x\right) \mathrel{:=} y_i-\phi \left(t_i; x\right),i = 1,\ldots,n_{\mathrm{res}}`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residuals, :math:`r_i`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**rx** : float, array-like, shape :math:`\left(\mathrm{nres}\right)`
The value of the residual vector, :math:`r\left(x\right)`, evaluated at :math:`x`.
**inform** : int
May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**lsqgrd** : callable inform = lsqgrd(x, nres, rdx, inform, data=None)
:math:`\mathrm{lsqgrd}` evaluates the residual gradients, :math:`\nabla r_i\left(x\right)`, at a specified point :math:`x`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the residual gradients, :math:`\nabla r_i\left(x\right)`, are to be evaluated.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**rdx** : float, ndarray, shape :math:`\left(\textit{nnzrd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the vector containing the nonzero residual gradients evaluated at :math:`x`,
.. math::
\nabla r\left(x\right) = \left[\nabla r_1\left(x\right),\nabla r_2\left(x\right),\ldots,\nabla r_{n_{\textit{res}}}\left(x\right)\right]\text{,}
where
.. math::
\nabla r_i\left(x\right) = \left[\frac{{\partial r_i\left(x\right)}}{{\partial x_1}},\ldots,\frac{{\partial r_i\left(x\right)}}{{\partial x_{n_{\textit{var}}}}}\right]^\mathrm{T}\text{.}
The elements must be stored in the same order as the defined sparsity pattern provided in the call to :meth:`handle_set_nlnls`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting :math:`\mathrm{inform}` to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 25.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**nres** : int
:math:`n_{\textit{res}}`, the current number of residuals in the model.
**confun** : None or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{confun}` must calculate the values of the :math:`m_g`-element vector :math:`g_i\left(x\right)` of nonlinear constraint functions at a specified value of the :math:`n_{\textit{var}}`-element vector of :math:`x` variables.
If there are no nonlinear constraints then :math:`\mathrm{confun}` will never be called by ``handle_solve_nldf`` and it may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the constraint functions are to be evaluated.
**ncnln** : int
:math:`m_g`, the number of nonlinear constraints, as specified in an earlier call to :meth:`handle_set_nlnconstr`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**gx** : float, array-like, shape :math:`\left(\mathrm{ncnln}\right)`
The :math:`m_g` values of the nonlinear constraint functions at :math:`x`.
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{confun}`. Specifically, if the value is negative, then the value of :math:`\mathrm{gx}` will be discarded and the solver will either attempt to find a different trial point or terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, the solver will proceed normally.
**congrd** : None or callable inform = congrd(x, gdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{congrd}` must calculate the nonzero values of the sparse Jacobian of the nonlinear constraint functions, :math:`\frac{{\partial g_i}}{{\partial x}}`, at a specified value of the :math:`n_{\textit{var}}`-element vector of :math:`x` variables.
If there are no nonlinear constraints, :math:`\mathrm{congrd}` will never be called by ``handle_solve_nldf`` and :math:`\mathrm{congrd}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the Jacobian of the constraint functions is to be evaluated.
**gdx** : float, ndarray, shape :math:`\left(\textit{nnzgd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by :math:`\textit{irowgd}` and :math:`\textit{icolgd}` in an earlier call to :meth:`handle_set_nlnconstr`. :math:`\mathrm{gdx}[\textit{i}-1]` will be the gradient :math:`\frac{{\partial g_j}}{{\partial x_k}}`, where :math:`j = {\textit{irowgd}}[\textit{i}-1]` and :math:`k = {\textit{icolgd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
Must be set to a value describing the action to be taken by the solver on return from :math:`\mathrm{congrd}`. Specifically, if the value is negative the solution of the current problem will terminate immediately with :math:`\mathrm{errno}` = 25; otherwise, computations will continue.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
:math:`\mathrm{monit}` is called at the end of every :math:`i`\ th step where :math:`i` is controlled by the option 'NLDF Monitor Frequency' (if the value is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The current best point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Best objective value computed and various indicators (the values are as described in the main argument :math:`\mathrm{rinfo}`).
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rx** : float, ndarray, shape :math:`\left(\mathrm{nres}\right)`
The values of the residuals at the final point given in :math:`\mathrm{x}`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are:
.. rst-class:: nag-rules-none nag-align-left
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value, :math:`f\left(x\right)` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#eq:df>`__. |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Loss function value, :math:`\sum_{{i = 1}}^{n_{\textit{res}}}\chi \left(r_i\left(x\right)\right)` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#eq:df>`__.|
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Regularization term value, :math:`\sum_{{i = 1}}^{n_{\textit{var}}}\psi \left(x_i\right)` in `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#eq:df>`__. |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Solution optimality measure. |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4-99`|Reserved for future use. |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration:
.. rst-class:: nag-rules-none nag-align-left
+------------+-----------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations performed. |
+------------+-----------------------------------------------------------------------------------------------+
|:math:`1` |Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.|
+------------+-----------------------------------------------------------------------------------------------+
|:math:`2-99`|Reserved for future use. |
+------------+-----------------------------------------------------------------------------------------------+
.. _e04gn-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'NLDF Iteration Limit'** : int
Default :math:`= 10000000`
The maximum number of iterations to be performed by ``handle_solve_nldf``.
If this limit is reached, then the solver will terminate with :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘NLDF Iteration Limit'} \geq 1`.
**'NLDF Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘NLDF Monitor Frequency'} > 0`, the user-supplied function :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘NLDF Monitor Frequency'} \geq 0`.
**'NLDF Stop Tolerance'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This argument sets the value of :math:`\epsilon_{\textit{tol}}` which specifies the tolerance for the optimality measure.
When both loss function and regularization are differentiable, and with only simple bound constraints, the optimality measures are defined by `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit1>`__ and 'NLDF Stop Tolerance' is passed to the solver :meth:`handle_solve_bounds_foas` as 'FOAS Stop Tolerance'.
When any of the loss function or regularization is non-differentiable, or there presents linear, quadratic or general nonlinear constraint, the optimality measure is defined by `(5) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04stf.html#eqn5>`__ and 'NLDF Stop Tolerance' is passed to the solver :meth:`handle_solve_ipopt` as 'Stop Tolerance 1'.
Constraint: :math:`0\leq \epsilon_{\textit{tol}} < 1`.
**'NLDF Loss Function Type'** : str
Default :math:`=`'L2'
This argument sets the loss function type used in the objective function.
Constraint: :math:`\text{‘NLDF Loss Function Type'} = \texttt{'HUBER'}`, :math:`\texttt{'L2'}`, :math:`\texttt{'CAUCHY'}`, :math:`\texttt{'ATAN'}`, :math:`\texttt{'SMOOTHL1'}`, :math:`\texttt{'LINF'}`, :math:`\texttt{'L1'}` or :math:`\texttt{'QUANTILE'}`.
**'Reg Term Type'** : str
Default :math:`=`'OFF'
This argument sets the regularization function type used in the objective function.
**Note:** if there is no residual in the model, regularization will be turned off.
Constraint: :math:`\text{‘Reg Term Type'} = \texttt{'OFF'}`, :math:`\texttt{'L2'}` or :math:`\texttt{'L1'}`.
**'NLDF Huber Function Width'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{\textit{hub}}` defined in the Huber loss function `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#func_huber>`__.
Constraint: :math:`\text{‘NLDF Huber Function Width'} > 0`.
**'NLDF Cauchy Function Sharpness'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{\textit{cau}}` defined in the Cauchy loss function `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#func_cauchy>`__.
Constraint: :math:`\text{‘NLDF Cauchy Function Sharpness'} > 0`.
**'NLDF SmoothL1 Function Width'** : float
Default :math:`\text{} = 1.0`
Sets the parameter :math:`d_{{\textit{sl}1}}` defined in the SmoothL1 loss function `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#func_smoothl1>`__.
Constraint: :math:`\text{‘NLDF SmoothL1 Function Width'} > 0`.
**'NLDF Quantile Parameter'** : float
Default :math:`\text{} = 0.5`
Sets the parameter :math:`d_{\textit{qnt}}` defined in the Quantile loss function `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#func_quantile>`__.
Constraint: :math:`0 < \text{‘NLDF Quantile Parameter'} < 1`.
**'Reg Coefficient'** : float
Default :math:`\text{} = 1.0`
Sets the regularization coefficient :math:`\rho` in the definition of the objective function `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#eq:df>`__. **Note:** if set to :math:`0`, regularization will be turned off.
Constraint: :math:`\text{‘Reg Coefficient'}\geq 0`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as for 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the initialization of the options, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}` or :math:`\texttt{'X'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timing.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If at the end of an iteration this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04gn-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`-199`)
``handle_solve_nldf`` is not available in this implementation.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{nres} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{nres}` must match the current number of residuals defined in the :math:`\mathrm{handle}`.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{confun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{congrd}` function.
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`28`)
The solver terminated with not enough degrees of freedom.
(`errno` :math:`51`)
The solver detected an infeasible problem.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Problem was solved to an acceptable level; full accuracy was not achieved.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user function.
(`errno` :math:`28`)
The solver terminated after failure during line search.
(`errno` :math:`28`)
The solver terminated after an error in the step computation.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04gn-py2-py-notes:
**Notes**
``handle_solve_nldf`` solves a data-fitting problem of the form
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^{n_{\textit{var}}}}} & f\left(x\right) = \sum_{{i = 1}}^{n_{\textit{res}}} \chi \left(r_i\left(x\right)\right) + \rho \sum_{{i = 1}}^{n_{\textit{var}}} \psi \left(x_i\right) \\ \text{subject to} & l_g \leq g\left(x\right) \leq u_g \text{,} \\& \frac{1}{2} x^\mathrm{T} Q_ix + p_i^\mathrm{T}x + s_i \leq 0 \text{, }\quad 1 \leq i \leq m_Q \text{,} \\& l_B \leq Bx \leq u_B \text{,} \\& l_x \leq x \leq u_x \text{,} \end{array}
where
:math:`n_{\textit{var}}` is the number of decision variables,
:math:`m_g` is the number of the nonlinear constraints and :math:`g\left(x\right)`, :math:`l_g` and :math:`u_g` are :math:`m_g`-dimensional vectors,
:math:`m_Q` is the number of quadratic constraints,
:math:`m_B` is the number of the linear constraints and :math:`B` is a :math:`m_B\times n_{\textit{var}}` matrix, :math:`l_B` and :math:`u_B` are :math:`m_B`-dimensional vectors,
there are :math:`n_{\textit{var}}` box constraints and :math:`l_x` and :math:`u_x` are :math:`n_{\textit{var}}`-dimensional vectors.
Here,
:math:`x` is an :math:`n_{\textit{var}}`-dimensional vector representing the model parameters,
:math:`\chi` is the loss function,
:math:`\psi` is the regularization function,
:math:`\rho` is the regularization coefficient,
:math:`n_{\textit{res}}` is the number of residuals and :math:`r_i` is the :math:`i`\ th residual, which is defined as
.. math::
r_i\left(x\right) = y_i-\phi \left(t_i; x\right)\text{, }\quad i = 1,\ldots,n_{\textit{res}}
where :math:`\phi \left(t_i; x\right)` is the predicted value of the :math:`i`\ th data point, given :math:`x`.
For the :math:`i`\ th data point, :math:`y_i` and :math:`t_i` are the observed values of the independent and dependant variables respectively.
The available loss and regularization function types are summarized in Table [label omitted], where :math:`d` is the function parameter and :math:`I\left(L\right)` denotes an indicator function taking the value :math:`1` if the logical expression :math:`L` is true and :math:`0` otherwise.
Loss function and regularization types can be specified by options 'NLDF Loss Function Type' and 'Reg Term Type', respectively.
For example, set :math:`\text{‘NLDF Loss Function Type'} = \texttt{'LINF'}` and :math:`\text{‘Reg Term Type'} = \texttt{'L2'}` to use :math:`l_{\infty }`-norm loss function with :math:`l_2`-norm (Ridge) regularization.
See `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gnf.html#algdetails>`__ for more details on the loss functions.
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Loss function |:math:`\chi \left(r_i\right)` |'NLDF Loss Function Type'|
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_2`-norm |:math:`r_i^2` |'L2' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_1`-norm |:math:`\left\lvert r_i\right\rvert` |'L1' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|:math:`l_{\infty }`-norm |:math:`\mathrm{max}_{{1\leq j\leq n_{\textit{res}}}}\left\lvert r_j\right\rvert /n_{\textit{res}}` |'LINF' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Huber (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#func_huber>`__) |:math:`\left\{\begin{array}{ll} 0.5 * r_i^2 & \text{if } \left\lvert r_i\right\rvert < d \\ d * \left(\left\lvert r_i\right\rvert -0.5*d\right) &\text{otherwise}\end{array}\right.`|'HUBER' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Cauchy (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#func_cauchy>`__) |:math:`\mathrm{ln}\left(1+\left(r_i/d\right)^2\right)` |'CAUCHY' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Atan |:math:`\arctan\left(r_i^2\right)` |'ATAN' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|SmoothL1 (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#func_smoothl1>`__)|:math:`\left\{\begin{array}{ll} 0.5 * r_i^2 / d & \text{if } \left\lvert r_i\right\rvert < d \\ \left\lvert r_i\right\rvert - 0.5 * d &\text{otherwise}\end{array}\right.` |'SMOOTHL1' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Quantile (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04gnf.html#func_quantile>`__)|:math:`r_i*\left(d-I_{\left(r_i < 0\right)}\right)` |'QUANTILE' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Regularization |:math:`\psi \left(x_i\right)` |'Reg Term Type' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Lasso (:math:`l_1`-norm) |:math:`\left\lvert x_i\right\rvert` |'L1' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
|Ridge (:math:`l_2`-norm) |:math:`x_i^2` |'L2' |
+---------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+
``handle_solve_nldf`` serves as a solver for problems stored as a :math:`\mathrm{handle}`.
The :math:`\mathrm{handle}` points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. After the :math:`\mathrm{handle}` has been initialized (e.g., :meth:`handle_init` has been called), :meth:`handle_set_nlnls` can be used to add a model and define its residual sparsity structure. :meth:`handle_set_qconstr` and :meth:`handle_set_qconstr_fac` may be used to set or modify quadratic constraints.
Linear constraints :math:`l_B`, :math:`B`, :math:`u_B` are handled by :meth:`handle_set_linconstr`.
Variable box bounds :math:`l_x` and :math:`u_x` can be specified with :meth:`handle_set_simplebounds`, and :meth:`handle_set_nlnconstr` can set or modify nonlinear constraints.
Once the problem is fully described, the :math:`\mathrm{handle}` may be passed to the solver ``handle_solve_nldf``.
When the :math:`\mathrm{handle}` is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
Nonlinear Programming (NLP) solvers :meth:`handle_solve_bounds_foas` and :meth:`handle_solve_ipopt` are used as solver engines by ``handle_solve_nldf``, which defines the selected loss function and regularization, then transforms the problem into standard form that the NLP solvers allow.
For best performance, when the objective function :math:`f\left(x\right)` is differentiable and without any constraint other than simple bound constraints, :meth:`handle_solve_bounds_foas` is used.
For non-differentiable objective functions or cases where constraints other than simple variable bounds are present, :meth:`handle_solve_ipopt` is used.
See `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#algdetails>`__ and :meth:`handle_solve_ipopt` for more details on algorithmic details.
The algorithm behaviour can be modified by various options (see :ref:`Other Parameters <e04gn-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the :math:`\mathrm{handle}` by e.g., :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
Option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_nldf_ex.main`
"""
raise NotImplementedError
[docs]def lsq_uncon_quasi_deriv_easy(m, lsfun2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_quasi_deriv_easy`` is an easy-to-use quasi-Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. deprecated:: 28.3.0.0
``lsq_uncon_quasi_deriv_easy`` is deprecated.
Please use :meth:`handle_solve_bxnl` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04gy-py2-py-doc:
For full information please refer to the NAG Library document for e04gy
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gyf.html
.. _e04gy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_quasi_deriv_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the first derivatives calculated by :math:`\mathrm{lsfun2}` at the starting point and so is more likely to detect an error in your function if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04gy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04gy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_quasi_deriv_easy`` is similar to the function LSFDQ2 in the NPL Algorithms Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left({F\left(x\right)}\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.) You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`.
Before attempting to minimize the sum of squares, the algorithm checks the function for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04gy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv_easy(m, lsfun2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv_easy`` is an easy-to-use modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04gz-py2-py-doc:
For full information please refer to the NAG Library document for e04gz
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04gzf.html
.. _e04gz-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv_easy``.
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[i-1]` must be set to the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must be set to the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the first derivatives calculated by :math:`\mathrm{lsfun2}` at the starting point and so is more likely to detect any error in your functions if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04gz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04gz-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv_easy`` is similar to the function LSFDN2 in the NPL Algorithms NAG Library.
It is applicable to problems of the form
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`.
Before attempting to minimize the sum of squares, the algorithm checks the function for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04gz-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def check_deriv(funct, x, data=None):
r"""
``check_deriv`` checks that a function for evaluating an objective function and its first derivatives produces derivative values which are consistent with the function values calculated.
.. _e04hc-py2-py-doc:
For full information please refer to the NAG Library document for e04hc
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hcf.html
.. _e04hc-py2-py-parameters:
**Parameters**
**funct** : callable (iflag, fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function and its first derivatives at a given point. (The minimization functions mentioned in :ref:`Notes <e04hc-py2-py-notes>` gives you the option of resetting arguments of :math:`\mathrm{funct}` to cause the minimization process to terminate immediately. ``check_deriv`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
Will be set to :math:`2`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which :math:`F` and its derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you reset :math:`\mathrm{iflag}` to a negative number in :math:`\mathrm{funct}` and return control to ``check_deriv``, ``check_deriv`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fc** : float
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{fc}` must be set to the value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must be set to the coordinates of a suitable point at which to check the derivatives calculated by :math:`\mathrm{funct}`. 'Obvious' settings, such as :math:`0.0` or :math:`1.0`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. Similarly, it is preferable that no two elements of :math:`\mathrm{x}` should be the same.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**f** : float
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{f}` contains the value of the objective function :math:`F\left(x\right)` at the point given by you in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{g}[\textit{j}-1]` contains the value of the derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{funct}`, for :math:`\textit{j} = 1,2,\ldots,n`.
.. _e04hc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
(`errno` :math:`2`)
Large errors were found in the derivatives of :math:`F` computed by :math:`\mathrm{funct}`.
.. _e04hc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
Functions for minimizing a function of several variables may require you to supply a function to evaluate the objective function :math:`F\left(x_1, x_2, \ldots, x_n\right)` and its first derivatives. ``check_deriv`` is designed to check the derivatives calculated by such functions.
As well as the function to be checked ( :math:`\mathrm{funct}`), you must supply a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the check will be made.
Note that ``check_deriv`` checks functions of the form required for :meth:`bounds_mod_deriv_comp` and :meth:`bounds_mod_deriv2_comp`.
``check_deriv`` first calls :math:`\mathrm{funct}` to evaluate :math:`F` and its first derivatives :math:`g_{\textit{j}} = \frac{{\partial F}}{{\partial x_{\textit{j}}}}`, for :math:`\textit{j} = 1,2,\ldots,n` at :math:`x`.
The components of the user-supplied derivatives along two orthogonal directions (defined by unit vectors :math:`p_1` and :math:`p_2`, say) are then calculated; these will be :math:`g^\mathrm{T}p_1` and :math:`g^\mathrm{T}p_2` respectively.
The same components are also estimated by finite differences, giving quantities
.. math::
v_k = \frac{{F\left(x+hp_k\right)-F\left(x\right)}}{h}\text{, }\quad k = 1,2
where :math:`h` is a small positive scalar.
If the relative difference between :math:`v_1` and :math:`g^\mathrm{T}p_1` or between :math:`v_2` and :math:`g^\mathrm{T}p_2` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def check_deriv2(funct, h, x, lh, data=None):
r"""
``check_deriv2`` checks that a function for calculating second derivatives of an objective function is consistent with a function for calculating the corresponding first derivatives.
.. _e04hd-py2-py-doc:
For full information please refer to the NAG Library document for e04hd
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hdf.html
.. _e04hd-py2-py-parameters:
**Parameters**
**funct** : callable (iflag, fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function and its first derivatives at a given point. (:meth:`bounds_mod_deriv2_comp` gives you the option of resetting arguments of :math:`\mathrm{funct}` to cause the minimization process to terminate immediately. ``check_deriv2`` will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
**Parameters**
**iflag** : int
To :math:`\mathrm{funct}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and first derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If you set :math:`\mathrm{iflag}` to some negative number in :math:`\mathrm{funct}` and return control to ``check_deriv2``, ``check_deriv2`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fc** : float
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{fc}` must be set to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{funct}` resets :math:`\mathrm{iflag}`, :math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**h** : callable (iflag, fhesl, fhesd) = h(iflag, xc, lh, fhesd, data=None)
:math:`\mathrm{h}` must evaluate the second derivatives of the function at a given point. (As with :math:`\mathrm{funct}`, an argument can be set to cause immediate termination.)
**Parameters**
**iflag** : int
Is set to a non-negative number.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the second derivatives of :math:`F\left(x\right)` are required.
**lh** : int
The length of the array :math:`\mathrm{fhesl}`.
**fhesd** : float, ndarray, shape :math:`\left(n\right)`
Contains the value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`. Functions written to take advantage of a similar feature of :meth:`bounds_mod_deriv2_comp` can be tested as they stand by ``check_deriv2``.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{h}` resets :math:`\mathrm{iflag}` to a negative number, ``check_deriv2`` will terminate immediately with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fhesl** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
Unless :math:`\mathrm{iflag}` is reset, :math:`\mathrm{h}` must place the strict lower triangle of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesl}`, stored by rows, i.e., :math:`\mathrm{fhesl}[\left(\textit{i}-1\right)\left(\textit{i}-2\right)/2+\textit{j}-1]` must be set to the value of :math:`\frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,\textit{i}-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**fhesd** : float, array-like, shape :math:`\left(n\right)`
Unless :math:`\mathrm{iflag}` is reset, :math:`\mathrm{h}` must place the diagonal elements of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesd}`, i.e., :math:`\mathrm{fhesd}[\textit{j}-1]` must be set to the value of :math:`\frac{{\partial^2F}}{{\partial x_{\textit{j}}^2}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]`, for :math:`\textit{j} = 1,2,\ldots,n`, must contain the coordinates of a suitable point at which to check the derivatives calculated by :math:`\mathrm{funct}`. 'Obvious' settings, such as :math:`0.0` or :math:`1.0`, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. Similarly, it is advisable that no two elements of :math:`\mathrm{x}` should be the same.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**g** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in the first call of :math:`\mathrm{funct}`, :math:`\mathrm{g}[\textit{j}-1]` contains the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point given in :math:`\mathrm{x}`, as calculated by :math:`\mathrm{funct}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
Unless you set :math:`\mathrm{iflag}` negative in :math:`\mathrm{h}`, :math:`\mathrm{hesl}` contains the strict lower triangle of the second derivative matrix of :math:`F`, as evaluated by :math:`\mathrm{h}` at the point given in :math:`\mathrm{x}`, stored by rows.
**hesd** : float, ndarray, shape :math:`\left(n\right)`
Unless you set :math:`\mathrm{iflag}` negative in :math:`\mathrm{h}`, :math:`\mathrm{hesd}` contains the diagonal elements of the second derivative matrix of :math:`F`, as evaluated by :math:`\mathrm{h}` at the point given in :math:`\mathrm{x}`.
.. _e04hd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \mathrm{max}\left(1, {n\times \frac{{n-1}}{2}}\right)`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
(`errno` :math:`2`)
Large errors were found in the derivatives of :math:`F` computed by :math:`\mathrm{funct}`.
.. _e04hd-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
Functions for minimizing a function :math:`F\left(x_1, x_2, \ldots, x_n\right)` of the variables :math:`x_1,x_2,\ldots,x_n` may require you to provide a function to evaluate the second derivatives of :math:`F`. ``check_deriv2`` is designed to check the second derivatives calculated by such functions.
As well as the function to be checked (:math:`\mathrm{h}`), you must supply a function (:math:`\mathrm{funct}`) to evaluate the first derivatives, and a point :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` at which the checks will be made.
Note that ``check_deriv2`` checks functions of the form required for :meth:`bounds_mod_deriv2_comp`.
``check_deriv2`` first calls functions :math:`\mathrm{funct}` and :math:`\mathrm{h}` to evaluate the first and second derivatives of :math:`F` at :math:`x`.
The user-supplied Hessian matrix (:math:`H`, say) is projected onto two orthogonal vectors :math:`y` and :math:`z` to give the scalars :math:`y^\mathrm{T}Hy` and :math:`z^\mathrm{T}Hz` respectively.
The same projections of the Hessian matrix are also estimated by finite differences, giving
.. math::
\begin{array}{ll}p = \left(y^\mathrm{T}g\left(x+hy\right)-y^\mathrm{T}g\left(x\right)\right)/h&\text{and}\\&q = \left(z^\mathrm{T}g\left(x+hz\right)-z^\mathrm{T}g\left(x\right)\right)/h\end{array}
respectively, where :math:`g\left(\right)` denotes the vector of first derivatives at the point in brackets and :math:`h` is a small positive scalar.
If the relative difference between :math:`p` and :math:`y^\mathrm{T}Hy` or between :math:`q` and :math:`z^\mathrm{T}Hz` is judged too large, an error indicator is set.
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv2_comp(m, lsqfun, lsqhes, xtol, x, lsqmon=None, iprint=1, maxcal=None, eta=None, stepmx=100000.0, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv2_comp`` is a comprehensive modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First and second derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04he-py2-py-doc:
For full information please refer to the NAG Library document for e04he
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hef.html
.. _e04he-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsqfun** : callable (iflag, fvec, fjac) = lsqfun(iflag, m, xc, data=None)
:math:`\mathrm{lsqfun}` must calculate the vector of values :math:`f_i\left(x\right)` and Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`. (However, if you do not wish to calculate the residuals or first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``lsq_uncon_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
To :math:`\mathrm{lsqfun}`, :math:`\mathrm{iflag}` will be set to :math:`2`.
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If it is not possible to evaluate the :math:`f_i\left(x\right)` or their first derivatives at the point given in :math:`\mathrm{xc}` (or if it wished to stop the calculations for any other reason), you should reset :math:`\mathrm{iflag}` to some negative number and return control to ``lsq_uncon_mod_deriv2_comp``. ``lsq_uncon_mod_deriv2_comp`` will then terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fvec}[\textit{i}-1]` must contain the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must contain the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**lsqhes** : callable (iflag, b) = lsqhes(iflag, fvec, xc, lb, data=None)
:math:`\mathrm{lsqhes}` must calculate the elements of the symmetric matrix
.. math::
B\left(x\right) = \sum_{{i = 1}}^mf_i\left(x\right)G_i\left(x\right)\text{,}
at any point :math:`x`, where :math:`G_i\left(x\right)` is the Hessian matrix of :math:`f_i\left(x\right)`. (As with :math:`\mathrm{lsqfun}`, there is the option of causing ``lsq_uncon_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Is set to a non-negative number.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The value of the residual :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`, so that the values of the :math:`f_{\textit{i}}` can be used in the calculation of the elements of :math:`\mathrm{b}`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the elements of :math:`\mathrm{b}` are to be evaluated.
**lb** : int
The length of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**iflag** : int
If :math:`\mathrm{lsqhes}` resets :math:`\mathrm{iflag}` to some negative number, ``lsq_uncon_mod_deriv2_comp`` will terminate immediately, with :math:`\textit{errno}` set to your setting of :math:`\mathrm{iflag}`.
**b** : float, array-like, shape :math:`\left(\mathrm{lb}\right)`
Unless :math:`\mathrm{iflag}` is reset to a negative number, :math:`\mathrm{b}` must contain the lower triangle of the matrix :math:`\mathrm{b}[x-1]`, evaluated at the point :math:`x`, stored by rows. (The upper triangle is not required because the matrix is symmetric.) More precisely, :math:`\mathrm{b}[\textit{j}\left(\textit{j}-1\right)/2+\textit{k}-1]` must contain :math:`\sum_{{\textit{i} = 1}}^mf_{\textit{i}}\frac{{\partial^2f_{\textit{i}}}}{{\partial x_{\textit{j}}\partial x_{\textit{k}}}}` evaluated at the point :math:`x`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**xtol** : float
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that
.. math::
\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)\text{,}
where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus and if :math:`\mathrm{xtol} = 1.0e-5`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hef.html#accuracy>`__.)
If :math:`F\left(x\right)` and the variables are scaled roughly as described in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hef.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then a setting of order :math:`\mathrm{xtol} = \sqrt{\epsilon }` will usually be appropriate.
If :math:`\mathrm{xtol}` is set to :math:`0.0` or some positive value less than :math:`10\epsilon`, ``lsq_uncon_mod_deriv2_comp`` will use :math:`10\epsilon` instead of :math:`\mathrm{xtol}`, since :math:`10\epsilon` is probably the smallest reasonable setting.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lsqmon** : None or callable lsqmon(xc, fvec, fjac, s, igrade, niter, nf, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{lsqmon}` which is suitable for monitoring the minimization process. :math:`\mathrm{lsqmon}` must not change the values of any of its arguments.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The values of the residuals :math:`f_i` at the current point :math:`x`.
**fjac** : float, ndarray, shape :math:`\left(m, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` contains the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the current Jacobian matrix. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem. (If :math:`\mathrm{iprint} > 0`, :math:`\mathrm{lsqmon}` is called at the initial point before the singular values have been calculated, so the elements of :math:`\mathrm{s}` are set to zero for the first call of :math:`\mathrm{lsqmon}`.)
**igrade** : int
``lsq_uncon_mod_deriv2_comp`` estimates the dimension of the subspace for which the Jacobian matrix can be used as a valid approximation to the curvature (see Gill and Murray (1978)). This estimate is called the grade of the Jacobian matrix, and :math:`\mathrm{igrade}` gives its current value.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv2_comp``.
**nf** : int
The number of times that :math:`\mathrm{lsqfun}` has been called so far. Thus :math:`\mathrm{nf}` gives the number of evaluations of the residuals and the Jacobian matrix.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**iprint** : int, optional
Specifies the frequency with which :math:`\mathrm{lsqmon}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{lsqmon}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``lsq_uncon_mod_deriv2_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{lsqmon}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{lsqmon}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
This argument is present so as to enable you to limit the number of times that :math:`\mathrm{lsqfun}` is called by ``lsq_uncon_mod_deriv2_comp``. There will be an error exit (see :ref:`Exceptions <e04he-py2-py-errors>`) after :math:`\mathrm{maxcal}` calls of :math:`\mathrm{lsqfun}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.5 }`.
Every iteration of ``lsq_uncon_mod_deriv2_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` with respect to :math:`\alpha^{\left(k\right)}`). :math:`\mathrm{eta}` must lie in the range :math:`0.0\leq \mathrm{eta} < 1.0`, and specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha^{\left(k\right)}` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations performed by ``lsq_uncon_mod_deriv2_comp``, they will increase the number of calls of :math:`\mathrm{lsqfun}` made each iteration.
On balance it is usually more efficient to perform a low accuracy minimization.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency, a slight overestimate is preferable.)
``lsq_uncon_mod_deriv2_comp`` will ensure that, for each iteration
.. math::
\sum_{{j = 1}}^n{\left(x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right)}^2\leq \left(\mathrm{stepmx}\right)^2\text{,}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``lsq_uncon_mod_deriv2_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can help avoid overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsqfun}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**fsumsq** : float
The value of :math:`F\left(x\right)`, the sum of squares of the residuals :math:`f_i\left(x\right)`, at the final point given in :math:`\mathrm{x}`.
**fvec** : float, ndarray, shape :math:`\left(\mathrm{m}\right)`
The value of the residual :math:`f_{\textit{i}}\left(x\right)` at the final point given in :math:`\mathrm{x}`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, ndarray, shape :math:`\left(\mathrm{m}, n\right)`
The value of the first derivative :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` evaluated at the final point given in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**s** : float, ndarray, shape :math:`\left(n\right)`
The singular values of the Jacobian matrix at the final point. Thus :math:`\mathrm{s}` may be useful as information about the structure of your problem.
**v** : float, ndarray, shape :math:`\left(n, n\right)`
The matrix :math:`V` associated with the singular value decomposition
.. math::
J = USV^\mathrm{T}
of the Jacobian matrix at the final point, stored by columns. This matrix may be useful for statistical purposes, since it is the matrix of orthonormalized eigenvectors of :math:`J^\mathrm{T}J`.
**niter** : int
The number of iterations which have been performed in ``lsq_uncon_mod_deriv2_comp``.
**nf** : int
The number of times that the residuals and Jacobian matrix have been evaluated (i.e., number of calls of :math:`\mathrm{lsqfun}`).
.. _e04he-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle` calls to :math:`\mathrm{lsqfun}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{lsqfun}` or :math:`\mathrm{lsqhes}`.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
.. _e04he-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv2_comp`` is essentially identical to the function LSQSDN in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply functions to calculate the values of the :math:`f_i\left(x\right)` and their first derivatives and second derivatives at any point :math:`x`.
From a starting point :math:`x^{\left(1\right)}` supplied by you, the function generates a sequence of points :math:`x^{\left(2\right)},x^{\left(3\right)},\ldots`, which is intended to converge to a local minimum of :math:`F\left(x\right)`.
The sequence of points is given by
.. math::
x^{\left(k+1\right)} = x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}
where the vector :math:`p^{\left(k\right)}` is a direction of search, and :math:`\alpha^{\left(k\right)}` is chosen such that :math:`F\left(x^{\left(k\right)}+\alpha^{\left(k\right)}p^{\left(k\right)}\right)` is approximately a minimum with respect to :math:`\alpha^{\left(k\right)}`.
The vector :math:`p^{\left(k\right)}` used depends upon the reduction in the sum of squares obtained during the last iteration.
If the sum of squares was sufficiently reduced, then :math:`p^{\left(k\right)}` is the Gauss--Newton direction; otherwise the second derivatives of the :math:`f_i\left(x\right)` are taken into account.
The method is designed to ensure that steady progress is made whatever the starting point, and to have the rapid ultimate convergence of Newton's method.
.. _e04he-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def lsq_uncon_mod_deriv2_easy(m, lsfun2, lshes2, x, data=None, spiked_sorder='C'):
r"""
``lsq_uncon_mod_deriv2_easy`` is an easy-to-use modified Gauss--Newton algorithm for finding an unconstrained minimum of a sum of squares of :math:`m` nonlinear functions in :math:`n` variables :math:`\left(m\geq n\right)`.
First and second derivatives are required.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04hy-py2-py-doc:
For full information please refer to the NAG Library document for e04hy
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04hyf.html
.. _e04hy-py2-py-parameters:
**Parameters**
**m** : int
The number :math:`m` of residuals, :math:`f_i\left(x\right)`, and the number :math:`n` of variables, :math:`x_j`.
**lsfun2** : callable (fvec, fjac) = lsfun2(m, xc, data=None)
You must supply this function to calculate the vector of values :math:`f_i\left(x\right)` and the Jacobian matrix of first derivatives :math:`\frac{{\partial f_i}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv2_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**m** : int
:math:`m`, the numbers of residuals.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the values of the :math:`f_i` and the :math:`\frac{{\partial f_i}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fvec** : float, array-like, shape :math:`\left(\mathrm{m}\right)`
:math:`\mathrm{fvec}[i-1]` must be set to the value of :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`.
**fjac** : float, array-like, shape :math:`\left(\mathrm{m}, n\right)`
:math:`\mathrm{fjac}[\textit{i}-1,\textit{j}-1]` must be set to the value of :math:`\frac{{\partial f_{\textit{i}}}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`, for :math:`\textit{i} = 1,2,\ldots,m`.
**lshes2** : callable b = lshes2(fvec, xc, lb, data=None)
You must supply this function to calculate the elements of the symmetric matrix
.. math::
B\left(x\right) = \sum_{{i = 1}}^mf_i\left(x\right)G_i\left(x\right)\text{,}
at any point :math:`x`, where :math:`G_i\left(x\right)` is the Hessian matrix of :math:`f_i\left(x\right)`.
It should be tested separately before being used in conjunction with ``lsq_uncon_mod_deriv2_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**fvec** : float, ndarray, shape :math:`\left(m\right)`
The value of the residual :math:`f_{\textit{i}}` at the point :math:`x`, for :math:`\textit{i} = 1,2,\ldots,m`, so that the values of the :math:`f_{\textit{i}}` can be used in the calculation of the elements of :math:`\mathrm{b}`.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the elements of :math:`\mathrm{b}` are to be evaluated.
**lb** : int
The length of the array :math:`\mathrm{b}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**b** : float, array-like, shape :math:`\left(\mathrm{lb}\right)`
Must contain the lower triangle of the matrix :math:`B\left(x\right)`, evaluated at the point :math:`x`, stored by rows. (The upper triangle is not required because the matrix is symmetric.) More precisely, :math:`\mathrm{b}[\textit{j}\left(\textit{j}-1\right)/2+\textit{k}-1]` must contain :math:`\sum_{{\textit{i} = 1}}^mf_{\textit{i}}\frac{{\partial^2f_{\textit{i}}}}{{\partial x_{\textit{j}}\partial x_{\textit{k}}}}` evaluated at the point :math:`x`, for :math:`\textit{k} = 1,2,\ldots,\textit{j}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks :math:`\mathrm{lsfun2}` and :math:`\mathrm{lshes2}` at the starting point and so is more likely to detect any error in your functions if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**spiked_sorder** : str, optional
If :math:`\mathrm{fjac}` in :math:`\mathrm{lsfun2}` is spiked (i.e., has unit extent in all but one dimension, or has size :math:`1`), :math:`\mathrm{spiked\_sorder}` selects the storage order to associate with it in the NAG Engine:
spiked_sorder = :math:`\texttt{'C'}`
row-major storage will be used;
spiked_sorder = :math:`\texttt{'F'}`
column-major storage will be used.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**fsumsq** : float
The value of the sum of squares, :math:`F\left(x\right)`, corresponding to the final point stored in :math:`\mathrm{x}`.
**comm** : dict, communication object
Communication structure.
.. _e04hy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq n`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` calls to :math:`\mathrm{lsfun2}`.
(`errno` :math:`4`)
Failure in computing SVD of Jacobian matrix.
(`errno` :math:`9`)
It is very likely that you have made an error in forming the derivatives in :math:`\mathrm{lsfun2}`.
(`errno` :math:`10`)
It is very likely that you have made an error in setting up the array :math:`\mathrm{b}` in :math:`\mathrm{lshes2}`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04hy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_uncon_mod_deriv2_easy`` is similar to the function LSSDN2 in the NPL Algorithms Library.
It is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x\right) = \sum_{{i = 1}}^m{\left[f_i\left(x\right)\right]}^2
where :math:`x = \left(x_1, x_2, \ldots, x_n\right)^\mathrm{T}` and :math:`m\geq n`. (The functions :math:`f_i\left(x\right)` are often referred to as 'residuals'.)
You must supply a function to evaluate the residuals and their first derivatives at any point :math:`x`, and a function to evaluate the elements of the second derivative term of the Hessian matrix of :math:`F\left(x\right)`.
Before attempting to minimize the sum of squares, the algorithm checks the functions for consistency.
Then, from a starting point supplied by you, a sequence of points is generated which is intended to converge to a local minimum of the sum of squares.
These points are generated using estimates of the curvature of :math:`F\left(x\right)`.
.. _e04hy-py2-py-references:
**References**
Gill, P E and Murray, W, 1978, `Algorithms for the solution of the nonlinear least squares problem`, SIAM J. Numer. Anal. (15), 977--992
"""
raise NotImplementedError
[docs]def bounds_bobyqa_func(objfun, npt, x, bl, bu, rhobeg, rhoend, maxcal, monfun=None, data=None):
r"""
``bounds_bobyqa_func`` is an easy-to-use algorithm that uses methods of quadratic approximation to find a minimum of an objective function :math:`F` over :math:`\mathbf{x} \in R^n`, subject to fixed lower and upper bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`.
Derivatives of :math:`F` are not required.
The function is intended for functions that are continuous and that have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
Efficiency is maintained for large :math:`n`.
.. deprecated:: 27.0.0.0
``bounds_bobyqa_func`` is deprecated.
Please use :meth:`handle_solve_dfno` and :meth:`handle_solve_dfno_rcomm` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04jc-py2-py-doc:
For full information please refer to the NAG Library document for e04jc
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jcf.html
.. _e04jc-py2-py-parameters:
**Parameters**
**objfun** : callable f = objfun(x, data=None)
:math:`\mathrm{objfun}` must evaluate the objective function :math:`F` at a specified vector :math:`\mathbf{x}`.
**Parameters**
**x** : float, ndarray, shape :math:`\left(n\right)`
:math:`\mathbf{x}`, the vector at which the objective function is to be evaluated.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**f** : float
Must be set to the value of the objective function at :math:`\mathbf{x}`.
**npt** : int
:math:`m`, the number of interpolation conditions imposed on the quadratic approximation at each iteration.
`Suggested value`: :math:`\mathrm{npt} = 2\times n_r+1`, where :math:`n_r` denotes the number of non-fixed variables.
**x** : float, array-like, shape :math:`\left(n\right)`
An estimate of the position of the minimum. If any component is out-of-bounds it is replaced internally by the bound it violates.
**bl** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bl}` must contain the fixed vector of lower bounds, :math:`\ell`.
**bu** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{bu}` must contain the fixed vector of upper bounds, :math:`\mathbf{u}`.
**rhobeg** : float
An initial lower bound on the value of the trust region radius.
`Suggested value`: :math:`\mathrm{rhobeg}` should be about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`\mathrm{x}` of length :math:`\mathrm{rhobeg}` along each coordinate direction.
**rhoend** : float
A final lower bound on the value of the trust region radius.
`Suggested value`: :math:`\mathrm{rhoend}` should indicate the absolute accuracy that is required in the final values of the variables.
**maxcal** : int
The maximum permitted number of calls to :math:`\mathrm{objfun}`.
**monfun** : None or callable monfun(nf, x, f, rho, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monfun}` may be used to monitor the optimization process.
It is invoked every time a new trust region radius is chosen.
If no monitoring is required, :math:`\mathrm{monfun}` may be **None**.
**Parameters**
**nf** : int
The cumulative number of calls made to :math:`\mathrm{objfun}`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The current best point.
**f** : float
The value of :math:`\mathrm{objfun}` at :math:`\mathrm{x}`.
**rho** : float
A lower bound on the current trust region radius.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}` is the position of the minimum.
**f** : float
The function value at the lowest point found (:math:`\mathrm{x}`).
**nf** : int
Unless :math:`\mathrm{errno}` = 1 or ``NagMemoryError`` on exit, the total number of calls made to :math:`\mathrm{objfun}`.
.. _e04jc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and :math:`\mathrm{npt} = \langle\mathit{\boldsymbol{value}}\rangle` on entry.
Constraint: :math:`n_r+2\leq \mathrm{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
(`errno` :math:`1`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds.
Constraint: :math:`n_r\geq 2`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[i-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{bl}[i-1]\neq \mathrm{bu}[i-1]` in coordinate :math:`i`, :math:`\mathrm{bu}[i-1]-\mathrm{bl}[i-1]\geq 2\times \mathrm{rhobeg}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhobeg} > 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhoend} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhoend}\geq \textit{macheps}`, where :math:`\textit{macheps} = \texttt{machine.precision}\left(\right)`, the machine precision.
(`errno` :math:`1`)
On entry, :math:`\mathrm{rhobeg} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{rhoend} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{rhoend}\leq \mathrm{rhobeg}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
**Warns**
**NagAlgorithmicMajorWarning**
(`errno` :math:`2`)
The function evaluations limit was reached: :math:`\mathrm{objfun}` has been called :math:`\mathrm{maxcal}` times.
(`errno` :math:`3`)
The predicted reduction in a trust region step was non-positive. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`4`)
A rescue procedure has been called in order to correct damage from rounding errors when computing an update to a quadratic approximation of :math:`F`, but no further progess could be made. Check your specification of :math:`\mathrm{objfun}` and whether the function needs rescaling. Try a different initial :math:`\mathrm{x}`.
**NagCallbackTerminateWarning**
(`errno` :math:`5`)
User-supplied objective function requested termination.
(`errno` :math:`5`)
User-supplied monitoring function requested termination.
.. _e04jc-py2-py-notes:
**Notes**
``bounds_bobyqa_func`` is applicable to problems of the form:
.. math::
\textit{minimize}_{{\mathbf{x} \in R^n}}F\left(\mathbf{x}\right)\quad \text{ subject to }\quad \ell \leq \mathbf{x}\leq \mathbf{u}\quad \text{ and }\quad \ell \leq \mathbf{u}\text{,}
where :math:`F` is a nonlinear scalar function whose derivatives may be unavailable, and where the bound vectors are elements of :math:`R^n`.
Relational operators between vectors are interpreted elementwise.
Fixing variables (that is, setting :math:`\ell_i = u_i` for some :math:`i`) is allowed in ``bounds_bobyqa_func``.
You must supply a function to calculate the value of :math:`F` at any given point :math:`\mathbf{x}`.
The method used by ``bounds_bobyqa_func`` is based on BOBYQA, the method of Bound Optimization BY Quadratic Approximation described in Powell (2009).
In particular, each iteration of ``bounds_bobyqa_func`` generates a quadratic approximation :math:`Q` to :math:`F` that agrees with :math:`F` at :math:`m` automatically chosen interpolation points.
The value of :math:`m` is a constant prescribed by you.
Updates to the independent variables mostly occur from approximate solutions to trust region subproblems, using the current quadratic model.
.. _e04jc-py2-py-references:
**References**
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
"""
raise NotImplementedError
[docs]def handle_solve_dfno(handle, x, objfun=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_dfno`` is a forward communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFNO) for small to medium-scale nonlinear problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04jd-py2-py-doc:
For full information please refer to the NAG Library document for e04jd
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jdf.html
.. _e04jd-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfno``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` calculates the value of the objective function :math:`f\left(x\right)` at a specified point :math:`x`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objfun}` will never be called by ``handle_solve_dfno`` and may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
May be used to indicate that the requested objective value could not be computed. Specifically, it can be set to a negative value:
:math:`\mathrm{inform} = -1`
The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with :math:`\mathrm{errno}` = 17.
:math:`\mathrm{inform} = -2`
The solver will cleanly exit with :math:`\mathrm{errno}` = 20 and return the best available point as well as the solve statistics.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the 'DFO Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
If no monitoring is required, :math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
:math:`x`, the vector of decision variables at the current iteration.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in the main argument :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument :math:`\mathrm{stats}`).
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+----------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+---------------------+----------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region.|
+---------------------+----------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+---------------------+----------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+---------------------+----------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+----------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04jd-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFNO Detect Unbounded'** : str
Default :math:`= {\texttt{'YES'}}`
The solver can try to detect whether the problem is unbounded.
This option allows you to turn on or off the unboundedness detection heuristic.
Constraint: :math:`\text{‘DFNO Detect Unbounded'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'DFNO Objective Limit'** : float
Default :math:`\text{} = {-\infty }`
This option sets an additional convergence criterion.
The solver will stop if it finds a point for which the function value is lower than this parameter.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, :math:`\mathrm{monit}` will be called at the end of every :math:`i`\ th step for monitoring purposes.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function if :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jdf.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+1\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model.
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04jd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is different from the total set by 'DFO Number Interp Points'.
Growing the interpolation set is not supported for this solver.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points could not be provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
The solver failed at the model building phase and the maximum number of restarts was reached. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`20`)
User requested termination after a call to the objective function.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
(`errno` :math:`54`)
The problem seems to be unbounded.
The unboundedness detection heuristic can be turned off with the option 'DFNO Detect Unbounded'.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04jd-py2-py-notes:
**Notes**
``handle_solve_dfno`` is aimed at minimizing a nonlinear objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & {f\left(x\right)} \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here :math:`f` is a smooth nonlinear function and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
``handle_solve_dfno`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnobj` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfno`` always assumes that the gradient of the objective is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnobj` will have no effect.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jdf.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04jd-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFNO (Derivative-free Nonlinear Optimization).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfno`` is the same as the one used by :meth:`handle_solve_dfno_rcomm`. ``handle_solve_dfno`` serves as a forward communication interface to the derivative-free solver for nonlinear objective functions.
.. _e04jd-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_dfno_ex.main`
"""
raise NotImplementedError
[docs]def handle_solve_dfno_rcomm(handle, irevcm, x, f, io_manager=None):
r"""
``handle_solve_dfno_rcomm`` is a reverse communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFNO) for small to medium-scale nonlinear problems with bound constraints.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04je-py2-py-doc:
For full information please refer to the NAG Library document for e04je
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jef.html
.. _e04je-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_dfno_rcomm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**irevcm** : int
Does not need to be set on the first call of ``handle_solve_dfno_rcomm``. On subsequent calls, :math:`\mathrm{irevcm}` must be set to a positive integer if all the required function evaluations have been correctly provided in :math:`\mathrm{f}`. Otherwise, if a problem occurred during a monitoring step or while providing objective values, it is possible to set it to a negative value:
:math:`\mathrm{irevcm} = -1`
If function evaluations were required, the solver will attempt a rescue procedure and request an alternative point. If no function were required (monitoring step), the solver will stop with :math:`\mathrm{errno}` = 20.
:math:`\mathrm{irevcm} \leq -2`
The solver will cleanly exit and return the best available point as a well as the solve statistics.
**x** : float, array-like, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
The first column contains :math:`x_0`, the initial estimates of the variables :math:`x`.
**f** : float, array-like, shape :math:`\left(\textit{maxeval}\right)`
Does not need to be set on the first call to ``handle_solve_dfno_rcomm``.
If :math:`\mathrm{irevcm} = 1` after the last call of ``handle_solve_dfno_rcomm``, the first :math:`\mathrm{neval}` elements must contain the function values of the requested points.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**irevcm** : int
Indicates what action is to be performed before the next call to ``handle_solve_dfno_rcomm``.
:math:`\mathrm{irevcm} = 0`
Final exit of the solver.
:math:`\mathrm{irevcm} = 1`
:math:`\mathrm{neval}` objective evaluations are required.
:math:`\mathrm{irevcm} = 2`
Monitoring step, no evaluation is required, :math:`\mathrm{x}` and :math:`\mathrm{f}` contain the best evaluation of the objective yet.
**neval** : int
Indicates the number of objective evaluations required for the next call of ``handle_solve_dfno_rcomm`` in :math:`\mathrm{f}`. The coordinates of the points to evaluate are provided in the first :math:`\mathrm{neval}` columns of :math:`\mathrm{x}`.
**x** : float, ndarray, shape :math:`\left(\textit{nvar}, \textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first column contains the best computed estimate of the solution.
If :math:`\mathrm{irevcm} = 1`, the first :math:`\mathrm{neval}` columns contain the coordinates of the points to evaluate.
**f** : float, ndarray, shape :math:`\left(\textit{maxeval}\right)`
If :math:`\mathrm{irevcm} = 0` or :math:`2`, the first element contains the function value of the best computed point.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+----------------------------------------------------------+
|:math:`0` |Objective function value :math:`f\left(x\right)`. |
+---------------------+----------------------------------------------------------+
|:math:`1` |:math:`\rho`, the current lower bound of the trust region.|
+---------------------+----------------------------------------------------------+
|:math:`2` |:math:`\Delta`, the current size of the trust region. |
+---------------------+----------------------------------------------------------+
|:math:`3` |The number of interpolation points used by the solver. |
+---------------------+----------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+----------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------+
|:math:`0` |Number of calls to the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`1` |Total time spent in the solver (including time spent evaluating the objective).|
+---------------------+-------------------------------------------------------------------------------+
|:math:`2` |Total time spent evaluating the objective function. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`3` |Number of steps. |
+---------------------+-------------------------------------------------------------------------------+
|:math:`4`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------+
.. _e04je-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'DFNO Detect Unbounded'** : str
Default :math:`= {\texttt{'YES'}}`
The solver can try to detect whether the problem is unbounded.
This option allows you to turn on or off the unboundedness detection heuristic.
Constraint: :math:`\text{‘DFNO Detect Unbounded'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'DFNO Objective Limit'** : float
Default :math:`\text{} = {-\infty }`
This option sets an additional convergence criterion.
The solver will stop if it finds a point for which the function value is lower than this parameter.
**'DFO Maximum Slow Steps'** : int
Default :math:`= {20}`
If :math:`\text{‘DFO Maximum Slow Steps'} > 0`, this argument defines the maximum number of consecutive slow iterations :math:`n_{\textit{slow}}` allowed.
Set :math:`\text{‘DFO Maximum Slow Steps'} = 0` to deactivate the slow iteration detection.
The algorithm can stop in two situations:
(i) :math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho < \text{‘DFO Trust Region Slow Tol'}` with :math:`\mathrm{errno}` = 50,
(#) :math:`n_{\textit{slow}} > 5\times \text{‘DFO Maximum Slow Steps'}` with :math:`\mathrm{errno}` = 24.
Constraint: :math:`\text{‘DFO Maximum Slow Steps'} \geq 0`.
**'DFO Max Objective Calls'** : int
Default :math:`= {500}`
A limit on the number of objective function evaluations the solver is allowed to compute.
If the limit is reached, the solver stops with :math:`\mathrm{errno}` = 21.
Constraint: :math:`\text{‘DFO Max Objective Calls'} \geq 1`.
**'DFO Max Soft Restarts'** : int
Default :math:`= {5}`
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Soft Restarts'} \geq 1`.
**'DFO Max Unsucc Soft Restarts'** : int
Default :math:`= {3}`
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy (:math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`).
Constraint: :math:`\text{‘DFO Max Unsucc Soft Restarts'} \geq 1`.
**'DFO Monitor Frequency'** : int
Default :math:`= {0}`
If :math:`\text{‘DFO Monitor Frequency'} > 0`, the solver will stop at the end of every :math:`i`\ th step for monitoring purposes. ``handle_solve_dfno_rcomm`` needs to be called again to continue the optimization.
Constraint: :math:`\text{‘DFO Monitor Frequency'} \geq 0`.
**'DFO Noise Level'** : float
Default :math:`= {0.0}`
Indicates the noise level expected when evaluating the objective function.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`.
Constraint: :math:`\text{‘DFO Noise Level'} \geq 0.0`.
**'DFO Noisy Problem'** : str
Default :math:`= {\texttt{'NO'}}`
Indicates if the function evaluations provided to the solver are noisy.
If :math:`\text{‘DFO Noisy Problem'} = \texttt{'YES'}`, some algorithmic features will be activated:
(i) The trust region update becomes slower to reflect the decreased confidence in the objective values.
(#) Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see 'DFO Max Soft Restarts', 'DFO Max Unsucc Soft Restarts', 'DFO Number Soft Restarts Pts' to control the restart characteristics).
(#) In addition, if :math:`\text{‘DFO Noise Level'} > 0.0`, the solver will trigger a soft restart if all the function values are within the noise level.
**'DFO Number Interp Points'** : int
Default :math:`= {0}`
The maximum number of interpolation points in :math:`Y_k` `(9) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jef.html#eqn9>`__ used to build the linear models of the residuals.
If :math:`\text{‘DFO Number Interp Points'} = 0`, the number of points is chosen to be :math:`n_r+1` where :math:`n_r` is the number of non-fixed variables.
Constraint: :math:`\text{‘DFO Number Interp Points'} \geq 0`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 6 if not met:
:math:`n_r+2\leq \text{‘DFO Number Interp Points'}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
**'DFO Number Soft Restarts Pts'** : int
Default :math:`= {3}`
The number of interpolation points that are replaced during a soft restart.
Constraint: :math:`\text{‘DFO Number Soft Restarts Pts'} \geq 1`.
**'DFO Print Frequency'** : int
Default :math:`= {1}`
If :math:`\text{‘DFO Print Frequency'} > 0`, the solver prints the iteration log to the appropriate units at the end of every :math:`i`\ th step.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq 0`.
**'DFO Random Seed'** : int
Default :math:`\text{} = -1`
The random seed used to generate the random points used to build the initial model.
If :math:`\text{‘DFO Random Seed'} < 0`, the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run.
Set it to a positive value to get fully reproducible runs.
Constraint: :math:`\text{‘DFO Print Frequency'} \geq -1`.
**'DFO Starting Trust Region'** : float
Default :math:`= 0.1`
:math:`\rho_{\textit{beg}}`, the initial trust region radius.
This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial :math:`x` of length :math:`\rho_{\textit{beg}}` along each coordinate direction.
The default value assumes that the variables have an order of magnitude :math:`1`.
Constraint: :math:`\text{‘DFO Starting Trust Region'} > \epsilon`.
Consistency constraints, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'}\leq \text{‘DFO Trust Region Tolerance'}`.
:math:`\text{‘DFO Starting Trust Region'}\leq \frac{1}{2}\mathrm{min}_i\left(u_x\left(i\right)-l_x\left(i\right)\right)`.
**'DFO Trust Region Slow Tol'** : float
Default :math:`\text{} = \epsilon^{0.25}`
The minimal acceptable trust region radius for the solution to be declared as acceptable.
The solver stops if:
:math:`n_{\textit{slow}} > \text{‘DFO Maximum Slow Steps'}` and :math:`\rho_k < \text{‘DFO Trust Region Slow Tol'}`.
Constraint: :math:`\text{‘DFO Trust Region Slow Tol'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Trust Region Slow Tol'} > \text{‘DFO Trust Region Tolerance'}`.
**'DFO Trust Region Tolerance'** : float
Default :math:`= \epsilon^{0.37}`
:math:`\rho_{\textit{end}}`, the requested trust region radius.
The algorithm declares convergence when the trust region radius reaches this limit.
It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: :math:`\text{‘DFO Trust Region Tolerance'} > \epsilon`.
Consistency constraint, the solver stops with :math:`\mathrm{errno}` = 5 if not met:
:math:`\text{‘DFO Starting Trust Region'} > \text{‘DFO Trust Region Tolerance'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If :math:`\text{‘Monitoring File'} = -1`, no secondary output is provided.
The information output to this unit is controlled by 'Monitoring Level'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by 'Print Level'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
.. rst-class:: nag-rules-none nag-align-left
+------------------------------------------+--------------------------------+
|:math:`i` |Output |
+==========================================+================================+
|:math:`0` |No output from the solver. |
+------------------------------------------+--------------------------------+
|:math:`1` |The Header and Summary. |
+------------------------------------------+--------------------------------+
|:math:`2`, :math:`3`, :math:`4`, :math:`5`|Additionally, the Iteration log.|
+------------------------------------------+--------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the solution will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
A limit to the number of seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23.
Constraint: :math:`\text{‘Time Limit'} > 0`.
.. _e04je-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{maxeval} > 0`.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`4`)
The information supplied does not match with that previously stored.
On entry, :math:`\textit{maxeval} = \langle\mathit{\boldsymbol{value}}\rangle` must match that given during the first call of the function, i.e., :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Inconsistent options 'DFO Trust Region Tolerance' :math:`\rho_{\textit{end}}` and 'DFO Trust Region Slow Tol' :math:`\rho_{\textit{tol}}`.
Constraint: :math:`\rho_{\textit{end}} < \rho_{\textit{tol}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`5`)
Option 'DFO Starting Trust Region' :math:`\rho_{\textit{beg}} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`l_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`u_x\left(i\right) = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`l_x\left(i\right)\neq u_x\left(i\right)` in coordinate :math:`i`, then :math:`u_x\left(i\right)-l_x\left(i\right)\geq 2\times \rho_{\textit{beg}}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
There were :math:`n_r = \langle\mathit{\boldsymbol{value}}\rangle` unequal bounds and the option 'DFO Number Interp Points' :math:`\textit{npt} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n_r+1\leq \textit{npt}\leq \frac{{\left(n_r+1\right)\times \left(n_r+2\right)}}{2}`.
Use :meth:`handle_opt_set` to set compatible option values.
(`errno` :math:`6`)
The number of initial interpolation points is different from the total set by 'DFO Number Interp Points'.
Growing the interpolation set is not supported for this solver.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
The problem was solved to an acceptable level after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow iterations.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
**NagAlgorithmicMajorWarning**
(`errno` :math:`17`)
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`17`)
Some initial interpolation points could not be provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`18`)
The predicted reduction in a trust region step was non-positive. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`19`)
The solver failed at the model building phase and the maximum number of restarts was reached. Check the specification of your objective and whether it needs rescaling. Try a different initial :math:`\mathrm{x}`.
(`errno` :math:`21`)
Maximum number of function evaluations exceeded.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exceeded.
(`errno` :math:`24`)
No progress, the solver was stopped after :math:`\langle\mathit{\boldsymbol{value}}\rangle` consecutive slow steps.
Use the option 'DFO Maximum Slow Steps' to modify the maximum number of slow steps accepted.
(`errno` :math:`54`)
The problem seems to be unbounded.
The unboundedness detection heuristic can be turned off with the option 'DFNO Detect Unbounded'.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
(`errno` :math:`20`)
User requested termination during an objective evaluation step.
.. _e04je-py2-py-notes:
**Notes**
``handle_solve_dfno_rcomm`` uses reverse communication for function evaluations and monitoring steps.
Every time the solver requires an evaluation of the objective function, it pauses its progress, exits and waits for the function to be called again with the objective value provided in the argument :math:`\mathrm{f}`.
``handle_solve_dfno_rcomm`` is aimed at minimizing a nonlinear objective function subject to bound constraints:
.. math::
\begin{array}{ll} \mathrm{minimize}_{{x \in ℝ^n}} & {f\left(x\right)} \\\text{subject to}& l_x \leq x \leq u_x \text{.} \end{array}
Here :math:`f` is a smooth nonlinear function and :math:`l_x` and :math:`u_x` are :math:`n`-dimensional vectors defining bounds on the variables.
``handle_solve_dfno_rcomm`` serves as a solver for compatible problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
To define a compatible problem handle, you must call :meth:`handle_init` followed by :meth:`handle_set_nlnobj` to initialize it and optionally call :meth:`handle_set_simplebounds` to define bounds on the variables.
If :meth:`handle_set_simplebounds` is not called, all the variables will be considered free by the solver.
It should be noted that ``handle_solve_dfno_rcomm`` always assumes that the gradient of the objective is dense, therefore, defining a sparse structure for the residuals in the call to :meth:`handle_set_nlnobj` will have no effect.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver allows fixing variables with the definition of the bounds.
However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable :math:`x_i`, the value of :math:`u_x\left(i\right)-l_x\left(i\right)` must be at least twice the starting trust region radius (see the consistency constraint of the option 'DFO Starting Trust Region').
The solver is based on a derivative-free trust region framework.
This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy.
For a detailed description of the algorithm see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jef.html#algdetails>`__.
The algorithm behaviour and solver strategy can be modified by various options (see :ref:`Other Parameters <e04je-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` at any time between the initialization of the handle by :meth:`handle_init` and a call to the solver.
The options' names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFNO (Derivative-free Nonlinear Optimization).
The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem.
In particular, if the objective function is known to be noisy, it is highly recommended to set the option 'DFO Noisy Problem' to 'YES'.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
The underlying algorithm implemented for ``handle_solve_dfno_rcomm`` is the same as the one used by :meth:`handle_solve_dfno`. ``handle_solve_dfno_rcomm`` serves as a reverse communication interface to the derivative-free solver for nonlinear objective functions.
.. _e04je-py2-py-references:
**References**
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers`, Technical Report, University of Oxford
Conn, A R, Scheinberg, K and Vicente, L N, 2009, `Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization`, MPS/SIAM, Philadelphia
Powell, M J D, 2009, `The BOBYQA algorithm for bound constrained optimization without derivatives`, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
"""
raise NotImplementedError
[docs]def bounds_quasi_func_easy(ibound, funct1, bl, bu, x, liw=None, lw=None, data=None):
r"""
``bounds_quasi_func_easy`` is an easy-to-use quasi-Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds of the independent variables :math:`x_1,x_2,\ldots,x_n`, using function values only.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04jy-py2-py-doc:
For full information please refer to the NAG Library document for e04jy
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04jyf.html
.. _e04jy-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used.
It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct1** : callable fc = funct1(xc, data=None)
You must supply :math:`\mathrm{funct1}` to calculate the value of the function :math:`F\left(x\right)` at any point :math:`x`.
It should be tested separately before being used with ``bounds_quasi_func_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function value is required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_quasi_func_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_quasi_func_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to an estimate of the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**liw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`n+2`.
The dimension of the array :math:`\mathrm{iw}`.
**lw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`\max\left({n\times \left(n-1\right) /2+12\times n},13\right)`.
The dimension of the array :math:`\mathrm{w}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_quasi_func_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_quasi_func_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**iw** : int, ndarray, shape :math:`\left(\mathrm{liw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, the first :math:`\textit{n}` elements of :math:`\mathrm{iw}` contain information about which variables are currently on their bounds and which are free. Specifically, if :math:`x_i` is:
- fixed on its upper bound, :math:`\mathrm{iw}[i-1]` is :math:`-1`;
- fixed on its lower bound, :math:`\mathrm{iw}[i-1]` is :math:`-2`;
- effectively a constant (i.e., :math:`l_j = u_j`), :math:`\mathrm{iw}[i-1]` is :math:`-3`;
- free, :math:`\mathrm{iw}[i-1]` gives its position in the sequence of free variables.
In addition, :math:`\mathrm{iw}[n]` contains the number of free variables (i.e., :math:`n_z`).
The rest of the array is used as workspace.
**w** : float, ndarray, shape :math:`\left(\mathrm{lw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, :math:`\mathrm{w}[i-1]` contains a finite difference approximation to the :math:`\textit{i}`\ th element of the projected gradient vector :math:`g_z`, for :math:`\textit{i} = 1,2,\ldots,n`. In addition, :math:`\mathrm{w}[n]` contains an estimate of the condition number of the projected Hessian matrix (i.e., :math:`k`). The rest of the array is used as workspace.
.. _e04jy-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{liw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{liw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, either :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`, or :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`2`)
There have been :math:`400\times n` calls to :math:`\mathrm{funct1}`.
(`errno` :math:`4`)
An overflow occurred during computation.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct1}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
One of the forward differences was negative.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been met but a lower point could not be found and the algorithm has failed.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04jy-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_quasi_func_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}F\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when derivatives of :math:`F\left(x\right)` are unavailable.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the value of :math:`F\left(x\right)` at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the gradient and the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
An attempt is made to verify that the final point is a minimum.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The projected gradient vector :math:`g_z`, whose elements are finite difference approximations to the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
A unit lower triangular matrix :math:`L` and a diagonal matrix :math:`D` (both of dimension :math:`n_z`), such that :math:`LDL^\mathrm{T}` is a positive definite approximation of the matrix of second derivatives with respect to the free variables (i.e., the projected Hessian) are also held.
The equations
.. math::
LDL^\mathrm{T}p_z = -g_z
are solved to give a search direction :math:`p_z`, which is expanded to an :math:`n`-vector :math:`p` by an insertion of appropriate zero elements.
Then :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; :math:`x` is replaced by :math:`x+\alpha p`, and the matrices :math:`L` and :math:`D` are updated so as to be consistent with the change produced in the estimated gradient by the step :math:`\alpha p`.
If any variable actually reaches a bound during the search along :math:`p`, it is fixed and :math:`n_z` is reduced for the next iteration.
Most iterations calculate :math:`g_z` using forward differences, but central differences are used when they seem necessary.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace provided that this is practicable.
When it is not, or when the stronger convergence criteria are already satisfied, then, if one or more Lagrange multiplier estimates are close to zero, a slight perturbation is made in the values of the corresponding variables in turn until a lower function value is obtained.
The normal algorithm is then resumed from the perturbed point.
If a saddle point is suspected, a local search is carried out with a view to moving away from the saddle point.
A local search is also performed when a point is found which is thought to be a constrained minimum.
.. _e04jy-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv_comp(funct, monit, eta, ibound, bl, bu, x, lh, iprint=1, maxcal=None, xtol=0.0, delta=0.0, stepmx=100000.0, data=None, io_manager=None):
r"""
``bounds_mod_deriv_comp`` is a comprehensive modified Newton algorithm for finding:
- an unconstrained minimum of a function of several variables;
- a minimum of a function of several variables subject to fixed upper and/or lower bounds on the variables.
First derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04kd-py2-py-doc:
For full information please refer to the NAG Library document for e04kd
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kdf.html
.. _e04kd-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(iflag, xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at a specified point. (However, if you do not wish to calculate :math:`F` or its first derivatives at a particular :math:`x`, there is the option of setting an argument to cause ``bounds_mod_deriv_comp`` to terminate immediately.)
**Parameters**
**iflag** : int
Will have been set to :math:`1` or :math:`2`. The value :math:`1` indicates that only the first derivatives of :math:`F` need be supplied, and the value :math:`2` indicates that both :math:`F` itself and its first derivatives must be calculated.
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the :math:`\frac{{\partial F}}{{\partial x_j}}`, or :math:`F` and the :math:`\frac{{\partial F}}{{\partial x_j}}`, are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
Unless :math:`\mathrm{iflag} = 1` on entry, :math:`\mathrm{funct}` must set :math:`\mathrm{fc}` to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{funct}` must set :math:`\mathrm{gc}[\textit{j}-1]` to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**monit** : callable monit(xc, fc, gc, istate, gpjnrm, cond, posdef, niter, nf, data=None)
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{monit}` which is suitable for monitoring the minimization process. :math:`\mathrm{monit}` must not change the values of any of its arguments.
If :math:`\mathrm{iprint} < 0`, a :math:`\mathrm{monit}` with the correct argument list must still be supplied, although it will not be called.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fc** : float
The value of :math:`F\left(x\right)` at the current point :math:`x`.
**gc** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently fixed on their bounds and which are free.
If :math:`\mathrm{istate}[j-1]` is negative, :math:`x_j` is currently:
- fixed on its upper bound if :math:`\mathrm{istate}[j-1] = -1`
- fixed on its lower bound if :math:`\mathrm{istate}[j-1] = -2`
- effectively a constant (i.e., :math:`l_j = u_j`) if :math:`\mathrm{istate}[j-1] = -3`
If :math:`\mathrm{istate}[j-1]` is positive, its value gives the position of :math:`x_j` in the sequence of free variables.
**gpjnrm** : float
The Euclidean norm of the current projected gradient vector :math:`g_z`.
**cond** : float
The ratio of the largest to the smallest elements of the diagonal factor :math:`D` of the approximated projected Hessian matrix. This quantity is usually a good estimate of the condition number of the projected Hessian matrix. (If no variables are currently free, :math:`\mathrm{cond}` is set to zero.)
**posdef** : bool
Specifies :math:`\mathbf{True}` or :math:`\mathbf{False}` according to whether or not the approximation to the second derivative matrix for the current subspace, :math:`H`, is positive definite.
**niter** : int
The number of iterations (as outlined in :ref:`Notes <e04kd-py2-py-notes>`) which have been performed by ``bounds_mod_deriv_comp`` so far.
**nf** : int
The number of evaluations of :math:`F\left(x\right)` so far, i.e., the number of calls of :math:`\mathrm{funct}` with :math:`\mathrm{iflag}` set to :math:`2`. Each such call of :math:`\mathrm{funct}` also calculates the first derivatives of :math:`F`. (In addition to these calls monitored by :math:`\mathrm{nf}`, :math:`\mathrm{funct}` is called with :math:`\mathrm{iflag}` set to :math:`1` not more than :math:`\textit{n}` times per iteration.)
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**eta** : float
Every iteration of ``bounds_mod_deriv_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x+\alpha p\right)` with respect to :math:`\alpha`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations (and hence the number of calls of :math:`\mathrm{funct}` to estimate the second derivatives), they will tend to increase the number of calls of :math:`\mathrm{funct}` needed for each linear minimization.
On balance, it is usually more efficient to perform a low accuracy linear minimization when :math:`n` is small and a high accuracy minimization when :math:`n` is large.
`Suggested value`:
:math:`\mathrm{eta} = 0.5` if :math:`1 < n < 10`
:math:`\mathrm{eta} = 0.1` if :math:`10\leq n\leq 20`
:math:`\mathrm{eta} = 0.01` if :math:`n > 20`
**If** :math:`n = 1` **, eta should be set to** :math:`0.0` (also when the problem is effectively one-dimensional even though :math:`n > 1`; i.e., if for all except one of the variables the lower and upper bounds are equal).
**ibound** : int
Indicates whether the problem is unconstrained or bounded. If there are bounds on the variables, :math:`\mathrm{ibound}` can be used to indicate whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If the variables are bounded and you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If the problem is unconstrained.
:math:`\mathrm{ibound} = 2`
If the variables are bounded, but all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If all the variables are bounded, and :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
:math:`\mathrm{ibound} = 4`
If the problem is unconstrained. (The :math:`\mathrm{ibound} = 4` option is provided for consistency with other functions. In ``bounds_mod_deriv_comp`` it produces the same effect as :math:`\mathrm{ibound} = 1\text{.}`)
**bl** : float, array-like, shape :math:`\left(n\right)`
The fixed lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to a large negative number, e.g., :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv_comp`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bl}` will be initialized by ``bounds_mod_deriv_comp``.
**bu** : float, array-like, shape :math:`\left(n\right)`
The fixed upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any variable, the corresponding :math:`\mathrm{bu}[j-1]` should be set to a large positive number, e.g., :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv_comp`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bu}` will be initialized by ``bounds_mod_deriv_comp``.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**iprint** : int, optional
The frequency with which :math:`\mathrm{monit}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{monit}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``bounds_mod_deriv_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{monit}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{monit}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
The maximum permitted number of evaluations of :math:`F\left(x\right)`, i.e., the maximum permitted number of calls of :math:`\mathrm{funct}` with :math:`\mathrm{iflag}` set to :math:`2`. It should be borne in mind that, in addition to the calls of :math:`\mathrm{funct}` which are limited directly by :math:`\mathrm{maxcal}`, there will be calls of :math:`\mathrm{funct}` (with :math:`\mathrm{iflag}` set to :math:`1`) to evaluate only first derivatives.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that :math:`\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)` where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus, and if :math:`\mathrm{xtol}` is set to :math:`10^{-5}`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kdf.html#accuracy>`__.)
If the problem is scaled as described in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kdf.html#fcomments2>`__ and :math:`\epsilon` is the machine precision, then :math:`\sqrt{\epsilon }` is probably the smallest reasonable choice for :math:`\mathrm{xtol}`.
This is because, normally, to machine accuracy, :math:`F\left(x+\sqrt{\epsilon }e_j\right) = F\left(x\right)`, for any :math:`j` where :math:`e_j` is the :math:`j`\ th column of the identity matrix.
If you set :math:`\mathrm{xtol}` to :math:`0.0` (or any positive value less than :math:`\epsilon`), ``bounds_mod_deriv_comp`` will use :math:`10.0\times \sqrt{\epsilon }` instead of :math:`\mathrm{xtol}`.
**delta** : float, optional
The differencing interval to be used for approximating the second derivatives of :math:`F\left(x\right)`. Thus, for the finite difference approximations, the first derivatives of :math:`F\left(x\right)` are evaluated at points which are :math:`\mathrm{delta}` apart. If :math:`\epsilon` is the machine precision, :math:`\sqrt{\epsilon }` will usually be a suitable setting for :math:`\mathrm{delta}`. If you set :math:`\mathrm{delta}` to :math:`0.0` (or to any positive value less than :math:`\epsilon`), ``bounds_mod_deriv_comp`` will automatically use :math:`\sqrt{\epsilon }` as the differencing interval.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency a slight overestimate is preferable.)
``bounds_mod_deriv_comp`` will ensure that, for each iteration,
.. math::
\sqrt{\sum_{{j = 1}}^n\left[x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right]^2}\leq \mathrm{stepmx}\text{,}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``bounds_mod_deriv_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can also help to avoid possible overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bl}[0] = \mathrm{bl}[1] = \cdots = \mathrm{bl}[n-1] = 0.0`.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bu}[0] = \mathrm{bu}[1] = \cdots = \mathrm{bu}[n-1] = 10^6`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04kd-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv_comp`` (see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kdf.html#accuracy>`__).
**hesd** : float, ndarray, shape :math:`\left(n\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04kd-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv_comp`` (see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kdf.html#accuracy>`__).
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently on their bounds and which are free. If :math:`\mathrm{istate}[j-1]` is:
- equal to :math:`-1`, :math:`x_j` is fixed on its upper bound;
- equal to :math:`-2`, :math:`x_j` is fixed on its lower bound;
- equal to :math:`-3`, :math:`x_j` is effectively a constant (i.e., :math:`l_j = u_j`);
- positive, :math:`\mathrm{istate}[j-1]` gives the position of :math:`x_j` in the sequence of free variables.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The first derivative vector corresponding to the final point given in :math:`\mathrm{x}`. The components of :math:`\mathrm{g}` corresponding to free variables should normally be close to zero.
.. _e04kd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 4`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{delta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{delta}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal}` function evaluations.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
No further progress can be made.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}`.
.. _e04kd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv_comp`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n\text{.}
Special provision is made for unconstrained minimization (i.e., problems which actually have no bounds on the :math:`x_j`), problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
It is possible to specify that a particular :math:`x_j` should be held constant.
You must supply a starting point, and a :math:`\mathrm{funct}` to calculate the value of :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from their bounds.
The vector :math:`g_z`, whose elements are the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
The matrix of second derivatives with respect to the free variables, :math:`H`, is estimated by finite differences. (Note that :math:`g_z` and :math:`H` are both of dimension :math:`n_z`.) The equations
.. math::
\left(H+E\right)p_z = -g_z
are solved to give a search direction :math:`p_z`. (The matrix :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`p_z` is then expanded to an :math:`n`-vector :math:`p` by the insertion of appropriate zero elements, :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; and :math:`x` is replaced by :math:`x+\alpha p`. (If a saddle point is found, a special search is carried out so as to move away from the saddle point.) If any variable actually reaches a bound, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace until the stronger convergence criteria are satisfied.
If at this point there are no negative or near-zero Lagrange multiplier estimates, the process is terminated.
If you specify that the problem is unconstrained, ``bounds_mod_deriv_comp`` sets the :math:`l_j` to :math:`{-10^6}` and the :math:`u_j` to :math:`10^6`.
Thus, provided that the problem has been sensibly scaled, no bounds will be encountered during the minimization process and ``bounds_mod_deriv_comp`` will act as an unconstrained minimization algorithm.
.. _e04kd-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
Gill, P E and Murray, W, 1974, `Newton-type methods for unconstrained and linearly constrained optimization`, Math. Programming (7), 311--350
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def handle_solve_bounds_foas(handle, x, objfun=None, objgrd=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_bounds_foas`` is a solver from the NAG optimization modelling suite for bound-constrained large-scale Nonlinear Programming (NLP) problems.
It is a first-order active-set method (FOAS) that has low memory requirements and thus is suitable for very large-scale problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04kf-py2-py-doc:
For full information please refer to the NAG Library document for e04kf
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html
.. _e04kf-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_bounds_foas``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : float, array-like, shape :math:`\left(\textit{nvar}\right)`
:math:`x_0`, the initial estimates of the variables, :math:`x`.
**objfun** : None or callable (fx, inform) = objfun(x, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objfun}` must calculate the value of the nonlinear objective function :math:`f\left(x\right)` at a specified point :math:`x`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objfun}` will never be called by ``handle_solve_bounds_foas`` and may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function is to be evaluated.
**inform** : int
A non-negative value.
In some cases, it is known beforehand that the evaluations of the objective function and its gradient are required at the same point :math:`x`, in such cases, :math:`\mathrm{inform} = 1`.
This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient; the objective function is always evaluated before the objective gradient.
This notification parameter may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fx** : float
The value of the objective function at :math:`x`.
**inform** : int
May be used to indicate that the function cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. Returning NaN or :math:`\pm \infty` in :math:`\mathrm{fx}` has the same effect. The algorithm will try to recover if the objective cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 25.
**objgrd** : None or callable inform = objgrd(x, fdx, inform, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{objgrd}` must calculate the values of the nonlinear objective function gradient :math:`\frac{{\partial f}}{{\partial x}}` at a specified point :math:`x`.
Every call to :math:`\mathrm{objgrd}` is preceded by a call to :math:`\mathrm{objfun}` at the same point, if this is known in advance, both functions will be notified via :math:`\mathrm{inform} = 1`.
If there is no nonlinear objective (e.g., :meth:`handle_set_linobj` or :meth:`handle_set_quadobj` was called to define a linear or quadratic objective function), :math:`\mathrm{objgrd}` will never be called by ``handle_solve_bounds_foas`` and :math:`\mathrm{objgrd}` may be **None**.
If the option :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'YES'}`, then after returning from :math:`\mathrm{objgrd}` the gradient vector is checked for missing entries which you have not supplied.
Missing entries are estimated using the finite difference method, see option 'FOAS Estimate Derivatives' description for more details.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of variable values at which the objective function gradient is to be evaluated.
**fdx** : float, ndarray, shape :math:`\left(\textit{nnzfd}\right)`, to be modified in place
`On entry`: the elements should only be assigned and not referenced.
`On exit`: the values of the nonzero elements in the sparse gradient vector of the objective function, in the order specified by :math:`\textit{idxfd}` in a previous call to :meth:`handle_set_nlnobj`. :math:`\mathrm{fdx}[\textit{i}-1]` will store the gradient element :math:`\frac{{\partial f}}{{\partial x_j}}`, where :math:`j = {\textit{idxfd}}[\textit{i}-1]`.
**inform** : int
A non-negative value.
If :math:`\mathrm{inform} = 1`, then the previous call to :math:`\mathrm{objfun}` was also made at the same point :math:`x` with :math:`\mathrm{inform} = 1`.
This may help to optimize your code in order to avoid recalculations of common quantities when evaluating both the objective function and gradient.
This notification parameter may be safely ignored if such optimization is not required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**inform** : int
May be used to inform that the gradient cannot be evaluated at the requested point :math:`x` by setting :math:`\mathrm{inform} < 0`. Returning NaN or :math:`\pm \infty` in any element of :math:`\mathrm{fdx}` has the same effect. The algorithm will try to recover if the gradient cannot be evaluated; if recovery is not possible it will stop with :math:`\mathrm{errno}` = 25.
**monit** : None or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization and optionally to terminate the solver early if necessary.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the 'FOAS Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
:math:`\mathrm{monit}` may be **None**.
**Parameters**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The vector :math:`x` of decision variables at the current iteration.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables, :math:`x`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Objective function value :math:`\mathrm{f}\left(x\right)`. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Norm of inactive gradient, the objective gradient over the current search space. If the problem is unconstrained, then elements :math:`1`--:math:`3` coincide. See `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#stoppingcriteria>`__ for details. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Norm of projected direction, used in `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#projdir>`__, see `Constrained Subproblem (NPG) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#constrainedsubproblem>`__. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Norm of objective gradient. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Last step size (:math:`\alpha_k`) used in `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#npgstep>`__ and `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#iterstep>`__, see `Constrained Subproblem (NPG) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#constrainedsubproblem>`__ and `Unconstrained Subproblem (CG) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#unconstrainedsubproblem>`__.|
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Progress score, a positive value that measures the progress of the solver. A low score close to zero indicates poor progress, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#moncrit1>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#stoppingcriteria>`__. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6`--:math:`99`|Reserved for future use. |
+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the final iteration as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of function evaluations performed by NPG, see `Constrained Subproblem (NPG) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#constrainedsubproblem>`__. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Number of gradient evaluations performed by NPG. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Number of function evaluations performed by CG, see `Unconstrained Subproblem (CG) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04kff.html#unconstrainedsubproblem>`__.|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Number of gradient evaluations performed by CG. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Number of function evaluations performed by LCG. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Number of gradient evaluations performed by LCG. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Number of function evaluations used by the finite difference method to estimate missing components of the gradient. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Number of iterations. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent in the solver (including user-supplied function calls). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent in user-supplied objective function. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Total time spent in user-supplied objective gradient. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11`--:math:`99`|Reserved for future use. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. _e04kf-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'FOAS Estimate Derivatives'** : str
Default :math:`\text{} = \texttt{'NO'}`
This option indicates whether to check for and estimate missing entries of the user-supplied gradient vector.
Since the associated cost of estimating missing elements can be high, this option should only be used if strictly necessary.
In general terms, if the gradient is not provided (in its entirety or for arbitrary points) potential degradation in the progress of the solver is to be expected.
Depending on the complexity of the objective, the function may not achieve the desired optimality accuracy or even terminate with no possible further progress error :math:`\mathrm{errno}` = 24, it is advisable to increase the values of 'FOAS Stop Tolerance' and 'FOAS Rel Stop Tolerance' when using this option.
Missing elements from the gradient vector are estimated by finite-differences using the perturbation interval specified by the option 'FOAS Finite Diff Interval'.
If :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'NO'}`, the entries are not checked and all derivative elements need to be provided.
Constraint: :math:`\text{‘FOAS Estimate Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'FOAS Finite Diff Interval'** : float
Default :math:`= \sqrt{\epsilon }`
Specifies the relative perturbation size used to estimate a derivative using the forward (or backward) finite-difference method.
Setting the value too small or too big may lead ``handle_solve_bounds_foas`` to terminated with :math:`\mathrm{errno}` = 24 or 25.
Constraint: :math:`10^{-12}\leq \text{‘FOAS Finite Diff Interval'}\leq 10^{-1}`.
**'FOAS Iteration Limit'** : int
Default :math:`= 10^7`
This argument sets the maximum number of iterations to be performed by ``handle_solve_bounds_foas``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘FOAS Iteration Limit'} \geq 1`.
**'FOAS Memory'** : int
Default :math:`= 11`
This argument specifies the maximum number of memory vectors to use in the LCG solver.
Constraint: :math:`0\leq \text{‘FOAS Memory'}\leq 100`.
**'FOAS Monitor Frequency'** : int
Default :math:`= 0`
This argument specifies the frequency on which to call the monitor function :math:`\mathrm{monit}`.
If zero, the monitor function will not be called.
Constraint: :math:`\text{‘FOAS Monitor Frequency'} \geq 0`.
**'FOAS Print Frequency'** : int
Default :math:`= \texttt{'1'}`
This argument specifies the frequency with which to print information regarding each iteration to 'Print File' and/or 'Monitoring File'.
By default, it will print information of every iteration.
Constraint: :math:`\text{‘FOAS Print Frequency'} \geq 1`.
**'FOAS Progress Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{3}{4}}`
Specifies the tolerance for :math:`\epsilon_{\mathrm{prog}}` (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#moncrit1>`__) for which the function characterises a poor rate of progress given that it deems to be far from a solution.
If this behaviour is persistent, then the function asserts that no substantial further progress can be achieved and the process is terminated with :math:`\mathrm{errno}` = 24.
Setting a high tolerance can lead to misinterpret reasonable progress for unsatisfactory progress or even issue a premature stop, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#moncrit1>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0 < \text{‘FOAS Progress Tolerance'} < 1`.
**'FOAS Rel Stop Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{3}{4}}`
This argument sets the value of :math:`\epsilon_{\textit{rel}}` which specifies the relative tolerance for the convergence measures in the stopping criteria, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0\leq \epsilon_{\textit{rel}} < 1`.
**'FOAS Restart Factor'** : float
Default :math:`= 6.0`
This factor specifies the frequency :math:`\textit{nvar}\times \text{‘FOAS Restart Factor'}` with which the CG/LCG directions are replaced by the steepest descent direction (:math:`d_k = -g_k`).
Setting the value too small can potentially slow the convergence speed.
Constraint: :math:`\text{‘FOAS Restart Factor'} \geq 0`.
**'FOAS Slow Tolerance'** : float
Default :math:`\text{} = \epsilon^{\frac{1}{8}}`
Specifies the tolerance for :math:`\epsilon_{\mathrm{slow}}` (see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#moncrit2>`__) for which the function characterises a slow rate of convergence.
If this behaviour is deemed permanent, then the function asserts that no substantial improvement can be achieved and the process is terminated with :math:`\mathrm{errno}` = 50.
Setting a large tolerance can lead to incorrectly identifying a suboptimal solution, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#moncrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`\text{‘FOAS Slow Tolerance'} > 0`.
**'FOAS Stop Tolerance'** : float
Default :math:`= \mathrm{max}\left(10^{-6}, \sqrt{\epsilon }\right)`
This argument sets the value of :math:`\epsilon_{\textit{tol}}` which specifies the tolerance for the convergence measures in the stopping criteria, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit1>`__ and `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stopcrit2>`__ in `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stoppingcriteria>`__.
Constraint: :math:`0\leq \epsilon_{\textit{tol}} < 1`.
**'FOAS Tolerance Norm'** : str
Default :math:`= \texttt{'INFINITY'}`
This argument specifies the norm used to measure some stopping metrics, such as optimality tolerances (see `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#stoppingcriteria>`__).
It is possible to choose between 2-norm and ∞-norm.
Solving problems using ∞-norm generally has lower computational costs than those based on 2-norm.
Constraint: :math:`\text{‘FOAS Tolerance Norm'} = \texttt{'INFINITY'}` or :math:`\texttt{'TWO'}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'} \geq 1000`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options;
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'} \geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the option's initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but each iteration line is longer and includes step length and progress measure |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'YES'}`, the final values of the solution vector are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
**'Time Limit'** : float
Default :math:`\text{} = 10^6`
This argument specifies a limit in seconds that the solver can use to solve one problem.
If during the convergence check this limit is exceeded, the solver will terminate with :math:`\mathrm{errno}` = 23 error message.
Constraint: :math:`\text{‘Time Limit'} > 0`.
**'Verify Derivatives'** : str
Default :math:`= \texttt{'NO'}`
This argument specifies whether the function should perform numerical checks on the consistency of the user-supplied gradient function.
If any discrepancies are found, :math:`\mathrm{errno}` = 26 is returned.
It is recommended that such checks are enabled when first developing the formulation of the problem, however, the verification process results in a significant increase of the number of the function evaluations and thus it shouldn't be used in production code.
Constraint: :math:`\text{‘Verify Derivatives'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
.. _e04kf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objfun}` function.
(`errno` :math:`7`)
Please provide a proper :math:`\mathrm{objgrd}` function.
(`errno` :math:`21`)
The current starting point is unusable.
(`errno` :math:`26`)
User-provided gradient is likely to be incorrect.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Problem was solved to an acceptable level; full accuracy was not achieved.
(`errno` :math:`54`)
The problem seems to be unbounded and the algorithm was stopped.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations reached.
(`errno` :math:`23`)
The solver terminated after the maximum time allowed was exhausted.
(`errno` :math:`24`)
The solver was terminated because no further progress could be achieved.
(`errno` :math:`25`)
Invalid number detected in user-supplied function and recovery failed.
(`errno` :math:`25`)
Invalid number detected in user-supplied gradient and recovery failed.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04kf-py2-py-notes:
**Notes**
``handle_solve_bounds_foas`` solves large-scale bound-constrained nonlinear optimization problems of the form
.. math::
\textit{minimize}_{{x \in R^n}}f\left(x\right)\quad \text{ subject to }\quad \ell_x\leq x\leq u_x\text{,}
where :math:`n` is the number of decision variables, :math:`\ell_x`, :math:`u_x`, and :math:`x` are :math:`n`-dimensional vectors, and the nonlinear objective function :math:`f\left(x\right)` is assumed to be sufficiently smooth.
The solver is a first-order method (i.e., uses only first derivatives) that has very low memory requirements and, therefore, is suitable for very large bound-constrained problems.
It is based on an active-set method coupled to a nonmonotone projected gradient algorithm (NPG), nonlinear conjugate gradient method (CG) and its limited-memory variant (LCG).
The active-set method is based on alternating between both solvers, the NPG step handles the box constraints and identifies a suitable search space while the CG step explores it for a solution.
For a detailed description of the algorithm see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#algdetails>`__.
Under standard assumptions on the problem (smoothness of the first derivative of the objective) the algorithm converges to a local solution or to a critical point.
``handle_solve_bounds_foas`` serves as a solver for problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_nlnobj` or :meth:`handle_set_simplebounds` may be called to formulate the objective and to define (simple) box constraints for the problem.
Once the problem is fully described, the handle may be passed to the solver ``handle_solve_bounds_foas``.
When the handle is no longer needed, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The algorithm behaviour can be modified by various options (see :ref:`Other Parameters <e04kf-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various starting points and/or options.
Option getter :meth:`handle_opt_get` can be called to retrieve the current value of any option.
The option 'Task' may be used to switch the problem to maximization, while 'FOAS Estimate Derivatives' can be used to complete missing elements from the gradient. Option 'Verify Derivatives' may help verify the correctness of the gradient vector before starting to solve a problem.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended that you experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kff.html#algdetails>`__ and :ref:`Other Parameters <e04kf-py2-py-other_params>` for further details.
.. _e04kf-py2-py-references:
**References**
Dai, Y-H and Kou, C-X, 2013, `A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search`, SIAM J. Optim. (23(1)), 296--320
Gill, P E and Leonard, M W, 2003, `Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization`, SIAM J. Optim. (14(2)), 380--401
Hager, W W and Zhang, H, 2005, `A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search`, SIAM J. Optim. (16(1)), 170--192
Hager, W W and Zhang, H, 2006, `Algorithm 851: CG DESCENT, a Conjugate Gradient Method with Guaranteed Descent`, ACM Trans. Math. Software (32(1)), 113--137
Hager, W W and Zhang, H, 2006, `A New Active Set Algorithm for Box Constrained Optimization`, SIAM J. Optim. (17(2)), 525--557
Hager, W W and Zhang, H, 2013, `The Limited Memory Conjugate Gradient Method`, SIAM J. Optim. (23(4)), 2150--2168
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_solve_bounds_foas_ex.main`
"""
raise NotImplementedError
[docs]def bounds_quasi_deriv_easy(ibound, funct2, bl, bu, x, liw=None, lw=None, data=None):
r"""
``bounds_quasi_deriv_easy`` is an easy-to-use quasi-Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`, when first derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04ky-py2-py-doc:
For full information please refer to the NAG Library document for e04ky
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kyf.html
.. _e04ky-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply :math:`\mathrm{funct2}` to calculate the values of the function :math:`F\left(x\right)` and its first derivative :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_quasi_deriv_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_quasi_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_quasi_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**liw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`n+2`.
The dimension of the array :math:`\mathrm{iw}`.
**lw** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`\max\left({10\times n+n\times \left(n-1\right) /2},11\right)`.
The dimension of the array :math:`\mathrm{w}`.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_quasi_deriv_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_quasi_deriv_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
**iw** : int, ndarray, shape :math:`\left(\mathrm{liw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, the first :math:`\textit{n}` elements of :math:`\mathrm{iw}` contain information about which variables are currently on their bounds and which are free. Specifically, if :math:`x_i` is:
- fixed on its upper bound, :math:`\mathrm{iw}[i-1]` is :math:`-1`;
- fixed on its lower bound, :math:`\mathrm{iw}[i-1]` is :math:`-2`;
- effectively a constant (i.e., :math:`l_j = u_j`), :math:`\mathrm{iw}[i-1]` is :math:`-3`;
- free, :math:`\mathrm{iw}[i-1]` gives its position in the sequence of free variables.
In addition, :math:`\mathrm{iw}[n]` contains the number of free variables (i.e., :math:`n_z`).
The rest of the array is used as workspace.
**w** : float, ndarray, shape :math:`\left(\mathrm{lw}\right)`
If the function exits successfully or :math:`\mathrm{errno}` = 3 or 5, :math:`\mathrm{w}[\textit{i}-1]` contains the :math:`\textit{i}`\ th element of the projected gradient vector :math:`g_z`, for :math:`\textit{i} = 1,2,\ldots,n`. In addition, :math:`\mathrm{w}[n]` contains an estimate of the condition number of the projected Hessian matrix (i.e., :math:`k`). The rest of the array is used as workspace.
.. _e04ky-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{liw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{liw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lw} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lw}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`100\times n` function evaluations.
(`errno` :math:`4`)
An overflow occurred during computation.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct2}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04ky-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_quasi_deriv_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first derivatives are available.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
An attempt is made to verify that the final point is a minimum.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The projected gradient vector :math:`g_z`, whose elements are the derivatives of :math:`F\left(x\right)` with respect to the free variables, is known.
A unit lower triangular matrix :math:`L` and a diagonal matrix :math:`D` (both of dimension :math:`n_z`), such that :math:`LDL^\mathrm{T}` is a positive definite approximation of the matrix of second derivatives with respect to the free variables (i.e., the projected Hessian) are also held.
The equations
.. math::
LDL^\mathrm{T}p_z = -g_z
are solved to give a search direction :math:`p_z`, which is expanded to an :math:`n`-vector :math:`p` by an insertion of appropriate zero elements.
Then :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`; :math:`x` is replaced by :math:`x+\alpha p`, and the matrices :math:`L` and :math:`D` are updated so as to be consistent with the change produced in the gradient by the step :math:`\alpha p`.
If any variable actually reaches a bound during the search along :math:`p`, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all the active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise minimization continues in the current subspace provided that this is practicable.
When it is not, or when the stronger convergence criteria are already satisfied, then, if one or more Lagrange multiplier estimates are close to zero, a slight perturbation is made in the values of the corresponding variables in turn until a lower function value is obtained.
The normal algorithm is then resumed from the perturbed point.
If a saddle point is suspected, a local search is carried out with a view to moving away from the saddle point.
A local search is also performed when a point is found which is thought to be a constrained minimum.
.. _e04ky-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv_easy(ibound, funct2, bl, bu, x, data=None):
r"""
``bounds_mod_deriv_easy`` is an easy-to-use modified Newton algorithm for finding a minimum of a function :math:`F\left(x_1, x_2, \ldots, x_n\right)`, subject to fixed upper and lower bounds on the independent variables :math:`x_1,x_2,\ldots,x_n`, when first derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04kz-py2-py-doc:
For full information please refer to the NAG Library document for e04kz
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04kzf.html
.. _e04kz-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply this function to calculate the values of the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv_easy`` (see submodule ``opt``).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`,
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for a particular :math:`x_{\textit{j}}`, the corresponding :math:`\mathrm{bl}[\textit{j}-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for a particular :math:`x_j`, the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
.. _e04kz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` function evaluations.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in :math:`\mathrm{funct2}`, your problem has no finite solution, or the problem needs rescaling.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
.. _e04kz-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first derivatives are known.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x`.
From a starting point you supplied there is generated, on the basis of estimates of the gradient of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
.. _e04kz-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv2_comp(funct, h, monit, ibound, bl, bu, x, lh, iprint=1, maxcal=None, eta=None, xtol=0.0, stepmx=100000.0, data=None, io_manager=None):
r"""
``bounds_mod_deriv2_comp`` is a comprehensive modified Newton algorithm for finding:
an unconstrained minimum of a function of several variables
a minimum of a function of several variables subject to fixed upper and/or lower bounds on the variables.
First and second derivatives are required.
The function is intended for functions which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04lb-py2-py-doc:
For full information please refer to the NAG Library document for e04lb
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lbf.html
.. _e04lb-py2-py-parameters:
**Parameters**
**funct** : callable (fc, gc) = funct(xc, data=None)
:math:`\mathrm{funct}` must evaluate the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which :math:`F` and the :math:`\frac{{\partial F}}{{\partial x_j}}` are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
:math:`\mathrm{funct}` must set :math:`\mathrm{fc}` to the value of the objective function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{funct}` must set :math:`\mathrm{gc}[j-1]` to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**h** : callable (fhesl, fhesd) = h(xc, lh, fhesd, data=None)
:math:`\mathrm{h}` must calculate the second derivatives of :math:`F` at any point :math:`x`. (As with :math:`\mathrm{funct}`, there is the option of causing ``bounds_mod_deriv2_comp`` to terminate immediately.)
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the second derivatives of :math:`F` are required.
**lh** : int
The length of the array :math:`\mathrm{fhesl}`.
**fhesd** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
These values may be useful in the evaluation of the second derivatives.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fhesl** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
:math:`\mathrm{h}` must place the strict lower triangle of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesl}`, stored by rows, i.e., set :math:`\mathrm{fhesl}[ \left(\textit{i}-1\right) \left(\textit{i}-2\right) / 2 +\textit{j} -1] = \left. \frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}\right|_{\mathrm{xc}}`, for :math:`\textit{j} = 1,2,\ldots,i-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**fhesd** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{h}` must place the diagonal elements of the second derivative matrix of :math:`F` (evaluated at the point :math:`x`) in :math:`\mathrm{fhesd}`, i.e., set :math:`\mathrm{fhesd}[j-1] = \left. \frac{{\partial^2F}}{{\partial x_j^2}}\right|_{\mathrm{xc}}`, :math:`j = 1,2,\ldots,n`.
**monit** : callable monit(xc, fc, gc, istate, gpjnrm, cond, posdef, niter, nf, data=None)
If :math:`\mathrm{iprint}\geq 0`, you must supply :math:`\mathrm{monit}` which is suitable for monitoring the minimization process. :math:`\mathrm{monit}` must not change the values of any of its arguments.
If :math:`\mathrm{iprint} < 0`, a :math:`\mathrm{monit}` with the correct argument list should still be supplied, although it will not be called.
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The coordinates of the current point :math:`x`.
**fc** : float
The value of :math:`F\left(x\right)` at the current point :math:`x`.
**gc** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the current point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently fixed on their bounds and which are free.
If :math:`\mathrm{istate}[j-1]` is negative, :math:`x_j` is currently:
- fixed on its upper bound if :math:`\mathrm{istate}[j-1] = -1`;
- fixed on its lower bound if :math:`\mathrm{istate}[j-1] = -2`;
- effectively a constant (i.e., :math:`l_j = u_j`) if :math:`\mathrm{istate}[j-1] = -3`.
If :math:`\mathrm{istate}` is positive, its value gives the position of :math:`x_j` in the sequence of free variables.
**gpjnrm** : float
The Euclidean norm of the projected gradient vector :math:`g_z`.
**cond** : float
The ratio of the largest to the smallest elements of the diagonal factor :math:`D` of the projected Hessian matrix (see specification of :math:`\mathrm{h}`). This quantity is usually a good estimate of the condition number of the projected Hessian matrix. (If no variables are currently free, :math:`\mathrm{cond}` is set to zero.)
**posdef** : bool
Is set :math:`\mathbf{True}` or :math:`\mathbf{False}` according to whether the second derivative matrix for the current subspace, :math:`H`, is positive definite or not.
**niter** : int
The number of iterations (as outlined in :ref:`Notes <e04lb-py2-py-notes>`) which have been performed by ``bounds_mod_deriv2_comp`` so far.
**nf** : int
The number of times that :math:`\mathrm{funct}` has been called so far. Thus :math:`\mathrm{nf}` is the number of function and gradient evaluations made so far.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**ibound** : int
Specifies whether the problem is unconstrained or bounded. If there are bounds on the variables, :math:`\mathrm{ibound}` can be used to indicate whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If the variables are bounded and you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If the problem is unconstrained.
:math:`\mathrm{ibound} = 2`
If the variables are bounded, but all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If all the variables are bounded, and :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
:math:`\mathrm{ibound} = 4`
If the problem is unconstrained. (The :math:`\mathrm{ibound} = 4` option is provided purely for consistency with other functions. In ``bounds_mod_deriv2_comp`` it produces the same effect as :math:`\mathrm{ibound} = 1`.)
**bl** : float, array-like, shape :math:`\left(n\right)`
The fixed lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bl}[\textit{j}-1]` to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to a large negative number, e.g., :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv2_comp`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bl}` will be initialized by ``bounds_mod_deriv2_comp``.
**bu** : float, array-like, shape :math:`\left(n\right)`
The fixed upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, you must set :math:`\mathrm{bu}[\textit{j}-1]` to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any variable, the corresponding :math:`\mathrm{bu}[j-1]` should be set to a large positive number, e.g., :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv2_comp`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
If :math:`\mathrm{ibound}` is set to :math:`1`, :math:`2` or :math:`4`, :math:`\mathrm{bu}` will then be initialized by ``bounds_mod_deriv2_comp``.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`.
**lh** : int
The dimension of the array :math:`\mathrm{hesl}`.
**iprint** : int, optional
The frequency with which :math:`\mathrm{monit}` is to be called.
:math:`\mathrm{iprint} > 0`
:math:`\mathrm{monit}` is called once every :math:`\mathrm{iprint}` iterations and just before exit from ``bounds_mod_deriv2_comp``.
:math:`\mathrm{iprint} = 0`
:math:`\mathrm{monit}` is just called at the final point.
:math:`\mathrm{iprint} < 0`
:math:`\mathrm{monit}` is not called at all.
:math:`\mathrm{iprint}` should normally be set to a small positive number.
**maxcal** : None or int, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:`50\times n`.
The maximum permitted number of evaluations of :math:`F\left(x\right)`, i.e., the maximum permitted number of calls of :math:`\mathrm{funct}`.
**eta** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:`n = 1`: :math:`{ 0.0 }`; otherwise: :math:`{ 0.9 }`.
Every iteration of ``bounds_mod_deriv2_comp`` involves a linear minimization (i.e., minimization of :math:`F\left(x+\alpha p\right)` with respect to :math:`\alpha`). :math:`\mathrm{eta}` specifies how accurately these linear minimizations are to be performed. The minimum with respect to :math:`\alpha` will be located more accurately for small values of :math:`\mathrm{eta}` (say, :math:`0.01`) than for large values (say, :math:`0.9`).
Although accurate linear minimizations will generally reduce the number of iterations of ``bounds_mod_deriv2_comp``, this usually results in an increase in the number of function and gradient evaluations required for each iteration.
On balance, it is usually more efficient to perform a low accuracy linear minimization.
**xtol** : float, optional
The accuracy in :math:`x` to which the solution is required.
If :math:`x_{\mathrm{true}}` is the true value of :math:`x` at the minimum, then :math:`x_{\mathrm{sol}}`, the estimated position before a normal exit, is such that :math:`\left\lVert x_{\mathrm{sol}}-x_{\mathrm{true}}\right\rVert < \mathrm{xtol}\times \left(1.0+\left\lVert x_{\mathrm{true}}\right\rVert \right)`, where :math:`\left\lVert y\right\rVert = \sqrt{\sum_{{j = 1}}^ny_j^2}`.
For example, if the elements of :math:`x_{\mathrm{sol}}` are not much larger than :math:`1.0` in modulus, and if :math:`\mathrm{xtol}` is set to :math:`10^{-5}`, then :math:`x_{\mathrm{sol}}` is usually accurate to about five decimal places. (For further details see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lbf.html#accuracy>`__.)
If the problem is scaled roughly as described in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lbf.html#fcomments>`__ and :math:`\epsilon` is the machine precision, then :math:`\sqrt{\epsilon }` is probably the smallest reasonable choice for :math:`\mathrm{xtol}`. (This is because, normally, to machine accuracy, :math:`F\left({x+\sqrt{\epsilon }}, e_j\right) = F\left(x\right)` where :math:`e_j` is any column of the identity matrix.)
If you set :math:`\mathrm{xtol}` to :math:`0.0` (or any positive value less than :math:`\epsilon`), ``bounds_mod_deriv2_comp`` will use :math:`10.0\times \sqrt{\epsilon }` instead of :math:`\mathrm{xtol}`.
**stepmx** : float, optional
An estimate of the Euclidean distance between the solution and the starting point supplied by you. (For maximum efficiency a slight overestimate is preferable.)
``bounds_mod_deriv2_comp`` will ensure that, for each iteration,
.. math::
\sqrt{\sum_{{j = 1}}^n\left[x_j^{\left(k\right)}-x_j^{\left(k-1\right)}\right]^2}\leq \mathrm{stepmx}
where :math:`k` is the iteration number.
Thus, if the problem has more than one solution, ``bounds_mod_deriv2_comp`` is most likely to find the one nearest to the starting point.
On difficult problems, a realistic choice can prevent the sequence of :math:`x^{\left(k\right)}` entering a region where the problem is ill-behaved and can also help to avoid possible overflow in the evaluation of :math:`F\left(x\right)`.
However, an underestimate of :math:`\mathrm{stepmx}` can lead to inefficiency.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv2_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bl}[0] = \mathrm{bl}[1] = \cdots = \mathrm{bl}[n-1] = 0.0`.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv2_comp``, e.g., if :math:`\mathrm{ibound} = 2`, :math:`\mathrm{bu}[0] = \mathrm{bu}[1] = \cdots = \mathrm{bu}[n-1] = 10^6`.
**x** : float, ndarray, shape :math:`\left(n\right)`
The final point :math:`x^{\left(k\right)}`. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the estimated position of the minimum.
**hesl** : float, ndarray, shape :math:`\left(\mathrm{lh}\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04lb-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full estimated second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final estimated second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the results produced by ``bounds_mod_deriv2_comp`` (see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lbf.html#accuracy>`__).
**hesd** : float, ndarray, shape :math:`\left(n\right)`
During the determination of a direction :math:`p_z` (see :ref:`Notes <e04lb-py2-py-notes>`), :math:`H+E` is decomposed into the product :math:`LDL^\mathrm{T}`, where :math:`L` is a unit lower triangular matrix and :math:`D` is a diagonal matrix. (The matrices :math:`H`, :math:`E`, :math:`L` and :math:`D` are all of dimension :math:`n_z`, where :math:`n_z` is the number of variables free from their bounds. :math:`H` consists of those rows and columns of the full second derivative matrix which relate to free variables. :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` are used to store the factors :math:`L` and :math:`D`.
The elements of the strict lower triangle of :math:`L` are stored row by row in the first :math:`n_z\left(n_z-1\right)/2` positions of :math:`\mathrm{hesl}`.
The diagonal elements of :math:`D` are stored in the first :math:`n_z` positions of :math:`\mathrm{hesd}`.
In the last factorization before a normal exit, the matrix :math:`E` will be zero, so that :math:`\mathrm{hesl}` and :math:`\mathrm{hesd}` will contain, on exit, the factors of the final second derivative matrix :math:`H`.
The elements of :math:`\mathrm{hesd}` are useful for deciding whether to accept the result produced by ``bounds_mod_deriv2_comp`` (see `Accuracy <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lbf.html#accuracy>`__).
**istate** : int, ndarray, shape :math:`\left(n\right)`
Information about which variables are currently on their bounds and which are free. If :math:`\mathrm{istate}[j-1]` is:
- equal to :math:`-1`, :math:`x_j` is fixed on its upper bound;
- equal to :math:`-2`, :math:`x_j` is fixed on its lower bound;
- equal to :math:`-3`, :math:`x_j` is effectively a constant (i.e., :math:`l_j = u_j`);
- positive, :math:`\mathrm{istate}[j-1]` gives the position of :math:`x_j` in the sequence of free variables.
**f** : float
The function value at the final point given in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The first derivative vector corresponding to the final point given in :math:`\mathrm{x}`. The components of :math:`\mathrm{g}` corresponding to free variables should normally be close to zero.
.. _e04lb-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 4`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{maxcal} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxcal}\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{stepmx} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{stepmx}\geq \mathrm{xtol}`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{eta} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0.0\leq \mathrm{eta} < 1.0`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{xtol} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xtol}\geq 0.0`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{lh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{lh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`2`)
There have been :math:`\mathrm{maxcal}` function evaluations.
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
No further progress can be made.
**NagCallbackTerminateWarning**
(`errno` :math:`i < 0`)
User requested termination by setting :math:`\mathrm{iflag}` negative in :math:`\mathrm{funct}` or :math:`\mathrm{h}`.
.. _e04lb-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``bounds_mod_deriv2_comp`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}F\left(x_1, x_2, \ldots, x_n\right)\text{subject to }l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n\text{.}
Special provision is made for unconstrained minimization (i.e., problems which actually have no bounds on the :math:`x_j`), problems which have only non-negativity bounds, and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
It is possible to specify that a particular :math:`x_j` should be held constant.
You must supply a starting point, a :math:`\mathrm{funct}` to calculate the value of :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`, and a :math:`\mathrm{h}` to calculate the second derivatives :math:`\frac{{\partial^2F}}{{\partial x_i\partial x_j}}`.
A typical iteration starts at the current point :math:`x` where :math:`n_z` (say) variables are free from both their bounds.
The vector of first derivatives of :math:`F\left(x\right)` with respect to the free variables, :math:`g_z`, and the matrix of second derivatives with respect to the free variables, :math:`H`, are obtained. (These both have dimension :math:`n_z`.)
The equations
.. math::
\left(H+E\right)p_z = -g_z
are solved to give a search direction :math:`p_z`. (The matrix :math:`E` is chosen so that :math:`H+E` is positive definite.)
:math:`p_z` is then expanded to an :math:`n`-vector :math:`p` by the insertion of appropriate zero elements; :math:`\alpha` is found such that :math:`F\left(x+\alpha p\right)` is approximately a minimum (subject to the fixed bounds) with respect to :math:`\alpha`, and :math:`x` is replaced by :math:`x+\alpha p`. (If a saddle point is found, a special search is carried out so as to move away from the saddle point.)
If any variable actually reaches a bound, it is fixed and :math:`n_z` is reduced for the next iteration.
There are two sets of convergence criteria -- a weaker and a stronger.
Whenever the weaker criteria are satisfied, the Lagrange multipliers are estimated for all active constraints.
If any Lagrange multiplier estimate is significantly negative, then one of the variables associated with a negative Lagrange multiplier estimate is released from its bound and the next search direction is computed in the extended subspace (i.e., :math:`n_z` is increased).
Otherwise, minimization continues in the current subspace until the stronger criteria are satisfied.
If at this point there are no negative or near-zero Lagrange multiplier estimates, the process is terminated.
If you specify that the problem is unconstrained, ``bounds_mod_deriv2_comp`` sets the :math:`l_j` to :math:`{-10^6}` and the :math:`u_j` to :math:`10^6`.
Thus, provided that the problem has been sensibly scaled, no bounds will be encountered during the minimization process and ``bounds_mod_deriv2_comp`` will act as an unconstrained minimization algorithm.
.. _e04lb-py2-py-references:
**References**
Gill, P E and Murray, W, 1973, `Safeguarded steplength algorithms for optimization using descent methods`, NPL Report NAC 37, National Physical Laboratory
Gill, P E and Murray, W, 1974, `Newton-type methods for unconstrained and linearly constrained optimization`, Math. Programming (7), 311--350
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def bounds_mod_deriv2_easy(ibound, funct2, hess2, bl, bu, x, data=None):
r"""
``bounds_mod_deriv2_easy`` is an easy-to-use modified-Newton algorithm for finding a minimum of a function, :math:`F\left(x_1, x_2, \ldots, x_n\right)` subject to fixed upper and lower bounds on the independent variables, :math:`x_1,x_2,\ldots,x_n` when first and second derivatives of :math:`F` are available.
It is intended for functions which are continuous and which have continuous first and second derivatives (although it will usually work even if the derivatives have occasional discontinuities).
.. _e04ly-py2-py-doc:
For full information please refer to the NAG Library document for e04ly
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04lyf.html
.. _e04ly-py2-py-parameters:
**Parameters**
**ibound** : int
Indicates whether the facility for dealing with bounds of special forms is to be used. It must be set to one of the following values:
:math:`\mathrm{ibound} = 0`
If you are supplying all the :math:`l_j` and :math:`u_j` individually.
:math:`\mathrm{ibound} = 1`
If there are no bounds on any :math:`x_j`.
:math:`\mathrm{ibound} = 2`
If all the bounds are of the form :math:`0\leq x_j`.
:math:`\mathrm{ibound} = 3`
If :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
**funct2** : callable (fc, gc) = funct2(xc, data=None)
You must supply this function to calculate the values of the function :math:`F\left(x\right)` and its first derivatives :math:`\frac{{\partial F}}{{\partial x_j}}` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv2_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the function and its derivatives are required.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**fc** : float
The value of the function :math:`F` at the current point :math:`x`.
**gc** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{gc}[\textit{j}-1]` must be set to the value of the first derivative :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` at the point :math:`x`, for :math:`\textit{j} = 1,2,\ldots,n`.
**hess2** : callable (heslc, hesdc) = hess2(xc, lh, data=None)
You must supply this function to evaluate the elements :math:`H_{{ij}} = \frac{{\partial^2F}}{{\partial x_i\partial x_j}}` of the matrix of second derivatives of :math:`F\left(x\right)` at any point :math:`x`.
It should be tested separately before being used in conjunction with ``bounds_mod_deriv2_easy`` (see `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html>`__).
**Parameters**
**xc** : float, ndarray, shape :math:`\left(n\right)`
The point :math:`x` at which the derivatives are required.
**lh** : int
The length of the array :math:`\mathrm{heslc}`.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**Returns**
**heslc** : float, array-like, shape :math:`\left(\mathrm{lh}\right)`
:math:`\mathrm{hess2}` must place the strict lower triangle of the second derivative matrix :math:`H` in :math:`\mathrm{heslc}`, stored by rows, i.e., set :math:`\mathrm{heslc}[\left(\textit{i}-1\right)\left(\textit{i}-2\right)/2+\textit{j}-1] = \frac{{\partial^2F}}{{\partial x_{\textit{i}}\partial x_{\textit{j}}}}`, for :math:`\textit{j} = 1,2,\ldots,\textit{i}-1`, for :math:`\textit{i} = 2,3,\ldots,n`. (The upper triangle is not required because the matrix is symmetric.)
**hesdc** : float, array-like, shape :math:`\left(n\right)`
Must contain the diagonal elements of the second derivative matrix, i.e., set :math:`\mathrm{hesdc}[\textit{j}-1] = \frac{{\partial^2F}}{{\partial x_{\textit{j}}^2}}`, for :math:`\textit{j} = 1,2,\ldots,n`.
**bl** : float, array-like, shape :math:`\left(n\right)`
The lower bounds :math:`l_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, :math:`\mathrm{bl}[\textit{j}-1]` must be set to :math:`l_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If a lower bound is not specified for any :math:`x_j`, the corresponding :math:`\mathrm{bl}[j-1]` should be set to :math:`{-10^6}`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bl}[0]` to :math:`l_1`; ``bounds_mod_deriv2_easy`` will then set the remaining elements of :math:`\mathrm{bl}` equal to :math:`\mathrm{bl}[0]`.
**bu** : float, array-like, shape :math:`\left(n\right)`
The upper bounds :math:`u_j`.
If :math:`\mathrm{ibound}` is set to :math:`0`, :math:`\mathrm{bu}[\textit{j}-1]` must be set to :math:`u_{\textit{j}}`, for :math:`\textit{j} = 1,2,\ldots,n`. (If an upper bound is not specified for any :math:`x_j` the corresponding :math:`\mathrm{bu}[j-1]` should be set to :math:`10^6`.)
If :math:`\mathrm{ibound}` is set to :math:`3`, you must set :math:`\mathrm{bu}[0]` to :math:`u_1`; ``bounds_mod_deriv2_easy`` will then set the remaining elements of :math:`\mathrm{bu}` equal to :math:`\mathrm{bu}[0]`.
**x** : float, array-like, shape :math:`\left(n\right)`
:math:`\mathrm{x}[\textit{j}-1]` must be set to a guess at the :math:`\textit{j}`\ th component of the position of the minimum, for :math:`\textit{j} = 1,2,\ldots,n`. The function checks the gradient and the Hessian matrix at the starting point, and is more likely to detect any error in your programming if the initial :math:`\mathrm{x}[j-1]` are nonzero and mutually distinct.
**data** : arbitrary, optional
User-communication data for callback functions.
**Returns**
**bl** : float, ndarray, shape :math:`\left(n\right)`
The lower bounds actually used by ``bounds_mod_deriv2_easy``.
**bu** : float, ndarray, shape :math:`\left(n\right)`
The upper bounds actually used by ``bounds_mod_deriv2_easy``.
**x** : float, ndarray, shape :math:`\left(n\right)`
The lowest point found during the calculations. Thus, if no exception or warning is raised on exit, :math:`\mathrm{x}[j-1]` is the :math:`j`\ th component of the position of the minimum.
**f** : float
The value of :math:`F\left(x\right)` corresponding to the final point stored in :math:`\mathrm{x}`.
**g** : float, ndarray, shape :math:`\left(n\right)`
The value of :math:`\frac{{\partial F}}{{\partial x_{\textit{j}}}}` corresponding to the final point stored in :math:`\mathrm{x}`, for :math:`\textit{j} = 1,2,\ldots,n`; the value of :math:`\mathrm{g}[j-1]` for variables not on a bound should normally be close to zero.
.. _e04ly-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 0` and :math:`\mathrm{bl}[\textit{j}-1] > \mathrm{bu}[\textit{j}-1]` for some :math:`j`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = 3` and :math:`\mathrm{bl}[0] > \mathrm{bu}[0]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ibound} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ibound}\leq 3`.
(`errno` :math:`1`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n\geq 1`.
(`errno` :math:`2`)
There have been :math:`50\times n` function evaluations.
(`errno` :math:`9`)
The modulus of a variable has become very large. There may be a mistake in your supplied functions, your problem has no finite solution, or the problem needs rescaling.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`3`)
The conditions for a minimum have not all been satisfied, but a lower point could not be found.
(`errno` :math:`5`)
It is probable that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`6`)
It is possible that a local minimum has been found, but it cannot be guaranteed.
(`errno` :math:`7`)
It is unlikely that a local minimum has been found.
(`errno` :math:`8`)
It is very unlikely that a local minimum has been found.
(`errno` :math:`10`)
It is very likely that you have made an error forming the gradient.
(`errno` :math:`11`)
It is very likely that you have made an error forming the 2nd derivatives.
.. _e04ly-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``bounds_mod_deriv2_easy`` is applicable to problems of the form:
.. math::
\mathrm{Minimize}\left(F\right)\left(x_1, x_2, \ldots, x_n\right)\quad \text{ subject to }\quad l_j\leq x_j\leq u_j\text{, }\quad j = 1,2,\ldots,n
when first and second derivatives of :math:`F\left(x\right)` are available.
Special provision is made for problems which actually have no bounds on the :math:`x_j`, problems which have only non-negativity bounds and problems in which :math:`l_1 = l_2 = \cdots = l_n` and :math:`u_1 = u_2 = \cdots = u_n`.
You must supply a function to calculate the values of :math:`F\left(x\right)` and its first derivatives at any point :math:`x` and a function to calculate the second derivatives.
From a starting point you supplied there is generated, on the basis of estimates of the curvature of :math:`F\left(x\right)`, a sequence of feasible points which is intended to converge to a local minimum of the constrained function.
.. _e04ly-py2-py-references:
**References**
Gill, P E and Murray, W, 1976, `Minimization subject to bounds on the variables`, NPL Report NAC 72, National Physical Laboratory
"""
raise NotImplementedError
[docs]def lp_solve(bl, bu, x, comm, a=None, cvec=None, istate=None, io_manager=None):
r"""
``lp_solve`` solves general linear programming problems.
It is not intended for large sparse problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`lp_option_file`, :meth:`lp_option_string`, :meth:`nlp1_init`.
.. _e04mf-py2-py-doc:
For full information please refer to the NAG Library document for e04mf
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mff.html
.. _e04mf-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**a** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; if :math:`\textit{nclin}=0`: :math:`1`; otherwise: :math:`0`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{a}` must contain the coefficients of the :math:`\textit{i}`\ th general linear constraint, for :math:`\textit{i} = 1,2,\ldots,m_L`.
**cvec** : None or float, array-like, shape :math:`\left(n\right)`, optional
The coefficients of the objective function when the problem is of type LP.
If the problem is of type FP, :math:`\mathrm{cvec}` is not referenced.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{istate}` specifies the desired status of the constraints at the start of the feasibility phase.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, and the next :math:`m_L` elements refer to the general linear constraints (if any).
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+========================================================================================================================================================+
|0 |The corresponding constraint should `not` be in the initial working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraint should be in the initial working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraint should be in the initial working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The constraint should be in the initial working set as an equality. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be reset to zero by the function.
If ``lp_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\textit{nclin}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The status of the constraints in the working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+======================================================================================================================================================================================================================+
|:math:`-2` |The constraint violates its lower bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |The constraint violates its upper bound by more than the feasibility tolerance. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the working set. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the working set at its lower bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the working set at its upper bound. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |This constraint is included in the working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`. |
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |This corresponds to optimality being declared with :math:`\mathrm{x}[j-1]` being temporarily fixed at its current value. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{errno}` = 1 on exit.|
+----------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**x** : float, ndarray, shape :math:`\left(n\right)`
The point at which ``lp_solve`` terminated. If the function exits successfully or :math:`\mathrm{errno}` = 1 or 4, :math:`\mathrm{x}` contains an estimate of the solution.
**itera** : int
The total number of iterations performed.
**obj** : float
The value of the objective function at :math:`x` if :math:`x` is feasible, or the sum of infeasibiliites at :math:`x` otherwise. If the problem is of type FP and :math:`x` is feasible, :math:`\mathrm{obj}` is set to zero.
**ax** : None or float, ndarray, shape :math:`\left(\max\left(1,\textit{nclin}\right)\right)`
The final values of the linear constraints :math:`Ax`.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The values of the Lagrange multipliers for each constraint with respect to the current working set. The first :math:`n` elements contain the multipliers for the bound constraints on the variables, and the next :math:`m_L` elements contain the multipliers for the general linear constraints (if any). If :math:`\mathrm{istate}[j-1] = 0` (i.e., constraint :math:`j` is not in the working set), :math:`\mathrm{clamda}[j-1]` is zero. If :math:`x` is optimal, :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1`, non-positive if :math:`\mathrm{istate}[j-1] = 2` and zero if :math:`\mathrm{istate}[j-1] = 4`.
.. _e04mf-py2-py-other_params:
**Other Parameters**
**'Check Frequency'** : int
Default :math:`\text{} = 50`
Every :math:`i`\ th iteration, a numerical test is made to see if the current solution :math:`x` satisfies the constraints in the working set.
If the largest residual of the constraints in the working set is judged to be too large, the current working set is refactorized and the variables are recomputed to satisfy the constraints more accurately.
If :math:`i\leq 0`, the default value is used.
**'Cold Start'** : valueless
Default
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lp_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lp_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lp_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lp_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lp_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lp_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``lp_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`a_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert a_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Expand Frequency'** : int
Default :math:`\text{} = 5`
This option is part of an anti-cycling procedure designed to guarantee progress even on highly degenerate problems.
The strategy is to force a positive step at every iteration, at the expense of violating the constraints by a small amount.
Suppose that the value of the option 'Feasibility Tolerance' is :math:`\delta`.
Over a period of :math:`i` iterations, the feasibility tolerance actually used by ``lp_solve`` (i.e., the `working` feasibility tolerance) increases from :math:`0.5\delta` to :math:`\delta` (in steps of :math:`0.5\delta /i`).
At certain stages the following 'resetting procedure' is used to remove constraint infeasibilities.
First, all variables whose upper or lower bounds are in the working set are moved exactly onto their bounds.
A count is kept of the number of nontrivial adjustments made.
If the count is positive, iterative refinement is used to give variables that satisfy the working set to (essentially) machine precision.
Finally, the working feasibility tolerance is reinitialized to :math:`0.5\delta`.
If a problem requires more than :math:`i` iterations, the resetting procedure is invoked and a new cycle of :math:`i` iterations is started with :math:`i` incremented by :math:`10`. (The decision to resume the feasibility phase or optimality phase is based on comparing any constraint infeasibilities with :math:`\delta`.)
The resetting procedure is also invoked when ``lp_solve`` reaches an apparently optimal, infeasible or unbounded solution, unless this situation has already occurred twice.
If any nontrivial adjustments are made, iterations are continued.
If :math:`i\leq 0`, the default value is used.
If :math:`i\geq 9999999`, no anti-cycling procedure is invoked.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
If :math:`r\geq \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point.
For example, if the variables and the coefficients in the general constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-6}`.
If :math:`0\leq r < \epsilon`, the default value is used.
``lp_solve`` attempts to find a feasible solution before optimizing the objective function.
If the sum of infeasibilities cannot be reduced to zero, the option 'Minimum Sum of Infeasibilities' can be used to find the minimum value of the sum.
Let Sinf be the corresponding sum of infeasibilities.
If Sinf is quite small, it may be appropriate to raise :math:`r` by a factor of :math:`10` or :math:`100`.
Otherwise, some error in the data should be suspected.
Note that a 'feasible solution' is a solution that satisfies the current constraints to within the tolerance :math:`r`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the problem is of type LP.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+m_L\right)}\right)`
The value of :math:`i` specifies the maximum number of iterations allowed before termination.
With :math:`i = 0` and :math:`\text{‘Print Level'} > 0`, the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i < 0`, the default value is used.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Minimum Sum of Infeasibilities'** : str
Default :math:`= \mathrm{NO}`
If no feasible point exists for the constraints, this option is used to control whether or not ``lp_solve`` will calculate a point that minimizes the constraint violations.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'NO'}`, ``lp_solve`` will terminate as soon as it is evident that no feasible point exists for the constraints.
The final point will generally not be the point at which the sum of infeasibilities is minimized.
If :math:`\text{‘Minimum Sum of Infeasibilities'} = \texttt{'YES'}`, ``lp_solve`` will continue until the sum of infeasibilities is minimized.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Print Level'} \geq 5`, monitoring information produced by ``lp_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} < 5`, no monitoring information is produced.
**'Optimality Tolerance'** : float
Default :math:`\text{} = \epsilon^{0.8}`
If :math:`r\geq \epsilon`, :math:`r` defines the tolerance used to determine if the bounds and general constraints have the right 'sign' for the solution to be judged to be optimal.
If :math:`0\leq r < \epsilon`, the default value is used.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``lp_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mff.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mff.html#monitoring>`__ (monitoring information at each iteration).
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+===================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mff.html#fc-printedoutput>`__) for each iteration (no printout of the final solution).|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+==========================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mff.html#monitoring>`__) for each iteration (no printout of the final solution). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each iteration, the Lagrange multipliers, the variables :math:`x`, the constraint values :math:`Ax` and the constraint status (see :math:`\mathrm{istate}`). |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each iteration, the diagonal elements of the upper triangular matrix :math:`T` associated with the :math:`TQ` factorization `(3) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mff.html#eqn3>`__ (see `Definition of Search Direction <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mff.html#ad-search>`__) of the working set.|
+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Problem Type'** : str
Default :math:`=` LP
This option specifies the type of objective function to be minimized during the optimality phase.
The following is the optional keyword and the dimensions of the array that must be specified in order to define the objective function:
.. rst-class:: nag-rules-none
+--+------------------------------------------------------------------------------+
|LP|length-:math:`\textit{n}` :math:`\mathrm{cvec}` required. |
+--+------------------------------------------------------------------------------+
|FP|the objective function is omitted and :math:`\mathrm{cvec}` is not referenced.|
+--+------------------------------------------------------------------------------+
For problems of type FP, the objective function is omitted and :math:`\mathrm{cvec}` is not referenced.
The following keywords are also acceptable.
.. rst-class:: nag-rules-none nag-align-left
+---------+------+
|:math:`a`|Option|
+=========+======+
|Linear |LP |
+---------+------+
|Feasible |FP |
+---------+------+
.. _e04mf-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`6`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Cold Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
'Problem Type' not recognized. Problem abandoned.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Weak :math:`\langle\mathit{\boldsymbol{value}}\rangle` solution.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` solution is unbounded.
(`errno` :math:`3`)
No feasible point for the linear constraints.
(`errno` :math:`3`)
Cannot satisfy the working-set constraints to the accuracy requested.
.. _e04mf-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lp_solve`` is designed to solve Linear Programming (LP) problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}c^\mathrm{T}x\text{, subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{,}
where :math:`c` is an :math:`n`-element vector and :math:`A` is an :math:`m_L\times n` matrix.
This is the default type of problem, referred to as type LP.
The option 'Problem Type' may be used to specify an alternative problem type FP, in which the objective function is omitted and the function attempts to find a feasible point for the set of constraints.
The constraints involving :math:`A` are called the `general` constraints.
Note that upper and lower bounds are specified for all the variables and for all the general constraints.
An `equality` constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
You must supply an initial estimate of the solution.
The method used by ``lp_solve`` is described in detail in `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mff.html#algdetails>`__.
.. _e04mf-py2-py-references:
**References**
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E and Murray, W, 1978, `Numerically stable methods for quadratic programming`, Math. Programming (14), 349--372
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1989, `A practical anti-cycling procedure for linearly constrained optimization`, Math. Programming (45), 437--474
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1991, `Inertia-controlling methods for general quadratic programming`, SIAM Rev. (33), 1--36
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
"""
raise NotImplementedError
[docs]def lp_option_file(ioptns, comm, io_manager=None):
r"""
``lp_option_file`` may be used to supply options to :meth:`lp_solve` from an external file.
.. _e04mg-py2-py-doc:
For full information please refer to the NAG Library document for e04mg
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mgf.html
.. _e04mg-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mg-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04mg-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lp_option_file`` may be used to supply values for options to :meth:`lp_solve`. ``lp_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lp_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lp_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lp_option_string(optstr, comm, io_manager=None):
r"""
``lp_option_string`` may be used to supply individual options to :meth:`lp_solve`.
.. _e04mh-py2-py-doc:
For full information please refer to the NAG Library document for e04mh
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mhf.html
.. _e04mh-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04mh-py2-py-notes>` and in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mh-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04mh-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lp_option_string`` may be used to supply values for options to :meth:`lp_solve`.
It is only necessary to call ``lp_option_string`` for those arguments whose values are to be different from their default values.
One call to ``lp_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
Printing of user-specified options is turned off by default.
It may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lp_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lp_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lp_solve <e04mf-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def handle_solve_lp_ipm(handle, x=None, u=None, monit=None, data=None, io_manager=None):
r"""
``handle_solve_lp_ipm`` is a solver from the NAG optimization modelling suite for large-scale Linear Programming (LP) problems based on an interior point method (IPM).
Note: this function uses optional algorithmic parameters, see also: :meth:`handle_opt_set`, :meth:`handle_opt_get`.
.. _e04mt-py2-py-doc:
For full information please refer to the NAG Library document for e04mt
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html
.. _e04mt-py2-py-parameters:
**Parameters**
**handle** : Handle
The handle to the problem. It needs to be initialized (e.g., by :meth:`handle_init`) and to hold a problem formulation compatible with ``handle_solve_lp_ipm``. It **must not** be changed between calls to the NAG optimization modelling suite.
**x** : None or float, array-like, shape :math:`\left(\textit{nvar}\right)`, optional
The input of :math:`\mathrm{x}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**u** : None or float, array-like, shape :math:`\left(\textit{nnzu}\right)`, optional
Note: if :math:`\textit{nnzu} > 0`, :math:`\mathrm{u}` holds Lagrange multipliers (dual variables) for the bound constraints and linear constraints. If :math:`\textit{nnzu} = 0`, :math:`\mathrm{u}` will not be referenced.
The input of :math:`\mathrm{u}` is reserved for future releases of the NAG Library and it is ignored at the moment.
**monit** : None or callable monit(handle, rinfo, stats, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.
:math:`\mathrm{monit}` is provided to enable you to monitor the progress of the optimization.
It is invoked at the end of every :math:`i`\ th iteration where :math:`i` is given by the option 'LPIPM Monitor Frequency' (the default is :math:`0`, :math:`\mathrm{monit}` is not called).
**Parameters**
**handle** : Handle
The handle to the problem as provided on entry to ``handle_solve_lp_ipm``. It may be used to query the model during the solve, and extract the current approximation of the solution by :meth:`handle_set_get_real`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators at the end of the current iteration as described in :math:`\mathrm{rinfo}`.
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics at the end of the current iteration as described in :math:`\mathrm{stats}`, however, elements :math:`2`, :math:`3`, :math:`5`, :math:`9`, :math:`10`, :math:`11` and :math:`15` refer to the quantities in the last iteration rather than accumulated over all iterations through the whole algorithm run.
**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.
**data** : arbitrary, optional
User-communication data for callback functions.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**x** : float, ndarray, shape :math:`\left(\textit{nvar}\right)`
The final values of the variables :math:`x`.
**u** : float, ndarray, shape :math:`\left(\textit{nnzu}\right)`
The final values of the variables :math:`u`.
**rinfo** : float, ndarray, shape :math:`\left(100\right)`
Error measures and various indicators of the algorithm (see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails>`__ for details) as given in the table below:
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Value of the primal objective. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Value of the dual objective. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Flag indicating the system formulation used by the solver, :math:`0`: augmented system, :math:`1`: normal equation. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Factorization type, :math:`3`: Cholesky, :math:`4`: Bunch--Parlett. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4`--:math:`13` |Primal-Dual specific information (will be :math:`0` if the Self-Dual algorithm is chosen). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`4` |Relative dual feasibility (optimality), see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#pdopt>`__. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`5` |Relative primal feasibility, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#pdfeas>`__. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`6` |Relative duality gap (complementarity), see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#pdcompl>`__. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`7` |Average complementarity error :math:`\mu` (see `The Infeasible-interior-point Primal-Dual Algorithm <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#algdetails_pd>`__).||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`8` |Centring parameter :math:`\sigma` (see `The Infeasible-interior-point Primal-Dual Algorithm <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#algdetails_pd>`__). ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`9` |Primal step length. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`10` |Dual step length. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
| ||:math:`11`--:math:`13`|Reserved for future use. ||
| |+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14`--:math:`23`|Self-Dual specific information (will be :math:`0` if the Primal-Dual algorithm is chosen). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`14` |Relative primal infeasibility, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#sdfeas>`__.| |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`15` |Relative dual infeasibility, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#sdopt>`__. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`16` |Relative duality gap, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#sdgap>`__. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`17` |Accuracy, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#sdacc>`__. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`18` |:math:`\tau`, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#hlf>`__. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`19` |:math:`\kappa`, see `[equation] <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04mtf.html#hlf>`__. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`20` |Step length. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
| ||:math:`21`--:math:`23`|Reserved for future use. | |
| |+----------------------+-----------------------------------------------------------------------------------------------------------------------------+ |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`24`--:math:`99`|Reserved for future use. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**stats** : float, ndarray, shape :math:`\left(100\right)`
Solver statistics as given in the table below. Note that time statistics are provided only if 'Stats Time' is set (the default is 'NO'), the measured time is returned in seconds.
.. rst-class:: nag-rules-none nag-align-left
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |Number of iterations. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Total number of centrality correction steps performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Total number of iterative refinements performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |Value of the perturbation added to the diagonal in the normal equation formulation or on the zero block in the augmented system formulation.|
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |Total number of factorizations performed. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |Total time spent in the solver. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`6` |Time spent in the presolve phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`7` |Time spent in the last iteration. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`8` |Total time spent factorizing the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`9` |Total time spent backsolving the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`10` |Total time spent in the multiple centrality correctors phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`11` |Time spent in the initialization phase. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`12` |Number of nonzeros in the system matrix. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`13` |Number of nonzeros in the system matrix factor. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`14` |Maximum error of the backsolve. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`15` |Number of columns in :math:`A` considered dense by the solver. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`16` |Maximum number of centrality corrector steps. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`17`--:math:`99`|Reserved for future use. |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
.. _e04mt-py2-py-other_params:
**Other Parameters**
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
Any value given with this keyword will be ignored.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
This defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: :math:`\text{‘Infinite Bound Size'}\geq 1000`.
**'LP Presolve'** : str
Default :math:`= \texttt{'FULL'}`
This argument allows you to reduce the level of presolving of the problem or turn it off completely.
If the presolver is turned off, the solver will try to handle the problem you have given.
In such a case, the presence of fixed variables or linear dependencies in the constraint matrix can cause numerical instabilities to occur.
In normal circumstances, it is recommended to use the full presolve which is the default.
Constraint: :math:`\text{‘LP Presolve'} = \texttt{'FULL'}`, :math:`\texttt{'BASIC'}` or :math:`\texttt{'NO'}`.
**'LPIPM Algorithm'** : str
Default :math:`= \texttt{'PRIMAL-DUAL'}`
As described in `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails>`__, ``handle_solve_lp_ipm`` implements the infeasible Primal-Dual algorithm, see `The Infeasible-interior-point Primal-Dual Algorithm <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_pd>`__, and the homogeneous Self-Dual algorithm, see `Homogeneous Self-Dual Algorithm <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_sd>`__.
This argument controls which one to use.
Constraint: :math:`\text{‘LPIPM Algorithm'} = \texttt{'PRIMAL-DUAL'}`, :math:`\texttt{'PD'}`, :math:`\texttt{'SELF-DUAL'}` or :math:`\texttt{'SD'}`.
**'LPIPM Centrality Correctors'** : int
Default :math:`\text{} = 6`
This argument controls the number of centrality correctors (see `Weighted Multiple Centrality Correctors <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_mcc>`__) used at each iteration.
Each corrector step attempts to improve the current iterate for the price of additional solve(s) of the factorized system matrix in order to reduce the total number of iterations.
Therefore, it trades the additional solves of the system with the number of factorizations.
The more expensive the factorization is with respect to the solve, the more corrector steps should be allowed.
If :math:`i > 0`, the maximum number of corrector steps will be computed by timing heuristics (the ratio between the times of the factorization and the solve in the first iteration) but will not be greater than :math:`i`.
The number computed by the heuristic can be recovered after the solve or during a monitoring step in :math:`\mathrm{stats}`.
This might cause non-repeatable results.
If :math:`i < 0`, the maximum number of corrector steps will be set to :math:`\left\lvert i\right\rvert`.
If it is set to :math:`0`, no additional centrality correctors will be used and the algorithm reverts to Mehrotra's predictor-corrector.
**'LPIPM Iteration Limit'** : int
Default :math:`\text{} = 100`
The maximum number of iterations to be performed by ``handle_solve_lp_ipm``.
Setting the option too low might lead to :math:`\mathrm{errno}` = 22.
Constraint: :math:`\text{‘LPIPM Iteration Limit'} \geq 1`.
**'LPIPM Max Iterative Refinement'** : int
Default :math:`\text{} = 5`
This argument controls the maximum number of iterative refinement iterations (see `Solving the KKT System <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_system>`__) used at each main iteration when :math:`\text{‘LPIPM System Formulation'} = \texttt{'Normal Equations'}`.
When solving the Normal Equations linear system for numerically challenging problems, mixed-precision iterative refinement may be used until the roundoff errors are reduced to an acceptable level or until the number of refinements reached its maximum value set by this argument.
Constraint: :math:`\text{‘LPIPM Max Iterative Refinement'} \geq 0`.
**'LPIPM Scaling'** : str
Default :math:`= \texttt{'ARITHMETIC'}`
This argument controls the type of scaling to be applied on the constraint matrix :math:`A` before solving the problem.
More precisely, the scaling procedure will try to find diagonal matrices :math:`D_1` and :math:`D_2` such that the values in :math:`D_1AD_2` are of a similar order of magnitude.
The solver is less likely to run into numerical difficulties when the constraint matrix is well scaled.
Constraint: :math:`\text{‘LPIPM Scaling'} = \texttt{'ARITHMETIC'}`, :math:`\texttt{'GEOMETRIC'}` or :math:`\texttt{'NONE'}`.
**'LPIPM Monitor Frequency'** : int
Default :math:`\text{} = 0`
This argument defines the frequency of how often function :math:`\mathrm{monit}` is called.
If :math:`i > 0`, the solver calls :math:`\mathrm{monit}` at the end of every :math:`i`\ th iteration.
If it is set to :math:`0`, the function is not called at all.
Constraint: :math:`\text{‘LPIPM Monitor Frequency'} \geq 0`.
**'LPIPM Stop Tolerance'** : float
Default :math:`= \sqrt{\epsilon }`
This argument sets the value :math:`\epsilon_1` which is the tolerance for the convergence measures in the stopping criteria, see `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘LPIPM Stop Tolerance'} > \epsilon`.
**'LPIPM Stop Tolerance 2'** : float
Default :math:`= \epsilon^{0.6}`
This argument sets the additional tolerance :math:`\epsilon_2` used in the stopping criteria for the Self-Dual algorithm, see `Stopping Criteria <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_stopcrit>`__.
Constraint: :math:`\text{‘LPIPM Stop Tolerance 2'} > \epsilon`.
**'LPIPM System Formulation'** : str
Default :math:`= \texttt{'AUTO'}`
As described in `Solving the KKT System <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_system>`__, ``handle_solve_lp_ipm`` can internally work either with the normal equations formulation `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#normaleq>`__ or with the augmented system `[equation] <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#augmented>`__.
A brief discussion of advantages and disadvantages is presented in `Solving the KKT System <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails_system>`__.
Option 'AUTO' leaves the decision to the solver based on the structure of the constraints and it is the recommended option.
This will typically lead to the normal equations formulation unless there are many dense columns or the system is significantly cheaper to factorize as the augmented system.
Note that in some cases even if :math:`\text{‘LPIPM System Formulation'} = \texttt{'NORMAL EQUATIONS'}` the solver might switch the formulation through the computation to the augmented system due to numerical instabilities or computational cost.
Constraint: :math:`\text{‘LPIPM System Formulation'} = \texttt{'AUTO'}`, :math:`\texttt{'AUGMENTED SYSTEM'}`, :math:`\texttt{'AS'}`, :math:`\texttt{'NORMAL EQUATIONS'}` or :math:`\texttt{'NE'}`.
**'Monitoring File'** : int
Default :math:`= -1`
If :math:`i\geq 0`, the unit number for the secondary (monitoring) output.
If set to :math:`-1`, no secondary output is provided.
The following information is output to the unit:
- a listing of the options if set by 'Print Options';
- problem statistics, the iteration log and the final status as set by 'Monitoring Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Monitoring File'}\geq -1`.
**'Monitoring Level'** : int
Default :math:`= 4`
This argument sets the amount of information detail that will be printed by the solver to the secondary output.
The meaning of the levels is the same as with 'Print Level'.
Constraint: :math:`0\leq \text{‘Monitoring Level'}\leq 5`.
**'Print File'** : int
Default :math:`= \text{advisory message unit number}`
If :math:`i\geq 0`, the unit number for the primary output of the solver.
If :math:`\text{‘Print File'} = -1`, the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The following information is output to the unit:
- a listing of options if set by 'Print Options';
- problem statistics, the iteration log and the final status from the solver as set by 'Print Level';
- the solution if set by 'Print Solution'.
Constraint: :math:`\text{‘Print File'} \geq -1`.
**'Print Level'** : int
Default :math:`= 2`
This argument defines how detailed information should be printed by the solver to the primary output.
.. rst-class:: nag-rules-none nag-align-left
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+===========+=====================================================================================================================================================+
|:math:`0` |No output from the solver |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |Only the final status and the primal and dual objective value |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |Problem statistics, one line per iteration showing the progress of the solution with respect to the convergence measures, final status and statistics|
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |As level :math:`2` but each iteration line is longer, including step lengths and errors |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4,5`|As level :math:`3` but further details of each iteration are presented |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
Constraint: :math:`0\leq \text{‘Print Level'}\leq 5`.
**'Print Options'** : str
Default :math:`= \texttt{'YES'}`
If :math:`\text{‘Print Options'} = \texttt{'YES'}`, a listing of options will be printed to the primary and secondary output.
Constraint: :math:`\text{‘Print Options'} = \texttt{'YES'}` or :math:`\texttt{'NO'}`.
**'Print Solution'** : str
Default :math:`= \texttt{'NO'}`
If :math:`\text{‘Print Solution'} = \texttt{'X'}`, the final values of the primal variables are printed on the primary and secondary outputs.
If :math:`\text{‘Print Solution'} = \texttt{'YES'}` or :math:`\texttt{'ALL'}`, in addition to the primal variables, the final values of the dual variables are printed on the primary and secondary outputs.
Constraint: :math:`\text{‘Print Solution'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'X'}` or :math:`\texttt{'ALL'}`.
**'Stats Time'** : str
Default :math:`= \texttt{'NO'}`
This argument allows you to turn on timings of various parts of the algorithm to give a better overview of where most of the time is spent.
This might be helpful for a choice of different solving approaches.
It is possible to choose between CPU and wall clock time.
Choice 'YES' is equivalent to 'WALL CLOCK'.
Constraint: :math:`\text{‘Stats Time'} = \texttt{'YES'}`, :math:`\texttt{'NO'}`, :math:`\texttt{'CPU'}` or :math:`\texttt{'WALL CLOCK'}`.
**'Task'** : str
Default :math:`= \texttt{'MINIMIZE'}`
This argument specifies the required direction of the optimization.
If :math:`\text{‘Task'} = \texttt{'FEASIBLE POINT'}`, the objective function (if set) is ignored and the algorithm stops as soon as a feasible point is found with respect to the given tolerance.
If no objective function is set, 'Task' reverts to 'FEASIBLE POINT' automatically.
Constraint: :math:`\text{‘Task'} = \texttt{'MINIMIZE'}`, :math:`\texttt{'MAXIMIZE'}` or :math:`\texttt{'FEASIBLE POINT'}`.
.. _e04mt-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized.
(`errno` :math:`1`)
:math:`\mathrm{handle}` does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
(`errno` :math:`1`)
:math:`\mathrm{handle}` has not been initialized properly or is corrupted.
(`errno` :math:`2`)
This solver does not support the model defined in the handle.
(`errno` :math:`2`)
The problem is already being solved.
(`errno` :math:`4`)
On entry, :math:`\textit{nvar} = \langle\mathit{\boldsymbol{value}}\rangle`, expected :math:`\mathrm{value} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nvar}` must match the current number of variables of the model in the :math:`\mathrm{handle}`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is either :math:`0` or :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`5`)
On entry, :math:`\textit{nnzu} = \langle\mathit{\boldsymbol{value}}\rangle`.
:math:`\textit{nnzu}` does not match the size of the Lagrangian multipliers for constraints.
The correct value is :math:`0` for no constraints.
(`errno` :math:`51`)
The problem was found to be primal infeasible.
(`errno` :math:`52`)
The problem was found to be dual infeasible.
(`errno` :math:`53`)
The problem seems to be primal or dual infeasible, the algorithm was stopped.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`50`)
Suboptimal solution.
**NagAlgorithmicMajorWarning**
(`errno` :math:`22`)
Maximum number of iterations exceeded.
(`errno` :math:`24`)
No progress, stopping early.
**NagCallbackTerminateWarning**
(`errno` :math:`20`)
User requested termination during a monitoring step.
.. _e04mt-py2-py-notes:
**Notes**
``handle_solve_lp_ipm`` solves a large-scale linear optimization problem in the following form:
.. math::
\begin{array}{lll} \mathrm{minimize}_{{x \in ℝ^n}} & c^\mathrm{T}x &\quad \text{ (a)}\\ \text{subject to }\quad &l_A\leq Ax\leq u_A&\quad \text{ (b)}\\& l_x\leq x\leq u_x \text{,} &\quad \text{ (c)}\end{array}
where :math:`n` is the number of decision variables and :math:`m` is the number of linear constraints.
Here :math:`c`, :math:`x`, :math:`l_x`, :math:`u_x` are :math:`n`-dimensional vectors, :math:`A` is an :math:`m\times n` sparse matrix and :math:`l_A`, :math:`u_A` are :math:`m`-dimensional vectors.
``handle_solve_lp_ipm`` implements two algorithmic variants of the interior point method for solving linear optimization problems: the infeasible Primal-Dual interior point method and homogeneous Self-Dual interior point method.
In general, the Self-Dual algorithm has a slightly higher price per iteration, however, it is able to declare infeasibility or unboundness of the problem, whereas the Primal-Dual relies, in this case, on heuristics.
For a detailed description of both algorithms see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails>`__.
The algorithm is chosen by the 'LPIPM Algorithm', the default is Primal-Dual.
``handle_solve_lp_ipm`` solves linear programming problems stored as a handle.
The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite.
First, the problem handle is initialized by calling :meth:`handle_init`.
Then some of the functions :meth:`handle_set_linobj`, :meth:`handle_set_quadobj`, :meth:`handle_set_simplebounds` or :meth:`handle_set_linconstr` may be called to formulate the objective function, bounds of the variables, and the block of linear constraints, respectively.
Once the problem is fully set, the handle may be passed to the solver.
When the handle is not needed anymore, :meth:`handle_free` should be called to destroy it and deallocate the memory held within.
See `the E04 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04intro.html#optsuite>`__ for more details about the NAG optimization modelling suite.
The solver method can be modified by various options (see :ref:`Other Parameters <e04mt-py2-py-other_params>`) which can be set by :meth:`handle_opt_set` and :meth:`handle_opt_set_file` anytime between the initialization of the handle by :meth:`handle_init` and a call to the solver.
Once the solver has finished, options may be modified for the next solve.
The solver may be called repeatedly with various options.
The option 'Task' may be used to switch the problem to maximization or to ignore the objective function and find only a feasible point.
Several options may have significant impact on the performance of the solver.
Even if the defaults were chosen to suit the majority of problems, it is recommended to experiment in order to find the most suitable set of options for a particular problem, see `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mtf.html#algdetails>`__ and :ref:`Other Parameters <e04mt-py2-py-other_params>` for further details.
**Structure of the Lagrangian Multipliers**
The algorithm works internally with estimates of both the decision variables, denoted by :math:`x`, and the Lagrangian multipliers (dual variables), denoted by :math:`u`.
The multipliers :math:`u` are stored in the array :math:`\mathrm{u}` and conform to the structure of the constraints.
If the simple bounds have been defined (:meth:`handle_set_simplebounds` was successfully called), the first :math:`2n` elements of :math:`\mathrm{u}` belong to the corresponding Lagrangian multipliers, interleaving a multiplier for the lower and the upper bound for each :math:`x_i`.
If any of the bounds were set to infinity, the corresponding Lagrangian multipliers are set to :math:`0` and may be ignored.
Similarly, the following :math:`2m` elements of :math:`\mathrm{u}` belong to multipliers for the linear constraints (if :meth:`handle_set_linconstr` has been successfully called).
The organization is the same, i.e., the multipliers for each constraint for the lower and upper bounds are alternated and zeros are used for any missing (infinite bound) constraints.
Some solvers merge multipliers for both lower and upper inequality into one element whose sign determines the inequality.
Negative multipliers are associated with the upper bounds and positive with the lower bounds.
An equivalent result can be achieved with this storage scheme by subtracting the upper bound multiplier from the lower one.
This is also consistent with equality constraints.
.. _e04mt-py2-py-references:
**References**
Andersen, E D, Gondzio, J, Mészáros, C and Xu, X, 1996, `Implementation of interior point methods for large scale linear programming`, HEC/Université de Genève
Colombo, M and Gondzio, J, 2008, `Further development of multiple centrality correctors for interior point methods`, Computational Optimization and Algorithms (41(3)), 277--305
Goldfard, D and Scheinberg, K, 2004, `A product-form Cholesky factorization method for handling dense columns in interior point methods for linear programming`, Mathematical Programming (99(1)), 1--34
Gondzio, J, 1996, `Multiple centrality corrections in a primal-dual method for linear programming`, Computational Optimization and Algorithms (6(2)), 137--156
Gondzio, J, 2012, `Interior point methods 25 years later`, European Journal of Operations Research (218(3)), 587--601
Hogg, J D and Scott, J A, 2011, `HSL MA97: a bit-compatible multifrontal code for sparse symmetric systems`, RAL Technical Report. RAL-TR-2011-024
HSL, 2011, `A collection of Fortran codes for large-scale scientific computation`, http://www.hsl.rl.ac.uk/
Karypis, G and Kumar, V, 1998, `A fast and high quality multilevel scheme for partitioning irregular graphs`, SIAM J. Sci. Comput. (20(1)), 359--392
Mészáros, C, 1996, `The efficient implementation of interior point methods for linear programming and their applications`, PhD Thesis, Eötvös Loránd University of Science, Budapest
Nocedal, J and Wright, S J, 2006, `Numerical Optimization`, (2nd Edition), Springer Series in Operations Research, Springer, New York
Wright, S W, 1997, `Primal-dual interior point methods`, SIAM, Philadelphia
Xu, X, Hung, P-F and Ye, Y, 1996, `A simplified homogeneous and self-dual linear programming algorithm and its implementation`, Annals of Operations Research (62(1)), 151--171
See Also
--------
:meth:`naginterfaces.library.examples.opt.handle_add_vars_ex.main`
:meth:`naginterfaces.library.examples.opt.handle_solve_lp_ipm_ex.main`
"""
raise NotImplementedError
[docs]def miqp_mps_write(outfile, n, m, ncolh, iobj, bl, bu, pnames, crname, minmax, idxc=None, c=None, a=None, irowa=None, iccola=None, h=None, irowh=None, iccolh=None, intvar=None, io_manager=None):
r"""
``miqp_mps_write`` writes data for sparse linear programming, mixed integer linear programming, quadratic programming or mixed integer quadratic programming problems to a file in MPS format.
.. _e04mw-py2-py-doc:
For full information please refer to the NAG Library document for e04mw
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mwf.html
.. _e04mw-py2-py-parameters:
**Parameters**
**outfile** : int
The ID of the file to store the problem data as associated by a call to the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`.
**n** : int
:math:`n`, the number of variables in the problem.
**m** : int
:math:`m`, the number of constraints in the problem. This is the number of rows in the linear constraint matrix :math:`A`, including the free row (if any; see :math:`\mathrm{iobj}`).
**ncolh** : int
The number of leading nonzero columns of the Hessian matrix :math:`H`.
If :math:`\mathrm{ncolh} = 0`, the quadratic term :math:`H` of the objective function is considered zero (e.g., LP problems), and arrays :math:`\mathrm{h}`, :math:`\mathrm{irowh}` and :math:`\mathrm{iccolh}` will not be referenced.
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the linear terms of the objective function. In this case :math:`\textit{nnzc}` is set to :math:`0`.
If :math:`\mathrm{iobj} = 0`, there is no free row in :math:`A`, and the linear terms might be supplied in array :math:`\mathrm{c}`.
**bl** : float, array-like, shape :math:`\left(\mathrm{n}+\mathrm{m}\right)`
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` contains the lower bounds :math:`l` and the upper bounds :math:`u`, respectively.
The first :math:`\mathrm{n}` elements refer to the bounds for the variables :math:`x` and the rest to the bounds for the linear constraints (including the objective row :math:`\mathrm{iobj}` if present).
To specify a nonexistent lower bound (i.e., :math:`l_j = -\mathrm{inf}`), set :math:`\mathrm{bl}[j-1]\leq -10^{20}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq 10^{20}`.
**bu** : float, array-like, shape :math:`\left(\mathrm{n}+\mathrm{m}\right)`
:math:`\mathrm{bl}` and :math:`\mathrm{bu}` contains the lower bounds :math:`l` and the upper bounds :math:`u`, respectively.
The first :math:`\mathrm{n}` elements refer to the bounds for the variables :math:`x` and the rest to the bounds for the linear constraints (including the objective row :math:`\mathrm{iobj}` if present).
To specify a nonexistent lower bound (i.e., :math:`l_j = -\mathrm{inf}`), set :math:`\mathrm{bl}[j-1]\leq -10^{20}`; to specify a nonexistent upper bound, set :math:`\mathrm{bu}[j-1]\geq 10^{20}`.
**pnames** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
The names can be composed only from 'printable' characters (ASCII codes between :math:`32` and :math:`126`).
If any of the names are blank, the default name is used.
:math:`\mathrm{pnames}[0]`
Contains the name of the problem.
:math:`\mathrm{pnames}[1]`
Contains the name of the objective row if the objective is provided in :math:`\mathrm{c}` instead of :math:`\mathrm{iobj}` and all names :math:`\mathrm{crname}` are given. The name must be nonempty and unique. In all other cases :math:`\mathrm{pnames}[1]` is not used.
:math:`\mathrm{pnames}[2]`
Contains the name of the RHS set.
:math:`\mathrm{pnames}[3]`
Contains the name of the RANGE.
:math:`\mathrm{pnames}[4]`
Contains the name of the BOUNDS.
**crname** : str, length 8, array-like, shape :math:`\left(\textit{nname}\right)`
The names of all the variables and constraints in the problem in that order.
The names can be composed only from 'printable' characters and must be unique.
**minmax** : int
:math:`\mathrm{minmax}` defines the direction of optimization problem.
:math:`\mathrm{minmax} = -1`
Minimization.
:math:`\mathrm{minmax} = 1`
Maximization.
**idxc** : None or int, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of sparse vector :math:`c`. :math:`\mathrm{idxc}[\textit{i}-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`.
The elements are stored in ascending order.
**c** : None or float, array-like, shape :math:`\left(\textit{nnzc}\right)`, optional
The nonzero elements of sparse vector :math:`c`. :math:`\mathrm{idxc}[\textit{i}-1]` must contain the index of :math:`\mathrm{c}[\textit{i}-1]` in the vector, for :math:`\textit{i} = 1,2,\ldots,\textit{nnzc}`.
The elements are stored in ascending order.
**a** : None or float, array-like, shape :math:`\left(\textit{nnza}\right)`, optional
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**irowa** : None or int, array-like, shape :math:`\left(\textit{nnza}\right)`, optional
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**iccola** : None or int, array-like, shape :math:`\left(:\right)`, optional
Note: the required length for this argument is determined as follows: if :math:`\textit{nnza} > 0`: :math:`{ \mathrm{n}+1 }`; otherwise: :math:`0`.
The nonzero elements of matrix :math:`A` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). Arrays :math:`\mathrm{irowa}` and :math:`\mathrm{a}` store the row indices and the values of the nonzero elements, respectively. The elements are sorted by columns and within each column in nondecreasing order. Duplicate entries are not allowed. :math:`\mathrm{iccola}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{a}` and :math:`\mathrm{irowa}`.
If :math:`\textit{nnza} = 0`, :math:`\mathrm{a}` and :math:`\mathrm{irowa}` are not referenced.
**h** : None or float, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{h}` is not referenced.
**irowh** : None or int, array-like, shape :math:`\left(\textit{nnzh}\right)`, optional
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{irowh}` is not referenced.
**iccolh** : None or int, array-like, shape :math:`\left(:\right)`, optional
Note: the required length for this argument is determined as follows: if :math:`\mathrm{ncolh} > 0`: :math:`{ \mathrm{ncolh}+1 }`; otherwise: :math:`0`.
The nonzero elements of the Hessian matrix :math:`H` in compressed column storage (see `the F11 Introduction <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/f11/f11intro.html#background12>`__). The Hessian matrix, :math:`H`, is symmetric and its elements are stored in a lower triangular matrix.
Arrays :math:`\mathrm{irowh}` and :math:`\mathrm{h}` store the row indices and the values of the nonzero elements, respectively.
The elements are sorted by columns and within each column in nondecreasing order.
Duplicate entries are not allowed. :math:`\mathrm{iccolh}` contains the (one-based) indices to the beginning of each column in :math:`\mathrm{h}` and :math:`\mathrm{irowh}`.
If :math:`\mathrm{ncolh} = 0`, :math:`\mathrm{iccolh}` is not referenced.
**intvar** : None or int, array-like, shape :math:`\left(\textit{lintvar}\right)`, optional
:math:`\mathrm{intvar}` contains the indices :math:`k` of variables :math:`x_k` which are defined as integers. Duplicate indices are not allowed.
If :math:`\textit{lintvar} = 0`, :math:`\mathrm{intvar}` is not referenced.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04mw-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
On entry, :math:`\mathrm{outfile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{outfile}\geq 0`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{n}\geq 1`.
(`errno` :math:`2`)
On entry, :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{m}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nnzc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnzc}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nnza} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nnza}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{lintvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{lintvar}\geq 0`.
(`errno` :math:`3`)
On entry, :math:`\textit{nname} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nname} = 0` or :math:`\mathrm{n}+\mathrm{m}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ncolh}\leq \mathrm{n}`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncolh} > 0`, :math:`\textit{nnzh} > 0`.
(`errno` :math:`4`)
On entry, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: if :math:`\mathrm{ncolh} = 0`, :math:`\textit{nnzh} = 0`.
(`errno` :math:`5`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{idxc}[\textit{j}-1]\leq \mathrm{n}`.
(`errno` :math:`5`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{idxc}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{idxc}[j] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{idxc}[j-1] < \mathrm{idxc}[j]`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{minmax} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{minmax} = -1` or :math:`1`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{iobj}\leq \mathrm{m}`.
(`errno` :math:`7`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzc} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: at most one of :math:`\mathrm{iobj}` or :math:`\textit{nnzc}` may be nonzero.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` are incorrect.
Constraint: :math:`\mathrm{bl}[\textit{j}-1]\leq \mathrm{bu}[\textit{j}-1]`.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1]` is incorrect.
Constraint: :math:`\mathrm{bl}[\textit{j}-1] < {1e+20}`.
(`errno` :math:`8`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bu}[\textit{j}-1]` is incorrect.
Constraint: :math:`\mathrm{bu}[\textit{j}-1] > {-1e+20}`.
(`errno` :math:`8`)
On entry, :math:`\mathrm{iobj} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, if :math:`\mathrm{iobj} > 0` the bounds must be infinite.
Constraints: :math:`\mathrm{bl}[\textit{j}-1]\leq {-1e+20}`, :math:`\mathrm{bu}[\textit{j}-1]\geq 1e+20`.
(`errno` :math:`8`)
On entry, :math:`\textit{j} = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{bl}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle`, the integer variable :math:`\textit{j}` requires at least one bound finite.
Constraint: at least one of the following conditions must be met for integer variable j: :math:`\mathrm{bl}[\textit{j}-1] > {-1e+20}`, :math:`\mathrm{bu}[\textit{j}-1] < 1e+20`.
(`errno` :math:`9`)
On entry, :math:`\mathrm{crname}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` is incorrect.
Constraint: the names in :math:`\mathrm{crname}` must consist only of printable characters.
(`errno` :math:`9`)
On entry, :math:`\mathrm{pnames}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` is incorrect.
Constraint: the names in :math:`\mathrm{pnames}` must consist only of printable characters.
(`errno` :math:`9`)
On entry, :math:`\mathrm{crname}[\textit{j}-1]` for :math:`j = \langle\mathit{\boldsymbol{value}}\rangle` has been already used.
Constraint: the names in :math:`\mathrm{crname}` must be unique.
(`errno` :math:`9`)
The name specified in :math:`\mathrm{pnames}[1]` is empty or has been already used among row names.
Constraint: the names in :math:`\mathrm{pnames}[1]` must be unique and nonempty if :math:`\mathrm{crname}` is provided and :math:`\textit{nnzc} > 0`.
(`errno` :math:`10`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{intvar}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{lintvar} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{intvar}[\textit{j}-1]\leq \textit{lintvar}`.
(`errno` :math:`10`)
On entry, :math:`\mathrm{intvar}[\langle\mathit{\boldsymbol{value}}\rangle] = \mathrm{intvar}[\langle\mathit{\boldsymbol{value}}\rangle] =` :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: all entries in :math:`\mathrm{intvar}` must be unique.
(`errno` :math:`11`)
On entry, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{irowa}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`1\leq \mathrm{irowa}[\textit{i}-1]\leq \mathrm{m}`.
(`errno` :math:`11`)
On entry, more than one element of :math:`\mathrm{a}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{a}` must have a unique row and column index.
(`errno` :math:`12`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iccola}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{iccola}[j] = \langle\mathit{\boldsymbol{value}}\rangle`, the values of :math:`\mathrm{iccola}` must be nondecreasing.
Constraint: :math:`\mathrm{iccola}[\textit{j}-1]\leq \mathrm{iccola}[j]`.
(`errno` :math:`12`)
On entry, :math:`\mathrm{iccola}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccola}[0] = 1`.
(`errno` :math:`12`)
On entry, :math:`\mathrm{iccola}[\mathrm{n}] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnza} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccola}[\mathrm{n}] = \textit{nnza}+1`.
(`errno` :math:`13`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`i = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{ncolh} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{irowh}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle`
Constraint: :math:`j\leq \mathrm{irowh}[\textit{i}-1]\leq \mathrm{ncolh}` (within the lower triangle).
(`errno` :math:`13`)
On entry, more than one element of :math:`\mathrm{h}` has row index :math:`\langle\mathit{\boldsymbol{value}}\rangle` and column index :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: each element of :math:`\mathrm{h}` must have a unique row and column index.
(`errno` :math:`14`)
On entry, :math:`j = \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{iccolh}[\textit{j}-1] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{iccolh}[j] = \langle\mathit{\boldsymbol{value}}\rangle`, the values of :math:`\mathrm{iccolh}` must be nondecreasing.
Constraint: :math:`\mathrm{iccolh}[\textit{j}-1]\leq \mathrm{iccolh}[j]`.
(`errno` :math:`14`)
On entry, :math:`\mathrm{iccolh}[0] = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccolh}[0] = 1`.
(`errno` :math:`14`)
On entry, :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{nnzh} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \textit{nnzh}+1`.
(`errno` :math:`15`)
An error occurred when writing to file.
.. _e04mw-py2-py-notes:
**Notes**
``miqp_mps_write`` writes data for Linear Programming (LP) or Quadratic Programming (QP) problems (or their mixed integer variants) from an optimization problem to a MPS output file, see `MPS Input Format for miqp_mps_read <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#mpsinput>`__ for the format description.
The problem is expected in the form
.. math::
\mathrm{minimize}_x\left({c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx}\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
Where :math:`n` is the number of variables, :math:`m` is the number of general linear constraints, :math:`A` is the linear constraint matrix with dimension :math:`m\times n`, the vectors :math:`l` and :math:`u` are the lower and upper bounds, respectively. :math:`H` is the Hessian matrix with dimension :math:`n\times n`, however, only leading :math:`\mathrm{ncolh}` columns might contain nonzero elements and the rest is assumed to be zero.
Note that the linear term of the objective function :math:`c` might be supplied either as :math:`\mathrm{c}` or via :math:`\mathrm{iobj}`.
If :math:`\mathrm{c}` is supplied then :math:`\mathrm{idxc}` contains the indices of the nonzero elements of sparse vector :math:`c`, whereas if :math:`\mathrm{iobj}` is supplied (:math:`\mathrm{iobj} > 0`), row :math:`\mathrm{iobj}` of matrix :math:`A` is a free row storing the nonzero elements of :math:`c`.
Note: that this function uses fixed MPS format, see IBM (1971).
.. _e04mw-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def miqp_mps_read(infile, maxn, maxm, maxnnz, maxncolh, maxnnzh, maxlintvar, pnames, mpslst=0, io_manager=None):
r"""
``miqp_mps_read`` reads data for sparse linear programming, mixed integer linear programming, quadratic programming or mixed integer quadratic programming problems from an external file which is in standard or compatible MPS input format.
.. _e04mx-py2-py-doc:
For full information please refer to the NAG Library document for e04mx
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html
.. _e04mx-py2-py-parameters:
**Parameters**
**infile** : int
The ID of the MPSX data file to be read as returned by a call to the ``FileObjManager`` method :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`.
**maxn** : int
An upper limit for the number of variables in the problem.
If :math:`\mathrm{maxn} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxm** : int
An upper limit for the number of general linear constraints (including the objective row) in the problem.
If :math:`\mathrm{maxm} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxnnz** : int
An upper limit for the number of nonzeros (including the objective row) in the problem.
If :math:`\mathrm{maxnnz} < 1`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxncolh** : int
An upper limit for the dimension of the matrix :math:`H`.
If :math:`\mathrm{maxncolh} < 0`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxnnzh** : int
An upper limit for the number of nonzeros of the matrix :math:`H`.
If :math:`\mathrm{maxnnzh} < 0`, ``miqp_mps_read`` will start in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__).
**maxlintvar** : int
If :math:`\mathrm{maxlintvar}\geq 0`, an upper limit for the number of integer variables.
If :math:`\mathrm{maxlintvar} < 0`, ``miqp_mps_read`` will treat all integer variables in the file as continuous variables.
**pnames** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
:math:`\mathrm{pnames}[0]`
Must either contain the name of the problem or be blank.
:math:`\mathrm{pnames}[1]`
Must either be blank or contain the name of the objective row (in which case it overrides OBJNAME section and the default choice of the first objective free row).
:math:`\mathrm{pnames}[2]`
Must either contain the name of the RHS set to be used or be blank (in which case the first RHS set is used).
:math:`\mathrm{pnames}[3]`
Must either contain the name of the RANGE set to be used or be blank (in which case the first RANGE set (if any) is used).
:math:`\mathrm{pnames}[4]`
Must either contain the name of the BOUNDS set to be used or be blank (in which case the first BOUNDS set (if any) is used).
**mpslst** : int, optional
If :math:`\mathrm{mpslst} \neq 0`, summary messages are sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`) as ``miqp_mps_read`` reads through the data file. This can be useful for debugging the file. If :math:`\mathrm{mpslst} = 0`, then no summary is produced.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**n** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of variables of the problem. Otherwise, :math:`n`, the actual number of variables in the problem.
**m** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of general linear constraints in the problem (including the objective row). Otherwise, :math:`m`, the actual number of general linear constraints of the problem.
**nnz** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of nonzeros in the problem (including the objective row). Otherwise, the actual number of nonzeros in the problem (including the objective row).
**ncolh** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the value of :math:`\textit{ncolh}` required by :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve`. In this context :math:`\mathrm{ncolh}` is the number of leading nonzero columns of the Hessian matrix :math:`H`. Otherwise, the actual dimension of the matrix :math:`H`.
**nnzh** : int
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of nonzeros of the matrix :math:`H`. Otherwise, the actual number of nonzeros of the matrix :math:`H`.
**lintvar** : int
If on entry :math:`\mathrm{maxlintvar} < 0`, all integer variables are treated as continuous and :math:`\mathrm{lintvar} = -1`.
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, an upper estimate of the number of integer variables of the problem.
Otherwise, the actual number of integer variables of the problem.
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the vector :math:`c`.
If :math:`\mathrm{iobj} = 0`, the coefficients of :math:`c` are assumed to be zero.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__) :math:`\mathrm{iobj}` is not referenced.
**a** : float, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{a}` is not referenced.
**irowa** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The row indices of the nonzero elements stored in :math:`\mathrm{a}`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{irowa}` is not referenced.
**iccola** : int, ndarray, shape :math:`\left(:\right)`
A set of pointers to the beginning of each column of :math:`A`. More precisely, :math:`\mathrm{iccola}[\textit{i}-1]` contains the index in :math:`\mathrm{a}` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{n}`. Note that :math:`\mathrm{iccola}[0] = 1` and :math:`\mathrm{iccola}[\mathrm{n}] = \mathrm{nnz}+1`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{iccola}` is not referenced.
**bl** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bl}` contains the vector :math:`l` (the lower bounds) for all the variables and constraints
**bu** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bu}` contains the vector :math:`u` (the upper bounds) for all the variables and constraints
**pnames** : str, length 8, ndarray, shape :math:`\left(5\right)`
A set of names associated with the problem as defined in the MPSX data file as follows:
:math:`\mathrm{pnames}[0]`
Contains the name of the problem (or blank if none).
:math:`\mathrm{pnames}[1]`
Contains the name of the objective row (or blank if none).
:math:`\mathrm{pnames}[2]`
Contains the name of the RHS set (or blank if none).
:math:`\mathrm{pnames}[3]`
Contains the name of the RANGE set (or blank if none).
:math:`\mathrm{pnames}[4]`
Contains the name of the BOUNDS set (or blank if none).
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{pnames}` is not referenced.
**nname** : int
:math:`n+m`, the total number of variables and constraints in the problem (including the objective row).
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{nname}` is not set.
**crname** : str, length 8, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
The MPS names of all the variables and constraints in the problem in the following order. The first :math:`\mathrm{n}` elements contain the MPS names for the variables and the next :math:`\mathrm{m}` elements contain the MPS names for the objective row and general linear constraints (if any). Note that the MPS name for the objective row is stored in :math:`\mathrm{crname}[\mathrm{n}+\mathrm{iobj}-1]`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{crname}` is not referenced.
**h** : float, ndarray, shape :math:`\left(\mathrm{maxnnzh}\right)`
The :math:`\mathrm{nnzh}` nonzero elements of :math:`H`, arranged by increasing column index.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{h}` is not referenced.
**irowh** : int, ndarray, shape :math:`\left(\mathrm{maxnnzh}\right)`
The :math:`\mathrm{nnzh}` row indices of the elements stored in :math:`H`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{irowh}` is not referenced.
**iccolh** : int, ndarray, shape :math:`\left(:\right)`
A set of pointers to the beginning of each column of :math:`H`. More precisely, :math:`\mathrm{iccolh}[\textit{i}-1]` contains the index in :math:`H` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{ncolh}`. Note that :math:`\mathrm{iccolh}[0] = 1` and :math:`\mathrm{iccolh}[\mathrm{ncolh}] = \mathrm{nnzh}+1`.
If ``miqp_mps_read`` is run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), :math:`\mathrm{iccolh}` is not referenced.
**minmax** : int
:math:`\mathrm{minmax}` defines the direction of the optimization as read from the MPS file. By default the function assumes the objective function should be minimized and will return :math:`\mathrm{minmax} = -1`. If the function discovers in the OBJSENSE section that the objective function should be maximized it will return :math:`\mathrm{minmax} = 1`. If the function discovers that there is neither the linear objective term :math:`c` (the objective row) nor the Hessian matrix :math:`H`, the problem is considered as a feasible point problem and :math:`\mathrm{minmax} = 0` is returned.
If ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{minmax}` is not set.
**intvar** : int, ndarray, shape :math:`\left(\mathrm{maxlintvar}\right)`
If :math:`\mathrm{maxlintvar} > 0` on entry, :math:`\mathrm{intvar}` contains pointers to the columns that are defined as integer variables. More precisely, :math:`\mathrm{intvar}[\textit{i}-1] = k`, where :math:`k` is the index of a column that is defined as an integer variable, for :math:`\textit{i} = 1,2,\ldots,\mathrm{lintvar}`.
If :math:`\mathrm{maxlintvar}\leq 0` on entry, or ``miqp_mps_read`` was run in query mode (see `Query Mode <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#querymode>`__), or it returned with :math:`\mathrm{errno}` = 2, :math:`\mathrm{intvar}` is not set.
.. _e04mx-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`3`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
OBJNAME indicator line found after ROWS indicator line.
(`errno` :math:`4`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
COLUMNS indicator line found before ROWS indicator line.
(`errno` :math:`5`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RHS indicator line found before COLUMNS indicator line.
(`errno` :math:`6`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
RANGES indicator line found before RHS indicator line.
(`errno` :math:`7`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
BOUNDS indicator line found before COLUMNS indicator line.
(`errno` :math:`8`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before BOUNDS indicator line.
(`errno` :math:`9`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Incorrect ordering of indicator lines.
QUADOBJ indicator line found before COLUMNS indicator line.
(`errno` :math:`10`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown indicator line :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`12`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Indicator line ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been found more than once in the MPS file.
(`errno` :math:`13`)
End of file found before ENDATA indicator line.
(`errno` :math:`14`)
No indicator line found in file. It may be an empty file.
(`errno` :math:`15`)
At least one mandatory section not found in MPS file.
(`errno` :math:`16`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: An illegal line was detected in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
This is neither a comment nor a valid data line.
(`errno` :math:`17`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown inequality key :math:`\langle\mathit{\boldsymbol{value}}\rangle` in ROWS section.
Expected '``N``', '``G``', '``L``' or '``E``'.
(`errno` :math:`18`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Empty ROWS section.
Neither the objective row nor the constraints were defined.
(`errno` :math:`19`)
The supplied name, in :math:`\mathrm{pnames}[1]` or in OBJNAME, of the objective row was not found among the free rows in the ROWS section.
(`errno` :math:`20`)
The supplied name, in :math:`\mathrm{pnames}[4]`, of the BOUNDS set to be used was not found in the BOUNDS section.
(`errno` :math:`21`)
The supplied name, in :math:`\mathrm{pnames}[2]`, of the RHS set to be used was not found in the RHS section.
(`errno` :math:`22`)
The supplied name, in :math:`\mathrm{pnames}[3]`, of the RANGES set to be used was not found in the RANGES section.
(`errno` :math:`23`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal row name.
Row names must consist of printable characters only.
(`errno` :math:`24`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal column name.
Column names must consist of printable characters only.
(`errno` :math:`25`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` has been defined more than once in the ROWS section.
(`errno` :math:`26`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' has been defined more than once in the COLUMNS section. Column definitions must be continuous. (See `COLUMNS Section <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#columnssection>`__).
(`errno` :math:`27`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` marker within ``'INTORG'`` to ``'INTEND'`` range.
(`errno` :math:`28`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTEND'`` marker without previous marker being ``'INTORG'``.
(`errno` :math:`29`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Found ``'INTORG'`` but not ``'INTEND'`` before the end of the COLUMNS section.
(`errno` :math:`30`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Illegal marker type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
Should be either ``'INTORG'`` or ``'INTEND'``.
(`errno` :math:`31`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All row names must be specified in the ROWS section.
(`errno` :math:`32`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` in :math:`\langle\mathit{\boldsymbol{value}}\rangle` section.
All column names must be specified in the COLUMNS section.
(`errno` :math:`33`)
Inconsistent bounds for row ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`33`)
Inconsistent bounds for column ':math:`\langle\mathit{\boldsymbol{value}}\rangle`'.
(`errno` :math:`33`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle`: Unknown bound type :math:`\langle\mathit{\boldsymbol{value}}\rangle` in BOUNDS section.
(`errno` :math:`34`)
More than one nonzero of :math:`A` has row name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' and column name ':math:`\langle\mathit{\boldsymbol{value}}\rangle`' in the COLUMNS section.
(`errno` :math:`35`)
Field :math:`\langle\mathit{\boldsymbol{value}}\rangle` did not contain a number (see :ref:`Notes <e04mx-py2-py-notes>`).
(`errno` :math:`36`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{infile}\geq 0`.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Warning: MPS file not strictly fixed format, although the problem was read anyway. The data may have been read incorrectly. You should set :math:`\mathrm{mpslst} = 1` and repeat the call to ``miqp_mps_read`` for more details.
(`errno` :math:`2`)
At least one of :math:`\mathrm{maxm}`, :math:`\mathrm{maxn}`, :math:`\mathrm{maxnnz}`, :math:`\mathrm{maxnnzh}` or :math:`\mathrm{maxncolh}` is too small. You must provide :math:`\mathrm{maxm}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxn}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnz}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnzh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxncolh}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
At least one of :math:`\mathrm{maxm}`, :math:`\mathrm{maxn}`, :math:`\mathrm{maxnnz}`, :math:`\mathrm{maxnnzh}`, :math:`\mathrm{maxncolh}` or :math:`\mathrm{maxlintvar}` is too small. You must provide :math:`\mathrm{maxm}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxn}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnz}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxnnzh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxncolh}\geq \langle\mathit{\boldsymbol{value}}\rangle`, :math:`\mathrm{maxlintvar}\geq \langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04mx-py2-py-notes:
**Notes**
``miqp_mps_read`` reads data for Linear Programming (LP) or Quadratic Programming (QP) problems (or their mixed integer variants) from an external file which is prepared in standard or compatible MPS (see IBM (1971)) input format.
It then initializes :math:`n` (the number of variables), :math:`m` (the number of general linear constraints), the :math:`m\times n` matrix :math:`A`, the vectors :math:`l`, :math:`u`, :math:`c` (stored in row :math:`\mathrm{iobj}` of :math:`A`) and the :math:`n\times n` Hessian matrix :math:`H` for use with :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve`. These functions are designed to solve problems of the form
.. math::
\mathrm{minimize}_x\left({c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx}\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
**MPS input format**
The input file of data may only contain two types of lines:
(1) Indicator lines (specifying the type of data which is to follow).
(#) Data lines (specifying the actual data).
A `section` is a combination of an indicator line and its corresponding data line(s).
Any characters beyond column 80 are ignored.
Indicator lines must not contain leading blank characters (in other words they must begin in column 1).
The following displays the order in which the indicator lines must appear in the file:
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------+----------+
|NAME |user-supplied name|(optional)|
+--------+------------------+----------+
|OBJSENSE| |(optional)|
+--------+------------------+----------+
| |data line | |
+--------+------------------+----------+
|OBJNAME | |(optional)|
+--------+------------------+----------+
| |data line | |
+--------+------------------+----------+
|ROWS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|COLUMNS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|RHS | | |
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|RANGES | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|BOUNDS | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|QUADOBJ | |(optional)|
+--------+------------------+----------+
| |data line(s) | |
+--------+------------------+----------+
|ENDATA | | |
+--------+------------------+----------+
A data line follows a fixed format, being made up of fields as defined below.
The contents of the fields may have different significance depending upon the section of data in which they appear.
.. rst-class:: nag-rules-none
+--------+-----------+------------+-------------+-------------+-------------+-------------+
| |Field 1 |Field 2 |Field 3 |Field 4 |Field 5 |Field 6 |
+========+===========+============+=============+=============+=============+=============+
|Columns |:math:`2-3`|:math:`5-12`|:math:`15-22`|:math:`25-36`|:math:`40-47`|:math:`50-61`|
+--------+-----------+------------+-------------+-------------+-------------+-------------+
|Contents|Code |Name |Name |Value |Name |Value |
+--------+-----------+------------+-------------+-------------+-------------+-------------+
Each name and code must consist of 'printable' characters only; names and codes supplied must match the case used in the following descriptions.
Values are read using a field width of :math:`12`.
This allows values to be entered in several equivalent forms.
For example, :math:`1.2345678`, :math:`1.2345678e+0`, :math:`123.45678e-2` and :math:`12345678e-07` all represent the same number.
It is safest to include an explicit decimal point.
Lines with an asterisk (:math:`*`) in column :math:`1` will be considered comment lines and will be ignored by the function.
Columns outside the six fields must be blank, except for columns 72--80, whose contents are ignored by the function.
A non-blank character outside the predefined six fields and columns 72--80 is considered to be a major error (:math:`\mathrm{errno}` = 16; see :ref:`Exceptions <e04mx-py2-py-errors>`), unless it is part of a comment.
`NAME Section (optional)`
The NAME section is the only section where the data must be on the same line as the indicator.
The 'user-supplied name' must be in field :math:`3` but may be blank.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-------------------+
|Field |Required|Description |
+=========+========+===================+
|:math:`3`|No |Name of the problem|
+---------+--------+-------------------+
`OBJSENSE Section (optional)`
The data line in this section can be used to specify the sense of the objective function.
If this section is present it must contain only one data line.
If the section is missing or empty, minimization is assumed.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-------------------------------+
|Field |Required|Description |
+=========+========+===============================+
|:math:`2`|No |Sense of the objective function|
+---------+--------+-------------------------------+
Field 2 may contain either ``MIN``, ``MAX``, ``MINIMIZE`` or ``MAXIMIZE``.
`OBJNAME Section (optional)`
The data line in this section can be used to specify the name of a free row (see `ROWS Section <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#rowssection>`__) that should be used as the objective function.
If this section is present it must contain only one data line.
If the section is missing or is empty, the first free row will be chosen instead.
Alternatively, OBJNAME can be overridden by setting nonempty :math:`\mathrm{pnames}[1]` (see :ref:`Parameters <e04mx-py2-py-parameters>`).
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------------------------------+
|Field |Required|Description |
+=========+========+=============================================+
|:math:`2`|No |Row name to be used as the objective function|
+---------+--------+---------------------------------------------+
Field 2 must contain a valid row name.
`ROWS Section`
The data lines in this section specify unique row (constraint) names and their inequality types (i.e., unconstrained, :math:`=`, :math:`\geq` or :math:`\leq`).
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+--------------+
|Field |Required|Description |
+=========+========+==============+
|:math:`1`|Yes |Inequality key|
+---------+--------+--------------+
|:math:`2`|Yes |Row name |
+---------+--------+--------------+
The inequality key specifies each row's type.
It must be ``E``, ``G``, ``L`` or ``N`` and can be in either column :math:`2` or :math:`3`.
.. rst-class:: nag-rules-none nag-align-left
+--------------+------------------------+---------------+--------------+
|Inequality Key|Description |:math:`l` |:math:`u` |
+==============+========================+===============+==============+
|``N`` |Free row |:math:`-\infty`|:math:`\infty`|
+--------------+------------------------+---------------+--------------+
|``G`` |Greater than or equal to|finite |:math:`\infty`|
+--------------+------------------------+---------------+--------------+
|``L`` |Less than or equal to |:math:`-\infty`|finite |
+--------------+------------------------+---------------+--------------+
|``E`` |Equal to |finite |:math:`l` |
+--------------+------------------------+---------------+--------------+
Row type ``N`` stands for 'Not binding'.
It can be used to define the objective row.
The objective row is a free row that specifies the vector :math:`c` in the linear objective term :math:`c^\mathrm{T}x`.
If there is more than one free row, the first free row is chosen, unless another free row name is specified by OBJNAME (see `OBJNAME Section (optional) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#objnamesection>`__) or :math:`\mathrm{pnames}[1]` (see :ref:`Parameters <e04mx-py2-py-parameters>`).
Note that :math:`c` is assumed to be zero if either the chosen row does not appear in the COLUMNS section (i.e., has no nonzero elements) or there are no free rows defined in the ROWS section.
`COLUMNS Section`
Data lines in this section specify the names to be assigned to the variables (columns) in the general linear constraint matrix :math:`A`, and define, in terms of column vectors, the actual values of the corresponding matrix elements.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |Column name|
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
Each data line in the COLUMNS section defines the nonzero elements of :math:`A` or :math:`c`.
Any elements of :math:`A` or :math:`c` that are undefined are assumed to be zero.
Nonzero elements of :math:`A` must be grouped by column, that is to say that all of the nonzero elements in the jth column of :math:`A` must be specified before those in the :math:`\textit{j}+1`\ th column, for :math:`\textit{j} = 1,2,\ldots,n-1`.
Rows may appear in any order within the column.
`Integer Markers`
For backward compatibility ``miqp_mps_read`` allows you to define the integer variables within the COLUMNS section using integer markers, although this is not recommended as markers can be treated differently by different MPS readers; you should instead define any integer variables in the BOUNDS section (see below).
Each marker line must have the following format:
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|No |Marker ID |
+---------+--------+-----------+
|:math:`3`|Yes |Marker tag |
+---------+--------+-----------+
|:math:`5`|Yes |Marker type|
+---------+--------+-----------+
The marker tag must be ``'MARKER'``.
The marker type must be ``'INTORG'`` to start reading integer variables and ``'INTEND'`` to finish reading integer variables.
This implies that a row cannot be named ``'MARKER'``, ``'INTORG'`` or ``'INTEND'``.
Please note that both marker tag and marker type comprise of :math:`8` characters as a ``'`` is the mandatory first and last character in the string.
You may wish to have several integer marker sections within the COLUMNS section, in which case each marker section must begin with an ``'INTORG'`` marker and end with an ``'INTEND'`` marker and there should not be another marker between them.
Field 2 is ignored by ``miqp_mps_read``.
When an integer variable is declared it will keep its default bounds unless they are changed in the BOUNDS section.
This may vary between different MPS readers.
`RHS Section`
This section specifies the right-hand side values (if any) of the general linear constraint matrix :math:`A`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |RHS name |
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
The MPS file may contain several RHS sets distinguished by RHS name.
If an RHS name is defined in :math:`\mathrm{pnames}[2]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then ``miqp_mps_read`` will read in only that RHS vector, otherwise the first RHS set will be used.
Only the nonzero RHS elements need to be specified.
Note that if an RHS is given to the objective function it will be ignored by ``miqp_mps_read``.
An RHS given to the objective function is dealt with differently by different MPS readers, therefore, it is safer to not define an RHS of the objective function in your MPS file.
Note that this section may be empty, in which case the RHS vector is assumed to be zero.
`RANGES Section (optional)`
Ranges are used to modify the interpretation of constraints defined in the ROWS section (see `ROWS Section <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mxf.html#rowssection>`__) to the form :math:`l\leq Ax\leq u`, where both :math:`l` and :math:`u` are finite.
The range of the constraint is :math:`r = u-l`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+-----------+
|Field |Required|Description|
+=========+========+===========+
|:math:`2`|Yes |Range name |
+---------+--------+-----------+
|:math:`3`|Yes |Row name |
+---------+--------+-----------+
|:math:`4`|Yes |Value |
+---------+--------+-----------+
|:math:`5`|No |Row name |
+---------+--------+-----------+
|:math:`6`|No |Value |
+---------+--------+-----------+
The range of each constraint implies an upper and lower bound dependent on the inequality key of each constraint, on the RHS :math:`b` of the constraint (as defined in the RHS section), and on the range :math:`r`.
.. rst-class:: nag-rules-none nag-align-left
+--------------+-----------------+-----------------------------------+-----------------------------------+
|Inequality Key|Sign of :math:`r`|:math:`l` |:math:`u` |
+==============+=================+===================================+===================================+
|``E`` |:math:`+` |:math:`b` |:math:`b+r` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``E`` |:math:`-` |:math:`b+r` |:math:`b` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``G`` |:math:`+/-` |:math:`b` |:math:`b+\left\lvert r\right\rvert`|
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``L`` |:math:`+/-` |:math:`b-\left\lvert r\right\rvert`|:math:`b` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
|``N`` |:math:`+/-` |:math:`-\infty` |:math:`+\infty` |
+--------------+-----------------+-----------------------------------+-----------------------------------+
If a range name is defined in :math:`\mathrm{pnames}[3]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then the function will read in only the range set of that name, otherwise the first set will be used.
`BOUNDS Section (optional)`
These lines specify limits on the values of the variables (the quantities :math:`l` and :math:`u` in :math:`l\leq x\leq u`).
If a variable is not specified in the bound set then it is automatically assumed to lie between :math:`0` and :math:`{+\infty }`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------+
|Field |Required|Description |
+=========+========+=====================+
|:math:`1`|Yes |Bound type identifier|
+---------+--------+---------------------+
|:math:`2`|Yes |Bound name |
+---------+--------+---------------------+
|:math:`3`|Yes |Column name |
+---------+--------+---------------------+
|:math:`4`|Yes/No |Value |
+---------+--------+---------------------+
Note: field 4 is required only if the bound type identifier is one of ``UP``, ``LO``, ``FX``, ``UI`` or ``LI`` in which case it gives the value :math:`k` below. If the bound type identifier is ``FR``, ``MI``, ``PL`` or ``BV``, field 4 is ignored and it is recommended to leave it blank.
The table below describes the acceptable bound type identifiers and how each determines the variables' bounds.
.. rst-class:: nag-rules-none nag-align-left
+---------------------+---------------+--------------+-----------------+
|Bound Type Identifier|:math:`l` |:math:`u` |Integer Variable?|
+=====================+===============+==============+=================+
|``UP`` |unchanged |:math:`k` |No |
+---------------------+---------------+--------------+-----------------+
|``LO`` |:math:`k` |unchanged |No |
+---------------------+---------------+--------------+-----------------+
|``FX`` |:math:`k` |:math:`k` |No |
+---------------------+---------------+--------------+-----------------+
|``FR`` |:math:`-\infty`|:math:`\infty`|No |
+---------------------+---------------+--------------+-----------------+
|``MI`` |:math:`-\infty`|unchanged |No |
+---------------------+---------------+--------------+-----------------+
|``PL`` |unchanged |:math:`\infty`|No |
+---------------------+---------------+--------------+-----------------+
|``BV`` |:math:`0` |:math:`1` |Yes |
+---------------------+---------------+--------------+-----------------+
|``UI`` |unchanged |:math:`k` |Yes |
+---------------------+---------------+--------------+-----------------+
|``LI`` |:math:`k` |unchanged |Yes |
+---------------------+---------------+--------------+-----------------+
If a bound name is defined in :math:`\mathrm{pnames}[4]` (see :ref:`Parameters <e04mx-py2-py-parameters>`) then the function will read in only the bound set of that name, otherwise the first set will be used.
`QUADOBJ Section (optional)`
The QUADOBJ section defines nonzero elements of the upper or lower triangle of the Hessian matrix :math:`H`.
.. rst-class:: nag-rules-none nag-align-left
+---------+--------+---------------------------+
|Field |Required|Description |
+=========+========+===========================+
|:math:`2`|Yes |Column name (HColumn Index)|
+---------+--------+---------------------------+
|:math:`3`|Yes |Column name (HRow Index) |
+---------+--------+---------------------------+
|:math:`4`|Yes |Value |
+---------+--------+---------------------------+
|:math:`5`|No |Column name (HRow Index) |
+---------+--------+---------------------------+
|:math:`6`|No |Value |
+---------+--------+---------------------------+
Each data line in the QUADOBJ section defines one (or optionally two) nonzero elements :math:`H_{{ij}}` of the matrix :math:`H`.
Each element :math:`H_{{ij}}` is given as a triplet of row index :math:`i`, column index :math:`j` and a value.
The column names (as defined in the COLUMNS section) are used to link the names of the variables and the indices :math:`i` and :math:`j`.
More precisely, the matrix :math:`H` on output will have a nonzero element
.. math::
H_{{ij}} = \text{Value}
where index :math:`j` belongs to HColumn Index and index :math:`i` to one of the HRow Indices such that
:math:`\mathrm{crname}[j-1] = \text{Column name (HColumn Index)}` and
:math:`\mathrm{crname}[i-1] = \text{Column name (HRow Index)}`.
It is only necessary to define either the upper or lower triangle of the :math:`H` matrix; either will suffice.
Any elements that have been defined in the upper triangle of the matrix will be moved to the lower triangle of the matrix, then any repeated nonzeros will be summed.
Note: it is much more efficient for :meth:`qpconvex1_sparse_solve` and :meth:`qpconvex2_sparse_solve` to have the :math:`H` matrix defined by the first :math:`\mathrm{ncolh}` column names. If the nonzeros of :math:`H` are defined by any columns that are not in the first :math:`\mathrm{ncolh}` of :math:`\mathrm{n}` then ``miqp_mps_read`` will rearrange the matrices :math:`A` and :math:`H` so that they are.
**Query Mode**
``miqp_mps_read`` offers a 'query mode' to quickly give upper estimates on the sizes of user arrays.
In this mode any expensive checks of the data and of the file format are skipped, providing a prompt count of the number of variables, constraints and matrix nonzeros.
This might be useful in the common case where the size of the problem is not known in advance.
You may activate query mode by setting any of the following: :math:`\mathrm{maxn} < 1`, :math:`\mathrm{maxm} < 1`, :math:`\mathrm{maxnnz} < 1`, :math:`\mathrm{maxncolh} < 0` or :math:`\mathrm{maxnnzh} < 0`.
If no major formatting error is detected in the data file, no exception or warning is raised is returned and the upper estimates are given as stated in Table [label omitted].
Alternatively, the function switches to query mode while the file is being read if it is discovered that the provided space is insufficient (that is, if :math:`\mathrm{n} > \mathrm{maxn}`, :math:`\mathrm{m} > \mathrm{maxm}`, :math:`\mathrm{nnz} > \mathrm{maxnnz}`, :math:`\mathrm{ncolh} > \mathrm{maxncolh}`, :math:`\mathrm{nnzh} > \mathrm{maxnnzh}` or :math:`\mathrm{lintvar} > \mathrm{maxlintvar}`).
In this case :math:`\mathrm{errno}` = 2 is returned.
.. rst-class:: nag-rules-none
+------------------------+---------------------------+
|Argument Name |Upper Estimate for |
+========================+===========================+
|:math:`\mathrm{n}` |:math:`\mathrm{maxn}` |
+------------------------+---------------------------+
|:math:`\mathrm{m}` |:math:`\mathrm{maxm}` |
+------------------------+---------------------------+
|:math:`\mathrm{nnz}` |:math:`\mathrm{maxnnz}` |
+------------------------+---------------------------+
|:math:`\mathrm{ncolh}` |:math:`\mathrm{maxncolh}` |
+------------------------+---------------------------+
|:math:`\mathrm{nnzh}` |:math:`\mathrm{maxnnzh}` |
+------------------------+---------------------------+
|:math:`\mathrm{lintvar}`|:math:`\mathrm{maxlintvar}`|
+------------------------+---------------------------+
.. _e04mx-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def qpconvex1_sparse_mps(infile, maxn, maxm, maxnnz, xbldef, xbudef, mpslst, names, io_manager=None):
r"""
``qpconvex1_sparse_mps`` reads data for a sparse linear programming or quadratic programming problem from an external file which is in standard or compatible MPSX input format.
.. deprecated:: 26.2.0.0
``qpconvex1_sparse_mps`` is deprecated.
Please use :meth:`miqp_mps_read` instead.
See also the :ref:`Replacement Calls <replace>` document.
.. _e04mz-py2-py-doc:
For full information please refer to the NAG Library document for e04mz
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04mzf.html
.. _e04mz-py2-py-parameters:
**Parameters**
**infile** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) associated with the MPSX data file.
**maxn** : int
An upper limit for the number of variables in the problem.
**maxm** : int
An upper limit for the number of constraints (including the objective row) in the problem.
**maxnnz** : int
An upper limit for the number of nonzeros (including the objective row) in the problem.
**xbldef** : float
The default lower bound to be used for the variables in the problem when none is specified in the BOUNDS section of the MPSX data file. For a standard LP or QP problem :math:`\mathrm{xbldef}` would normally be set to zero.
**xbudef** : float
The default upper bound to be used for the variables in the problem when none is specified in the BOUNDS section of the MPSX data file. For a standard LP or QP problem :math:`\mathrm{xbudef}` would normally be set to 'infinity' (i.e., :math:`\mathrm{xbudef}\geq 10^{20}`).
**mpslst** : bool
If :math:`\mathrm{mpslst} = \mathbf{True}`, a listing of the input data is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`). This can be useful for debugging the MPSX data file. If :math:`\mathrm{mpslst} = \mathbf{False}`, no listing is produced.
**names** : str, length 8, array-like, shape :math:`\left(5\right)`
A set of names associated with the MPSX form of the problem.
:math:`\mathrm{names}[0]`
Must contain either the name of the problem or be blank.
:math:`\mathrm{names}[1]`
Must contain either the name of the objective row or be blank (in which case the first objective free row is used).
:math:`\mathrm{names}[2]`
Must contain either the name of the RHS set to be used or be blank (in which case the first RHS set is used).
:math:`\mathrm{names}[3]`
Must contain either the name of the RANGE set to be used or be blank (in which case the first RANGE set (if any) is used).
:math:`\mathrm{names}[4]`
Must contain either the name of the BOUNDS set to be used or be blank (in which case the first BOUNDS set (if any) is used).
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**n** : int
:math:`n`, the actual number of variables in the problem.
**m** : int
:math:`m`, the actual number of general linear constraints in the problem (including the objective row).
**nnz** : int
The actual number of nonzeros in the problem (including the objective row).
**iobj** : int
If :math:`\mathrm{iobj} > 0`, row :math:`\mathrm{iobj}` of :math:`A` is a free row containing the nonzero coefficients of the vector :math:`c`.
If :math:`\mathrm{iobj} = 0`, the coefficients of :math:`c` are assumed to be zero.
If :math:`\mathrm{iobj} = -1`, no such row was found and the function terminates with :math:`\mathrm{errno}` = 4 or 5 (see :ref:`Exceptions <e04mz-py2-py-errors>`).
**ncolh** : int
:math:`\mathrm{ncolh} = 0`. For QP problems, :math:`\mathrm{ncolh}` is the number of leading nonzero columns of the Hessian matrix :math:`H` and must, therefore, be set :math:`\text{} > 0` before calling :meth:`qpconvex1_sparse_solve`.
**a** : float, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The nonzero elements of :math:`A`, ordered by increasing column index.
**ha** : int, ndarray, shape :math:`\left(\mathrm{maxnnz}\right)`
The row indices of the nonzero elements stored in :math:`\mathrm{a}`.
**ka** : int, ndarray, shape :math:`\left(\mathrm{maxn}+1\right)`
A set of pointers to the beginning of each column of :math:`A`. More precisely, :math:`\mathrm{ka}[\textit{i}-1]` contains the index in :math:`\mathrm{a}` of the start of the :math:`\textit{i}`\ th column, for :math:`\textit{i} = 1,2,\ldots,\mathrm{n}`. Note that :math:`\mathrm{ka}[0] = 1` and :math:`\mathrm{ka}[\mathrm{n}] = \mathrm{nnz}+1`.
**bl** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bl}` contains the vector :math:`l` (the lower bounds) for all the variables and constraints
**bu** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
:math:`\mathrm{bu}` contains the vector :math:`u` (the upper bounds) for all the variables and constraints
**start** : str, length 1
:math:`\mathrm{start} = \texttt{'C'}` and an internal Crash procedure will be used by :meth:`qpconvex1_sparse_solve` to choose an initial basis.
**names** : str, length 8, ndarray, shape :math:`\left(5\right)`
A set of names associated with the problem as defined in the MPSX data file as follows:
:math:`\mathrm{names}[0]`
Contains the name of the problem (or blank if none).
:math:`\mathrm{names}[1]`
Contains the name of the objective row (or blank if none).
:math:`\mathrm{names}[2]`
Contains the name of the RHS set (or blank if none).
:math:`\mathrm{names}[3]`
Contains the name of the RANGE set (or blank if none).
:math:`\mathrm{names}[4]`
Contains the name of the BOUNDS set (or blank if none).
**nname** : int
:math:`n+m`, the total number of variables and constraints in the problem.
**crname** : str, length 8, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
The MPSX names of all the variables and constraints in the problem in the following order. The first :math:`\mathrm{n}` elements contain the MPSX names for the variables and the next :math:`\mathrm{m}` elements contain the MPSX names for the objective row and general linear constraints (if any). Note that the MPSX name for the objective row is stored in :math:`\mathrm{crname}[\mathrm{n}+\mathrm{iobj}-1]`.
**xs** : float, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
A set of initial values for the variables and constraints in the problem. More precisely, :math:`\mathrm{xs}[\textit{j}-1] = \mathrm{min}\left(\mathrm{max}\left(0.0, {\mathrm{bl}[\textit{j}-1]}\right), \mathrm{bu}[\textit{j}-1]\right)`, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nname}`.
**istate** : int, ndarray, shape :math:`\left(\mathrm{maxn}+\mathrm{maxm}\right)`
A set of initial states for the variables and constraints in the problem. More precisely, :math:`\mathrm{istate}[\textit{j}-1] = 1` if :math:`\mathrm{xs}[\textit{j}-1] = \mathrm{bu}[\textit{j}-1]` and :math:`0` otherwise, for :math:`\textit{j} = 1,2,\ldots,\mathrm{nname}`.
.. _e04mz-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
Too many rows. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`2`)
Too many columns. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`3`)
Too many nonzeros. Limit is :math:`\langle\mathit{\boldsymbol{value}}\rangle`, but the actual number required is :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`4`)
No objective row found.
(`errno` :math:`5`)
Objective row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the ROWS section.
(`errno` :math:`6`)
No rows specified.
(`errno` :math:`7`)
Illegal constraint type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`8`)
Row name with leading blank or non-alphanumeric character: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`9`)
Column name with leading blank or non-alphanumeric character: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`10`)
Illegal bound type :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`11`)
Column name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the COLUMNS section.
(`errno` :math:`12`)
The last line must be the ENDATA indicator line.
(`errno` :math:`13`)
Line :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not a comment nor a valid line.
(`errno` :math:`14`)
Row name :math:`\langle\mathit{\boldsymbol{value}}\rangle` is not defined in the ROWS section.
(`errno` :math:`15`)
No columns specified.
(`errno` :math:`16`)
BOUNDS name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`16`)
RANGES name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`16`)
RHS name :math:`\langle\mathit{\boldsymbol{value}}\rangle` was not found.
(`errno` :math:`17`)
On entry, :math:`\mathrm{xbldef} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{xbudef} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{xbldef}\leq \mathrm{xbudef}`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxnnz} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxnnz}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxm} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxm}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{maxn} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\mathrm{maxn}\geq 1`.
(`errno` :math:`17`)
On entry, :math:`\mathrm{infile} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{infile}\leq 2147483647`.
.. _e04mz-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``qpconvex1_sparse_mps`` reads Linear Programming (LP) or Quadratic Programming (QP) problem data from an external file which is prepared in standard or compatible MPSX (see IBM (1971)) input format and then initializes :math:`n` (the number of variables), :math:`m` (the number of general linear constraints), the :math:`m\times n` matrix :math:`A`, and the vectors :math:`l`, :math:`u` and :math:`c` (stored in row :math:`\mathrm{iobj}` of :math:`A`) for use with :meth:`qpconvex1_sparse_solve`, which is designed to solve problems of the form
.. math::
\mathrm{minimize}_{{x \in R^n}}c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Hx\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Ax\end{array}\right\}\leq u\text{.}
For LP problems, :math:`H = 0`.
For QP problems, you must set :math:`\mathrm{ncolh} > 0` (see :ref:`Parameters <e04mz-py2-py-parameters>`) and provide a function to :meth:`qpconvex1_sparse_solve` to compute :math:`Hx` for any given vector :math:`x`.
The option 'Maximize' may be used to specify an alternative problem in which the objective function is maximized (see :ref:`Other Parameters for qpconvex1_sparse_solve <e04nk-py2-py-other_params>`).
**MPSX input format**
The input file of data may only contain two types of lines:
(1) Indicator lines (specifying the type of data which is to follow).
(#) Data lines (specifying the actual data).
The input file must not contain any blank lines.
Any characters beyond column 80 are ignored.
Indicator lines must not contain leading blank characters (in other words they must begin in column 1).
The following displays the order in which the indicator lines must appear in the file:
.. rst-class:: nag-rules-none nag-align-left
+-------+------------+------------------+
|NAME | |user-supplied name|
+-------+------------+------------------+
|ROWS | | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|COLUMNS| | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|RHS | | |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|RANGES | |(optional) |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|BOUNDS | |(optional) |
+-------+------------+------------------+
| |data line(s)| |
+-------+------------+------------------+
|ENDATA | | |
+-------+------------+------------------+
The 'user-supplied name' specifies a name for the problem and must occupy columns :math:`15-22`.
The name can either be blank or up to a maximum of :math:`8` characters.
A data line follows the same fixed format made up of fields defined below.
The contents of the fields may have different significance depending upon the section of data in which they appear.
.. rst-class:: nag-rules-none
+--------+-----------+------------+-------------+-------------+-------------+-------------+
| |Field 1 |Field 2 |Field 3 |Field 4 |Field 5 |Field 6 |
+========+===========+============+=============+=============+=============+=============+
|Columns |:math:`2-3`|:math:`5-12`|:math:`15-22`|:math:`25-36`|:math:`40-47`|:math:`50-61`|
+--------+-----------+------------+-------------+-------------+-------------+-------------+
|Contents|Code |Name |Name |Value |Name |Value |
+--------+-----------+------------+-------------+-------------+-------------+-------------+
The names and codes consist of 'alphanumeric' characters (i.e., a--z, A--Z, :math:`0`--:math:`9`, :math:`+`, :math:`-`, :math:`*`, blank (), :, $ or full stop (.) only) and the names must not contain leading blank characters.
Values are read using Fortran format :math:`e12.0`.
This allows values to be entered in several equivalent forms.
For example, :math:`1.2345678`, :math:`1.2345678e+0`, :math:`123.45678e-2` and :math:`12345678e-07` all represent the same number.
It is safest to include an explicit decimal point.
Note that in order to ensure numeric values are interpreted as intended, they should be `right-justified` in the :math:`12`-character field, with no trailing blanks.
This is because in some situations trailing blanks may be interpreted as zeros and this can dramatically affect the interpretation of the value.
This is relevant if the value contains an exponent, or if it contains neither an exponent nor an explicit decimal point.
For example, the fields
::
%%%%1.23e-2%
%%%%%%%123%%
may be interpreted as :math:`1.23e-20` and :math:`12300` respectively (where ``%`` denotes a blank).
The actual behaviour is system-dependent.
Comment lines are allowed in the data file.
These must have an asterisk (``*``) in column 1 and any characters in columns 2--80.
In any data line, a dollar sign (``$``) as the first character in Field 3 or 5 indicates that the information from that point through column 80 consists of comments.
Columns outside the six fields must be blank, except for columns 72--80, whose contents are ignored by the function.
These columns may be used to enter a sequence number.
A non-blank character outside the predefined six fields and columns 72--80 is considered to be a major error (:math:`\mathrm{errno}` = 13; see :ref:`Exceptions <e04mz-py2-py-errors>`), unless it is part of a comment.
**ROWS Data Lines**
These lines specify row (constraint) names and their inequality types (i.e., :math:`=`, :math:`\geq` or :math:`\leq`).
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------------------------------------------------------------------------+
|Field 1:|defines the constraint type. It may be in column 2 or column 3. |
+--------+------------------------------------------------------------------------------------+
| |+-----+----------------------------------------------------------------------------+|
| ||``N``|free row, that is no constraint. It may be used to define the objective row.||
| |+-----+----------------------------------------------------------------------------+|
| ||``G``|greater than or equal to (i.e., :math:`\geq`). ||
| |+-----+----------------------------------------------------------------------------+|
| ||``L``|less than or equal to (i.e., :math:`\leq`). ||
| |+-----+----------------------------------------------------------------------------+|
| ||``E``|exactly equal to (i.e., :math:`=`). ||
| |+-----+----------------------------------------------------------------------------+|
+--------+------------------------------------------------------------------------------------+
|Field 2:|defines the row name. |
+--------+------------------------------------------------------------------------------------+
Row type ``N`` stands for 'Not binding', also known as 'Free'.
It can be used to define the objective row.
The objective row is a free row that specifies the vector :math:`c` in the linear objective term :math:`c^\mathrm{T}x`.
It is taken to be the first free row, unless some other free row name is specified by the :math:`\mathrm{names}` array (see :ref:`Parameters <e04mz-py2-py-parameters>`).
Note that :math:`c` is assumed to be zero if (for example) the line
::
%N%%DUMMYROW
(where ``%`` denotes a blank) appears in the ROWS section of the MPSX data file, and the row name ``DUMMYROW`` is omitted from the COLUMNS section.
**COLUMNS Data Lines**
These lines specify the names to be assigned to the variables (columns) in the general linear constraint matrix :math:`A`, and define, in terms of column vectors, the actual values of the corresponding matrix elements.
.. rst-class:: nag-rules-none nag-align-left
+--------+--------------------------------------------------------------------------------------------+
|Field 1:|blank (ignored). |
+--------+--------------------------------------------------------------------------------------------+
|Field 2:|gives the name of the column associated with the elements specified in the following fields.|
+--------+--------------------------------------------------------------------------------------------+
|Field 3:|contains the name of a row. |
+--------+--------------------------------------------------------------------------------------------+
|Field 4:|used in conjunction with Field 3 contains the value of the matrix element. |
+--------+--------------------------------------------------------------------------------------------+
|Field 5:|is optional (may be used like Field 3). |
+--------+--------------------------------------------------------------------------------------------+
|Field 6:|is optional (may be used like Field 4). |
+--------+--------------------------------------------------------------------------------------------+
Note that only the nonzero elements of :math:`A` and :math:`c` need to be specified in the COLUMNS section, as any zero elements of :math:`A` are removed and any unspecified elements of :math:`c` are assumed to be zero.
In addition, any nonzero elements in the :math:`\textit{j}`\ th column of :math:`A` must be grouped together before those in the :math:`\left(\textit{j}+1\right)`\ th column, for :math:`\textit{j} = 1,2,\ldots,n-1`.
Nonzero elements within a column may however appear in any order.
**RHS Data Lines**
This section specifies the right-hand side values of the general linear constraint matrix :math:`A` (if any).
The lines specify the name to be given to the right-hand side (RHS) vector along with the numerical values of the elements of the vector, which may appear in any order.
The data lines have exactly the same format as the COLUMNS data lines, except that the column name is replaced by the RHS name.
Only the nonzero elements need be specified.
Note that this section may be empty, in which case the RHS vector is assumed to be zero.
**RANGES Data Lines (optional)**
Ranges are used for constraints of the form :math:`l\leq Ax\leq u`, where both :math:`l` and :math:`u` are finite.
The range of the constraint is :math:`r = u-l`.
Either :math:`l` or :math:`u` must be specified in the RHS section and :math:`r` must be defined in this section.
The data lines have exactly the same format as the COLUMNS data lines, except that the column name is replaced by the RANGES name.
**BOUNDS Data Lines (optional)**
These lines specify limits on the values of the variables (:math:`l` and :math:`u` in :math:`l\leq x\leq u`).
If the variable is not specified in the bound set then it is automatically assumed to lie between default lower and upper bounds (usually :math:`0` and :math:`{+\infty }`).
Like an RHS column which is given a name, the set of variables in one bound set is also given a name.
.. rst-class:: nag-rules-none nag-align-left
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 1:|+--+---------------------------------------------------------------------+ |
| ||LO|lower bound | |
| |+--+---------------------------------------------------------------------+ |
| ||UP|upper bound | |
| |+--+---------------------------------------------------------------------+ |
| ||FX|fixed variable | |
| |+--+---------------------------------------------------------------------+ |
| ||FR|free variable (:math:`{-\infty }` to :math:`{+\infty }`) | |
| |+--+---------------------------------------------------------------------+ |
| ||MI|lower bound is :math:`{-\infty }` | |
| |+--+---------------------------------------------------------------------+ |
| ||PL|upper bound is :math:`{+\infty }`. This is the default variable type.| |
| |+--+---------------------------------------------------------------------+ |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 2:|identifies a name for the bound set. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 3:|identifies the column name of the variable belonging to this set. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 4:|identifies the value of the bound; this has a numerical value only in association with LO, UP, FX in Field 1, otherwise it is blank.|
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 5:|is blank and ignored. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
|Field 6:|is blank and ignored. |
+--------+------------------------------------------------------------------------------------------------------------------------------------+
Note that if RANGES and BOUNDS sections are both present, the RANGES section must appear first.
.. _e04mz-py2-py-references:
**References**
IBM, 1971, `MPSX -- Mathematical programming system`, Program Number 5734 XM4, IBM Trade Corporation, New York
"""
raise NotImplementedError
[docs]def lsq_lincon_solve(bl, bu, x, comm, c=None, cvec=None, istate=None, kx=None, a=None, b=None, io_manager=None):
r"""
``lsq_lincon_solve`` solves linearly constrained linear least squares problems and convex quadratic programming problems.
It is not intended for large sparse problems.
Note: this function uses optional algorithmic parameters, see also: :meth:`lsq_lincon_option_file`, :meth:`lsq_lincon_option_string`, :meth:`nlp1_init`.
.. _e04nc-py2-py-doc:
For full information please refer to the NAG Library document for e04nc
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html
.. _e04nc-py2-py-parameters:
**Parameters**
**bl** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bl}` must contain the lower bounds for all the constraints
**bu** : float, array-like, shape :math:`\left(n+\textit{nclin}\right)`
:math:`\mathrm{bu}` must contain the upper bounds for all the constraints
**x** : float, array-like, shape :math:`\left(n\right)`
An initial estimate of the solution.
Note: that it may be best to avoid the choice :math:`\mathrm{x} = 0.0`.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**c** : None or float, array-like, shape :math:`\left(\textit{nclin}, :\right)`, optional
Note: the required extent for this argument in dimension 2 is determined as follows: if :math:`\textit{nclin} > 0`: :math:`n`; otherwise: :math:`1`.
The :math:`\textit{i}`\ th row of :math:`\mathrm{c}` must contain the coefficients of the :math:`\textit{i}`\ th general constraint, for :math:`\textit{i} = 1,2,\ldots,\textit{nclin}`.
**cvec** : None or float, array-like, shape :math:`\left(n\right)`, optional
The coefficients of the explicit linear term of the objective function.
If the problem is of type FP, QP1, QP3, LS1 (the default) or LS3, :math:`\mathrm{cvec}` is not referenced.
**istate** : None or int, array-like, shape :math:`\left(n+\textit{nclin}\right)`, optional
Need not be set if the (default) option 'Cold Start' is used.
If the option 'Warm Start' has been chosen, :math:`\mathrm{istate}` specifies the desired status of the constraints at the start of the feasibility phase.
More precisely, the first :math:`n` elements of :math:`\mathrm{istate}` refer to the upper and lower bounds on the variables, and the next :math:`n_L` elements refer to the general linear constraints (if any).
Possible values for :math:`\mathrm{istate}[j-1]` are as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+========================================================================================================================================================+
|0 |The constraint should `not` be in the initial working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|1 |The constraint should be in the initial working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|2 |The constraint should be in the initial working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|3 |The constraint should be in the initial working set as an equality. This value must not be specified unless :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
The values :math:`-2`, :math:`-1` and :math:`4` are also acceptable but will be reset to zero by the function.
If ``lsq_lincon_solve`` has been called previously with the same values of :math:`\textit{n}` and :math:`\textit{nclin}`, :math:`\mathrm{istate}` already contains satisfactory information. (See also the description of the option 'Warm Start'.) The function also adjusts (if necessary) the values supplied in :math:`\mathrm{x}` to be consistent with :math:`\mathrm{istate}`.
**kx** : None or int, array-like, shape :math:`\left(n\right)`, optional
Need not be initialized for problems of type FP, LP, QP1, QP2, LS1 (the default) or LS2.
For problems QP3, QP4, LS3 or LS4, :math:`\mathrm{kx}` must specify the order of the columns of the matrix :math:`A` with respect to the ordering of :math:`\mathrm{x}`.
Thus if column :math:`j` of :math:`A` is the column associated with the variable :math:`x_i` then :math:`\mathrm{kx}[j-1] = i`.
**a** : None or float, array-like, shape :math:`\left(m, n\right)`, optional
The matrix :math:`A`.
**b** : None or float, array-like, shape :math:`\left(m\right)`, optional
The :math:`m` elements of the vector of observations.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
**Returns**
**istate** : int, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The status of the constraints in the working set at the point returned in :math:`\mathrm{x}`. The significance of each possible value of :math:`\mathrm{istate}[j-1]` is as follows:
.. rst-class:: nag-rules-none nag-align-left
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\mathrm{istate}[j-1]`|Meaning |
+============================+====================================================================================================================================================================+
|:math:`-2` |The constraint violates its lower bound by more than the feasibility tolerance. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`-1` |The constraint violates its upper bound by more than the feasibility tolerance. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`0` |The constraint is satisfied to within the feasibility tolerance, but is not in the working set. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |This inequality constraint is included in the working set at its lower bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`2` |This inequality constraint is included in the working set at its upper bound. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`3` |The constraint is included in the working set as an equality. This value of :math:`\mathrm{istate}` can occur only when :math:`\mathrm{bl}[j-1] = \mathrm{bu}[j-1]`.|
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`4` |This corresponds to optimality being declared with :math:`\mathrm{x}[j-1]` being temporarily fixed at its current value. |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
**kx** : int, ndarray, shape :math:`\left(n\right)`
Defines the order of the columns of :math:`\mathrm{a}` with respect to the ordering of :math:`\mathrm{x}`, as described above.
**x** : float, ndarray, shape :math:`\left(n\right)`
The point at which ``lsq_lincon_solve`` terminated. If the function exits successfully or :math:`\mathrm{errno}` = 1 or 4, :math:`\mathrm{x}` contains an estimate of the solution.
**a** : float, ndarray, shape :math:`\left(m, n\right)`
The matrix :math:`A`.
**b** : None or float, ndarray, shape :math:`\left(m\right)`
The transformed residual vector of `(0) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#eqn10>`__ (see `Main Iteration <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#ad-mainiteration>`__).
If the problem is of type FP, LP, QP1, QP2, QP3 or QP4, :math:`\mathrm{b}` is not referenced.
**itera** : int
The total number of iterations performed.
**obj** : float
The value of the objective function at :math:`x` if :math:`x` is feasible, or the sum of infeasibiliites at :math:`x` otherwise. If the problem is of type FP and :math:`x` is feasible, :math:`\mathrm{obj}` is set to zero.
**clamda** : float, ndarray, shape :math:`\left(n+\textit{nclin}\right)`
The values of the Lagrange multipliers for each constraint with respect to the current working set. The first :math:`n` elements contain the multipliers for the bound constraints on the variables, and the next :math:`n_L` elements contain the multipliers for the general linear constraints (if any). If :math:`\mathrm{istate}[j-1] = 0` (i.e., constraint :math:`j` is not in the working set), :math:`\mathrm{clamda}[j-1]` is zero. If :math:`x` is optimal, :math:`\mathrm{clamda}[j-1]` should be non-negative if :math:`\mathrm{istate}[j-1] = 1`, non-positive if :math:`\mathrm{istate}[j-1] = 2` and zero if :math:`\mathrm{istate}[j-1] = 4`.
.. _e04nc-py2-py-other_params:
**Other Parameters**
**'Cold Start'** : valueless
Default
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lsq_lincon_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lsq_lincon_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_lincon_solve`` is called repeatedly to solve related problems.
**'Warm Start'** : valueless
This option specifies how the initial working set is chosen.
With a 'Cold Start', ``lsq_lincon_solve`` chooses the initial working set based on the values of the variables and constraints at the initial point.
Broadly speaking, the initial working set will include equality constraints and bounds or inequality constraints that violate or 'nearly' satisfy their bounds (to within 'Crash Tolerance').
With a 'Warm Start', you must provide a valid definition of every element of the array :math:`\mathrm{istate}`. ``lsq_lincon_solve`` will override your specification of :math:`\mathrm{istate}` if necessary, so that a poor choice of the working set will not cause a fatal error.
For instance, any elements of :math:`\mathrm{istate}` which are set to :math:`-2`, :math:`-1` or :math:`4` will be reset to zero, as will any elements which are set to :math:`3` when the corresponding elements of :math:`\mathrm{bl}` and :math:`\mathrm{bu}` are not equal.
A warm start will be advantageous if a good estimate of the initial working set is available -- for example, when ``lsq_lincon_solve`` is called repeatedly to solve related problems.
**'Crash Tolerance'** : float
Default :math:`\text{} = 0.01`
This value is used in conjunction with the option 'Cold Start' (the default value) when ``lsq_lincon_solve`` selects an initial working set.
If :math:`0\leq r\leq 1`, the initial working set will include (if possible) bounds or general inequality constraints that lie within :math:`r` of their bounds.
In particular, a constraint of the form :math:`c_j^\mathrm{T}x\geq l` will be included in the initial working set if :math:`\left\lvert c_j^\mathrm{T}x-l\right\rvert \leq r\left(1+\left\lvert l\right\rvert \right)`.
If :math:`r < 0` or :math:`r > 1`, the default value is used.
**'Defaults'** : valueless
This special keyword may be used to reset all options to their default values.
**'Feasibility Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
The scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. Option 'Optimality Phase Iteration Limit' is equivalent to option 'Iteration Limit'.
Setting :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Optimality Phase Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
The scalars :math:`i_1` and :math:`i_2` specify the maximum number of iterations allowed in the feasibility and optimality phases. Option 'Optimality Phase Iteration Limit' is equivalent to option 'Iteration Limit'.
Setting :math:`i_2 = 0` and :math:`\text{‘Print Level'} > 0` means that the workspace needed will be computed and printed, but no iterations will be performed.
If :math:`i_1 < 0` or :math:`i_2 < 0`, the default value is used.
**'Feasibility Tolerance'** : float
Default :math:`\text{} = \sqrt{\epsilon }`
If :math:`r > \epsilon`, :math:`r` defines the maximum acceptable `absolute` violation in each constraint at a 'feasible' point.
For example, if the variables and the coefficients in the general constraints are of order unity, and the latter are correct to about :math:`6` decimal digits, it would be appropriate to specify :math:`r` as :math:`10^{-6}`.
If :math:`0\leq r < \epsilon`, the default value is used.
Note that a 'feasible solution' is a solution that satisfies the current constraints to within the tolerance :math:`r`.
**'Hessian'** : str
Default :math:`\text{} = \mathrm{NO}`
This option controls the contents of the upper triangular matrix :math:`R` (see the description of :math:`\mathrm{a}` in :ref:`Parameters <e04nc-py2-py-parameters>`). ``lsq_lincon_solve`` works exclusively with the transformed and reordered matrix :math:`H_Q` `(8) <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#eqn8>`__, and hence extra computation is required to form the Hessian itself.
If :math:`\text{‘Hessian'} = \texttt{'NO'}`, :math:`\mathrm{a}` contains the Cholesky factor of the matrix :math:`H_Q` with columns ordered as indicated by :math:`\mathrm{kx}` (see :ref:`Parameters <e04nc-py2-py-parameters>`).
If :math:`\text{‘Hessian'} = \texttt{'YES'}`, :math:`\mathrm{a}` contains the Cholesky factor of the matrix :math:`H`, with columns ordered as indicated by :math:`\mathrm{kx}`.
**'Infinite Bound Size'** : float
Default :math:`\text{} = 10^{20}`
If :math:`r > 0`, :math:`r` defines the 'infinite' bound :math:`\textit{bigbnd}` in the definition of the problem constraints.
Any upper bound greater than or equal to :math:`\textit{bigbnd}` will be regarded as :math:`{+\infty }` (and similarly any lower bound less than or equal to :math:`{-\textit{bigbnd}}` will be regarded as :math:`{-\infty }`).
If :math:`r < 0`, the default value is used.
**'Infinite Step Size'** : float
Default :math:`\text{} = \mathrm{max}\left(\textit{bigbnd}, 10^{20}\right)`
If :math:`r > 0`, :math:`r` specifies the magnitude of the change in variables that will be considered a step to an unbounded solution. (Note that an unbounded solution can occur only when the Hessian is singular and the objective contains an explicit linear term.) If the change in :math:`x` during an iteration would exceed the value of :math:`r`, the objective function is considered to be unbounded below in the feasible region.
If :math:`r\leq 0`, the default value is used.
**'Iteration Limit'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Iters'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'Itns'** : int
Default :math:`\text{} = \mathrm{max}\left(50, {5\left(n+n_L\right)}\right)`
See option 'Feasibility Phase Iteration Limit'.
**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Nolist'** : valueless
Default :math:`\text{} = \text{‘Nolist'}`
Option 'List' enables printing of each option specification as it is supplied. 'Nolist' suppresses this printing.
**'Monitoring File'** : int
Default :math:`\text{} = -1`
If :math:`i\geq 0` and :math:`\text{‘Print Level'} \geq 5`, monitoring information produced by ``lsq_lincon_solve`` at every iteration is sent to a file with logical unit number :math:`i`.
If :math:`i < 0` and/or :math:`\text{‘Print Level'} < 5`, no monitoring information is produced.
**'Print Level'** : int
Default :math:`\text{} = 0`
The value of :math:`i` controls the amount of printout produced by ``lsq_lincon_solve``, as indicated below.
A detailed description of the printed output is given in `Further Comments <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#fc-printedoutput>`__ (summary output at each iteration and the final solution) and `Monitoring Information <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#monitoring>`__ (monitoring information at each iteration).
The following printout is sent to the file object associated with the advisory I/O unit (see :class:`~naginterfaces.base.utils.FileObjManager`):
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+===================================================================================================================================================================================================================================+
|:math:`0` |No output. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`1` |The final solution only. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`5` |One line of summary output (:math:`\text{} < 80` characters; see `Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ncf.html#fc-printedoutput>`__) for each iteration (no printout of the final solution).|
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 10`|The final solution and one line of summary output for each iteration. |
+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The following printout is sent to the unit number given by the option 'Monitoring File':
.. rst-class:: nag-rules-none nag-align-left
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`i` |Output |
+======================+=============================================================================================================================================================================================================================================================================================================================================================================================================+
|:math:`\text{} < 5` |No output. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 5` |One long line of output (:math:`\text{} > 80` characters; see `Monitoring Information <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ncf.html#monitoring>`__) for each iteration (no printout of the final solution). |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 20`|At each iteration, the Lagrange multipliers, the variables :math:`x`, the constraint values :math:`Cx` and the constraint status. |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:`\text{}\geq 30`|At each iteration, the diagonal elements of the matrix :math:`T` associated with the :math:`TQ` factorization `(4) <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ncf.html#eqn4>`__ (see `Definition of Search Direction <https://www.nag.com/numeric/nl/nagdoc_28.4/flhtml/e04/e04ncf.html#ad-search>`__) of the working set, and the diagonal elements of the upper triangular matrix :math:`R`.|
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
If :math:`\text{‘Print Level'} \geq 5` and the unit number defined by the option 'Monitoring File' is the advisory unit number, the summary output for each major iteration is suppressed.
**'Problem Type'** : str
Default :math:`=` LS1
This option specifies the type of objective function to be minimized during the optimality phase.
The following are the nine optional keywords and the dimensions of the arrays that must be specified in order to define the objective function:
.. rst-class:: nag-rules-none nag-align-left
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LP |:math:`\mathrm{a}` and :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP1|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` symmetric, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP2|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` symmetric, :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP3|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|QP4|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, :math:`\mathrm{b}` not referenced, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS1|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS2|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, length-:math:`\textit{n}` :math:`\mathrm{cvec}`; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS3|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, :math:`\mathrm{cvec}` not referenced; |
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|LS4|shape-:math:`\left(m, n\right)` :math:`\mathrm{a}` upper trapezoidal, length-:math:`\textit{n}` :math:`\mathrm{kx}`, length-:math:`\textit{m}` :math:`\mathrm{b}`, length-:math:`\textit{n}` :math:`\mathrm{cvec}`.|
+---+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
For problems of type FP, the objective function is omitted and :math:`\mathrm{a}`, :math:`\mathrm{b}` and :math:`\mathrm{cvec}` are not referenced.
The following keywords are also acceptable.
.. rst-class:: nag-rules-none nag-align-left
+---------+------+
|:math:`a`|Option|
+=========+======+
|Least |LS1 |
+---------+------+
|Quadratic|QP2 |
+---------+------+
|Linear |LP |
+---------+------+
In addition, the keywords LS and LSQ are equivalent to the default option LS1, and the keyword QP is equivalent to the option QP2.
If :math:`A = 0`, i.e., the objective function is purely linear, the efficiency of ``lsq_lincon_solve`` may be increased by specifying :math:`a` as LP.
**'Rank Tolerance'** : float
Default :math:`\text{} = 100\epsilon` or :math:`10\sqrt{\epsilon }` (see below)
Note that this option does not apply to problems of type FP or LP.
The default value of :math:`r` depends on the problem type.
If :math:`A` occurs as a least squares matrix, as it does in problem types QP1, LS1 and LS3, then the default value of :math:`r` is :math:`100\epsilon`.
In all other cases, :math:`A` is treated as the 'square root' of the Hessian matrix :math:`H` and :math:`r` has the default value :math:`10\sqrt{\epsilon }`.
This argument enables you to control the estimate of the triangular factor :math:`R_1` (see `Main Iteration <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#ad-mainiteration>`__).
If :math:`\rho_i` denotes the function :math:`\rho_i = \mathrm{max}\left\{\left\lvert R_{11}\right\rvert,\left\lvert R_{22}\right\rvert,\ldots,\left\lvert R_{{ii}}\right\rvert \right\}`, the rank of :math:`R` is defined to be smallest index `i` such that :math:`\left\lvert R_{{i+1,i+1}}\right\rvert \leq r\left\lvert \rho_{{i+1}}\right\rvert`.
If :math:`r\leq 0`, the default value is used.
.. _e04nc-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`4`)
Too many iterations.
(`errno` :math:`5`)
Too many iterations without changing :math:`x`.
(`errno` :math:`6`)
On entry, :math:`n = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`n > 0`.
(`errno` :math:`6`)
On entry, :math:`\textit{nclin} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`\textit{nclin}\geq 0`.
(`errno` :math:`6`)
On entry, :math:`m = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`m > 0`.
(`errno` :math:`6`)
On entry, :math:`\mathrm{kx}` has not been supplied as a valid permutation.
(`errno` :math:`6`)
On entry, the equal bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry with a Warm Start, :math:`\mathrm{istate}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the equal bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are infinite, because :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \textit{beta}`, but :math:`\left\lvert \textit{beta}\right\rvert \geq \textit{bigbnd}`: :math:`\textit{beta} = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\textit{bigbnd} = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on variable :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on linear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`6`)
On entry, the bounds on nonlinear constraint :math:`\langle\mathit{\boldsymbol{value}}\rangle` are inconsistent: :math:`\mathrm{bl}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle` and :math:`\mathrm{bu}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle`.
(`errno` :math:`7`)
The problem to be solved is of type QP1 or QP2, but the Hessian matrix supplied in :math:`\mathrm{a}` is not positive semidefinite.
**Warns**
**NagAlgorithmicWarning**
(`errno` :math:`1`)
Weak :math:`\langle\mathit{\boldsymbol{value}}\rangle` solution.
(`errno` :math:`2`)
:math:`\langle\mathit{\boldsymbol{value}}\rangle` solution is unbounded.
(`errno` :math:`3`)
Cannot satisfy the linear constraints.
.. _e04nc-py2-py-notes:
**Notes**
`In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.`
``lsq_lincon_solve`` is designed to solve a class of quadratic programming problems of the following general form:
.. math::
\mathrm{minimize}_{{x \in R^n}}F\left(x\right)\quad \text{ subject to }\quad l\leq \left\{\begin{array}{c}x\\Cx\end{array}\right\}\leq u
where :math:`\mathrm{c}` is an :math:`n_L\times n` matrix and the objective function :math:`F\left(x\right)` may be specified in a variety of ways depending upon the particular problem to be solved.
The available forms for :math:`F\left(x\right)` are listed in Table [label omitted], in which the prefixes FP, LP, QP and LS stand for 'feasible point', 'linear programming', 'quadratic programming' and 'least squares' respectively, :math:`c` is an :math:`n`-element vector, :math:`b` is an :math:`m` element vector and :math:`\left\lVert z\right\rVert` denotes the Euclidean length of :math:`z`.
+------------+---------------------------------------------------------------+-------------------------------------------------+
|Problem type|:math:`F\left(x\right)` |Matrix :math:`A` |
+============+===============================================================+=================================================+
|FP |None |Not applicable |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LP |:math:`c^\mathrm{T}x` |Not applicable |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP1 |:math:`\frac{1}{2}x^\mathrm{T}Ax` |:math:`n\times n` symmetric positive semidefinite|
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP2 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}Ax` |:math:`n\times n` symmetric positive semidefinite|
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP3 |:math:`\frac{1}{2}x^\mathrm{T}A^\mathrm{T}Ax` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|QP4 |:math:`c^\mathrm{T}x+\frac{1}{2}x^\mathrm{T}A^\mathrm{T}Ax` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS1 |:math:`\frac{1}{2}\left\lVert b-Ax\right\rVert^2` |:math:`m\times n` |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS2 |:math:`c^\mathrm{T}x+\frac{1}{2}\left\lVert b-Ax\right\rVert^2`|:math:`m\times n` |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS3 |:math:`\frac{1}{2}\left\lVert b-Ax\right\rVert^2` |:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
|LS4 |:math:`c^\mathrm{T}x+\frac{1}{2}\left\lVert b-Ax\right\rVert^2`|:math:`m\times n` upper trapezoidal |
+------------+---------------------------------------------------------------+-------------------------------------------------+
In the standard LS problem :math:`F\left(x\right)` will usually have the form LS1, and in the standard convex QP problem :math:`F\left(x\right)` will usually have the form QP2.
The default problem type is LS1 and other objective functions are selected by using the option 'Problem Type'.
When :math:`A` is upper trapezoidal it will usually be the case that :math:`m = n`, so that :math:`A` is upper triangular, but full generality has been allowed for in the specification of the problem.
The upper trapezoidal form is intended for cases where a previous factorization, such as a :math:`QR` factorization, has been performed.
The constraints involving :math:`\mathrm{c}` are called the `general` constraints.
Note that upper and lower bounds are specified for all the variables and for all the general constraints.
An equality constraint can be specified by setting :math:`l_i = u_i`.
If certain bounds are not present, the associated elements of :math:`l` or :math:`u` can be set to special values that will be treated as :math:`{-\infty }` or :math:`{+\infty }`. (See the description of the option 'Infinite Bound Size'.)
The defining feature of a quadratic function :math:`F\left(x\right)` is that the second-derivative matrix :math:`H` (the `Hessian matrix`) is constant.
For the LP case :math:`H = 0`; for QP1 and QP2, :math:`H = A`; for QP3 and QP4, :math:`H = A^\mathrm{T}A` and for LS1 (the default), LS2, LS3 and LS4, :math:`H = A^\mathrm{T}A`.
Problems of type QP3 and QP4 for which :math:`A` is not in upper trapezoidal form should be solved as types LS1 and LS2 respectively, with :math:`b = 0`.
For problems of type LS, we refer to :math:`A` as the `least squares` matrix, or the `matrix of observations` and to :math:`b` as the `vector of observations.`
You must supply an initial estimate of the solution.
If :math:`H` is nonsingular then ``lsq_lincon_solve`` will obtain the unique (global) minimum.
If :math:`H` is singular then the solution may still be a global minimum if all active constraints have nonzero Lagrange multipliers.
Otherwise the solution obtained will be either a weak minimum (i.e., with a unique optimal objective value, but an infinite set of optimal :math:`x`), or else the objective function is unbounded below in the feasible region.
The last case can only occur when :math:`F\left(x\right)` contains an explicit linear term (as in problems LP, QP2, QP4, LS2 and LS4).
The method used by ``lsq_lincon_solve`` is described in detail in `Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ncf.html#algdetails>`__.
.. _e04nc-py2-py-references:
**References**
Gill, P E, Hammarling, S, Murray, W, Saunders, M A and Wright, M H, 1986, `Users' guide for LSSOL (Version 1.0)`, Report SOL 86-1, Department of Operations Research, Stanford University
Gill, P E, Murray, W, Saunders, M A and Wright, M H, 1984, `Procedures for optimization problems with a mixture of bounds and general linear constraints`, ACM Trans. Math. Software (10), 282--298
Gill, P E, Murray, W and Wright, M H, 1981, `Practical Optimization`, Academic Press
Stoer, J, 1971, `On the numerical solution of constrained least squares problems`, SIAM J. Numer. Anal. (8), 382--411
"""
raise NotImplementedError
[docs]def lsq_lincon_option_file(ioptns, comm, io_manager=None):
r"""
``lsq_lincon_option_file`` may be used to supply options to :meth:`lsq_lincon_solve` from an external file.
.. _e04nd-py2-py-doc:
For full information please refer to the NAG Library document for e04nd
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04ndf.html
.. _e04nd-py2-py-parameters:
**Parameters**
**ioptns** : int
The unit number (see :meth:`~naginterfaces.base.utils.FileObjManager.unit_from_fileobj`) of the options file to be read.
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04nd-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`1`)
:math:`\mathrm{ioptns}` is not in the range :math:`\left[0, 2147483647\right]`.
(`errno` :math:`1`)
On entry, :math:`\mathrm{ioptns} = \langle\mathit{\boldsymbol{value}}\rangle`.
Constraint: :math:`0\leq \mathrm{ioptns}\leq 2147483647`.
(`errno` :math:`2`)
``Begin`` was found, but end-of-file was found before ``End`` was found.
(`errno` :math:`3`)
End-of-file was found before ``Begin`` was found.
(`errno` :math:`5`)
One or more lines of the options file is invalid.
.. _e04nd-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_lincon_option_file`` may be used to supply values for options to :meth:`lsq_lincon_solve`. ``lsq_lincon_option_file`` reads an external file and each line of the file defines a single option.
It is only necessary to supply values for those arguments whose values are to be different from their default values.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank strings and comments are ignored.
A comment begins with an asterisk (*) and all subsequent characters in the string are regarded as part of the comment.
The file containing the options must start with ``Begin`` and must finish with ``End``.
An example of a valid options file is:
::
Begin * Example options file
Print level = 5
End
Printing of user-supplied options is turned off by default, but may be turned on at any time using the keyword 'List'.
Option settings are preserved following a call to :meth:`lsq_lincon_solve` and so the keyword 'Defaults' is provided to allow you to reset all the options to their default values before a subsequent call to :meth:`lsq_lincon_solve`.
A complete list of options, their abbreviations, synonyms and default values is given in :ref:`Other Parameters for lsq_lincon_solve <e04nc-py2-py-other_params>`.
"""
raise NotImplementedError
[docs]def lsq_lincon_option_string(optstr, comm, io_manager=None):
r"""
``lsq_lincon_option_string`` may be used to supply individual options to :meth:`lsq_lincon_solve`.
.. _e04ne-py2-py-doc:
For full information please refer to the NAG Library document for e04ne
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04nef.html
.. _e04ne-py2-py-parameters:
**Parameters**
**optstr** : str
A single valid option string (as described in :ref:`Notes <e04ne-py2-py-notes>` and in :ref:`Other Parameters for lsq_lincon_solve <e04nc-py2-py-other_params>`).
**comm** : dict, communication object, modified in place
Communication structure.
This argument must have been initialized by a prior call to :meth:`nlp1_init`.
**io_manager** : FileObjManager, optional
Manager for I/O in this routine.
.. _e04ne-py2-py-errors:
**Raises**
**NagValueError**
(`errno` :math:`5`)
The supplied option string is invalid. Supplied value was: :math:`\langle\mathit{\boldsymbol{value}}\rangle`.
.. _e04ne-py2-py-notes:
**Notes**
`No equivalent traditional C interface for this routine exists in the NAG Library.`
``lsq_lincon_option_string`` may be used to supply values for options to :meth:`lsq_lincon_solve`.
It is only necessary to call ``lsq_lincon_option_string`` for those arguments whose values are to be different from their default values.
One call to ``lsq_lincon_option_string`` sets one argument value.
Each option is defined by a single character string, of up to :math:`72` characters, consisting of one or more items.
The items associated with a given option must be separated by spaces, or equals signs :math:`\left[ = \right]`.
Alphabetic characters may be upper or lower case.
The string
::
Print Level = 1
is an example of a string used to set an option.
For each option the string contains one or more of the following items:
- a mandatory keyword;
- a phrase that qualifies the keyword;
- a number that specifies an `int` or `float` value. Such numbers may be up to :math:`40` contiguous characters in Fortran's I, F, E or D formats, terminated by a space if this is not the last item on the line.
Blank 