naginterfaces.library.opt.handle_solve_nldf¶
- naginterfaces.library.opt.handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, confun=None, congrd=None, monit=None, data=None, io_manager=None)[source]¶
handle_solve_nldf
is a solver from the NAG optimization modelling suite for general nonlinear data-fitting problems with constraints. Various loss and regularization functions are supported.Note: this function uses optional algorithmic parameters, see also:
handle_opt_set()
,handle_opt_get()
.For full information please refer to the NAG Library document for e04gn
https://support.nag.com/numeric/nl/nagdoc_30.2/flhtml/e04/e04gnf.html
- Parameters
- handleHandle
The handle to the problem. It needs to be initialized (e.g., by
handle_init()
) and to hold a problem formulation compatible withhandle_solve_nldf
. It must not be changed between calls to the NAG optimization modelling suite.- lsqfuncallable (rx, inform) = lsqfun(x, nres, inform, data=None)
must evaluate the value of the nonlinear residuals, , at a specified point .
- Parameters
- xfloat, ndarray, shape
, the vector of variable values at which the residuals, , are to be evaluated.
- nresint
, the current number of residuals in the model.
- informint
A non-negative value.
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- Returns
- rxfloat, array-like, shape
The value of the residual vector, , evaluated at .
- informint
May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with = 25.
- lsqgrdcallable inform = lsqgrd(x, nres, rdx, inform, data=None)
evaluates the residual gradients, , at a specified point .
- Parameters
- xfloat, ndarray, shape
, the vector of variable values at which the residual gradients, , are to be evaluated.
- nresint
, the current number of residuals in the model.
- rdxfloat, ndarray, shape , to be modified in place
On entry: the elements should only be assigned and not referenced.
On exit: the vector containing the nonzero residual gradients evaluated at ,
where
The elements must be stored in the same order as the defined sparsity pattern provided in the call to
handle_set_nlnls()
.- informint
A non-negative value.
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- Returns
- informint
May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with = 25.
- xfloat, array-like, shape
, the initial estimates of the variables, .
- nresint
, the current number of residuals in the model.
- confunNone or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional
Note: if this argument is None then a NAG-supplied facility will be used.
must calculate the values of the -element vector of nonlinear constraint functions at a specified value of the -element vector of variables.
If there are no nonlinear constraints then will never be called by
handle_solve_nldf
and it may be None.- Parameters
- xfloat, ndarray, shape
The vector of variable values at which the constraint functions are to be evaluated.
- ncnlnint
, the number of nonlinear constraints, as specified in an earlier call to
handle_set_nlnconstr()
.- informint
A non-negative value.
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- Returns
- gxfloat, array-like, shape
The values of the nonlinear constraint functions at .
- informint
Must be set to a value describing the action to be taken by the solver on return from . Specifically, if the value is negative, then the value of will be discarded and the solver will either attempt to find a different trial point or terminate immediately with = 25; otherwise, the solver will proceed normally.
- congrdNone or callable inform = congrd(x, gdx, inform, data=None), optional
Note: if this argument is None then a NAG-supplied facility will be used.
must calculate the nonzero values of the sparse Jacobian of the nonlinear constraint functions, , at a specified value of the -element vector of variables.
If there are no nonlinear constraints, will never be called by
handle_solve_nldf
and may be None.- Parameters
- xfloat, ndarray, shape
The vector of variable values at which the Jacobian of the constraint functions is to be evaluated.
- gdxfloat, ndarray, shape , to be modified in place
On entry: the elements should only be assigned and not referenced.
On exit: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by and in an earlier call to
handle_set_nlnconstr()
. will be the gradient , where and .- informint
A non-negative value.
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- Returns
- informint
Must be set to a value describing the action to be taken by the solver on return from . Specifically, if the value is negative the solution of the current problem will terminate immediately with = 25; otherwise, computations will continue.
- monitNone or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is None then a NAG-supplied facility will be used.
is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, may be None.
is called at the end of every th step where is controlled by the option ‘NLDF Monitor Frequency’ (if the value is , is not called).
- Parameters
- xfloat, ndarray, shape
The current best point.
- rinfofloat, ndarray, shape
Best objective value computed and various indicators (the values are as described in the main argument ).
- statsfloat, ndarray, shape
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument ).
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- dataarbitrary, optional
User-communication data for callback functions.
- io_managerFileObjManager, optional
Manager for I/O in this routine.
- Returns
- xfloat, ndarray, shape
The final values of the variables, .
- rxfloat, ndarray, shape
The values of the residuals at the final point given in .
- rinfofloat, ndarray, shape
Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are:
Objective function value, in [equation].
Loss function value, in [equation].
Regularization term value, in [equation].
Solution optimality measure.
Reserved for future use.
- statsfloat, ndarray, shape
Solver statistics at monitoring steps or at the end of the final iteration:
Number of iterations performed.
Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.
Reserved for future use.
- Other Parameters
- ‘Defaults’valueless
This special keyword may be used to reset all options to their default values. Any value given with this keyword will be ignored.
- ‘NLDF Iteration Limit’int
Default
The maximum number of iterations to be performed by
handle_solve_nldf
. If this limit is reached, then the solver will terminate with = 22.Constraint: .
- ‘NLDF Monitor Frequency’int
Default
If , the user-supplied function will be called at the end of every th step for monitoring purposes.
Constraint: .
- ‘NLDF Stop Tolerance’float
Default
This argument sets the value of which specifies the tolerance for the optimality measure.
When both loss function and regularization are differentiable, and with only simple bound constraints, the optimality measures are defined by [equation] and [equation] and ‘NLDF Stop Tolerance’ is passed to the solver
handle_solve_bounds_foas()
as ‘FOAS Stop Tolerance’.When any of the loss function or regularization is non-differentiable, or there presents linear, quadratic or general nonlinear constraint, the optimality measure is defined by (5) and ‘NLDF Stop Tolerance’ is passed to the solver
handle_solve_ipopt()
as ‘Stop Tolerance 1’.Constraint: .
- ‘NLDF Loss Function Type’str
Default ’L2’
This argument sets the loss function type used in the objective function.
Constraint: , , , , , , or .
- ‘Reg Term Type’str
Default ’OFF’
This argument sets the regularization function type used in the objective function.
Note: if there is no residual in the model, regularization will be turned off.
Constraint: , , or .
- ‘NLDF Huber Function Width’float
Default
Sets the parameter defined in the Huber loss function [equation].
Constraint: .
- ‘NLDF Cauchy Function Sharpness’float
Default
Sets the parameter defined in the Cauchy loss function [equation].
Constraint: .
- ‘NLDF SmoothL1 Function Width’float
Default
Sets the parameter defined in the SmoothL1 loss function [equation].
Constraint: .
- ‘NLDF Quantile Parameter’float
Default
Sets the parameter defined in the Quantile loss function [equation].
Constraint: .
- ‘Reg Coefficient’float
Default
Sets the regularization coefficient in the definition of the objective function [equation]. Note: if set to , regularization will be turned off.
Constraint: .
- ‘NLDF Elastic Net Alpha’float
Default
Sets the regularization coefficient in the definition of the elastic net regularization function [equation].
Note: if set to , the regularization function reduces to the -norm. In this case, only the regularization coefficient ‘Reg Coefficient’ is used, and the value of ‘NLDF Elastic Net Beta’ is ignored.
Constraint: .
- ‘NLDF Elastic Net Beta’float
Default
Sets the regularization coefficient in the definition of the elastic net regularization function [equation].
Note: if set to , the regularization function reduces to the -norm. In this case, only the regularization coefficient ‘Reg Coefficient’ is used, and the value of ‘NLDF Elastic Net Alpha’ is ignored.
Constraint: .
- ‘Infinite Bound Size’float
Default
This defines the ‘infinite’ bound in the definition of the problem constraints. Any upper bound greater than or equal to will be regarded as (and similarly any lower bound less than or equal to will be regarded as ). Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: .
- ‘Monitoring File’int
Default
If , the unit number for the secondary (monitoring) output. If , no secondary output is provided. The information output to this unit is controlled by ‘Monitoring Level’.
Constraint: .
- ‘Monitoring Level’int
Default
This argument sets the amount of information detail that will be printed by the solver to the secondary output. The meaning of the levels is the same as for ‘Print Level’.
Constraint: .
- ‘Print File’int
Default
If , the unit number for the primary output of the solver. If , the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the initialization of the options, e.g., at the initialization of the handle. The information output to this unit is controlled by ‘Print Level’.
Constraint: .
- ‘Print Level’int
Default
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
Output
No output from the solver.
The Header and Summary.
, , ,
Additionally, the Iteration log.
Constraint: .
- ‘Print Options’str
Default
If , a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: or .
- ‘Print Solution’str
Default
If or , the final values of the primal variables are printed on the primary and secondary outputs.
Constraint: , or .
- ‘Stats Time’str
Default
This argument turns on timing. This might be helpful for a choice of different solving approaches. It is possible to choose between CPU and wall clock time. Choice ‘YES’ is equivalent to ‘WALL CLOCK’.
Constraint: , , or .
- ‘Time Limit’float
Default
A limit to the number of seconds that the solver can use to solve one problem. If at the end of an iteration this limit is exceeded, the solver will terminate with = 23.
Constraint: .
- Raises
- NagValueError
- (errno )
handle_solve_nldf
is not available in this implementation.- (errno )
has not been initialized.
- (errno )
does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
- (errno )
has not been initialized properly or is corrupted.
- (errno )
This solver does not support the model defined in the handle.
- (errno )
The problem is already being solved.
- (errno )
On entry, , expected .
Constraint: must match the current number of variables of the model in the .
- (errno )
On entry, , expected .
Constraint: must match the current number of residuals defined in the .
- (errno )
Please provide a proper function.
- (errno )
Please provide a proper function.
- (errno )
The current starting point is unusable.
- (errno )
The solver terminated with not enough degrees of freedom.
- (errno )
The solver detected an infeasible problem.
- Warns
- NagAlgorithmicWarning
- (errno )
Problem was solved to an acceptable level; full accuracy was not achieved.
- NagAlgorithmicMajorWarning
- (errno )
Maximum number of iterations reached.
- (errno )
The solver terminated after the maximum time allowed was exhausted.
- (errno )
The solver was terminated because no further progress could be achieved.
- (errno )
Invalid number detected in user function.
- (errno )
The solver terminated after failure during line search.
- (errno )
The solver terminated after an error in the step computation.
- NagCallbackTerminateWarning
- (errno )
User requested termination during a monitoring step.
- Notes
handle_solve_nldf
solves a data-fitting problem of the formwhere
is the number of decision variables,
is the number of the nonlinear constraints and , and are -dimensional vectors,
is the number of quadratic constraints,
is the number of the linear constraints and is a matrix, and are -dimensional vectors,
there are box constraints and and are -dimensional vectors.
Here,
is an -dimensional vector representing the model parameters,
is the loss function,
is the regularization function,
is the regularization coefficient,
is the number of residuals and is the th residual, which is defined as
where is the predicted value of the th data point, given . For the th data point, and are the observed values of the independent and dependant variables respectively.
The available loss and regularization function types are summarized in Table [label omitted], where is the function parameter and denotes an indicator function taking the value if the logical expression is true and otherwise. Loss function and regularization types can be specified by options ‘NLDF Loss Function Type’ and ‘Reg Term Type’, respectively. For example, set and to use -norm loss function with -norm (Ridge) regularization. See Algorithmic Details for more details on the loss functions.
Loss function
‘NLDF Loss Function Type’
-norm
‘L2’
-norm
‘L1’
-norm
‘LINF’
Huber (see [equation])
‘HUBER’
Cauchy (see [equation])
‘CAUCHY’
Atan
‘ATAN’
SmoothL1 (see [equation])
‘SMOOTHL1’
Quantile (see [equation])
‘QUANTILE’
Regularization
‘Reg Term Type’
Lasso (-norm)
‘L1’
Ridge (-norm)
‘L2’
Elastic net
‘ELASTIC NET’
handle_solve_nldf
serves as a solver for problems stored as a . The points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. After the has been initialized (e.g.,handle_init()
has been called),handle_set_nlnls()
can be used to add a model and define its residual sparsity structure.handle_set_qconstr()
andhandle_set_qconstr_fac()
may be used to set or modify quadratic constraints. Linear constraints , , are handled byhandle_set_linconstr()
. Variable box bounds and can be specified withhandle_set_simplebounds()
, andhandle_set_nlnconstr()
can set or modify nonlinear constraints. Once the problem is fully described, the may be passed to the solverhandle_solve_nldf
. When the is no longer needed,handle_free()
should be called to destroy it and deallocate the memory held within. See the E04 Introduction for more details about the NAG optimization modelling suite.Nonlinear Programming (NLP) solvers
handle_solve_bounds_foas()
andhandle_solve_ipopt()
are used as solver engines byhandle_solve_nldf
, which defines the selected loss function and regularization, then transforms the problem into standard form that the NLP solvers allow. For best performance, when the objective function is differentiable and without any constraint other than simple bound constraints,handle_solve_bounds_foas()
is used. For non-differentiable objective functions or cases where constraints other than simple variable bounds are present,handle_solve_ipopt()
is used. See Algorithmic Details andhandle_solve_ipopt()
for more details on algorithmic details.The algorithm behaviour can be modified by various options (see Other Parameters) which can be set by
handle_opt_set()
andhandle_opt_set_file()
anytime between the initialization of the by e.g.,handle_init()
and a call to the solver. Once the solver has finished, options may be modified for the next solve. The solver may be called repeatedly with various starting points and/or options. Option getterhandle_opt_get()
can be called to retrieve the current value of any option.