naginterfaces.library.opt.handle_solve_dfls¶
- naginterfaces.library.opt.handle_solve_dfls(handle, objfun, x, nres, monit=None, data=None, io_manager=None)[source]¶
handle_solve_dfls
is a forward communication Derivative-free Optimization (DFO) solver from the NAG optimization modelling suite (DFLS) for small to medium-scale nonlinear least squares problems with bound constraints.Note: this function uses optional algorithmic parameters, see also:
handle_opt_set()
,handle_opt_get()
.For full information please refer to the NAG Library document for e04ff
https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/e04/e04fff.html
- Parameters
- handleHandle
The handle to the problem. It needs to be initialized (e.g., by
handle_init()
) and to hold a problem formulation compatible withhandle_solve_dfls
. It must not be changed between calls to the NAG optimization modelling suite.- objfuncallable (rx, inform) = objfun(x, nres, inform, data=None)
must evaluate the value of the nonlinear residuals at a specified point .
- Parameters
- xfloat, ndarray, shape
, the vector of variable values at which the residuals, , are to be evaluated.
- nresint
, the number of residuals in the problem, as set during the initialization of the handle by
handle_set_nlnls()
.- informint
A non-negative value.
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- Returns
- rxfloat, array-like, shape
The value of the residuals at .
- informint
May be used to indicate that the requested objective value could not be computed. Specifically, it can be set to a negative value:
The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with = 17.
The solver will cleanly exit with = 20 and return the best available point as well as the solve statistics.
- xfloat, array-like, shape
, the initial estimates of the variables, .
- nresint
, the number of residuals in the problem. It must be unchanged from the value set during the definition of the objective structure by
handle_set_nlnls()
.- monitNone or callable monit(x, rinfo, stats, data=None), optional
Note: if this argument is None then a NAG-supplied facility will be used.
is provided to enable you to monitor the progress of the optimization and, if necessary, to halt the optimization process.
If no monitoring is required, may be None.
is called at the end of every th step where is controlled by the option ‘DFO Monitor Frequency’ (default value , is never called).
- Parameters
- xfloat, ndarray, shape
The current best point.
- rinfofloat, ndarray, shape
Best objective value computed and various indicators (the values are as described in the main argument ).
- statsfloat, ndarray, shape
Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument ).
- dataarbitrary, optional, modifiable in place
User-communication data for callback functions.
- dataarbitrary, optional
User-communication data for callback functions.
- io_managerFileObjManager, optional
Manager for I/O in this routine.
- Returns
- xfloat, ndarray, shape
The final values of the variables, .
- rxfloat, ndarray, shape
The values of the residuals at the final point given in .
- rinfofloat, ndarray, shape
Optimal objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are given in the table below:
Objective function value (sum of the squared residuals).
, the current lower bound of the trust region.
, the current size of the trust region.
The number of interpolation points used by the solver.
Reserved for future use.
- statsfloat, ndarray, shape
Solver statistics at monitoring steps or at the end of the final iteration as given in the table below:
Number of calls to the objective function.
Total time spent in the solver (including time spent evaluating the objective).
Total time spent evaluating the objective function.
Number of steps.
–
Reserved for future use.
- Other Parameters
- ‘Defaults’valueless
This special keyword may be used to reset all options to their default values. Any value given with this keyword will be ignored.
- ‘DFLS Small Residuals Tol’float
Default
This option defines the tolerance on the value of the residuals. Namely, the solver declares convergence if
Constraint: .
- ‘DFO Initial Interp Points’str
Default
Determines how the initial interpolation points are chosen. If , the interpolation points are chosen along the coordinate directions around the initial point. If , the initial interpolation points are chosen along random orthogonal directions around the initial point. Set ‘DFO Random Seed’ to a positive value to fix the random seed and get reproducible results.
Constraint: or .
- ‘DFO Maximum Slow Steps’int
Default
If , this argument defines the maximum number of consecutive slow iterations allowed. Set to deactivate the slow iteration detection. The algorithm can stop in two situations:
and with = 50,
with = 24.
Constraint: .
- ‘DFO Max Objective Calls’int
Default
A limit on the number of objective function evaluations the solver is allowed to compute. If the limit is reached, the solver stops with = 21.
Constraint: .
- ‘DFO Max Soft Restarts’int
Default
The maximum total number of soft restarts that can be performed if the objective function is declared as noisy ().
Constraint: .
- ‘DFO Max Unsucc Soft Restarts’int
Default
The maximum number of consecutive unsuccessful soft restarts that can be performed if the objective function is declared as noisy ().
Constraint: .
- ‘DFO Monitor Frequency’int
Default
If , will be called at the end of every th step for monitoring purposes.
Constraint: .
- ‘DFO Noise Level’float
Default
Indicates the noise level expected when evaluating the objective function if .
Constraint: .
- ‘DFO Noisy Problem’str
Default
Indicates if the function evaluations provided to the solver are noisy. If , some algorithmic features will be activated:
The trust region update becomes slower to reflect the decreased confidence in the objective values.
Soft restarts of the algorithm can be performed to ensure the algorithm did not get stuck because of the noise (see ‘DFO Max Soft Restarts’, ‘DFO Max Unsucc Soft Restarts’, ‘DFO Number Soft Restarts Pts’ to control the restart characteristics).
In addition, if , the solver will trigger a soft restart if all the function values are within the noise level.
- ‘DFO Number Initial Points’int
Default
The initial number of interpolation points in (9) used to build the linear models of the residuals. If , the number of points is chosen to be equal to the total number of interpolation points set by ‘DFO Number Interp Points’.
If this parameter is chosen to be lower than the maximum set by ‘DFO Number Interp Points’, the solver will progressively increase the number of interpolation points until it reaches that value. In this release, it is only possible to grow the interpolation set if ‘DFO Number Interp Points’ is set to the default value.
Constraint: .
Consistency constraint, the solver stops with = 6 if not met:
.
If , ‘DFO Number Interp Points’ must be set to the default value.
- ‘DFO Number Interp Points’int
Default
The maximum number of interpolation points in (9) used to build the linear models of the residuals. If , the number of points is chosen to be where is the number of non-fixed variables.
Constraint: .
Consistency constraint, the solver stops with = 6 if not met:
.
- ‘DFO Number Soft Restarts Pts’int
Default
The number of interpolation points that are replaced during a soft restart.
Constraint: .
- ‘DFO Print Frequency’int
Default
If , the solver prints the iteration log to the appropriate units at the end of every th step.
Constraint: .
- ‘DFO Random Seed’int
Default
The random seed used to generate the random points used to build the initial model or build the underdetermined models when the interpolation set has not fully grown (). If , the random seed will be based on values taken from the real-time clock, potentially resulting in the solver taking a different path each time it is run. Set it to a positive value to get fully reproducible runs.
Constraint: .
- ‘DFO Starting Trust Region’float
Default
, the initial trust region radius. This argument should be set to about one tenth of the greatest expected overall change to a variable: the initial quadratic model will be constructed by taking steps from the initial of length along each coordinate direction. The default value assumes that the variables have an order of magnitude .
Constraint: .
Consistency constraints, the solver stops with = 5 if not met:
.
.
- ‘DFO Trust Region Slow Tol’float
Default
The minimal acceptable trust region radius for the solution to be declared as acceptable. The solver stops if:
and .
Constraint: .
Consistency constraint, the solver stops with = 5 if not met:
.
- ‘DFO Trust Region Tolerance’float
Default
, the requested trust region radius. The algorithm declares convergence when the trust region radius reaches this limit. It should indicate the absolute accuracy that is required in the final values of the variables.
Constraint: .
Consistency constraints, the solver stops with = 5 if not met:
.
.
- ‘DFO Version’str
Default
At Mark 27, the underlying algorithm of
handle_solve_dfls
underwent significant changes. This option allows you to continue using the Mark 26 version if it is set to ‘’26’’. By default (recommended), the latest version of the code is called.Constraint: or .
- ‘Infinite Bound Size’float
Default
This defines the ‘infinite’ bound in the definition of the problem constraints. Any upper bound greater than or equal to will be regarded as (and similarly any lower bound less than or equal to will be regarded as ). Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.
Constraint: .
- ‘Monitoring File’int
Default
If , the unit number for the secondary (monitoring) output. If , no secondary output is provided. The information output to this unit is controlled by ‘Monitoring Level’.
Constraint: .
- ‘Monitoring Level’int
Default
This argument sets the amount of information detail that will be printed by the solver to the secondary output. The meaning of the levels is the same as with ‘Print Level’.
Constraint: .
- ‘Print File’int
Default
If , the unit number for the primary output of the solver. If , the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the options initialization, e.g., at the initialization of the handle. The information output to this unit is controlled by ‘Print Level’.
Constraint: .
- ‘Print Level’int
Default
This argument defines how detailed information should be printed by the solver to the primary and secondary output.
Output
No output from the solver.
The Header and Summary.
, , ,
Additionally, the Iteration log.
Constraint: .
- ‘Print Options’str
Default
If , a listing of options will be printed to the primary output and is always printed to the secondary output.
Constraint: or .
- ‘Print Solution’str
Default
If , the solution will be printed to the primary and secondary output.
Constraint: or .
- ‘Stats Time’str
Default
This argument turns on timings of various parts of the algorithm to give a better overview of where most of the time is spent. This might be helpful for a choice of different solving approaches. It is possible to choose between CPU and wall clock time. Choice ‘YES’ is equivalent to ‘WALL CLOCK’.
Constraint: , , or .
- ‘Time Limit’float
Default
A limit to the number of seconds that the solver can use to solve one problem. If during the convergence check this limit is exceeded, the solver will terminate with = 23.
Constraint: .
- Raises
- NagValueError
- (errno )
has not been initialized.
- (errno )
does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.
- (errno )
has not been initialized properly or is corrupted.
- (errno )
This solver does not support the model defined in the handle.
- (errno )
The problem is already being solved.
- (errno )
On entry, , expected .
Constraint: must match the current number of variables of the model in the .
- (errno )
The information supplied does not match with that previously stored.
On entry, must match that given during the definition of the objective in the , i.e., .
- (errno )
Inconsistent options ‘DFO Trust Region Tolerance’ and ‘DFO Starting Trust Region’ .
Constraint: .
Use
handle_opt_set()
to set compatible option values.- (errno )
Inconsistent options ‘DFO Trust Region Tolerance’ and ‘DFO Trust Region Slow Tol’ .
Constraint: .
Use
handle_opt_set()
to set compatible option values.- (errno )
Option ‘DFO Starting Trust Region’ , , and .
Constraint: if in coordinate , then .
Use
handle_opt_set()
to set compatible option values.- (errno )
There were unequal bounds and the option ‘DFO Number Interp Points’ .
Constraint: .
Use
handle_opt_set()
to set compatible option values.- (errno )
The number of initial interpolation points is greater than the maximum.
Use ‘DFO Number Interp Points’ and ‘DFO Number Initial Points’ to control the number of interpolation points.
- (errno )
Initial number of interpolation points , total number of interpolation points , number of variables .
Constraint: growing interpolation set is only supported for linear models ().
Use ‘DFO Number Interp Points’ and ‘DFO Number Initial Points’ to control the number of interpolation points.
- (errno )
Maximization is not possible for a nonlinear least squares problem.
- Warns
- NagAlgorithmicWarning
- (errno )
The problem was solved to an acceptable level after consecutive slow iterations.
Use the option ‘DFO Maximum Slow Steps’ to modify the maximum number of slow steps accepted.
- NagAlgorithmicMajorWarning
- (errno )
Rescue failed: the trust region could not be reduced further after some function evaluation could not be provided. Check the specification of your objective and whether it needs rescaling. Try a different initial .
- (errno )
Some initial interpolation points were not provided. Rescue cannot be attempted at this stage.
Check the specification of your objective and whether it needs rescaling. Try a different initial .
- (errno )
The predicted reduction in a trust region step was non-positive. Check your specification of and whether the function needs rescaling. Try a different initial .
- (errno )
A rescue procedure has been called in order to correct damage from rounding errors when computing an update to a quadratic approximation of , but no further progress could be made. Check your specification of and whether the function needs rescaling. Try a different initial .
- (errno )
User requested termination after a call to the objective function.
- (errno )
Maximum number of function evaluations exceeded.
- (errno )
The solver terminated after the maximum time allowed was exceeded.
- (errno )
No progress, the solver was stopped after consecutive slow steps.
Use the option ‘DFO Maximum Slow Steps’ to modify the maximum number of slow steps accepted.
- NagCallbackTerminateWarning
- (errno )
User requested termination during a monitoring step.
- Notes
handle_solve_dfls
is aimed at minimizing a sum of squares objective function subject to bound constraints:Here the are smooth nonlinear functions called residuals and and are -dimensional vectors defining bounds on the variables. Typically, in a calibration or data fitting context, the residuals will be defined as the difference between the data points and a nonlinear model (see the E04 Introduction).
handle_solve_dfls
serves as a solver for compatible problems stored as a handle. The handle points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. To define a compatible problem handle, you must callhandle_init()
followed byhandle_set_nlnls()
to initialize it and optionally callhandle_set_simplebounds()
to define bounds on the variables. Ifhandle_set_simplebounds()
is not called, all the variables will be considered free by the solver. It should be noted thathandle_solve_dfls
always assumes that the Jacobian of the residuals is dense, therefore, defining a sparse structure for the residuals in the call tohandle_set_nlnls()
will have no effect. See the E04 Introduction for more details about the NAG optimization modelling suite.The solver allows fixing variables with the definition of the bounds. However, the following constraint must be met in order to be able to call the solver:
for all non-fixed variable , the value of must be at least twice the starting trust region radius (see the consistency constraint of the option ‘DFO Starting Trust Region’).
The solver is based on a derivative-free trust region framework. This type of method is well suited for small to medium-scale problems (around 100 variables) for which the derivatives are unavailable or not easy to compute, and/or for which the function evaluations are expensive or noisy. For a detailed description of the algorithm see Algorithmic Details.
The algorithm behaviour and solver strategy can be modified by various options (see Other Parameters) which can be set by
handle_opt_set()
andhandle_opt_set_file()
at any time between the initialization of the handle byhandle_init()
and a call to the solver. The options’ names specific for this solver start either with the prefix DFO (Derivative-free Optimization) or DFLS (Derivative-free Least Squares). The default values for these options are chosen to work well in the general case, but it is recommended you tune them to your particular problem. In particular, if the objective function is known to be noisy, it is highly recommended to set the option ‘DFO Noisy Problem’ to ‘YES’. Once the solver has finished, options may be modified for the next solve. The solver may be called repeatedly with various starting points and/or options.The underlying algorithm implemented for
handle_solve_dfls
is the same as the one used byhandle_solve_dfls_rcomm()
.handle_solve_dfls
serves as a forward communication interface to the derivative-free solver for nonlinear least squares problems.
- References
Cartis, C, Fiala, J, Marteau, B and Roberts, L, 2018, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, Technical Report, University of Oxford
Cartis, C and Roberts, L, 2017, A Derivative-Free Gauss-Newton Method
Conn, A R, Scheinberg, K and Vicente, L N, 2009, Introduction to Derivative-Free Optimization, vol. 8 of MPS-SIAM Series on Optimization, MPS/SIAM, Philadelphia
Powell, M J D, 2009, The BOBYQA algorithm for bound constrained optimization without derivatives, Report DAMTP 2009/NA06, University of Cambridge, https://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf
Zhang, H, Conn, A R and Scheinberg, K, 2010, A Derivative-Free Algorithm for Least-Squares Minimization, SIAM J. Optim. (20(6)), 3555–3576