e04saf initializes a data structure for the NAG optimization modelling suite from a data file for problems such as, Linear Programming (LP), Quadratic Programming (QP), Second-order Cone Programming (SOCP), or linear Semidefinite Programming (SDP).
The routine may be called by the names e04saf or nagf_opt_handle_read_file.
3Description
e04saf reads a file in one of the supported file formats and stores the data in a new handle. The supported formats are MPS and its derivatives (for LP, QP, SOCP) and sparse SDPA for linear semidefinite programming problems. See Section 9 for more details on the supported file formats. It might be useful to refer to the file format you use to fully understand the possible error messages. Once the file is successfuly processed, the problem can be solved by a compatible solver from the suite or queried via the printing routine e04ryf. Once the problem is not needed any more, e04rzf should be called to destroy the handle and deallocate the memory held within. See Section 3.1 in the E04 Chapter Introduction for more details about the NAG optimization modelling suite. Also see Section 2.2 in the E04 Chapter Introduction for more details on the standard formulations of the optimization problems (e.g., LP, QP, SOCP and SDP) for better understanding on the notations used in this document.
4References
Borchers B (1999) SDPLIB 1.2, A Library of semidefinite programming test problems Optimization Methods and Software11(1) 683–690 http://euler.nmt.edu/~brian/sdplib/
Fujisawa K, Kojima M and Nakata K (1998) SDPA (Semidefinite Programming Algorithm) User's Manual Technical Report B-308 Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.
IBM (1971) MPSX – Mathematical programming system Program Number 5734 XM4 IBM Trade Corporation, New York
On exit: holds a handle to the internal data structure. It must not be changed between calls to the NAG optimization modelling suite.
2: – Character(*)Input
On entry: the name of the file to be opened.
Constraint:
must contain a valid filename for the computer system being used.
3: – Character(*)Input
On entry: the expected file format of the input file. ftype is case insensitive.
or
The input file is in MPS format.
or
The input file is in SDPA format.
Constraint:
, , or .
4: – Integer arrayOutput
On exit: problem sizes and statistics as given in the table below:
Number of variables (nvar).
Number of integer variables.
Number of binary variables.
Number of Lagrange multipliers (dual variables) for the bound constraints and linear constraints (nnzu).
Number of Lagrange multipliers (dual variables) for the cone constraints (nnzuc).
Number of nonzeros in all Lagrange multipliers for the matrix constraints (nnzua).
Flag indicating the objective function type, : no objective, : linear objective, : quadratic without linear term, : quadratic with linear term, : general nonlinear, : nonlinear least square with dense Jacobian, : nonlinear least square with sparse Jacobian.
Number of residuals defined in a nonlinear least square problem.
Definition of box constraints, : not defined, : defined.
Number of linear constraints.
Number of nonlinear constraints.
Definition of Hessian for nonlinear objective and constraints, : not defined, : defined as Lagrangian, : defined as individual Hessians of the objective and the constraints.
Number of cone constraints.
Number of matrix inequality constraints.
Number of quadratic constraints.
otherwise
Reserved for future use.
–
Reserved for future use.
nvar, nnzu, nnzuc and nnzua define the problem size and are required by other routines such as e04mtf, e04ptf and e04svf in the optimization suite.
5: – IntegerInput/Output
On entry: ifail must be set to , or to set behaviour on detection of an error; these values have no effect when no error is detected.
A value of causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of means that an error message is printed while a value of means that it is not.
If halting is not appropriate, the value or is recommended. If message printing is undesirable, then the value is recommended. Otherwise, the value is recommended. When the value or is used it is essential to test the value of ifail on exit.
On exit: unless the routine detects an error or a warning has been flagged (see Section 6).
6Error Indicators and Warnings
If on entry or , explanatory error messages are output on the current error message unit (as defined by x04aaf).
Errors or warnings detected by the routine:
On entry, . Constraint: , , or .
Cannot open file for reading.
Cannot close file.
The input file seems to be empty. No data was read.
Reading from the file caused an unknown error on line .
The token on line at position to was not recognized as a valid integer.
The token on line at position to was not recognized as a valid real number.
The token on line starting at position was too long and was not recognized.
An invalid number of variables was given on line .
The number stated there is and needs to be at least .
An invalid number of blocks was given on line .
The number stated there is and needs to be at least .
An invalid size of the block number was given on line .
The number stated there is and needs to be nonzero.
Not enough data was given on line specifying block sizes.
Expected
tokens but found only .
Not enough data was given on line specifying the objective function.
Expected tokens but found only .
Not enough data was given on line specifying nonzero matrix elements.
Expected tokens but found only .
Invalid structural data found on line .
The given matrix number is out of bounds. Its value must be between
and
(inclusive).
Invalid structural data found on line .
The given block number is out of bounds. Its value must be between
and
(inclusive).
Invalid structural data found on line .
The given row index is out of bounds, it must respect the size of the block. Its value must be between and (inclusive).
Invalid structural data found on line .
The given column index is out of bounds, it must respect the size of the block. Its value must be between and (inclusive).
Invalid structural data found on line .
The specified nonzero element is not in the upper triangle.
The row index is and column index is .
Invalid structural data found on line .
The specified element belongs to a diagonal block but is not diagonal.
The row index is and column index is .
An entry in the constraints with , , row index and column index was defined more than once. All entries need to be unique.
A premature end of the input stream. The part defining the dimensions of the blocks was not found.
A premature end of the input stream. The part defining the nonzero entries was not found.
A premature end of the input stream. The part defining the number of blocks was not found.
A premature end of the input stream. The part defining the number of variables was not found.
A premature end of the input stream. The part defining the objective function was not found.
Incorrect ordering of indicator lines.
OBJNAME indicator line found after ROWS indicator line.
Incorrect ordering of indicator lines.
COLUMNS indicator line found before ROWS indicator line.
Incorrect ordering of indicator lines.
RHS indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines.
RANGES indicator line found before RHS indicator line.
Incorrect ordering of indicator lines.
BOUNDS indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines.
QUADOBJ indicator line found before BOUNDS indicator line.
Incorrect ordering of indicator lines.
QUADOBJ indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines. CSECTION indicator line found before COLUMNS indicator line.
Unknown indicator line ‘’.
Indicator line ‘’ has been found more than once in the MPS file.
End of file found before ENDATA indicator line.
At least one mandatory section not found in MPS file.
An illegal line was detected in ‘’ section.
This is neither a comment nor a valid data line.
Unknown inequality key ‘’ in ROWS section.
Expected ‘N’, ‘G’, ‘L’ or ‘E’.
Empty ROWS section.
Neither the objective row nor the constraints were defined.
The supplied name in OBJNAME of the objective row was not found among the free rows in the ROWS section.
Illegal row name.
Row names must consist of printable characters only.
Illegal column name.
Column names must consist of printable characters only.
Row name ‘’ has been defined more than once in the ROWS section.
Column ‘’ has been defined more than once in the COLUMNS section. Column definitions must be continuous. (See Section 9.1.5).
Found 'INTORG' marker within 'INTORG' to 'INTEND' range.
Found 'INTEND' marker without previous marker being 'INTORG'.
Found 'INTORG' but not 'INTEND' before the end of the COLUMNS section.
Illegal marker type ‘’.
Should be either 'INTORG' or 'INTEND'.
Unknown row name ‘’ in section.
All row names must be specified in the ROWS section.
Unknown column name ‘’ in section.
All column names must be specified in the COLUMNS section.
Inconsistent bounds for column ‘’.
Inconsistent bounds for row ‘’.
Unknown bound type ‘’ in BOUNDS section.
Inconsistent bounds are reported when the lower bound is greater than or equal to or the upper bound is less than or equal to , or when the lower bound is greater than the upper bound. Any upper bound greater than or equal to will be regarded as (and similarly any lower bound less than or equal to will be regarded as ).
More than one nonzero of has row name ‘’ and column name ‘’ in the COLUMNS section.
Both quadratic objective and cone constraints found, not supported.
Rotated second-order cone ‘’ should have at least variables.
Second-order cone ‘’ should have at least variables.
Unknown cone type ‘’.
Illegal cone name. Cone names must consist of printable characters only.
Cone name ‘’ has been defined more than once.
Column name ‘’ has been defined more than once in cone constraint.
An unexpected error has been triggered by this routine. Please
contact NAG.
See Section 7 in the Introduction to the NAG Library FL Interface for further information.
Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library FL Interface for further information.
Dynamic memory allocation failed.
See Section 9 in the Introduction to the NAG Library FL Interface for further information.
7Accuracy
Not applicable.
8Parallelism and Performance
Background information to multithreading can be found in the Multithreading documentation.
e04saf is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
e04saf makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the Users' Note for your implementation for any additional implementation-specific information.
9Further Comments
9.1MPS input format
The input file of data may only contain two types of lines:
1.Indicator lines (specifying the type of data which is to follow).
2.Data lines (specifying the actual data).
A section is a combination of an indicator line and its corresponding data line(s). Any characters beyond column 80 are ignored. Indicator lines must not contain leading blank characters (in other words they must begin in column 1). The following displays the order in which the indicator lines must appear in the file:
NAME
user-supplied name
(optional)
OBJSENSE
(optional)
data line
OBJNAME
(optional)
data line
ROWS
data line(s)
COLUMNS
data line(s)
RHS
data line(s)
RANGES
(optional)
data line(s)
BOUNDS
(optional)
data line(s)
QUADOBJ
(optional)
data line(s)
CSECTION
user-supplied name
parameter
cone type
(optional)
data line(s)
ENDATA
A data line follows a fixed format, being made up of fields as defined below. The contents of the fields may have different significance depending upon the section of data in which they appear.
Field 1
Field 2
Field 3
Field 4
Field 5
Field 6
Columns
Contents
Code
Name
Name
Value
Name
Value
Each name and code must consist of ‘printable’ characters only; names and codes supplied must match the case used in the following descriptions. Values are read using a field width of . This allows values to be entered in several equivalent forms. For example, , , and all represent the same number. It is safest to include an explicit decimal point.
Lines with an asterisk () in column will be considered comment lines and will be ignored by the routine.
Columns outside the six fields must be blank, except for columns 72–80, whose contents are ignored by the routine. A non-blank character outside the predefined six fields and columns 72–80 is considered to be a major error (; see Section 6), unless it is part of a comment.
9.1.1NAME Section (optional)
The NAME section is the only section where the data must be on the same line as the indicator. The ‘user-supplied name’ must be in Field but may be blank.
Field
Required
Description
No
Name of the problem
9.1.2OBJSENSE Section (optional)
The data line in this section can be used to specify the sense of the objective function. If this section is present it must contain only one data line. If the section is missing or empty, minimization is assumed.
Field
Required
Description
No
Sense of the objective function
Field may contain either MIN, MAX, MINIMIZE or MAXIMIZE.
9.1.3OBJNAME Section (optional)
The data line in this section can be used to specify the name of a free row (see Section 9.1.4) that should be used as the objective function. If this section is present it must contain only one data line. If the section is missing or is empty, the first free row will be chosen instead.
Field
Required
Description
No
Row name to be used as the objective function
Field must contain a valid row name.
9.1.4ROWS Section
The data lines in this section specify unique row (constraint) names and their inequality types (i.e., unconstrained, , or ).
Field
Required
Description
Yes
Inequality key
Yes
Row name
The inequality key specifies each row's type. It must be E, G, L or N and can be in either column or .
Inequality Key
Description
N
Free row
G
Greater than or equal to
finite
L
Less than or equal to
finite
E
Equal to
finite
Row type N stands for ‘Not binding’. It can be used to define the objective row. The objective row is a free row that specifies the vector in the linear objective term . If there is more than one free row, the first free row is chosen, unless another free row name is specified by OBJNAME (see Section 9.1.3). Note that is assumed to be zero if either the chosen row does not appear in the COLUMNS section (i.e., has no nonzero elements) or there are no free rows defined in the ROWS section.
9.1.5COLUMNS Section
Data lines in this section specify the names to be assigned to the variables (columns) in the general linear constraint matrix , and define, in terms of column vectors, the actual values of the corresponding matrix elements.
Field
Required
Description
Yes
Column name
Yes
Row name
Yes
Value
No
Row name
No
Value
Each data line in the COLUMNS section defines the nonzero elements of or . Any elements of or that are undefined are assumed to be zero. Nonzero elements of must be grouped by column, that is to say that all of the nonzero elements in the jth column of must be specified before those in the th column, for . Rows may appear in any order within the column.
9.1.5.1Integer Markers
For backward compatibility e04saf allows you to define the integer variables within the COLUMNS section using integer markers, although this is not recommended as markers can be treated differently by different MPS readers; you should instead define any integer variables in the BOUNDS section (see below). Each marker line must have the following format:
Field
Required
Description
No
Marker ID
Yes
Marker tag
Yes
Marker type
The marker tag must be 'MARKER'. The marker type must be 'INTORG' to start reading integer variables and 'INTEND' to finish reading integer variables. This implies that a row cannot be named 'MARKER', 'INTORG' or 'INTEND'. Please note that both marker tag and marker type comprise of characters as a ' is the mandatory first and last character in the string. You may wish to have several integer marker sections within the COLUMNS section, in which case each marker section must begin with an 'INTORG' marker and end with an 'INTEND' marker and there should not be another marker between them.
Field 2 is ignored by e04saf. When an integer variable is declared it will keep its default bounds unless they are changed in the BOUNDS section. This may vary between different MPS readers.
9.1.6RHS Section
This section specifies the right-hand side values (if any) of the general linear constraint matrix .
Field
Required
Description
Yes
RHS name
Yes
Row name
Yes
Value
No
Row name
No
Value
The MPS file may contain several RHS sets distinguished by RHS name. However only the first RHS set will be used.
Only the nonzero RHS elements need to be specified. Note that if an RHS is given to the objective function it will be ignored by e04saf. An RHS given to the objective function is dealt with differently by different MPS readers, therefore, it is safer not to define an RHS of the objective function in your MPS file. Note that this section may be empty, in which case the RHS vector is assumed to be zero.
9.1.7RANGES Section (optional)
Ranges are used to modify the interpretation of constraints defined in the ROWS section (see Section 9.1.4) to the form , where both and are finite. The range of the constraint is .
Field
Required
Description
Yes
Range name
Yes
Row name
Yes
Value
No
Row name
No
Value
The range of each constraint implies an upper and lower bound dependent on the inequality key of each constraint, on the RHS of the constraint (as defined in the RHS section), and on the range .
Inequality Key
Sign of
E
E
G
L
N
Only the first range set will be used.
9.1.8BOUNDS Section (optional)
These lines specify limits on the values of the variables (the quantities and in ). If a variable is not specified in the bound set then it is automatically assumed to lie between and .
Field
Required
Description
Yes
Bound type identifier
Yes
Bound name
Yes
Column name
Yes/No
Value
Note: Field is required only if the bound type identifier is one of UP, LO, FX, UI or LI in which case it gives the value below. If the bound type identifier is FR, MI, PL or BV, Field is ignored and it is recommended to leave it blank.
The table below describes the acceptable bound type identifiers and how each determines the variables' bounds.
Bound Type Identifier
Integer Variable?
UP
unchanged
No
LO
unchanged
No
FX
No
FR
No
MI
unchanged
No
PL
unchanged
No
BV
Yes
UI
unchanged
Yes
LI
unchanged
Yes
Only the first bound set will be used.
9.1.9QUADOBJ Section (optional)
The QUADOBJ section defines nonzero elements of the upper or lower triangle of the Hessian matrix .
Field
Required
Description
Yes
Column name (HColumn Index)
Yes
Column name (HRow Index)
Yes
Value
No
Column name (HRow Index)
No
Value
Each data line in the QUADOBJ section defines one (or optionally two) nonzero elements of the matrix . Each element is given as a triplet of row index , column index and a value. The column names (as defined in the COLUMNS section) are used to link the names of the variables and the indices and .
It is only necessary to define either the upper or lower triangle of the matrix; either will suffice. Any elements that have been defined in the upper triangle of the matrix will be moved to the lower triangle of the matrix, then any repeated nonzeros will be summed.
9.1.10CSECTION Section (optional)
Data lines in this section specify the cone constraint that has the following form in Second-order Cone Programming (SOCP).
where is a subvector of decision variables indexed by , is either quadratic cone or rotated quadratic cone of dimension . This section may appear multiple times in the file to define various cone constraints .
The CSECTION has data on the same line as the indicator to store the name, parameter and type of the cone. Cone name is compulsory and must be in Field . The possible cone type key is either QUAD for quadratic cone or RQUAD for rotated quadratic cone. The parameter of cone is not used in all cases.
Field
Required
Description
Yes
Name of the cone
No
Parameter of the cone
Yes
Cone type
The data lines in this section specify variable names in the cone. Note that we require at least variables for quadratic cone and variables for rotated quadratic cone.
Field
Required
Description
Yes
A valid variable name
9.2Sparse SDPA file format
The problem data is written in an ASCII input file in a SDPA sparse format which was first introduced in Fujisawa et al. (1998).
In the description below we follow closely the specification from Borchers (1999).
The format is line oriented. If more elements are required on the line they need to be separated by a space, a tab, or any of the special characters ‘,’, ‘(’, ‘)’, ‘{’, or ‘}’. The file consists of six sections:
1.Comments. The file can begin with arbitrarily many lines of comments. Each line of comments must begin with ‘"’ or ‘*’.
2.The first line after the comments contains integer , the number of variables. The rest of this line is ignored.
3.The second line after the comments contains integer , the number of blocks in the block diagonal structure of the matrices. Additional text on this line after is ignored.
4.The third line after the comments contains a vector of integers that give the sizes of the individual blocks. Negative numbers may be used to indicate that a block is actually a diagonal submatrix. Thus a block size of ‘’ indicates a block in which only the diagonal elements are nonzero.
5.The fourth line after the comments contains an -dimensional real vector defining the objective function vector .
6.The remaining lines of the file contain nonzero entries in the constraint matrices, with one entry per line. The format for each line is
where is the number of the matrix to which this entry belongs and specifies the block number within this matrix. Together, they uniquely identify the block . Integers and are one-based indices which specify a location of the entry within the block. Note that since all matrices are assumed to be symmetric, only entries in the upper triangle of a matrix should be supplied. Finally, should give the real value of the entry in the matrix. Precisely, .
For SDPA format we use the word ‘token’ as a reference to a group of contiguous characters without a space or any other delimeters.
9.3Internal Changes
Internal changes have been made to this routine as follows:
At Mark 27.1: Previously all integer and binary variables defined in the file were relaxed to continuous variables automatically. This behavior was removed. Now e04saf reads integer and binary variables and stores them in the handle data structure.
For details of all known issues which have been reported for the NAG Library please refer to the Known Issues.
10Example
This example demonstrates how to load data of an SOCP problem from an MPS file and solve it using SOCP solver e04ptf.
The data file stores the following SOCP problem
subject to the bounds
the general linear constraints
and the cone constraint
The optimal solution (to five significant figures) is