NAG CL Interface
e04sac (handle_read_file)
1
Purpose
e04sac initializes a data structure for the NAG optimization modelling suite from a data file for problems such as, Linear Programming (LP), Quadratic Programming (QP), Second-order Cone Programming (SOCP), or linear Semidefinite Programming (SDP).
2
Specification
The function may be called by the names: e04sac or nag_opt_handle_read_file.
3
Description
e04sac reads a file in one of the supported file formats and stores the data in a new handle. The supported formats are MPS and its derivatives (for LP, QP, SOCP) and sparse SDPA for linear semidefinite programming problems. See
Section 9 for more details on the supported file formats. It might be useful to refer to the file format you use to fully understand the possible error messages. Once the file is successfuly processed, the problem can be solved by a compatible solver from the suite or queried via the printing function
e04ryc. Once the problem is not needed any more,
e04rzc should be called to destroy the handle and deallocate the memory held within. See
Section 4.1 in the
E04 Chapter Introduction for more details about the
NAG optimization modelling suite. Also see
Section 2.2 in the
E04 Chapter Introduction for more details on the standard formulations of the optimization problems (e.g., LP, QP, SOCP and SDP) for better understanding on the notations used in this document.
4
References
Borchers B (1999) SDPLIB 1.2, A Library of semidefinite programming test problems
Optimization Methods and Software 11(1) 683–690
http://euler.nmt.edu/~brian/sdplib/
Fujisawa K, Kojima M and Nakata K (1998) SDPA (Semidefinite Programming Algorithm) User's Manual Technical Report B-308 Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.
IBM (1971) MPSX – Mathematical programming system Program Number 5734 XM4 IBM Trade Corporation, New York
5
Arguments
-
1:
– void **
Output
-
Note: handle does not need to be set on input.
On exit: holds a handle to the internal data structure. It must not be changed between calls to the NAG optimization modelling suite.
-
2:
– const char *
Input
-
On entry: the name of the file to be opened.
Constraint:
must contain a valid filename for the computer system being used.
-
3:
– const char *
Input
-
On entry: the expected file format of the input file.
ftype is case insensitive.
- or
- The input file is in MPS format.
- or
- The input file is in SDPA format.
Constraint:
, , or .
-
4:
– Integer
Output
-
On exit: problem sizes and statistics as given in the table below:
|
Number of variables (nvar). |
|
Number of integer variables. |
|
Number of binary variables. |
|
Number of Lagrange multipliers (dual variables) for the bound constraints and linear constraints (nnzu). |
|
Number of Lagrange multipliers (dual variables) for the cone constraints (nnzuc). |
|
Number of nonzeros in all Lagrange multipliers for the matrix constraints (nnzua). |
|
Flag indicating the objective function type, : no objective, : linear objective, : quadratic without linear term, : quadratic with linear term, : general nonlinear, : nonlinear least square with dense Jacobian, : nonlinear least square with sparse Jacobian. |
|
Number of residuals defined in a nonlinear least square problem. |
|
Definition of box constraints, : not defined, : defined. |
|
Number of linear constraints. |
|
Number of nonlinear constraints. |
|
Definition of Hessian for nonlinear objective and constraints, : not defined, : defined as Lagrangian, : defined as individual Hessians of the objective and the constraints. |
|
Number of cone constraints. |
|
Number of matrix inequality constraints. |
|
Number of quadratic constraints. |
otherwise |
Reserved for future use. |
– |
Reserved for future use. |
nvar,
nnzu,
nnzuc and
nnzua define the problem size and are required by other functions such as
e04mtc,
e04ptc and
e04svc in the optimization suite.
-
5:
– NagError *
Input/Output
-
The
NAG error argument (see
Section 7 in the Introduction to the
NAG Library CL Interface).
6
Error Indicators and Warnings
- NE_ALLOC_FAIL
-
Dynamic memory allocation failed.
See
Section 3.1.2 in the Introduction to the
NAG Library CL Interface for further information.
- NE_BAD_PARAM
-
On entry, argument had an illegal value.
- NE_DIAG_ELEMENTS
-
Invalid structural data found on line .
The specified element belongs to a diagonal block but is not diagonal.
The row index is and column index is .
- NE_DUPLICATE_ELEMENT
-
An entry in the constraints with , , row index and column index was defined more than once. All entries need to be unique.
- NE_FILE_INCOMPLETE
-
A premature end of the input stream. The part defining the dimensions of the blocks was not found.
A premature end of the input stream. The part defining the nonzero entries was not found.
A premature end of the input stream. The part defining the number of blocks was not found.
A premature end of the input stream. The part defining the number of variables was not found.
A premature end of the input stream. The part defining the objective function was not found.
- NE_INT
-
An invalid number of blocks was given on line .
The number stated there is and needs to be at least .
An invalid number of variables was given on line .
The number stated there is and needs to be at least .
- NE_INT_ARRAY
-
An invalid size of the block number was given on line .
The number stated there is and needs to be nonzero.
- NE_INTERNAL_ERROR
-
An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact
NAG for assistance.
See
Section 7.5 in the Introduction to the
NAG Library CL Interface for further information.
- NE_INVALID_CS
-
Invalid structural data found on line .
The given column index is out of bounds, it must respect the size of the block. Its value must be between and (inclusive).
Invalid structural data found on line .
The given row index is out of bounds, it must respect the size of the block. Its value must be between and (inclusive).
Invalid structural data found on line .
The specified nonzero element is not in the upper triangle.
The row index is and column index is .
- NE_INVALID_FORMAT
-
The token on line at position to was not recognized as a valid integer.
The token on line at position to was not recognized as a valid real number.
The token on line starting at position was too long and was not recognized.
- NE_MPS_BOUNDS
-
Inconsistent bounds for column ‘’.
Inconsistent bounds for row ‘’.
Unknown bound type ‘’ in BOUNDS section.
Inconsistent bounds are reported when the lower bound is greater than or equal to or the upper bound is less than or equal to , or when the lower bound is greater than the upper bound. Any upper bound greater than or equal to will be regarded as (and similarly any lower bound less than or equal to will be regarded as ).
- NE_MPS_COLUMNS
-
Column ‘
’ has been defined more than once in the COLUMNS section. Column definitions must be continuous. (See
Section 9.1.5).
Unknown column name ‘’ in section.
All column names must be specified in the COLUMNS section.
- NE_MPS_ENDATA_NOT_FOUND
-
End of file found before ENDATA indicator line.
- NE_MPS_ILL_NCONEVAR
-
Rotated second-order cone ‘’ should have at least variables.
Second-order cone ‘’ should have at least variables.
- NE_MPS_ILLEGAL_DATA_LINE
-
An illegal line was detected in ‘’ section.
This is neither a comment nor a valid data line.
- NE_MPS_ILLEGAL_NUMBER
-
Field
did not contain a number (see
Section 9.1).
- NE_MPS_INDICATOR
-
Both quadratic objective and cone constraints found, not supported.
Incorrect ordering of indicator lines.
BOUNDS indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines.
COLUMNS indicator line found before ROWS indicator line.
Incorrect ordering of indicator lines.
CSECTION indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines.
OBJNAME indicator line found after ROWS indicator line.
Incorrect ordering of indicator lines.
QUADOBJ indicator line found before BOUNDS indicator line.
Incorrect ordering of indicator lines.
QUADOBJ indicator line found before COLUMNS indicator line.
Incorrect ordering of indicator lines.
RANGES indicator line found before RHS indicator line.
Incorrect ordering of indicator lines.
RHS indicator line found before COLUMNS indicator line.
Indicator line ‘’ has been found more than once in the MPS file.
Unknown cone type ‘’.
Unknown indicator line ‘’.
- NE_MPS_INVALID_INTORG_INTEND
-
Found 'INTEND' marker without previous marker being 'INTORG'.
Found 'INTORG' but not 'INTEND' before the end of the COLUMNS section.
Found 'INTORG' marker within 'INTORG' to 'INTEND' range.
Illegal marker type ‘’.
Should be either 'INTORG' or 'INTEND'.
- NE_MPS_MANDATORY
-
At least one mandatory section not found in MPS file.
- NE_MPS_OBJNAME
-
The supplied name in OBJNAME of the objective row was not found among the free rows in the ROWS section.
- NE_MPS_PRINTABLE
-
Illegal column name.
Column names must consist of printable characters only.
Illegal cone name.
Cone names must consist of printable characters only.
Illegal row name.
Row names must consist of printable characters only.
- NE_MPS_REPEAT_COLUMN
-
More than one nonzero of has row name ‘’ and column name ‘’ in the COLUMNS section.
- NE_MPS_REPEAT_CONE
-
Cone name ‘’ has been defined more than once.
- NE_MPS_REPEAT_CONEVAR
-
Column name ‘’ has been defined more than once in cone constraint.
- NE_MPS_REPEAT_ROW
-
Row name ‘’ has been defined more than once in the ROWS section.
- NE_MPS_ROWS
-
Unknown inequality key ‘’ in ROWS section.
Expected ‘N’, ‘G’, ‘L’ or ‘E’.
Unknown row name ‘’ in section.
All row names must be specified in the ROWS section.
- NE_MPS_ROWS_OR_CONS
-
Empty ROWS section.
Neither the objective row nor the constraints were defined.
- NE_NO_LICENCE
-
Your licence key may have expired or may not have been installed correctly.
See
Section 8 in the Introduction to the
NAG Library CL Interface for further information.
- NE_NOT_CLOSE_FILE
-
Cannot close file.
- NE_NOT_READ_FILE
-
The input file seems to be empty. No data was read.
- NE_OUT_OF_RANGE
-
Invalid structural data found on line .
The given block number is out of bounds. Its value must be between
and
(inclusive).
Invalid structural data found on line .
The given matrix number is out of bounds. Its value must be between
and
(inclusive).
- NE_READ_ERROR
-
Cannot open file for reading.
Reading from the file caused an unknown error on line .
- NE_STR_UNKNOWN
-
On entry, .
Constraint: , , or .
- NE_WRONG_NUM_TOKENS
-
Not enough data was given on line specifying block sizes.
Expected
tokens but found only .
Not enough data was given on line specifying nonzero matrix elements.
Expected tokens but found only .
Not enough data was given on line specifying the objective function.
Expected tokens but found only .
7
Accuracy
Not applicable.
8
Parallelism and Performance
Background information to multithreading can be found in the
Multithreading documentation.
e04sac is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
e04sac makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this function. Please also consult the
Users' Note for your implementation for any additional implementation-specific information.
The input file of data may only contain two types of lines:
-
1.Indicator lines (specifying the type of data which is to follow).
-
2.Data lines (specifying the actual data).
A
section is a combination of an indicator line and its corresponding data line(s). Any characters beyond column 80 are ignored. Indicator lines must not contain leading blank characters (in other words they must begin in column 1). The following displays the order in which the indicator lines must appear in the file:
NAME |
user-supplied name |
(optional) |
OBJSENSE |
|
(optional) |
|
data line |
|
OBJNAME |
|
(optional) |
|
data line |
|
ROWS |
|
|
|
data line(s) |
|
COLUMNS |
|
|
|
data line(s) |
|
RHS |
|
|
|
data line(s) |
|
RANGES |
|
(optional) |
|
data line(s) |
|
BOUNDS |
|
(optional) |
|
data line(s) |
|
QUADOBJ |
|
(optional) |
|
data line(s) |
|
CSECTION |
user-supplied name |
parameter |
cone type |
(optional) |
|
data line(s) |
|
ENDATA |
A data line follows a fixed format, being made up of fields as defined below. The contents of the fields may have different significance depending upon the section of data in which they appear.
|
Field 1 |
Field 2 |
Field 3 |
Field 4 |
Field 5 |
Field 6 |
Columns |
|
|
|
|
|
|
Contents |
Code |
Name |
Name |
Value |
Name |
Value |
Each name and code must consist of ‘printable’ characters only; names and codes supplied must match the case used in the following descriptions. Values are read using a field width of . This allows values to be entered in several equivalent forms. For example, , , and all represent the same number. It is safest to include an explicit decimal point.
Lines with an asterisk () in column will be considered comment lines and will be ignored by the function.
Columns outside the six fields must be blank, except for columns 72–80, whose contents are ignored by the function. A non-blank character outside the predefined six fields and columns 72–80 is considered to be a major error (
NE_MPS_ILLEGAL_DATA_LINE; see
Section 6), unless it is part of a comment.
9.1.1
NAME Section (optional)
The NAME section is the only section where the data must be on the same line as the indicator. The ‘user-supplied name’ must be in Field
but may be blank.
Field |
Required |
Description |
|
No |
Name of the problem |
9.1.2
OBJSENSE Section (optional)
The data line in this section can be used to specify the sense of the objective function. If this section is present it must contain only one data line. If the section is missing or empty, minimization is assumed.
Field |
Required |
Description |
|
No |
Sense of the objective function |
Field may contain either MIN, MAX, MINIMIZE or MAXIMIZE.
9.1.3
OBJNAME Section (optional)
The data line in this section can be used to specify the name of a free row (see
Section 9.1.4) that should be used as the objective function. If this section is present it must contain only one data line. If the section is missing or is empty, the first free row will be chosen instead.
Field |
Required |
Description |
|
No |
Row name to be used as the objective function |
Field must contain a valid row name.
9.1.4
ROWS Section
The data lines in this section specify unique row (constraint) names and their inequality types (i.e., unconstrained,
,
or
).
Field |
Required |
Description |
|
Yes |
Inequality key |
|
Yes |
Row name |
The inequality key specifies each row's type. It must be
E,
G,
L or
N and can be in either column
or
.
Inequality Key |
Description |
|
|
N |
Free row |
|
|
G |
Greater than or equal to |
finite |
|
L |
Less than or equal to |
|
finite |
E |
Equal to |
finite |
|
Row type
N stands for ‘Not binding’. It can be used to define the objective row. The objective row is a free row that specifies the vector
in the linear objective term
. If there is more than one free row, the first free row is chosen, unless another free row name is specified by OBJNAME (see
Section 9.1.3). Note that
is assumed to be zero if either the chosen row does not appear in the COLUMNS section (i.e., has no nonzero elements) or there are no free rows defined in the ROWS section.
9.1.5
COLUMNS Section
Data lines in this section specify the names to be assigned to the variables (columns) in the general linear constraint matrix
, and define, in terms of column vectors, the actual values of the corresponding matrix elements.
Field |
Required |
Description |
|
Yes |
Column name |
|
Yes |
Row name |
|
Yes |
Value |
|
No |
Row name |
|
No |
Value |
Each data line in the COLUMNS section defines the nonzero elements of or . Any elements of or that are undefined are assumed to be zero. Nonzero elements of must be grouped by column, that is to say that all of the nonzero elements in the jth column of must be specified before those in the th column, for . Rows may appear in any order within the column.
9.1.5.1
Integer Markers
For backward compatibility
e04sac allows you to define the integer variables within the COLUMNS section using integer markers, although this is not recommended as markers can be treated differently by different MPS readers; you should instead define any integer variables in the BOUNDS section (see below). Each marker line must have the following format:
Field |
Required |
Description |
|
No |
Marker ID |
|
Yes |
Marker tag |
|
Yes |
Marker type |
The marker tag must be 'MARKER'. The marker type must be 'INTORG' to start reading integer variables and 'INTEND' to finish reading integer variables. This implies that a row cannot be named 'MARKER', 'INTORG' or 'INTEND'. Please note that both marker tag and marker type comprise of characters as a ' is the mandatory first and last character in the string. You may wish to have several integer marker sections within the COLUMNS section, in which case each marker section must begin with an 'INTORG' marker and end with an 'INTEND' marker and there should not be another marker between them.
Field 2 is ignored by e04sac. When an integer variable is declared it will keep its default bounds unless they are changed in the BOUNDS section. This may vary between different MPS readers.
9.1.6
RHS Section
This section specifies the right-hand side values (if any) of the general linear constraint matrix
.
Field |
Required |
Description |
|
Yes |
RHS name |
|
Yes |
Row name |
|
Yes |
Value |
|
No |
Row name |
|
No |
Value |
The MPS file may contain several RHS sets distinguished by RHS name. However only the first RHS set will be used.
Only the nonzero RHS elements need to be specified. Note that if an RHS is given to the objective function it will be ignored by e04sac. An RHS given to the objective function is dealt with differently by different MPS readers, therefore, it is safer not to define an RHS of the objective function in your MPS file. Note that this section may be empty, in which case the RHS vector is assumed to be zero.
9.1.7
RANGES Section (optional)
Ranges are used to modify the interpretation of constraints defined in the ROWS section (see
Section 9.1.4) to the form
, where both
and
are finite. The range of the constraint is
.
Field |
Required |
Description |
|
Yes |
Range name |
|
Yes |
Row name |
|
Yes |
Value |
|
No |
Row name |
|
No |
Value |
The range of each constraint implies an upper and lower bound dependent on the inequality key of each constraint, on the RHS
of the constraint (as defined in the RHS section), and on the range
.
Inequality Key |
Sign of |
|
|
E |
|
|
|
E |
|
|
|
G |
|
|
|
L |
|
|
|
N |
|
|
|
Only the first range set will be used.
9.1.8
BOUNDS Section (optional)
These lines specify limits on the values of the variables (the quantities
and
in
). If a variable is not specified in the bound set then it is automatically assumed to lie between
and
.
Field |
Required |
Description |
|
Yes |
Bound type identifier |
|
Yes |
Bound name |
|
Yes |
Column name |
|
Yes/No |
Value |
Note: Field is required only if the bound type identifier is one of UP, LO, FX, UI or LI in which case it gives the value below. If the bound type identifier is FR, MI, PL or BV, Field is ignored and it is recommended to leave it blank.
The table below describes the acceptable bound type identifiers and how each determines the variables' bounds.
Bound Type Identifier |
|
|
Integer Variable? |
UP |
unchanged |
|
No |
LO |
|
unchanged |
No |
FX |
|
|
No |
FR |
|
|
No |
MI |
|
unchanged |
No |
PL |
unchanged |
|
No |
BV |
|
|
Yes |
UI |
unchanged |
|
Yes |
LI |
|
unchanged |
Yes |
Only the first bound set will be used.
9.1.9
QUADOBJ Section (optional)
The QUADOBJ section defines nonzero elements of the upper or lower triangle of the Hessian matrix
.
Field |
Required |
Description |
|
Yes |
Column name (HColumn Index) |
|
Yes |
Column name (HRow Index) |
|
Yes |
Value |
|
No |
Column name (HRow Index) |
|
No |
Value |
Each data line in the QUADOBJ section defines one (or optionally two) nonzero elements of the matrix . Each element is given as a triplet of row index , column index and a value. The column names (as defined in the COLUMNS section) are used to link the names of the variables and the indices and .
It is only necessary to define either the upper or lower triangle of the matrix; either will suffice. Any elements that have been defined in the upper triangle of the matrix will be moved to the lower triangle of the matrix, then any repeated nonzeros will be summed.
9.1.10
CSECTION Section (optional)
Data lines in this section specify the cone constraint that has the following form in Second-order Cone Programming (SOCP).
where
is a subvector of decision variables
indexed by
,
is either quadratic cone or rotated quadratic cone of dimension
. This section may appear multiple times in the file to define various cone constraints
.
The CSECTION has data on the same line as the indicator to store the name, parameter and type of the cone. Cone name is compulsory and must be in Field
. The possible cone type key is either QUAD for quadratic cone or RQUAD for rotated quadratic cone. The parameter of cone is not used in all cases.
Field |
Required |
Description |
|
Yes |
Name of the cone |
|
No |
Parameter of the cone |
|
Yes |
Cone type |
The data lines in this section specify variable names in the cone. Note that we require at least
variables for quadratic cone and
variables for rotated quadratic cone.
Field |
Required |
Description |
|
Yes |
A valid variable name |
9.2
Sparse SDPA file format
The problem data is written in an ASCII input file in a SDPA sparse format which was first introduced in
Fujisawa et al. (1998).
In the description below we follow closely the specification from
Borchers (1999).
The format is line oriented. If more elements are required on the line they need to be separated by a space, a tab, or any of the special characters ‘,’, ‘(’, ‘)’, ‘{’, or ‘}’. The file consists of six sections:
-
1.Comments. The file can begin with arbitrarily many lines of comments. Each line of comments must begin with ‘"’ or ‘*’.
-
2.The first line after the comments contains integer , the number of variables. The rest of this line is ignored.
-
3.The second line after the comments contains integer , the number of blocks in the block diagonal structure of the matrices. Additional text on this line after is ignored.
-
4.The third line after the comments contains a vector of integers that give the sizes of the individual blocks. Negative numbers may be used to indicate that a block is actually a diagonal submatrix. Thus a block size of ‘’ indicates a block in which only the diagonal elements are nonzero.
-
5.The fourth line after the comments contains an -dimensional real vector defining the objective function vector .
-
6.The remaining lines of the file contain nonzero entries in the constraint matrices, with one entry per line. The format for each line is
where is the number of the matrix to which this entry belongs and specifies the block number within this matrix. Together, they uniquely identify the block . Integers and are one-based indices which specify a location of the entry within the block. Note that since all matrices are assumed to be symmetric, only entries in the upper triangle of a matrix should be supplied. Finally, should give the real value of the entry in the matrix. Precisely, .
For SDPA format we use the word ‘token’ as a reference to a group of contiguous characters without a space or any other delimeters.
9.3
Internal Changes
Internal changes have been made to this function as follows:
- At Mark 27.1: Previously all integer and binary variables defined in the file were relaxed to continuous variables automatically. This behavior was removed. Now this function reads integer and binary variables and stores them in the handle data structure.
For details of all known issues which have been reported for the
NAG Library please refer to the
Known Issues.
10
Example
This example demonstrates how to load data of an SOCP problem from an MPS file and solve it using SOCP solver
e04ptc.
The data file stores the following SOCP problem
subject to the bounds
the general linear constraints
and the cone constraint
The optimal solution (to five significant figures) is
and the objective function value is
.
10.1
Program Text
10.2
Program Data
10.3
Program Results