NAG Library Routine Document
H05ABF
1 Purpose
Given a set of features and a scoring mechanism for any subset of those features, H05ABF selects the best subsets of size using a direct communication branch and bound algorithm.
2 Specification
SUBROUTINE H05ABF ( |
MINCR, M, IP, NBEST, LA, BSCORE, BZ, F, MINCNT, GAMMA, ACC, IUSER, RUSER, IFAIL) |
INTEGER |
MINCR, M, IP, NBEST, LA, BZ(M-IP,NBEST), MINCNT, IUSER(*), IFAIL |
REAL (KIND=nag_wp) |
BSCORE(NBEST), GAMMA, ACC(2), RUSER(*) |
EXTERNAL |
F |
|
3 Description
Given , a set of unique features and a scoring mechanism defined for all then H05ABF is designed to find , an optimal subset of size . Here denotes the cardinality of , the number of elements in the set.
The definition of the optimal subset depends on the properties of the scoring mechanism, if
then the optimal subset is defined as one of the solutions to
else if
then the optimal subset is defined as one of the solutions to
If neither of these properties hold then H05ABF cannot be used.
As well as returning the optimal subset,
, H05ABF can return the best
solutions of size
. If
denotes the
th best subset, for
, then the
th best subset is defined as the solution to either
or
depending on the properties of
.
The solutions are found using a branch and bound method, where each node of the tree is a subset of
. Assuming that
(1) holds then a particular node, defined by subset
, can be trimmed from the tree if
where
is the
th highest score we have observed so far for a subset of size
, i.e., our current best guess of the score for the
th best subset. In addition, because of
(1) we can also drop all nodes defined by any subset
where
, thus avoiding the need to enumerate the whole tree. Similar short cuts can be taken if
(2) holds. A full description of this branch and bound algorithm can be found in
Ridout (1988).
Rather than calculate the score at a given node of the tree H05ABF utilizes the fast branch and bound algorithm of
Somol et al. (2004), and attempts to estimate the score where possible. For each feature,
, two values are stored, a count
and
, an estimate of the contribution of that feature. An initial value of zero is used for both
and
. At any stage of the algorithm where both
and
have been calculated (as opposed to estimated), the estimated contribution of the feature
is updated to
and
is incremented by
, therefore at each stage
is the mean contribution of
observed so far and
is the number of observations used to calculate that mean.
As long as , for the user-supplied constant , then rather than calculating this routine estimates it using or if has been estimated, where is a user-supplied scaling factor. An estimated score is never used to trim a node or returned as the optimal score.
Setting
in this routine will cause the algorithm to always calculate the scores, returning to the branch and bound algorithm of
Ridout (1988). In most cases it is preferable to use the fast branch and bound algorithm, by setting
, unless the score function is iterative in nature, i.e.,
must have been calculated before
can be calculated.
H05ABF is a direct communication version of
H05AAF.
4 References
Narendra P M and Fukunaga K (1977) A branch and bound algorithm for feature subset selection IEEE Transactions on Computers 9 917–922
Ridout M S (1988) Algorithm AS 233: An improved branch and bound algorithm for feature subset selection Journal of the Royal Statistics Society, Series C (Applied Statistics) (Volume 37) 1 139–147
Somol P, Pudil P and Kittler J (2004) Fast branch and bound algorithms for optimal feature selection IEEE Transactions on Pattern Analysis and Machine Intelligence (Volume 26) 7 900–912
5 Parameters
- 1: – INTEGERInput
-
On entry: flag indicating whether the scoring function
is increasing or decreasing.
- , i.e., the subsets with the largest score will be selected.
- , i.e., the subsets with the smallest score will be selected.
For all
and
.
Constraint:
or .
- 2: – INTEGERInput
-
On entry: , the number of features in the full feature set.
Constraint:
.
- 3: – INTEGERInput
-
On entry: , the number of features in the subset of interest.
Constraint:
.
- 4: – INTEGERInput
-
On entry:
, the maximum number of best subsets required. The actual number of subsets returned is given by
LA on final exit. If on final exit
then
is returned.
Constraint:
.
- 5: – INTEGEROutput
-
On exit: the number of best subsets returned.
- 6: – REAL (KIND=nag_wp) arrayOutput
-
On exit: holds the score for the
LA best subsets returned in
BZ.
- 7: – INTEGER arrayOutput
-
On exit: the th best subset is constructed by dropping the features specified in
, for and , from the set of all features, . The score for the th best subset is given in .
- 8: – SUBROUTINE, supplied by the user.External Procedure
-
F must evaluate the scoring function
.
The specification of
F is:
INTEGER |
M, DROP, LZ, Z(LZ), LA, A(LA), IUSER(*), INFO |
REAL (KIND=nag_wp) |
SCORE(max(LA,1)), RUSER(*) |
|
- 1: – INTEGERInput
-
On entry: , the number of features in the full feature set.
- 2: – INTEGERInput
-
On entry: flag indicating whether the intermediate subsets should be constructed by dropping features from the full set (
) or adding features to the empty set (
). See
SCORE for additional details.
- 3: – INTEGERInput
-
On entry: the number of features stored in
Z.
- 4: – INTEGER arrayInput
-
On entry:
, for
, contains the list of features which, along with those specified in
A, define the subsets whose score is required. See
SCORE for additional details.
- 5: – INTEGERInput
-
On entry: if
, the number of subsets for which a score must be returned.
If
, the score for a single subset should be returned. See
SCORE for additional details.
- 6: – INTEGER arrayInput
-
On entry:
, for
, contains the list of features which, along with those specified in
Z, define the subsets whose score is required. See
SCORE for additional details.
- 7: – REAL (KIND=nag_wp) arrayOutput
-
On exit: the value
, for
, the score associated with the
th subset.
is constructed as follows:
- is constructed by dropping the features specified in the first LZ elements of Z and the single feature given in from the full set of features, . The subset will therefore contain features.
- is constructed by adding the features specified in the first LZ elements of Z and the single feature specified in to the empty set, . The subset will therefore contain features.
In both cases the individual features are referenced by the integers
to
M with
indicating the first feature,
the second, etc., for some arbitrary ordering of the features, chosen by you prior to calling H05ABF. For example,
might refer to the first variable in a particular set of data,
the second, etc..
If
, the score for a single subset should be returned. This subset is constructed by adding or removing only those features specified in the first
LZ elements of
Z. If
, this subset will either be
or
.
- 8: – INTEGER arrayUser Workspace
- 9: – REAL (KIND=nag_wp) arrayUser Workspace
-
F is called with the parameters
IUSER and
RUSER as supplied to H05ABF. You are free to use the arrays
IUSER and
RUSER to supply information to
F as an alternative to using COMMON global variables.
- 10: – INTEGERInput/Output
-
On entry: .
On exit: set
INFO to a nonzero value if you wish H05ABF to terminate with
.
F must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which H05ABF is called. Parameters denoted as
Input must
not be changed by this procedure.
- 9: – INTEGERInput
-
On entry:
, the minimum number of times the effect of each feature,
, must have been observed before
is estimated from
as opposed to being calculated directly.
If then is never estimated. If then is set to .
- 10: – REAL (KIND=nag_wp)Input
-
On entry: , the scaling factor used when estimating scores. If then is used.
- 11: – REAL (KIND=nag_wp) arrayInput
-
On entry: a measure of the accuracy of the scoring function,
.
Letting
, then when confirming whether the scoring function is strictly increasing or decreasing (as described in
MINCR), or when assessing whether a node defined by subset
can be trimmed, then any values in the range
are treated as being numerically equivalent.
If then , otherwise .
If then , otherwise .
In most situations setting both and to zero should be sufficient. Using a nonzero value, when one is not required, can significantly increase the number of subsets that need to be evaluated.
- 12: – INTEGER arrayUser Workspace
- 13: – REAL (KIND=nag_wp) arrayUser Workspace
-
IUSER and
RUSER are not used by H05ABF, but are passed directly to
F and may be used to pass information to this routine as an alternative to using COMMON global variables.
- 14: – INTEGERInput/Output
-
On entry:
IFAIL must be set to
,
. If you are unfamiliar with this parameter you should refer to
Section 3.3 in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value
is recommended. If the output of error messages is undesirable, then the value
is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is
.
When the value is used it is essential to test the value of IFAIL on exit.
On exit:
unless the routine detects an error or a warning has been flagged (see
Section 6).
6 Error Indicators and Warnings
If on entry
or
, explanatory error messages are output on the current error message unit (as defined by
X04AAF).
Errors or warnings detected by the routine:
-
On entry, .
Constraint: or .
-
On entry, .
Constraint: .
-
On entry, and .
Constraint: .
-
On entry, .
Constraint: .
-
On entry, .
But only best subsets could be calculated.
-
On exit from
F,
, which is inconsistent with the score for the parent node. Score for the parent node is
.
-
A nonzero value for
INFO has been returned:
.
An unexpected error has been triggered by this routine. Please
contact
NAG.
See
Section 3.8 in the Essential Introduction for further information.
Your licence key may have expired or may not have been installed correctly.
See
Section 3.7 in the Essential Introduction for further information.
Dynamic memory allocation failed.
See
Section 3.6 in the Essential Introduction for further information.
7 Accuracy
The subsets returned by H05ABF are guaranteed to be optimal up to the accuracy of the calculated scores.
8 Parallelism and Performance
H05ABF is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the
Users' Note for your implementation for any additional implementation-specific information.
The maximum number of unique subsets of size
from a set of
features is
. The efficiency of the branch and bound algorithm implemented in H05ABF comes from evaluating subsets at internal nodes of the tree, that is subsets with more than
features, and where possible trimming branches of the tree based on the scores at these internal nodes as described in
Narendra and Fukunaga (1977). Because of this it is possible, in some circumstances, for more than
subsets to be evaluated. This will tend to happen when most of the features have a similar effect on the subset score.
If multiple optimal subsets exist with the same score, and
NBEST is too small to return them all, then the choice of which of these optimal subsets is returned is arbitrary.
10 Example
This example finds the three linear regression models, with five variables, that have the smallest residual sums of squares when fitted to a supplied dataset. The data used in this example was simulated.
10.1 Program Text
Program Text (h05abfe.f90)
10.2 Program Data
Program Data (h05abfe.d)
10.3 Program Results
Program Results (h05abfe.r)