G03DCF (PDF version)
G03 Chapter Contents
G03 Chapter Introduction
NAG Library Manual

NAG Library Routine Document


Note:  before using this routine, please read the Users' Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent details.

+ Contents

    1  Purpose
    7  Accuracy

1  Purpose

G03DCF allocates observations to groups according to selected rules. It is intended for use after G03DAF.

2  Specification


3  Description

Discriminant analysis is concerned with the allocation of observations to groups using information from other observations whose group membership is known, Xt; these are called the training set. Consider p variables observed on ng populations or groups. Let x-j be the sample mean and Sj the within-group variance-covariance matrix for the jth group; these are calculated from a training set of n observations with nj observations in the jth group, and let xk be the kth observation from the set of observations to be allocated to the ng groups. The observation can be allocated to a group according to a selected rule. The allocation rule or discriminant function will be based on the distance of the observation from an estimate of the location of the groups, usually the group means. A measure of the distance of the observation from the jth group mean is given by the Mahalanobis distance, Dkj:
Dkj2=xk-x-jTSj-1xk-x-j. (1)
If the pooled estimate of the variance-covariance matrix S is used rather than the within-group variance-covariance matrices, then the distance is:
Dkj2=xk-x-jTS-1xk-x-j. (2)
Instead of using the variance-covariance matrices S and Sj, G03DCF uses the upper triangular matrices R and Rj supplied by G03DAF such that S=RTR and Sj=RjTRj. Dkj2 can then be calculated as zTz where RTjz=xk-xj or RTz=xk-x as appropriate.
In addition to the distances, a set of prior probabilities of group membership, πj, for j=1,2,,ng, may be used, with πj=1. The prior probabilities reflect your view as to the likelihood of the observations coming from the different groups. Two common cases for prior probabilities are π1=π2==πng, that is, equal prior probabilities, and πj=nj/n, for j=1,2,,ng, that is, prior probabilities proportional to the number of observations in the groups in the training set.
G03DCF uses one of four allocation rules. In all four rules the p variables are assumed to follow a multivariate Normal distribution with mean μj and variance-covariance matrix Σj if the observation comes from the jth group. The different rules depend on whether or not the within-group variance-covariance matrices are assumed equal, i.e., Σ1=Σ2==Σng, and whether a predictive or estimative approach is used. If p xk μj ,Σj  is the probability of observing the observation xk from group j, then the posterior probability of belonging to group j is:
p jxk,μj,Σj p xk μj ,Σj πj. (3)
In the estimative approach, the parameters μj and Σj in (3) are replaced by their estimates calculated from Xt. In the predictive approach, a non-informative prior distribution is used for the parameters and a posterior distribution for the parameters, p μj, Σj Xt , is found. A predictive distribution is then obtained by integrating p jxk,μj,Σj p μj, Σj X  over the parameter space. This predictive distribution then replaces p xk μj ,Σj  in (3). See Aitchison and Dunsmore (1975), Aitchison et al. (1977) and Moran and Murphy (1979) for further details.
The observation is allocated to the group with the highest posterior probability. Denoting the posterior probabilities, p jxk,μj,Σj , by qj, the four allocation rules are:
(i) Estimative with equal variance-covariance matrices – Linear Discrimination
(ii) Estimative with unequal variance-covariance matrices – Quadratic Discrimination
(iii) Predictive with equal variance-covariance matrices
q j - 1 n j +1 / n j p / 2 1 + n j / n - n g n j +1 D k j 2 n +1 - n g / 2
(iv) Predictive with unequal variance-covariance matrices
q j - 1 C n j 2 - 1 / n j S j p / 2 1 + n j / n j 2 - 1 D k j 2 n j / 2 ,
C=Γ12nj-p Γ12nj .
In the above the appropriate value of Dkj2 from (1) or (2) is used. The values of the qj are standardized so that,
Moran and Murphy (1979) show the similarity between the predictive methods and methods based upon likelihood ratio tests.
In addition to allocating the observation to a group, G03DCF computes an atypicality index, Ijxk. The predictive atypicality index is returned, irrespective of the value of the parameter TYP. This represents the probability of obtaining an observation more typical of group j than the observed xk (see Aitchison and Dunsmore (1975) and Aitchison et al. (1977)). The atypicality index is computed for unequal within-group variance-covariance matrices as:
where PBβ:a,b is the lower tail probability from a beta distribution and
and for equal within-group variance-covariance matrices as:
Ijxk=PBz : 12p,12n-ng-p+ 1,
If Ijxk is close to 1 for all groups it indicates that the observation may come from a grouping not represented in the training set. Moran and Murphy (1979) provide a frequentist interpretation of Ijxk.

4  References

Aitchison J and Dunsmore I R (1975) Statistical Prediction Analysis Cambridge
Aitchison J, Habbema J D F and Kay J W (1977) A critical comparison of two methods of statistical discrimination Appl. Statist. 26 15–25
Kendall M G and Stuart A (1976) The Advanced Theory of Statistics (Volume 3) (3rd Edition) Griffin
Krzanowski W J (1990) Principles of Multivariate Analysis Oxford University Press
Moran M A and Murphy B J (1979) A closer look at two alternative methods of statistical discrimination Appl. Statist. 28 223–232
Morrison D F (1967) Multivariate Statistical Methods McGraw–Hill

5  Parameters

1:     TYP – CHARACTER(1)Input
On entry: whether the estimative or predictive approach is used.
The estimative approach is used.
The predictive approach is used.
Constraint: TYP='E' or 'P'.
2:     EQUAL – CHARACTER(1)Input
On entry: indicates whether or not the within-group variance-covariance matrices are assumed to be equal and the pooled variance-covariance matrix used.
The within-group variance-covariance matrices are assumed equal and the matrix R stored in the first pp+1/2 elements of GC is used.
The within-group variance-covariance matrices are assumed to be unequal and the matrices Ri, for i=1,2,,ng, stored in the remainder of GC are used.
Constraint: EQUAL='E' or 'U'.
3:     PRIORS – CHARACTER(1)Input
On entry: indicates the form of the prior probabilities to be used.
Equal prior probabilities are used.
Prior probabilities proportional to the group sizes in the training set, nj, are used.
The prior probabilities are input in PRIOR.
Constraint: PRIORS='E', 'I' or 'P'.
4:     NVAR – INTEGERInput
On entry: p, the number of variables in the variance-covariance matrices.
Constraint: NVAR1.
5:     NG – INTEGERInput
On entry: the number of groups, ng.
Constraint: NG2.
6:     NIG(NG) – INTEGER arrayInput
On entry: the number of observations in each group in the training set, nj.
  • if EQUAL='E', NIGj>0 and j=1ngNIGj>NG+NVAR, for j=1,2,,ng;
  • if EQUAL='U', NIGj>NVAR, for j=1,2,,ng.
7:     GMN(LDGMN,NVAR) – REAL (KIND=nag_wp) arrayInput
On entry: the jth row of GMN contains the means of the p variables for the jth group, for j=1,2,,nj. These are returned by G03DAF.
8:     LDGMN – INTEGERInput
On entry: the first dimension of the array GMN as declared in the (sub)program from which G03DCF is called.
Constraint: LDGMNNG.
9:     GC(NG+1×NVAR×NVAR+1/2) – REAL (KIND=nag_wp) arrayInput
On entry: the first pp+1/2 elements of GC should contain the upper triangular matrix R and the next ng blocks of pp+1/2 elements should contain the upper triangular matrices Rj.
All matrices must be stored packed by column. These matrices are returned by G03DAF. If EQUAL='E' only the first pp+1/2 elements are referenced, if EQUAL='U' only the elements pp+1/2+1 to ng+1pp+1/2 are referenced.
  • if EQUAL='E', the diagonal elements of R must be 0.0;
  • if EQUAL='U', the diagonal elements of the Rj must be 0.0, for j=1,2,,ng.
10:   DET(NG) – REAL (KIND=nag_wp) arrayInput
On entry: if EQUAL='U'. the logarithms of the determinants of the within-group variance-covariance matrices as returned by G03DAF. Otherwise DET is not referenced.
11:   NOBS – INTEGERInput
On entry: the number of observations in X which are to be allocated.
Constraint: NOBS1.
12:   M – INTEGERInput
On entry: the number of variables in the data array X.
Constraint: MNVAR.
13:   ISX(M) – INTEGER arrayInput
On entry: ISXl indicates if the lth variable in X is to be included in the distance calculations.
If ISXl>0, the lth variable is included, for l=1,2,,M; otherwise the lth variable is not referenced.
Constraint: ISXl>0 for NVAR values of l.
14:   X(LDX,M) – REAL (KIND=nag_wp) arrayInput
On entry: Xkl must contain the kth observation for the lth variable, for k=1,2,,NOBS and l=1,2,,M.
15:   LDX – INTEGERInput
On entry: the first dimension of the array X as declared in the (sub)program from which G03DCF is called.
Constraint: LDXNOBS.
16:   PRIOR(NG) – REAL (KIND=nag_wp) arrayInput/Output
On entry: if PRIORS='I', the prior probabilities for the ng groups.
Constraint: if PRIORS='I', PRIORj>0.0 and 1- j=1 ng PRIORj 10×machine precision , for j=1,2,,ng.
On exit: if PRIORS='P', the computed prior probabilities in proportion to group sizes for the ng groups.
If PRIORS='I', the input prior probabilities will be unchanged.
If PRIORS='E', PRIOR is not set.
17:   P(LDP,NG) – REAL (KIND=nag_wp) arrayOutput
On exit: Pkj contains the posterior probability pkj for allocating the kth observation to the jth group, for k=1,2,,NOBS and j=1,2,,ng.
18:   LDP – INTEGERInput
On entry: the first dimension of the arrays P and ATI as declared in the (sub)program from which G03DCF is called.
Constraint: LDPNOBS.
19:   IAG(NOBS) – INTEGER arrayOutput
On exit: the groups to which the observations have been allocated.
20:   ATIQ – LOGICALInput
On entry: ATIQ must be .TRUE. if atypicality indices are required. If ATIQ is .FALSE. the array ATI is not set.
21:   ATI(LDP,*) – REAL (KIND=nag_wp) arrayOutput
Note: the second dimension of the array ATI must be at least NG if ATIQ=.TRUE., and at least 1 otherwise.
On exit: if ATIQ is .TRUE., ATIkj will contain the predictive atypicality index for the kth observation with respect to the jth group, for k=1,2,,NOBS and j=1,2,,ng.
If ATIQ is .FALSE., ATI is not set.
22:   WK(2×NVAR) – REAL (KIND=nag_wp) arrayWorkspace
23:   IFAIL – INTEGERInput/Output
On entry: IFAIL must be set to 0, -1​ or ​1. If you are unfamiliar with this parameter you should refer to Section 3.3 in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value -1​ or ​1 is recommended. If the output of error messages is undesirable, then the value 1 is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is 0. When the value -1​ or ​1 is used it is essential to test the value of IFAIL on exit.
On exit: IFAIL=0 unless the routine detects an error or a warning has been flagged (see Section 6).

6  Error Indicators and Warnings

If on entry IFAIL=0 or -1, explanatory error messages are output on the current error message unit (as defined by X04AAF).
Errors or warnings detected by the routine:
On entry,NVAR<1,
orTYP'E' or ‘P’,
orEQUAL'E' or ‘U’,
orPRIORS'E', ‘I’ or ‘P’.
On entry,the number of variables indicated by ISX is not equal to NVAR,
orEQUAL='E' and NIGj0, for some j,
orEQUAL='E' and j=1ngNIGjNG+NVAR,
orEQUAL='U' and NIGjNVAR for some j.
On entry,PRIORS='I' and PRIORj0.0 for some j,
orPRIORS='I' and j=1ngPRIORj is not within 10×machine precision of 1.
On entry,EQUAL='E' and a diagonal element of R is zero,
orEQUAL='U' and a diagonal element of Rj for some j is zero.

7  Accuracy

The accuracy of the returned posterior probabilities will depend on the accuracy of the input R or Rj matrices. The atypicality index should be accurate to four significant places.

8  Further Comments

The distances Dkj2 can be computed using G03DBF if other forms of discrimination are required.

9  Example

The data, taken from Aitchison and Dunsmore (1975), is concerned with the diagnosis of three ‘types’ of Cushing's syndrome. The variables are the logarithms of the urinary excretion rates (mg/24hr) of two steroid metabolites. Observations for a total of 21 patients are input and the group means and R matrices are computed by G03DAF. A further six observations of unknown type are input and allocations made using the predictive approach and under the assumption that the within-group covariance matrices are not equal. The posterior probabilities of group membership, qj, and the atypicality index are printed along with the allocated group. The atypicality index shows that observations 5 and 6 do not seem to be typical of the three types present in the initial 21 observations.

9.1  Program Text

Program Text (g03dcfe.f90)

9.2  Program Data

Program Data (g03dcfe.d)

9.3  Program Results

Program Results (g03dcfe.r)

G03DCF (PDF version)
G03 Chapter Contents
G03 Chapter Introduction
NAG Library Manual

© The Numerical Algorithms Group Ltd, Oxford, UK. 2012