NAG Library Function Document

nag_mv_discrim_group (g03dcc)

nag_mv_discrim_group (Nag_DiscrimMethod type, Nag_GroupCovars equal, Nag_PriorProbability priors, Integer nvar, Integer ng, const Integer nig[], const double gmean[], Integer tdg, const double gc[], const double det[], Integer nobs, Integer m, const Integer isx[], const double x[], Integer tdx, double prior[], double p[], Integer tdp, Integer iag[], Nag_Boolean atiq, double ati[], NagError *fail)

3 Description

Discriminant analysis is concerned with the allocation of observations to groups using information from other observations whose group membership is known,

X_{t}

; these are called the training set. Consider

p

variables observed on

n_{g}

populations or groups. Let

{\bar{x}}_{j}

be the sample mean and

S_{j}

the within-group variance-covariance matrix for the

j

th group; these are calculated from a training set of

n

observations with

n_{j}

observations in the

j

th group, and let

x_{k}

be the

k

th observation from the set of observations to be allocated to the

n_{g}

groups. The observation can be allocated to a group according to a selected rule. The allocation rule or discriminant function will be based on the distance of the observation from an estimate of the location of the groups, usually the group means. A measure of the distance of the observation from the

j

th group mean is given by the Mahalanobis distance,

D_{k j}^{2}

D_{k j}^{2} = {(x_{k} - {\bar{x}}_{j})}^{T} S_{j}^{- 1} (x_{k} - {\bar{x}}_{j}) .

(1)

If the pooled estimate of the variance-covariance matrix

S

is used rather than the within-group variance-covariance matrices, then the distance is:

D_{k j}^{2} = {(x_{k} - {\bar{x}}_{j})}^{T} S^{- 1} (x_{k} - {\bar{x}}_{j}) .

(2)

Instead of using the variance-covariance matrices

S

and

S_{j}

, nag_mv_discrim_group (g03dcc) uses the upper triangular matrices

R

and

R_{j}

supplied by nag_mv_discrim (g03dac) such that

S = R^{T} R

and

S_{j} = R_{j}^{T} R_{j}

D_{k j}^{2}

can then be calculated as

z^{T} z

where

R_{j} z = (x_{k} - {\bar{x}}_{j})

R z = (x_{k} - {\bar{x}}_{j})

as appropriate.

In addition to the distances, a set of prior probabilities of group membership,

π_{j}

, for

j = 1, 2, \dots, n_{g}

, may be used, with

\sum π_{j} = 1

. The prior probabilities reflect your view as to the likelihood of the observations coming from the different groups. Two common cases for prior probabilities are

π_{1} = π_{2} = \dots = π_{n_{g}}

, that is, equal prior probabilities, and

π_{j} = n_{j} / n

, for

j = 1, 2, \dots, n_{g}

, that is, prior probabilities proportional to the number of observations in the groups in the training set.

nag_mv_discrim_group (g03dcc) uses one of four allocation rules. In all four rules the

p

variables are assumed to follow a multivariate Normal distribution with mean

μ_{j}

and variance-covariance matrix

Σ_{j}

if the observation comes from the

j

th group. The different rules depend on whether or not the within-group variance-covariance matrices are assumed equal, i.e.,

Σ_{1} = Σ_{2} = \dots = Σ_{n_{g}}

, and whether a predictive or estimative approach is used. If

p (x_{k} ∣ μ_{j}, Σ_{j})

is the probability of observing the observation

x_{k}

from group

j

, then the posterior probability of belonging to group

j

is:

p (j ∣ x_{k} μ_{j}, Σ_{j}) \propto p (x_{k} ∣ μ_{j}, Σ_{j}) π_{j} .

(3)

In the estimative approach, the arguments

μ_{j}

and

Σ_{j}

in (3) are replaced by their estimates calculated from

X_{t}

. In the predictive approach, a non-informative prior distribution is used for the arguments and a posterior distribution for the arguments,

p (μ_{j}, Σ_{j} ∣ X_{t})

, is found. A predictive distribution is then obtained by integrating

p (j ∣ x_{k}, μ_{j}, Σ_{j}) p (μ_{j}, Σ_{j} ∣ X)

over the argument space. This predictive distribution then replaces

p (x_{k} ∣ μ_{j}, Σ_{j})

in (3). See Aitchison and Dunsmore (1975), Aitchison et al. (1977) and Moran and Murphy (1979) for further details.

The observation is allocated to the group with the highest posterior probability. Denoting the posterior probabilities,

p (j ∣ x_{k}, μ_{j}, Σ_{j})

, by

q_{j}

, the four allocation rules are:

(i)

Estimative with equal variance-covariance matrices – Linear Discrimination.

{\log (q)}_{j} \propto - \frac{1}{2} D_{k j}^{2} + {\log π}_{j}

(ii)

Estimative with unequal variance-covariance matrices – Quadratic Discrimination.

{\log (q)}_{j} \propto - \frac{1}{2} D_{k j}^{2} + {\log π}_{j} - \frac{1}{2} \log |S_{j}|

(iii)

Predictive with equal variance-covariance matrices.

q_{j}^{- 1} \propto {((n_{j} + 1) / n_{j})}^{p / 2} {\{1 + [n_{j} / (({n - n}_{g}) (n_{j} + 1))] D_{k j}^{2}\}}^{(n + 1 - n_{g}) / 2}

(iv)

Predictive with unequal variance-covariance matrices

q_{j}^{- 1} \propto C {\{((n_{j}^{2} - 1) / n_{j}) |S_{j}|\}}^{p / 2} {\{1 + (n_{j} / (n_{j}^{2} - 1)) D_{k j}^{2}\}}^{n_{j} / 2}

where

C = \frac{Γ (\frac{1}{2} (n_{j} - p))}{Γ (\frac{1}{2} n_{j})}

In the above the appropriate value of

D_{k j}^{2}

from (1) or (2) is used. The values of the

q_{j}

are standardized so that,

\sum_{j = 1}^{n_{g}} q_{j} = 1 .

Moran and Murphy (1979) show the similarity between the predictive methods and methods based upon likelihood ratio tests.

In addition to allocating the observation to a group, nag_mv_discrim_group (g03dcc) computes an atypicality index,

I_{j} (x_{k})

. This represents the probability of obtaining an observation more typical of group

j

than the observed

x_{k}

(see Aitchison and Dunsmore (1975) and Aitchison et al. (1977)). The atypicality index is computed as:

I_{j} (x_{k}) = P (B \leq z : \frac{1}{2} p, \frac{1}{2} (n_{j} - d))

where

P (B \leq β : a, b)

is the lower tail probability from a beta distribution where, for unequal within-group variance-covariance matrices,

z = D_{k j}^{2} / (D_{k j}^{2} + (n_{j}^{2} - 1) / n_{j}),

and for equal within-group variance-covariance matrices,

z = D_{k j}^{2} / (D_{k j}^{2} + ({n - n}_{g}) (n_{j} - 1) / n_{j}) .

I_{j} (x_{k})

is close to 1 for all groups it indicates that the observation may come from a grouping not represented in the training set. Moran and Murphy (1979) provide a frequentist interpretation of

I_{j} (x_{k})

4 References

Aitchison J and Dunsmore I R (1975) Statistical Prediction Analysis Cambridge

Aitchison J, Habbema J D F and Kay J W (1977) A critical comparison of two methods of statistical discrimination Appl. Statist. 26 15–25

Kendall M G and Stuart A (1976) The Advanced Theory of Statistics (Volume 3) (3rd Edition) Griffin

Krzanowski W J (1990) Principles of Multivariate Analysis Oxford University Press

Moran M A and Murphy B J (1979) A closer look at two alternative methods of statistical discrimination Appl. Statist. 28 223–232

Morrison D F (1967) Multivariate Statistical Methods McGraw–Hill

5 Arguments

1: $type$ – Nag_DiscrimMethodInput

On entry: indicates whether the estimative or predictive approach is to be used.

$type = Nag_DiscrimEstimate$: The estimative approach is used.
$type = Nag_DiscrimPredict$: The predictive approach is used.

Constraint:

type = Nag_DiscrimEstimate

Nag_DiscrimPredict

2: $equal$ – Nag_GroupCovarsInput

On entry: indicates whether or not the within-group variance-covariance matrices are assumed to be equal and the pooled variance-covariance matrix used.

$equal = Nag_EqualCovar$: The within-group variance-covariance matrices are assumed equal and the matrix $R$ stored in the first $p (p + 1) / 2$ elements of gc is used.
$equal = Nag_NotEqualCovar$: The within-group variance-covariance matrices are assumed to be unequal and the matrices $R_{i}$ , for $i = 1, 2, \dots, n_{g}$ , stored in the remainder of gc are used.

Constraint:

equal = Nag_EqualCovar

Nag_NotEqualCovar

3: $priors$ – Nag_PriorProbabilityInput

On entry: indicates the form of the prior probabilities to be used.

$priors = Nag_EqualPrior$: Equal prior probabilities are used.
$priors = Nag_GroupSizePrior$: Prior probabilities proportional to the group sizes in the training set, $n_{j}$ , are used.
$priors = Nag_UserPrior$: The prior probabilities are input in prior.

Constraint:

priors = Nag_EqualPrior

Nag_GroupSizePrior

Nag_UserPrior

4: $nvar$ – IntegerInput

On entry: the number of variables,

p

, in the variance-covariance matrices as specified to nag_mv_discrim (g03dac).

Constraint:

nvar \geq 1

5: $ng$ – IntegerInput

On entry: the number of groups,

n_{g}

Constraint:

ng \geq 2

6: $nig [ng]$ – const IntegerInput

On entry: the number of observations in each group training set,

n_{j}

Constraints:

if $equal = Nag_EqualCovar$ , $nig [j - 1] > 0$ and $\sum_{j = 1}^{n_{g}} nig [j - 1] > ng + nvar$ , for $j = 1, 2, \dots, n_{g}$ ;
if $equal = Nag_NotEqualCovar$ , $nig [j - 1] > nvar$ , for $j = 1, 2, \dots, n_{g}$ .

7: $gmean [ng \times tdg]$ – const doubleInput

Note: the

(i, j)

th element of the matrix is stored in

gmean [(i - 1) \times tdg + j - 1]

On entry: the

j

th row of gmean contains the means of the

p

variables for the

j

th group, for

j = 1, 2, \dots, n_{j}

. These are returned by nag_mv_discrim (g03dac).

8: $tdg$ – IntegerInput

On entry: the stride separating matrix column elements in the array gmean.

Constraint:

tdg \geq nvar

9: $gc [\dim]$ – const doubleInput

Note: the dimension, dim, of the array gc must be at least

(ng + 1) \times nvar \times (nvar + 1) / 2

On entry: the first

p (p + 1) / 2

elements of gc should contain the upper triangular matrix

R

and the next

n_{g}

blocks of

p (p + 1) / 2

elements should contain the upper triangular matrices

R_{j}

All matrices must be stored packed by column. These matrices are returned by nag_mv_discrim (g03dac). If

equal = Nag_EqualCovar

, only the first

p (p + 1) / 2

elements are referenced, if

equal = Nag_NotEqualCovar

, only the elements

p (p + 1) / 2

(n_{g} + 1) p (p + 1) / 2 - 1

are referenced.

Constraints:

if $equal = Nag_EqualCovar$ , the diagonal elements of $R$ must be $\neq 0.0$ ;
if $equal = Nag_NotEqualCovar$ , the diagonal elements of the $R_{j}$ must be $\neq 0.0$ , for $j = 1, 2, \dots, n_{g}$ .

10: $\det [ng]$ – const doubleInput

On entry: if

equal = Nag_NotEqualCovar

, the logarithms of the determinants of the within-group variance-covariance matrices as returned by nag_mv_discrim (g03dac). Otherwise det is not referenced.

11: $nobs$ – IntegerInput

On entry: the number of observations in x which are to be allocated.

Constraint:

nobs \geq 1

12: $m$ – IntegerInput

On entry: the number of variables in the data array x.

Constraint:

m \geq nvar

13: $isx [m]$ – const IntegerInput

On entry:

isx [l - 1]

indicates if the

l

th variable in x is to be included in the distance calculations. If

isx [l - 1] > 0

the

l

th variable is included, for

l = 1, 2, \dots, m

; otherwise the

l

th variable is not referenced.

Constraint:

isx [l - 1] > 0

for nvar values of

l

14: $x [nobs \times tdx]$ – const doubleInput

On entry:

x [(k - 1) \times tdx + l - 1]

must contain the

k

th observation for the

l

th variable, for

k = 1, 2, \dots, nobs

and

l = 1, 2, \dots, m

15: $tdx$ – IntegerInput

On entry: the stride separating matrix column elements in the array x.

Constraint:

tdx \geq m

16: $prior [ng]$ – doubleInput/Output

On entry: if

priors = Nag_UserPrior

the prior probabilities for the

n_{g}

groups.

Constraint: if

priors = Nag_UserPrior

prior [j - 1] > 0.0

and

|1 - \sum_{j = 1}^{n_{g}} prior [j - 1]| \leq 10 \times machine precision

, for

j = 1, 2, \dots, n_{g}

On exit: if

priors = Nag_GroupSizePrior

, the computed prior probabilities in proportion to group sizes for the

n_{g}

groups.

priors = Nag_UserPrior

, the input prior probabilities will be unchanged.

priors = Nag_EqualPrior

, prior is not set.

17: $p [nobs \times tdp]$ – doubleOutput

On exit:

p [(k - 1) \times tdp + j - 1]

contains the posterior probability

p_{k j}

for allocating the

k

th observation to the

j

th group , for

k = 1, 2, \dots, nobs

and

j = 1, 2, \dots, n_{g}

18: $tdp$ – IntegerInput

On entry: the stride separating matrix column elements in the arrays p, ati.

Constraint:

tdp \geq ng

19: $iag [nobs]$ – IntegerOutput

On exit: the groups to which the observations have been allocated.

20: $atiq$ – Nag_BooleanInput

On entry: atiq must be Nag_TRUE if atypicality indices are required. If atiq is Nag_FALSE, the array ati is not set.

21: $ati [nobs \times tdp]$ – doubleOutput

On exit: if atiq is Nag_TRUE,

ati [(k - 1) \times tdp + j - 1]

will contain the atypicality index for the

k

th observation with respect to the

j

th group, for

k = 1, 2, \dots, nobs

and

j = 1, 2, \dots, n_{g}

. If atiq is Nag_FALSE, ati is not set.

22: $fail$ – NagError *Input/Output

The NAG error argument (see Section 3.6 in the Essential Introduction).

6 Error Indicators and Warnings

NE_2_INT_ARG_LT: On entry, $m = 〈value〉$ while $nvar = 〈value〉$ . These arguments must satisfy $m \geq nvar$ .

On entry, $tdg = 〈value〉$ while $nvar = 〈value〉$ . These arguments must satisfy $tdg \geq nvar$ .

On entry, $tdp = 〈value〉$ while $ng = 〈value〉$ . These arguments must satisfy $tdp \geq ng$ .

On entry, $tdx = 〈value〉$ while $m = 〈value〉$ . These arguments must satisfy $tdx \geq m$ .
NE_ALLOC_FAIL: Dynamic memory allocation failed.
NE_BAD_PARAM: On entry, argument equal had an illegal value.

On entry, argument priors had an illegal value.

On entry, argument type had an illegal value.
NE_DIAG_0_COND: A diagonal element of R is zero when $equal = Nag_EqualCovar$ .
NE_DIAG_0_J_COND: A diagonal element of R is zero for some $j$ , when $equal = Nag_NotEqualCovar$
NE_GROUP_SUM: On entry, the $\sum_{j = 1}^{ng} nig [j - 1] = 〈value〉$ , $ng = 〈value〉$ , $nvar = 〈value〉$ .
Constraint: $\sum_{j = 1}^{ng} nig [j - 1] > ng + nvar$ when $equal = Nag_EqualCovar$ .
NE_INT_ARG_LT: On entry, $ng = 〈value〉$ .
Constraint: $ng \geq 2$ .

On entry, $nobs = 〈value〉$ .
Constraint: $nobs \geq 1$ .

On entry, $nvar = 〈value〉$ .
Constraint: $nvar \geq 1$ .
NE_INTARR: On entry, $nig [〈value〉] = 〈value〉$ .
Constraint: $nig [i - 1] > 0$ , for $i = 1, 2, \dots, ng$ , when $equal = Nag_EqualCovar$ .
NE_INTARR_INT: On entry, $nig [〈value〉] = 〈value〉$ , $nvar = 〈value〉$ .
Constraint: $nig [i - 1] > nvar$ , $i = 1, 2, \dots, ng$ when $equal = Nag_NotEqualCovar$ .
NE_INTERNAL_ERROR: An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
NE_PRIOR_SUM: On entry, $\sum_{j = 1}^{ng} prior [j - 1] = 〈value〉$ .
Constraint: $\sum_{j = 1}^{ng} prior [j - 1]$ must be within $10 \times$ machine precision of 1 when $priors = Nag_UserPrior$ .
NE_REALARR: On entry, $prior [〈value〉] = 〈value〉$ .
Constraint: $prior [j - 1] > 0$ , $j = 1, 2, \dots, ng$ when $priors = Nag_UserPrior$ .
NE_VAR_INCL_INDICATED: The number of variables, nvar in the analysis $= 〈value〉$ , while number of variables included in the analysis via array $isx = 〈value〉$ .
Constraint: these two numbers must be the same.

7 Accuracy

The accuracy of the returned posterior probabilities will depend on the accuracy of the input

R

R_{j}

matrices. The atypicality index should be accurate to four significant places.

8 Parallelism and Performance

Not applicable.

9 Further Comments

The distances

D_{k j}^{2}

can be computed using nag_mv_discrim_mahaldist (g03dbc) if other forms of discrimination are required.

10 Example

The data, taken from Aitchison and Dunsmore (1975), is concerned with the diagnosis of three ‘types’ of Cushing's syndrome. The variables are the logarithms of the urinary excretion rates (mg/24hr) of two steroid metabolites. Observations for a total of 21 patients are input and the group means and

R

matrices are computed by nag_mv_discrim (g03dac). A further six observations of unknown type are input and allocations made using the predictive approach and under the assumption that the within-group covariance matrices are not equal. The posterior probabilities of group membership,

q_{j}

, and the atypicality index are printed along with the allocated group. The atypicality index shows that observations 5 and 6 do not seem to be typical of the three types present in the initial 21 observations.

NAG Library Function Documentnag_mv_discrim_group (g03dcc)

▸▿ Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Parallelism and Performance

9 Further Comments

10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

NAG Library Function Document

nag_mv_discrim_group (g03dcc)