naginterfaces.library.mv.discrim_​group

naginterfaces.library.mv.discrim_group(typ, equal, priors, nig, gmn, gc, det, isx, x, prior, atiq)

discrim_group allocates observations to groups according to selected rules. It is intended for use after discrim().

For full information please refer to the NAG Library document for g03dc

https://support.nag.com/numeric/nl/nagdoc_30/flhtml/g03/g03dcf.html

Parameters

typstr, length 1

Whether the estimative or predictive approach is used.

$typ=\text{'E'}$

The estimative approach is used.

$typ=\text{'P'}$

The predictive approach is used.

equalstr, length 1

Indicates whether or not the within-group variance-covariance matrices are assumed to be equal and the pooled variance-covariance matrix used.

$equal=\text{'E'}$

The within-group variance-covariance matrices are assumed equal and the matrix $R$ stored in the first $p(p+1)/2$ elements of $gc$ is used.

$equal=\text{'U'}$

The within-group variance-covariance matrices are assumed to be unequal and the matrices $R_{\text{i}}$, for $\text{i}=1,2,\dots ,n_g$, stored in the remainder of $gc$ are used.

priorsstr, length 1

Indicates the form of the prior probabilities to be used.

$priors=\text{'E'}$

Equal prior probabilities are used.

$priors=\text{'P'}$

Prior probabilities proportional to the group sizes in the training set, $n_j$, are used.

$priors=\text{'I'}$

The prior probabilities are input in $prior$.

nigint, array-like, shape $\left(\text{ng}\right)$

The number of observations in each group in the training set, $n_j$.

gmnfloat, array-like, shape $(\text{ng},\text{nvar})$

The $\text{j}$th row of $gmn$ contains the means of the $p$ variables for the $\text{j}$th group, for $\text{j}=1,2,\dots ,n_j$. These are returned by discrim().

gcfloat, array-like, shape $((\text{ng}+1)\times \text{nvar}\times (\text{nvar}+1)/2)$

The first $p(p+1)/2$ elements of $gc$ should contain the upper triangular matrix $R$ and the next $n_g$ blocks of $p(p+1)/2$ elements should contain the upper triangular matrices $R_j$.

All matrices must be stored packed by column.

These matrices are returned by discrim().

If $equal=\text{'E'}$ only the first $p(p+1)/2$ elements are referenced, if $equal=\text{'U'}$ only the elements $p(p+1)/2+1$ to $(n_g+1)p(p+1)/2$ are referenced.

detfloat, array-like, shape $\left(\text{ng}\right)$

If $equal=\text{'U'}$. the logarithms of the determinants of the within-group variance-covariance matrices as returned by discrim(). Otherwise $det$ is not referenced.

isxint, array-like, shape $\left(m\right)$

$isx[l-1]$ indicates if the $l$th variable in $x$ is to be included in the distance calculations.

If $isx[\text{l}-1]>0$, the $\text{l}$th variable is included, for $\text{l}=1,2,\dots ,m$; otherwise the $\text{l}$th variable is not referenced.

xfloat, array-like, shape $(\text{nobs},m)$

$x[\text{k}-1,\text{l}-1]$ must contain the $\text{k}$th observation for the $\text{l}$th variable, for $\text{l}=1,2,\dots ,m$, for $\text{k}=1,2,\dots ,\text{nobs}$.

priorfloat, array-like, shape $\left(\text{ng}\right)$

If $priors=\text{'I'}$, the prior probabilities for the $n_g$ groups.

atiqbool

$atiq$ must be $True$ if atypicality indices are required. If $atiq$ is $False$ the array $ati$ is not set.

Returns

priorfloat, ndarray, shape $\left(\text{ng}\right)$

If $priors=\text{'P'}$, the computed prior probabilities in proportion to group sizes for the $n_g$ groups.

If $priors=\text{'I'}$, the input prior probabilities will be unchanged.

If $priors=\text{'E'}$, $prior$ is not set.

pfloat, ndarray, shape $(\text{nobs},\text{ng})$

$p[\text{k}-1,\text{j}-1]$ contains the posterior probability $p_{\text{k}\text{j}}$ for allocating the $\text{k}$th observation to the $\text{j}$th group, for $\text{j}=1,2,\dots ,n_g$, for $\text{k}=1,2,\dots ,\text{nobs}$.

iagint, ndarray, shape $\left(\text{nobs}\right)$

The groups to which the observations have been allocated.

atifloat, ndarray, shape $(\text{nobs},:)$

If $atiq$ is $True$, $ati[\text{k}-1,\text{j}-1]$ will contain the predictive atypicality index for the $\text{k}$th observation with respect to the $\text{j}$th group, for $\text{j}=1,2,\dots ,n_g$, for $\text{k}=1,2,\dots ,\text{nobs}$.

If $atiq$ is $False$, $ati$ is not set.

Raises

NagValueError

(errno $1$)

On entry, $typ=⟨value⟩$.

Constraint: $typ=\text{'E'}$ or $\text{'P'}$.

(errno $1$)

On entry, $priors=⟨value⟩$.

Constraint: $priors=\text{'E'}$, $\text{'I'}$ or $\text{'P'}$.

(errno $1$)

On entry, $equal=⟨value⟩$.

Constraint: $equal=\text{'E'}$ or $\text{'U'}$.

(errno $1$)

On entry, $m=⟨value⟩$ and $\text{nvar}=⟨value⟩$.

Constraint: $m\ge \text{nvar}$.

(errno $1$)

On entry, $\text{nobs}=⟨value⟩$.

Constraint: $\text{nobs}\ge 1$.

(errno $1$)

On entry, $\text{ng}=⟨value⟩$.

Constraint: $\text{ng}\ge 2$.

(errno $1$)

On entry, $\text{nvar}=⟨value⟩$.

Constraint: $\text{nvar}\ge 1$.

(errno $2$)

On entry, $i=⟨value⟩$ and $nig[i-1]=⟨value⟩$.

Constraint: $nig[i-1]>0$.

(errno $2$)

On entry, ${\sum }_i\left(nig\right[i\left]\right)\le \text{ng}+\text{nvar}$.

(errno $2$)

On entry, $i=⟨value⟩$, $nig[i-1]=⟨value⟩$ and $\text{nvar}=⟨value⟩$.

Constraint: $nig[i-1]<\text{nvar}$.

(errno $2$)

On entry, $\text{nvar}=⟨value⟩$ and $⟨value⟩$ values of $isx>0$.

Constraint: exactly $\text{nvar}$ elements of $isx>0$.

(errno $3$)

On entry, ${\sum }_j\left(prior\right[j\left]\right)\ne 1.0$.

(errno $3$)

On entry, $j=⟨value⟩$ and $prior[j-1]=⟨value⟩$.

Constraint: $prior[j-1]>0$.

(errno $4$)

On entry, a diagonal element of $R$ or $R_j$ is zero.

Notes

In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

Discriminant analysis is concerned with the allocation of observations to groups using information from other observations whose group membership is known, $X_t$; these are called the training set. Consider $p$ variables observed on $n_g$ populations or groups. Let ${\overline{x}}_j$ be the sample mean and $S_j$ the within-group variance-covariance matrix for the $j$th group; these are calculated from a training set of $n$ observations with $n_j$ observations in the $j$th group, and let $x_k$ be the $k$th observation from the set of observations to be allocated to the $n_g$ groups. The observation can be allocated to a group according to a selected rule. The allocation rule or discriminant function will be based on the distance of the observation from an estimate of the location of the groups, usually the group means. A measure of the distance of the observation from the $j$th group mean is given by the Mahalanobis distance, $D_{kj}$:

$$D_{kj}^2={(x_k-{\overline{x}}_j)}^T{S}_j^{-1}(x_k-{\overline{x}}_j)\text{.}$$

If the pooled estimate of the variance-covariance matrix $S$ is used rather than the within-group variance-covariance matrices, then the distance is:

$$D_{kj}^2={(x_k-{\overline{x}}_j)}^T{S}^{-1}(x_k-{\overline{x}}_j)\text{.}$$

Instead of using the variance-covariance matrices $S$ and $S_j$, discrim_group uses the upper triangular matrices $R$ and $R_j$ supplied by discrim() such that $S=R^{T}R$ and $S_j=R_j^T{R}_j$. $D_{kj}^2$ can then be calculated as $z^{T}z$ where $R_j^{T}z=(x_k-x_j)$ or $R^{T}z=(x_k-x)$ as appropriate.

In addition to the distances, a set of prior probabilities of group membership, ${\pi }_j$, for $j=1,2,\dots ,n_g$, may be used, with $\sum {\pi }_j=1$. The prior probabilities reflect your view as to the likelihood of the observations coming from the different groups. Two common cases for prior probabilities are ${\pi }_1={\pi }_2=\cdots ={\pi }_{n_g}$, that is, equal prior probabilities, and ${\pi }_{\text{j}}=n_{\text{j}}/n$, for $\text{j}=1,2,\dots ,n_g$, that is, prior probabilities proportional to the number of observations in the groups in the training set.

discrim_group uses one of four allocation rules. In all four rules the $p$ variables are assumed to follow a multivariate Normal distribution with mean ${\mu }_j$ and variance-covariance matrix ${\Sigma }_j$ if the observation comes from the $j$th group. The different rules depend on whether or not the within-group variance-covariance matrices are assumed equal, i.e., ${\Sigma }_1={\Sigma }_2=\cdots ={\Sigma }_{n_g}$, and whether a predictive or estimative approach is used. If $p(x_k|{\mu }_j,{\Sigma }_j)$ is the probability of observing the observation $x_k$ from group $j$, then the posterior probability of belonging to group $j$ is:

$$p(j|x_k,{\mu }_j,{\Sigma }_j)\propto p(x_k|{\mu }_j,{\Sigma }_j){\pi }_j\text{.}$$

In the estimative approach, the parameters ${\mu }_j$ and ${\Sigma }_j$ in (3) are replaced by their estimates calculated from $X_t$. In the predictive approach, a non-informative prior distribution is used for the parameters and a posterior distribution for the parameters, $p({\mu }_j,{\Sigma }_j|X_t)$, is found. A predictive distribution is then obtained by integrating $p(j|x_k,{\mu }_j,{\Sigma }_j)p({\mu }_j,{\Sigma }_j|X)$ over the parameter space. This predictive distribution then replaces $p(x_k|{\mu }_j,{\Sigma }_j)$ in (3). See Aitchison and Dunsmore (1975), Aitchison et al. (1977) and Moran and Murphy (1979) for further details.

The observation is allocated to the group with the highest posterior probability. Denoting the posterior probabilities, $p(j|x_k,{\mu }_j,{\Sigma }_j)$, by $q_j$, the four allocation rules are:

1. Estimative with equal variance-covariance matrices – Linear Discrimination

   $$\log \left(q_j\right)\propto -\frac{1}{2}{D}_{kj}^2+\log \left({\pi }_j\right)$$

2. Estimative with unequal variance-covariance matrices – Quadratic Discrimination

   $$\log \left(q_j\right)\propto -\frac{1}{2}{D}_{kj}^2+\log \left({\pi }_j\right)-\frac{1}{2}\log \left(\left|S_j\right|\right)$$

3. Predictive with equal variance-covariance matrices

   $$q_j^{-1}\propto {\left((n_j+1)/n_j\right)}^{p/2}{\{1+\left[n_j/\left((n-n_g)(n_j+1)\right)\right]D_{kj}^2\}}^{(n+1-n_g)/2}$$

4. Predictive with unequal variance-covariance matrices

   $$q_j^{-1}\propto C{\left\{\left((n_j^2-1)/n_j\right)\left|S_j\right|\right\}}^{p/2}{\{1+\left(n_j/(n_j^2-1)\right)D_{kj}^2\}}^{n_j/2}\text{,}$$

   where

   $$C=\frac{\Gamma \left(\frac{1}{2}(n_j-p)\right)}{\Gamma \left(\frac{1}{2}{n}_j\right)}\text{.}$$

In the above the appropriate value of $D_{kj}^2$ from (1) and (2) is used. The values of the $q_j$ are standardized so that,

$$\sum _{j=1}^{n_g}{q}_j=1\text{.}$$

Moran and Murphy (1979) show the similarity between the predictive methods and methods based upon likelihood ratio tests.

In addition to allocating the observation to a group, discrim_group computes an atypicality index, $I_j\left(x_k\right)$. The predictive atypicality index is returned, irrespective of the value of the parameter $typ$. This represents the probability of obtaining an observation more typical of group $j$ than the observed $x_k$ (see Aitchison and Dunsmore (1975) and Aitchison et al. (1977)). The atypicality index is computed for unequal within-group variance-covariance matrices as:

$$I_j\left(x_k\right)=P(B\le z:\frac{1}{2}p,\frac{1}{2}(n_j-p))$$

where $P(B\le \beta :a,b)$ is the lower tail probability from a beta distribution and

$$z=D_{kj}^2/(D_{kj}^2+(n_j^2-1)/n_j)\text{,}$$

and for equal within-group variance-covariance matrices as:

$$I_j\left(x_k\right)=P(B\le z:\frac{1}{2}p,\frac{1}{2}(n-n_g-p+1))\text{,}$$

with

$$z=D_{kj}^2/(D_{kj}^2+(n-n_g)(n_j+1)/n_j)\text{.}$$

If $I_j\left(x_k\right)$ is close to $1$ for all groups it indicates that the observation may come from a grouping not represented in the training set. Moran and Murphy (1979) provide a frequentist interpretation of $I_j\left(x_k\right)$.

References

Aitchison, J and Dunsmore, I R, 1975, Statistical Prediction Analysis, Cambridge

Aitchison, J, Habbema, J D F and Kay, J W, 1977, A critical comparison of two methods of statistical discrimination, Appl. Statist. (26), 15–25

Kendall, M G and Stuart, A, 1976, The Advanced Theory of Statistics (Volume 3), (3rd Edition), Griffin

Krzanowski, W J, 1990, Principles of Multivariate Analysis, Oxford University Press

Moran, M A and Murphy, B J, 1979, A closer look at two alternative methods of statistical discrimination, Appl. Statist. (28), 223–232

Morrison, D F, 1967, Multivariate Statistical Methods, McGraw–Hill

See also

naginterfaces.library.examples.mv.discrim_group_ex.main()