naginterfaces.library.univar.outlier_​peirce_​1var

naginterfaces.library.univar.outlier_peirce_1var(p, y, ldiff, mean=0.0, var=0.0)

outlier_peirce_1var identifies outlying values using Peirce’s criterion.

For full information please refer to the NAG Library document for g07ga

https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/g07/g07gaf.html

Parameters

pint

$p$, the number of parameters in the model used in obtaining the $y$. If $y$ is an observed set of values, as opposed to the residuals from fitting a model with $p$ parameters, then $p$ should be set to $1$, i.e., as if a model just containing the mean had been used.

yfloat, array-like, shape $\left(n\right)$

$y$, the data being tested.

ldiffint

The maximum number of values to be returned in arrays $diff$ and $llamb$.

If $ldiff\le 0$, arrays $diff$ and $llamb$ are not referenced.

meanfloat, optional

If $var>0.0$, $mean$ must contain $\mu$, the mean of $y$, otherwise $mean$ is not referenced and the mean is calculated from the data supplied in $y$.

varfloat, optional

If $var>0.0$, $var$ must contain ${\sigma }^2$, the variance of $y$, otherwise the variance is calculated from the data supplied in $y$.

Returns

ioutint, ndarray, shape $\left(n\right)$

The indices of the values in $y$ sorted in descending order of the absolute difference from the mean, therefore, $\left|y\right[iout[\text{i}-2]-1]-\mu |\ge \left|y\right[iout[\text{i}-1]-1]-\mu |$, for $\text{i}=2,3,\dots ,n$.

nioutint

The number of potential outliers. The indices for these potential outliers are held in the first $niout$ elements of $iout$. By construction there can be at most $n-p-1$ values flagged as outliers.

difffloat, ndarray, shape $\left(ldiff\right)$

$diff[\text{i}-1]$ holds $|y-\mu |-{\sigma }^{2}z$ for observation $y\left[iout\right[\text{i}-1]-1]$, for $\text{i}=1,2,\dots ,min(ldiff,niout+1,n-p-1)$.

llambfloat, ndarray, shape $\left(ldiff\right)$

$llamb[\text{i}-1]$ holds $\log \left({\lambda }^2\right)$ for observation $y\left[iout\right[\text{i}-1]-1]$, for $\text{i}=1,2,\dots ,min(ldiff,niout+1,n-p-1)$.

Raises

NagValueError

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n\ge 3$.

(errno $2$)

On entry, $p=⟨value⟩$ and $n=⟨value⟩$.

Constraint: $1\le p\le n-2$.

Notes

outlier_peirce_1var flags outlying values in data using Peirce’s criterion. Let

$y$ denote a vector of $n$ observations (for example the residuals) obtained from a model with $p$ parameters,

$m$ denote the number of potential outlying values,

$\mu$ and ${\sigma }^2$ denote the mean and variance of $y$ respectively,

$\tilde{y}$ denote a vector of length $n-m$ constructed by dropping the $m$ values from $y$ with the largest value of $|y_i-\mu |$,

${\tilde{\sigma }}^2$ denote the (unknown) variance of $\tilde{y}$,

$\lambda$ denote the ratio of $\tilde{\sigma }$ and $\sigma$ with $\lambda =\frac{\tilde{\sigma }}{\sigma }$.

Peirce’s method flags $y_i$ as a potential outlier if $|y_i-\mu |\ge x$, where $x={\sigma }^{2}z$ and $z$ is obtained from the solution of

$$R^m={\lambda }^{m-n}\frac{m^m{(n-m)}^{n-m}}{n^n}$$

where

$$R=2exp\left(\left(\frac{z^2-1}{2}\right)(1-\Phi \left(z\right))\right)$$

and $\Phi$ is the cumulative distribution function for the standard Normal distribution.

As ${\tilde{\sigma }}^2$ is unknown an assumption is made that the relationship between ${\tilde{\sigma }}^2$ and ${\sigma }^2$, hence $\lambda$, depends only on the sum of squares of the rejected observations and the ratio estimated as

$${\lambda }^2=\frac{n-p-m{z}^2}{n-p-m}$$

which gives

$$z^2=1+\frac{n-p-m}{m}(1-{\lambda }^2)$$

A value for the cutoff $x$ is calculated iteratively. An initial value of $R=0.2$ is used and a value of $\lambda$ is estimated using equation [equation]. Equation [equation] is then used to obtain an estimate of $z$ and then equation [equation] is used to get a new estimate for $R$. This process is then repeated until the relative change in $z$ between consecutive iterations is $\le \sqrt{ϵ}$, where $ϵ$ is machine precision.

By construction, the cutoff for testing for $m+1$ potential outliers is less than the cutoff for testing for $m$ potential outliers. Therefore, Peirce’s criterion is used in sequence with the existence of a single potential outlier being investigated first. If one is found, the existence of two potential outliers is investigated etc.

If one of a duplicate series of observations is flagged as an outlier, then all of them are flagged as outliers.

References

Gould, B A, 1855, On Peirce’s criterion for the rejection of doubtful observations, with tables for facilitating its application, The Astronomical Journal (45)

Peirce, B, 1852, Criterion for the rejection of doubtful observations, The Astronomical Journal (45)