naginterfaces.library.correg.robustm_​corr_​user_​deriv

naginterfaces.library.correg.robustm_corr_user_deriv(ucv, indm, x, a, theta, bl=0.9, bd=0.9, maxit=150, nitmon=0, tol=5e-05, data=None, io_manager=None)

robustm_corr_user_deriv calculates a robust estimate of the covariance matrix for user-supplied weight functions and their derivatives.

For full information please refer to the NAG Library document for g02hl

https://support.nag.com/numeric/nl/nagdoc_30.2/flhtml/g02/g02hlf.html

Parameters

ucvcallable (u, ud, w, wd) = ucv(t, data=None)

$ucv$ must return the values of the functions $u$ and $w$ and their derivatives for a given value of its argument.

Parameters

tfloat

The argument for which the functions $u$ and $w$ must be evaluated.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns

ufloat

The value of the $u$ function at the point $t$.

udfloat

The value of the derivative of the $u$ function at the point $t$.

wfloat

The value of the $w$ function at the point $t$.

wdfloat

The value of the derivative of the $w$ function at the point $t$.

indmint

Indicates which form of the function $v$ will be used.

$indm=1$

$v=1$.

$indm\ne 1$

$v=u$.

xfloat, array-like, shape $(n,m)$

$x[\text{i}-1,\text{j}-1]$ must contain the $\text{i}$th observation on the $\text{j}$th variable, for $\text{j}=1,2,\dots ,m$, for $\text{i}=1,2,\dots ,n$.

afloat, array-like, shape $(m\times (m+1)/2)$

An initial estimate of the lower triangular real matrix $A$. Only the lower triangular elements must be given and these should be stored row-wise in the array.

The diagonal elements must be $\ne 0$, and in practice will usually be $>0$.

If the magnitudes of the columns of $X$ are of the same order, the identity matrix will often provide a suitable initial value for $A$.

If the columns of $X$ are of different magnitudes, the diagonal elements of the initial value of $A$ should be approximately inversely proportional to the magnitude of the columns of $X$.

thetafloat, array-like, shape $\left(m\right)$

An initial estimate of the location parameter, ${\theta }_{\text{j}}$, for $\text{j}=1,2,\dots ,m$.

In many cases an initial estimate of ${\theta }_{\text{j}}=0$, for $\text{j}=1,2,\dots ,m$, will be adequate.

Alternatively medians may be used as given by univar.robust_1var_median.

blfloat, optional

The magnitude of the bound for the off-diagonal elements of $S_k$, $BL$.

bdfloat, optional

The magnitude of the bound for the diagonal elements of $S_k$, $BD$.

maxitint, optional

The maximum number of iterations that will be used during the calculation of $A$.

nitmonint, optional

Indicates the amount of information on the iteration that is printed.

$nitmon>0$

The value of $A$, $\theta$ and $\delta$ (see Accuracy) will be printed at the first and every $nitmon$ iterations.

$nitmon\le 0$

No iteration monitoring is printed.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see FileObjManager).

tolfloat, optional

The relative precision for the final estimates of the covariance matrix. Iteration will stop when maximum $\delta$ (see Accuracy) is less than $tol$.

dataarbitrary, optional

User-communication data for callback functions.

io_managerFileObjManager, optional

Manager for I/O in this routine.

Returns

covfloat, ndarray, shape $(m\times (m+1)/2)$

Contains a robust estimate of the covariance matrix, $C$. The upper triangular part of the matrix $C$ is stored packed by columns (lower triangular stored by rows), $C_{ij}$ is returned in $cov[(j\times (j-1)/2+i)-1]$, $i\le j$.

afloat, ndarray, shape $(m\times (m+1)/2)$

The lower triangular elements of the inverse of the matrix $A$, stored row-wise.

wtfloat, ndarray, shape $\left(n\right)$

$wt[\text{i}-1]$ contains the weights, ${\text{wt}}_{\text{i}}=u\left({\parallel z_{\text{i}}\parallel }_2\right)$, for $\text{i}=1,2,\dots ,n$.

thetafloat, ndarray, shape $\left(m\right)$

Contains the robust estimate of the location parameter, ${\theta }_{\text{j}}$, for $\text{j}=1,2,\dots ,m$.

nitint

The number of iterations performed.

Raises

NagValueError

(errno $1$)

On entry, $n=⟨value⟩$ and $m=⟨value⟩$.

Constraint: $n\ge m$.

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n\ge 2$.

(errno $1$)

On entry, $m=⟨value⟩$.

Constraint: $m\ge 1$.

(errno $2$)

On entry, $bd=⟨value⟩$.

Constraint: $bd>0.0$.

(errno $2$)

On entry, $bl=⟨value⟩$.

Constraint: $bl>0.0$.

(errno $2$)

On entry, $maxit=⟨value⟩$.

Constraint: $maxit>0$.

(errno $2$)

On entry, $tol=⟨value⟩$.

Constraint: $tol>0.0$.

(errno $2$)

On entry, $i=⟨value⟩$ and the $i$th diagonal element of $A$ is $0$.

Constraint: all diagonal elements of $A$ must be non-zero.

(errno $3$)

On entry, a variable has a constant value, i.e., all elements in column $⟨value⟩$ of $x$ are identical.

(errno $4$)

$w$ value returned by $ucv<0.0$: $w(⟨value⟩)=⟨value⟩$.

Constraint: $w\ge 0.0$.

(errno $4$)

$\text{u}$ value returned by $ucv<0.0$: $u(⟨value⟩)=⟨value⟩$.

Constraint: $u\ge 0.0$.

(errno $5$)

Iterations to calculate weights failed to converge.

Warns

NagAlgorithmicWarning

(errno $6$)

The sum ${\text{D}}_2$ is zero. Try either a larger initial estimate of $A$ or make $u$ and $w$ less strict.

(errno $6$)

The sum ${\text{D}}_3$ is zero. Try either a larger initial estimate of $A$ or make $u$ and $w$ less strict.

(errno $6$)

The sum ${\text{D}}_1$ is zero. Try either a larger initial estimate of $A$ or make $u$ and $w$ less strict.

Notes

For a set of $n$ observations on $m$ variables in a matrix $X$, a robust estimate of the covariance matrix, $C$, and a robust estimate of location, $\theta$, are given by:

$$C={\tau }^2{\left(A^{T}A\right)}^{-1}\text{,}$$

where ${\tau }^2$ is a correction factor and $A$ is a lower triangular matrix found as the solution to the following equations.

$$z_i=A(x_i-\theta )$$

$$\frac{1}{n}\sum _{i=1}^{n}w\left({\parallel z_i\parallel }_2\right)z_i=0$$

and

$$\frac{1}{n}\sum _{i=1}^{n}u\left({\parallel z_i\parallel }_2\right)z_i{z}_i^T-v\left({\parallel z_i\parallel }_2\right)I=0\text{,}$$

where	$x_i$ is a vector of length $m$ containing the elements of the $i$th row of $X$,
	$z_i$ is a vector of length $m$,
	$I$ is the identity matrix and $0$ is the zero matrix,
and	$w$ and $u$ are suitable functions.

robustm_corr_user_deriv covers two situations:

1. $v\left(t\right)=1$ for all $t$,

2. $v\left(t\right)=u\left(t\right)$.

The robust covariance matrix may be calculated from a weighted sum of squares and cross-products matrix about $\theta$ using weights ${\text{wt}}_i=u\left(\parallel z_i\parallel \right)$. In case (1) a divisor of $n$ is used and in case (2) a divisor of ${\sum }_{i=1}^n{\text{wt}}_i$ is used. If $w(.)=\sqrt{u(.)}$, then the robust covariance matrix can be calculated by scaling each row of $X$ by $\sqrt{{\text{wt}}_i}$ and calculating an unweighted covariance matrix about $\theta$.

In order to make the estimate asymptotically unbiased under a Normal model a correction factor, ${\tau }^2$, is needed. The value of the correction factor will depend on the functions employed (see Huber (1981) and Marazzi (1987)).

robustm_corr_user_deriv finds $A$ using the iterative procedure as given by Huber.

$$A_k=(S_k+I)A_{k-1}$$

and

$${\theta }_{j_k}=\frac{b_j}{D_1}+{\theta }_{j_{k-1}}\text{,}$$

where $S_k=\left(s_{jl}\right)$, for $\text{l}=1,2,\dots ,m$, for $\text{j}=1,2,\dots ,m$, is a lower triangular matrix such that:

$$\begin{array}{c}{s}_{jl}=\{\begin{array}{cc}-min[max(h_{jl}/D_3,-\text{BL}),\text{BL}]\text{,}& j>l\\ & \\ -min[max(\left(h_{jj}/(2D_3-D_4/D_2)\right),-\text{BD}),\text{BD}]\text{,}& j=l\end{array}\text{,}\end{array}$$

where

$D_1={\sum }_{i=1}^n\{w\left({\parallel z_i\parallel }_2\right)+\frac{1}{m}{w}^{\prime }\left({\parallel z_i\parallel }_2\right){\parallel z_i\parallel }_2\}$

$D_2={\sum }_{i=1}^n\{\frac{1}{p}(u^{\prime }\left({\parallel z_i\parallel }_2\right){\parallel z_i\parallel }_2+2u\left({\parallel z_i\parallel }_2\right)){\parallel z_i\parallel }_2-v^{\prime }\left({\parallel z_i\parallel }_2\right)\}{\parallel z_i\parallel }_2$

$D_3=\frac{1}{m+2}{\sum }_{i=1}^n\{\frac{1}{m}(u^{\prime }\left({\parallel z_i\parallel }_2\right){\parallel z_i\parallel }_2+2u\left({\parallel z_i\parallel }_2\right))+u\left({\parallel z_i\parallel }_2\right)\}{\parallel z_i\parallel }_2^2$

$D_4={\sum }_{i=1}^n\{\frac{1}{m}u\left({\parallel z_i\parallel }_2\right){\parallel z_i\parallel }_2^2-v\left({\parallel z_i\parallel }_2^2\right)\}$

$h_{jl}={\sum }_{i=1}^{n}u\left({\parallel z_i\parallel }_2\right)z_{ij}{z}_{il}$, for $j>l$

$h_{jj}={\sum }_{i=1}^{n}u\left({\parallel z_i\parallel }_2\right)(z_{ij}^2-{\Vert z_{ij}\Vert }_2^2/m)$

$b_j={\sum }_{i=1}^{n}w\left({\parallel z_i\parallel }_2\right)(x_{ij}-b_j)$

and $\text{BD}$ and $\text{BL}$ are suitable bounds.

robustm_corr_user_deriv is based on routines in ROBETH; see Marazzi (1987).

References

Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne