naginterfaces.library.univar.robust_​2var_​ci

naginterfaces.library.univar.robust_2var_ci(method, x, y, clevel)

robust_2var_ci calculates a rank based (nonparametric) estimate and confidence interval for the difference in location between two independent populations.

For full information please refer to the NAG Library document for g07eb

https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/g07/g07ebf.html

Parameters

methodstr, length 1

Specifies the method to be used.

$method=\text{'E'}$

The exact algorithm is used.

$method=\text{'A'}$

The iterative algorithm is used.

xfloat, array-like, shape $\left(n\right)$

The observations of the first sample, $x_{\text{i}}$, for $\text{i}=1,2,\dots ,n$.

yfloat, array-like, shape $\left(m\right)$

The observations of the second sample, $y_{\text{j}}$, for $\text{j}=1,2,\dots ,m$.

clevelfloat

The confidence interval required, $1-\alpha$; e.g., for a $95\%$ confidence interval set $clevel=0.95$.

Returns

thetafloat

The estimate of the difference in the location of the two populations, $\hat{\theta }$.

thetalfloat

The estimate of the lower limit of the confidence interval, ${\theta }_l$.

thetaufloat

The estimate of the upper limit of the confidence interval, ${\theta }_u$.

estclfloat

An estimate of the actual percentage confidence of the interval found, as a proportion between $(0.0,1.0)$.

ulowerfloat

The value of the Mann–Whitney $U$ statistic corresponding to the lower confidence limit, $U_l$.

uupperfloat

The value of the Mann–Whitney $U$ statistic corresponding to the upper confidence limit, $U_u$.

Raises

NagValueError

(errno $1$)

On entry, $m=⟨value⟩$.

Constraint: $m\ge 1$.

(errno $1$)

On entry, $clevel=⟨value⟩$.

Constraint: $0.0<clevel<1.0$.

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n\ge 1$.

(errno $1$)

On entry, $method=⟨value⟩$.

Constraint: $method=\text{'E'}$ or $\text{'A'}$.

Warns

NagAlgorithmicWarning

(errno $2$)

Not enough information to compute an interval estimate since each sample has identical values. The common difference is returned in $theta$, $thetal$ and $thetau$.

(errno $3$)

The iterative procedure used to estimate $\theta$ has not converged.

(errno $3$)

The iterative procedure used to estimate, ${\theta }_u$, the upper confidence limit has not converged.

(errno $3$)

The iterative procedure used to estimate, ${\theta }_l$, the lower confidence limit has not converged.

Notes

Consider two random samples from two populations which have the same continuous distribution except for a shift in the location. Let the random sample, $x={(x_1,x_2,\dots ,x_n)}^T$, have distribution $F\left(x\right)$ and the random sample, $y={(y_1,y_2,\dots ,y_m)}^T$, have distribution $F(x-\theta )$.

robust_2var_ci finds a point estimate, $\hat{\theta }$, of the difference in location $\theta$ together with an associated confidence interval. The estimates are based on the ordered differences $y_j-x_i$. The estimate $\hat{\theta }$ is defined by

$$\hat{\theta }=median\{y_j-x_i\text{,}i=1,2,\dots ,n\text{;}j=1,2,\dots ,m\}\text{.}$$

Let $d_{\text{k}}$, for $\text{k}=1,2,\dots ,nm$, denote the $nm$ (ascendingly) ordered differences $y_{\text{j}}-x_{\text{i}}$, for $\text{j}=1,2,\dots ,m$, for $\text{i}=1,2,\dots ,n$. Then

if $nm$ is odd, $\hat{\theta }=d_k$ where $k=(nm-1)/2$;

if $nm$ is even, $\hat{\theta }=(d_k+d_{k+1})/2$ where $k=nm/2$.

This estimator arises from inverting the two sample Mann–Whitney rank test statistic, $U\left({\theta }_0\right)$, for testing the hypothesis that $\theta ={\theta }_0$. Thus $U\left({\theta }_0\right)$ is the value of the Mann–Whitney $U$ statistic for the two independent samples $\{(x_i+{\theta }_0)\text{, for}i=1,2,\dots ,n\}$ and $\{y_j\text{, for}j=1,2,\dots ,m\}$. Effectively $U\left({\theta }_0\right)$ is a monotonically increasing step function of ${\theta }_0$ with

$$\begin{array}{c}\begin{array}{c}\text{mean}\left(U\right)=\mu =\frac{nm}{2}\text{,}\\ \\ var\left(U\right)={\sigma }^2\frac{nm(n+m+1)}{12}\text{.}\end{array}\end{array}$$

The estimate $\hat{\theta }$ is the solution to the equation $U\left(\hat{\theta }\right)=\mu$; two methods are available for solving this equation. These methods avoid the computation of all the ordered differences $d_k$; this is because for large $n$ and $m$ both the storage requirements and the computation time would be high.

The first is an exact method based on a set partitioning procedure on the set of all differences $y_{\text{j}}-x_{\text{i}}$, for $\text{j}=1,2,\dots ,m$, for $\text{i}=1,2,\dots ,n$. This is adapted from the algorithm proposed by Monahan (1984) for the computation of the Hodges–Lehmann estimator for a single population.

The second is an iterative algorithm, based on the Illinois method which is a modification of the regula falsi method, see McKean and Ryan (1977). This algorithm has proved suitable for the function $U\left({\theta }_0\right)$ which is asymptotically linear as a function of ${\theta }_0$.

The confidence interval limits are also based on the inversion of the Mann–Whitney test statistic.

Given a desired percentage for the confidence interval, $1-\alpha$, expressed as a proportion between $0.0$ and $1.0$ initial estimates of the upper and lower confidence limits for the Mann–Whitney $U$ statistic are found;

$$\begin{array}{c}\begin{array}{c}{U}_l=\mu -0.5+(\sigma \times {\Phi }^{-1}\left(\alpha /2\right))\\ \\ U_u=\mu +0.5+(\sigma \times {\Phi }^{-1}\left((1-\alpha )/2\right))\end{array}\end{array}$$

where ${\Phi }^{-1}$ is the inverse cumulative Normal distribution function.

$U_l$ and $U_u$ are rounded to the nearest integer values. These estimates are refined using an exact method, without taking ties into account, if $n+m\le 40$ and $max(n,m)\le 30$ and a Normal approximation otherwise, to find $U_l$ and $U_u$ satisfying

$$\begin{array}{c}\begin{array}{c}P(U\le U_l)\le \alpha /2\\ P(U\le U_l+1)>\alpha /2\end{array}\end{array}$$

and

$$\begin{array}{c}\begin{array}{c}P(U\ge U_u)\le \alpha /2\\ P(U\ge U_u-1)>\alpha /2\text{.}\end{array}\end{array}$$

The function $U\left({\theta }_0\right)$ is a monotonically increasing step function. It is the number of times a score in the second sample, $y_j$, precedes a score in the first sample, $x_i+\theta$, where we only count a half if a score in the second sample actually equals a score in the first.

Let $U_l=k$; then ${\theta }_l=d_{k+1}$. This is the largest value ${\theta }_l$ such that $U\left({\theta }_l\right)=U_l$.

Let $U_u=nm-k$; then ${\theta }_u=d_{nm-k}$. This is the smallest value ${\theta }_u$ such that $U\left({\theta }_u\right)=U_u$.

As in the case of $\hat{\theta }$, these equations may be solved using either the exact or iterative methods to find the values ${\theta }_l$ and ${\theta }_u$.

Then $({\theta }_l,{\theta }_u)$ is the confidence interval for $\theta$. The confidence interval is thus defined by those values of ${\theta }_0$ such that the null hypothesis, $\theta ={\theta }_0$, is not rejected by the Mann–Whitney two sample rank test at the $(100\times \alpha )\%$ level.

References

Lehmann, E L, 1975, Nonparametrics: Statistical Methods Based on Ranks, Holden–Day

McKean, J W and Ryan, T A, 1977, Algorithm 516: An algorithm for obtaining confidence intervals and point estimates based on ranks in the two-sample location problem, ACM Trans. Math. Software (10), 183–185

Monahan, J F, 1984, Algorithm 616: Fast computation of the Hodges–Lehman location estimator, ACM Trans. Math. Software (10), 265–270