nag_robust_m_regsn_estim (g02hac) (PDF version)
g02 Chapter Contents
g02 Chapter Introduction
NAG Library Manual

NAG Library Function Document

nag_robust_m_regsn_estim (g02hac)

 Contents

    1  Purpose
    7  Accuracy

1  Purpose

nag_robust_m_regsn_estim (g02hac) performs bounded influence regression (M-estimates). Several standard methods are available.

2  Specification

#include <nag.h>
#include <nagg02.h>
void  nag_robust_m_regsn_estim (Nag_RegType regtype, Nag_PsiFun psifun, Nag_SigmaEst sigma_est, Nag_CovMatrixEst covmat_est, Integer n, Integer m, double x[], Integer tdx, double y[], double cpsi, const double hpsi[], double cucv, double dchi, double theta[], double *sigma, double c[], Integer tdc, double rs[], double wt[], double tol, Integer max_iter, Integer print_iter, const char *outfile, double info[], NagError *fail)

3  Description

For the linear regression model
y = X θ + ε  
where y  is a vector of length n  of the dependent variable,
X  is a n  by m  matrix of independent variables of column rank k ,
θ  is a vector of length m  of unknown arguments,
and ε  is a vector of length n  of unknown errors with var ε i = σ 2 :
nag_robust_m_regsn_estim (g02hac) calculates the M-estimates given by the solution, θ ^ , to the equation
i=1 n ψ r i / σ w i w i x ij = 0 ,   j = 1 , 2 , , m (1)
where r i  is the i th residual, i.e., the i th element of r = y - X θ ^ ,
ψ  is a suitable weight function,
w i  are suitable weights,
and σ  may be estimated at each iteration by the median absolute deviation of the residuals:
σ ^ = med i r i / β 1
or as the solution to:
i=1 n χ r i / σ ^ w i w i 2 = n-k β 2  
for suitable weight function χ , where β 1  and β 2  are constants, chosen so that the estimator of σ  is asymptotically unbiased if the errors, ε i , have a Normal distribution. Alternatively σ  may be held at a constant value.
The above describes the Schweppe type regression. If the w i  are assumed to equal 1 for all i  then Huber type regression is obtained. A third type, due to Mallows, replaces (1) by
i=1 n ψ r i / σ w i x ij = 0 ,   j = 1 , 2 , , m .  
This may be obtained by use of the transformations
(see Marazzi (1987a)).
For Huber and Schweppe type regressions, β 1  is the 75th percentile of the standard Normal distribution. For Mallows type regression β 1  is the solution to
1 n i=1 n Φ β 1 / w i = . 75  
where Φ  is the standard Normal cumulative distribution function.
β 2  is given by:
β 2 = - χ z ϕ z dz , in Huber case; β 2 = 1 n i=1 n w i - χ z ϕ z dz , in Mallows case; β 2 = 1 n i=1 n w i 2 - χ z/ w i ϕ z dz , in Schweppe case;  
where ϕ  is the standard Normal density, i.e.,
1 2π exp - 1 2 x 2 .  
The calculation of the estimates of θ  can be formulated as an iterative weighted least squares problem with a diagonal weight matrix G  given by
G ii = ψ r i / σ w i r i / σ w i , r i 0 ψ 0 , r i = 0  
where ψ t  is the derivative of ψ  at the point t .
The value of θ  at each iteration is given by the weighted least squares regression of y  on X . This is carried out by first transforming the y  and X  by
and then obtaining the solution of the resulting least squares problem. If X  is of full column rank then an orthogonal-triangular (QR) decomposition is used, if not, a singular value decomposition is used.
The following functions are available for ψ  and χ  in nag_robust_m_regsn_estim (g02hac).
(a) Unit Weights
ψ t = t , χ t = t 2 2  
this gives least squares regression.
(b) Huber's Function
ψ t = max-c,minc,t , χ t = t 2 2 , t d d 2 2 , t > d  
(c) Hampel's Piecewise Linear Function
ψ h 1 , h 2 , h 3 t = - ψ h 1 , h 2 , h 3 -t = t , 0 t h 1 h 1 , h 1 t h 2 h 1 h 3 - t / h 3 - h 2 , h 2 t h 3 0 , h 3 < t .  
χ t = t 2 2 , t d d 2 2 , t > d  
(d) Andrew's Sine Wave Function
ψ t = sint , -π t π 0 , t > π χ t = t 2 2 , t d d 2 2 , t > d  
(e) Tukey's Bi-weight
ψ t = t 1 - t 2 2 , t 1 0 , t > 1 χ t = t 2 2 , t d d 2 2 , t > d  
where c , h 1 , h 2 , h 3 , and d  are given constants.
Several schemes for calculating weights have been proposed, see Hampel et al. (1986) and Marazzi (1987a). As the different independent variables may be measured on different scales, one group of proposed weights aims to bound a standardized measure of influence. To obtain such weights the matrix A  has to be found such that:
1 n i=1 n u z i 2 z i ziT = I  
and
z i = Ax i  
where x i  is a vector of length m  containing the i th row of X ,
A  is a m  by m  lower triangular matrix,
and u  is a suitable function.
The weights are then calculated as
w i = f z i 2  
for a suitable function f .
nag_robust_m_regsn_estim (g02hac) finds A  using the iterative procedure
A k = S k + I A k-1  
where S k = s jl ,
s jl = - minmax h jl / n ,-BL,BL j > - minmax 1 2 h jj /n-1 ,-BD,BD j = .  
and
h jl = i=1 n u z i 2 z ij z il  
and BL  and BD  are bounds set at 0.9.
Two weights are available in nag_robust_m_regsn_estim (g02hac):
(i) Krasker–Welsch weights
u t = g 1 c t  
where g 1 t = t 2 + 1 - t 2 2 Φ t - 1 - 2 t ϕ t , Φ t  is the standard Normal cumulative distribution function, ϕ t  is the standard Normal probability density function, and f t = 1 t .
These are for use with Schweppe type regression.
(ii) Maronna's proposed weights
u t = c / t 2 t > c 1 t c  
f t = u t .  
These are for use with Mallows type regression.
Finally the asymptotic variance-covariance matrix, C , of the estimates θ  is calculated.
For Huber type regression
C = f H XT X -1 σ ^ 2  
where
f H = 1 n-m i=1 n ψ 2 r i / σ ^ 1 n i=1 n ψ r i σ ^ 2 κ 2  
κ 2 = 1 + m n 1 n i=1 n ψ r i / σ ^ - 1 n i=1 n ψ r i / σ ^ 2 1 n i=1 n ψ r i σ ^ 2  
See Huber (1981) and Marazzi (1987b).
For Mallows and Schweppe type regressions C  is of the form
σ ^ n 2 S 1 -1 S 2 S 1 -1  
where S 1 = 1 n XT DX  and S 2 = 1 n XT PX .
D  is a diagonal matrix such that the i th element approximates E ψ r i / σ w i  in the Schweppe case and E ψ r i / σ w i  in the Mallows case.
P  is a diagonal matrix such that the i th element approximates E ψ 2 r i / σ w i w i 2  in the Schweppe case and E ψ 2 r i / σ w i 2  in the Mallows case.
Two approximations are available in nag_robust_m_regsn_estim (g02hac):
1. Average over the r i  
Schweppe Mallows D i = 1 n j=1 n ψ r j σ ^ w i w i D i = 1 n j=1 n ψ r j σ ^ w i  
2. Replace expected value by observed
Schweppe Mallows D i = ψ r i σ ^ w i w i D i = ψ r i σ ^ w i P i = ψ 2 r i σ ^ w i w i 2 P i = ψ 2 r i σ ^ w i 2  
See Hampel et al. (1986) and Marazzi (1987b).
Note: there is no explicit provision in the function for a constant term in the regression model. However, the addition of a dummy variable whose value is 1.0 for all observations will produce a value of θ ^  corresponding to the usual constant term.
nag_robust_m_regsn_estim (g02hac) is based on routines in ROBETH, see Marazzi (1987a).

4  References

Hampel F R, Ronchetti E M, Rousseeuw P J and Stahel W A (1986) Robust Statistics. The Approach Based on Influence Functions Wiley
Huber P J (1981) Robust Statistics Wiley
Marazzi A (1987a) Weights for bounded influence regression in ROBETH Cah. Rech. Doc. IUMSP, No. 3 ROB 3 Institut Universitaire de Médecine Sociale et Préventive, Lausanne
Marazzi A (1987b) Subroutines for robust and bounded influence regression in ROBETH Cah. Rech. Doc. IUMSP, No. 3 ROB 2 Institut Universitaire de Médecine Sociale et Préventive, Lausanne

5  Arguments

1:     regtype Nag_RegTypeInput
On entry: specifies the type of regression to be performed.
regtype=Nag_HuberReg
Huber type regression.
regtype=Nag_MallowsReg
Mallows type regression with Maronna's proposed weights.
regtype=Nag_SchweppeReg
Schweppe type regression with Krasker–Welsch weights.
Constraint: regtype=Nag_HuberReg, Nag_MallowsReg or Nag_SchweppeReg.
2:     psifun Nag_PsiFunInput
On entry: specifies which ψ  function is to be used.
psifun=Nag_Lsq
ψ t = t , i.e., least squares.
psifun=Nag_HuberFun
Huber's function.
psifun=Nag_HampelFun
Hampel's piecewise linear function.
psifun=Nag_AndrewFun
Andrew's sine wave.
psifun=Nag_TukeyFun
Tukey's bi-weight.
Constraint: psifun=Nag_Lsq, Nag_HuberFun, Nag_HampelFun, Nag_AndrewFun or Nag_TukeyFun.
3:     sigma_est Nag_SigmaEstInput
On entry: specifies how σ  is to be estimated.
sigma_est=Nag_SigmaRes
σ  is estimated by median absolute deviation of residuals.
sigma_est=Nag_SigmaConst
σ  is held constant at its initial value.
sigma_est=Nag_SigmaChi
σ  is estimated using the χ  function.
Constraint: sigma_est=Nag_SigmaRes, Nag_SigmaConst or Nag_SigmaChi.
4:     covmat_est Nag_CovMatrixEstInput
On entry: if regtypeNag_HuberReg, covmat_est specifies the approximations used in estimating the covariance matrix of θ ^ . covmat_est=Nag_CovMatAve, averaging over residuals. covmat_est=Nag_CovMatObs, replacing expected by observed.
If regtype=Nag_HuberReg then covmat_est is not referenced.
Constraint: covmat_est=Nag_CovMatAve or Nag_CovMatObs.
5:     n IntegerInput
On entry: the number of observations, n .
Constraint: n>1 .
6:     m IntegerInput
On entry: the number m , of independent variables.
Constraint: 1 m < n .
7:     x[n×tdx] doubleInput/Output
Note: the i,jth element of the matrix X is stored in x[i-1×tdx+j-1].
On entry: the values of the X  matrix, i.e., the independent variables. x i-1 j-1  must contain the ij th element of X , for i=1,2,,n and j=1,2,,m.
On exit: if regtype=Nag_MallowsReg, then during calculations the elements of x will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input x and the output x. Otherwise x is unchanged.
8:     tdx IntegerInput
On entry: the stride separating matrix column elements in the array x.
Constraint: tdxm .
9:     y[n] doubleInput/Output
On entry: the data values of the dependent variable. y[i-1]  must contain the value of y  for the i th observation, for i=1,2,,n.
On exit: if regtype=Nag_MallowsReg, then during calculations the elements of y will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input y and the output y. Otherwise y is unchanged.
10:   cpsi doubleInput
On entry: if psifun=Nag_HuberFun, cpsi must specify the argument, c , of Huber's ψ  function. Otherwise cpsi is not referenced.
Constraint: if psifun=Nag_HuberFun then cpsi>0.0 .
11:   hpsi[3] const doubleInput
On entry: if psifun=Nag_HampelFun then hpsi[0] , hpsi[1]  and hpsi[2]  must specify the arguments h 1 , h 2 , and h 3 , of Hampel's piecewise linear ψ  function. Otherwise the elements of hpsi are not referenced.
Constraint: if psifun=Nag_HampelFun, 0.0 hpsi[0] hpsi[1] hpsi[2]  and hpsi[2] > 0.0 .
12:   cucv doubleInput
On entry: if regtype=Nag_MallowsReg then cucv must specify the value of the constant, c , of the function u  for Maronna's proposed weights.
If regtype=Nag_SchweppeReg then cucv must specify the value of the function u  for the Krasker–Welsch weights.
If regtype=Nag_HuberReg then cucv is not referenced.
Constraints:
  • if regtype=Nag_MallowsReg, cucvm ;
  • if regtype=Nag_SchweppeReg, cucv m .
13:   dchi doubleInput
On entry: the constant, d , of the χ  function.
dchi is referenced only if psifunNag_Lsq and sigma_est=Nag_SigmaChi.
Constraint: if psifunNag_Lsq and sigma_est=Nag_SigmaChi, dchi>0.0 .
14:   theta[m] doubleInput/Output
On entry: starting values of the argument vector θ . These may be obtained from least squares regression.
Alternatively if sigma_est=Nag_SigmaRes and sigma=1  or if sigma_est=Nag_SigmaChi and sigma approximately equals the standard deviation of the dependent variable, y , then theta[i-1] = 0.0 , for i=1,2,,m may provide reasonable starting values.
On exit: theta[i-1]  contains the M-estimate of θ i , for i=1,2,,m.
15:   sigma double *Input/Output
On entry: a starting value for the estimation of σ .
sigma should be approximately the standard deviation of the residuals from the model evaluated at the value of θ  given by theta on entry.
On exit: sigma contains the final estimate of σ , unless sigma_est=Nag_SigmaConst.
Constraint: sigma>0.0 .
16:   c[m×tdc] doubleOutput
On exit: the diagonal elements of c contain the estimated asymptotic standard errors of the estimates of θ , i.e., c[i-1×tdc+i-1]  contains the estimated asymptotic standard error of the estimate contained in theta[i-1] , for i=1,2,,m.
The elements above the diagonal contain the estimated asymptotic correlation between the estimates of θ , i.e., c[i-1×tdc+j-1] , 1 i < j m  contains the asymptotic correlation between the estimates contained in theta[i-1]  and theta[j-1] .
The elements below the diagonal contain the estimated asymptotic covariance between the estimates of θ , i.e., c[i-1×tdc+j-1] , 1 j < i m  contains the estimated asymptotic covariance between the estimates contained in theta[i-1]  and theta[j-1] .
17:   tdc IntegerInput
On entry: the stride separating matrix column elements in the array c.
Constraint: tdcm .
18:   rs[n] doubleOutput
On exit: contains the residuals from the model evaluated at final value of theta, i.e., rs[i-1] , for i=1,2,,n, contains the vector y-X θ ^ .
19:   wt[n] doubleOutput
On exit: contains the vector of weights. wt[i-1]  contains the weight for the i th observation, for i=1,2,,n.
20:   tol doubleInput
On entry: the relative precision for the calculation of A  (if regtypeNag_HuberReg), the estimates of θ  and the estimate of σ  (if sigma_estNag_SigmaConst). Convergence is assumed when the relative change in all elements being considered is less than tol.
If regtype=Nag_MallowsReg and sigma_est=Nag_SigmaRes, tol is also used to determine the precision of β 1 .
It is advisable for tol to be greater than 100 ×  machine precision.
Constraint: tol>0.0 .
21:   max_iter IntegerInput
On entry: the maximum number of iterations that should be used in the calculation of A  (if regtypeNag_HuberReg), and of the estimates of θ  and σ , and of β 1  (if regtype=Nag_MallowsReg and sigma_est=Nag_SigmaRes)
Suggested value: A value of max_iter=50  should be adequate for most uses.
Constraint: max_iter>0 .
On entry: the amount of information that is printed on each iteration.
print_iter=0
No information is printed.
print_iter0
The current estimate of θ , the change in θ  during the current iteration and the current value of σ  are printed on the first and every absprint_iter  iterations.
Also, if regtypeNag_HuberReg and print_iter>0  then information on the iterations to calculate A  is printed. This is the current estimate of A  and the maximum value of S ij  (see Section 3).
23:   outfile const char *Input
On entry: a null terminated character string giving the name of the file to which results should be printed. If outfile is NULL or an empty string then the stdout stream is used. Note that the file will be opened in the append mode.
24:   info[4] doubleOutput
On exit: elements of info contain the following values:
  • info[0] = β 1  if sigma_est=Nag_SigmaRes,
  • or info[0] = β 2  if sigma_est=Nag_SigmaChi,
  • info[1] =  number of iterations used to calculate A .
  • info[2] =  number of iterations used to calculate final estimates of θ  and σ .
  • info[3] = k , the rank of the weighted least squares equations.
25:   fail NagError *Input/Output
The NAG error argument (see Section 2.7 in How to Use the NAG Library and its Documentation).

6  Error Indicators and Warnings

NE_2_INT_ARG_GE
On entry, m=value  while n=value . These arguments must satisfy m<n .
NE_2_INT_ARG_LT
On entry, tdc=value  while m=value . These arguments must satisfy tdcm .
On entry, tdx=value  while m=value . These arguments must satisfy tdxm .
NE_ALLOC_FAIL
Dynamic memory allocation failed.
NE_BAD_HAMPEL_PSI_FUN
On entry, psifun=Nag_HampelFun and hpsi[0] = value, hpsi[1] = value and hpsi[2] = value. For this value of psifun, the elements of hpsi must satisfy the condition 0.0 hpsi[0] hpsi[1] hpsi[2]  and hpsi[2] > 0.0 .
NE_BAD_PARAM
On entry, argument covmat_est had an illegal value.
On entry, argument psifun had an illegal value.
On entry, argument regtype had an illegal value.
On entry, argument sigma_est had an illegal value.
NE_BETA1_ITER_EXCEEDED
The number of iterations required to calculate β 1  exceeds max_iter. This is only applicable if regtype=Nag_MallowsReg and sigma_est=Nag_SigmaRes.
NE_COV_MAT_FACTOR_ZERO
In calculating the correlation factor for the asymptotic variance-covariance matrix, the factor for covariance matrix =0 .
For this error, either the value of
1 n i=1 n ψ r i / σ ^ = 0 ,  
or κ=0 ,
or i=1 n ψ 2 r i / σ ^ = 0 .
See Section 9. In this case c is returned as XT X -1 .
(This is only applicable if regtype=Nag_HuberReg).
NE_ERR_DOF_LEQ_ZERO
n=value , rank of x=value . The degrees of freedom for error, n -  (rank of x) must be > 0.0 .
NE_ESTIM_SIGMA_ZERO
The estimated value of σ  was 0.0  during an iteration.
NE_INT_ARG_LE
On entry, max_iter must not be less than or equal to 0: max_iter=value .
NE_INT_ARG_LT
On entry, m=value.
Constraint: m1.
On entry, n=value.
Constraint: n2.
NE_INTERNAL_ERROR
An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
NE_INVALID_DCHI_FUN
On entry, psifunNag_Lsq, sigma_est=Nag_SigmaChi and dchi=value . For these values of psifun and sigma_est, dchi must be > 0.0 .
NE_INVALID_HUBER_FUN
On entry, psifun=Nag_HuberFun and cpsi=value . For this value of psifun, cpsi must be > 0.0 .
NE_INVALID_MALLOWS_REG_C
On entry, regtype=Nag_MallowsReg, cucv=value and m=value . For this value of regtype, cucv must be m .
NE_INVALID_SCHWEPPE_REG_C
On entry, regtype=Nag_SchweppeReg, cucv=value and m=value . For this value of regtype, cucv must be m .
NE_LSQ_FAIL_CONV
The iterations to solve the weighted least squares equations failed to converge.
NE_NOT_APPEND_FILE
Cannot open file string  for appending.
NE_NOT_CLOSE_FILE
Cannot close file string .
NE_REAL_ARG_LE
On entry, sigma must not be less than or equal to 0.0: sigma=value .
On entry, tol must not be less than or equal to 0.0: tol=value .
NE_REG_MAT_SINGULAR
Failure to invert matrix while calculating covariance.
If regtype=Nag_HuberReg, then XT X  is almost singular.
If regtypeNag_HuberReg, then S 1  is singular or almost singular. This may be due to too many diagonal elements of the matrix being zero, see Section 9.
NE_THETA_ITER_EXCEEDED
The number of iterations required to calculate θ  and σ  exceeds max_iter. In this case, info[2] = max_iter  on exit.
NE_VAR_THETA_LEQ_ZERO
The estimated variance for an element of θ0 . In this case the diagonal element of c will contain the negative variance and the above diagonal elements in the row and the column corresponding to the element will be returned as zero.
This error may be caused by rounding errors or too many of the diagonal elements of p being zero. See Section 9.
NE_WT_ITER_EXCEEDED
The number of iterations required to calculate the weights exceeds max_iter. This is only applicable if regtypeNag_HuberReg.
NE_WT_LSQ_NOT_FULL_RANK
The weighted least squares equations are not of full rank.

7  Accuracy

The precision of the estimates is determined by tol, see Section 5. As a more stable method is used to calculate the estimates of θ  than is used to calculate the covariance matrix, it is possible for the least squares equations to be of full rank but the XT X  matrix to be too nearly singular to be inverted.

8  Parallelism and Performance

nag_robust_m_regsn_estim (g02hac) is not threaded in any implementation.

9  Further Comments

In cases when sigma_estNag_SigmaRes it is important for the value of sigma to be of a reasonable magnitude. Too small a value may cause too many of the winsorized residuals, i.e., ψ r i / σ  to be zero or a value of ψ r i / σ , used to estimate the asymptotic covariance matrix, to be zero. This can lead to errors with fail set to one of the following values:

10  Example

The number of observations and the number of x  variables are read in followed by the data. The option arguments are then read in (in this case giving: Schweppe type regression with Hampel's ψ  function and Huber's χ  function and then using the ‘replace expected by observed’ option in calculating the covariances). Finally a set of values for the constants are read in. After a call to nag_robust_m_regsn_estim (g02hac), θ ^ , its standard error and σ ^  are printed. In addition the weight and residual for each observation is printed.

10.1  Program Text

Program Text (g02hace.c)

10.2  Program Data

Program Data (g02hace.d)

10.3  Program Results

Program Results (g02hace.r)


nag_robust_m_regsn_estim (g02hac) (PDF version)
g02 Chapter Contents
g02 Chapter Introduction
NAG Library Manual

© The Numerical Algorithms Group Ltd, Oxford, UK. 2016