NAG Library Manual, Mark 28.6

Interfaces: FL CL CPP AD PY MB

NAG FL Interface Introduction

G02 (Correg) Chapter Contents

G02 (Correg) Chapter Introduction

g02ef: FL CL CPP AD PY MB

NAG FL Interface
g02eff (linregm_fit_stepwise)

Keyword Search:

NAG Library Manual, Mark 28.6

Interfaces: FL CL CPP AD PY MB

NAG FL Interface Introduction

G02 (Correg) Chapter Contents

G02 (Correg) Chapter Introduction

g02ef: FL CL CPP AD PY MB

▸▿ Contents

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

8 Parallelism and Performance

9 Further Comments

▸▿ 10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

© The Numerical Algorithms Group Ltd. 2022

1 Purpose

g02eff calculates a full stepwise selection from

p

variables by using Clarke's sweep algorithm on the correlation matrix of a design and data matrix,

Z

. The (weighted) variance-covariance, (weighted) means and sum of weights of

Z

must be supplied.

2 Specification

Fortran Interface

Subroutine g02eff (

m, n, wmean, c, sw, isx, fin, fout, tau, b, se, rsq, rms, df, monlev, monfun, iuser, ruser, ifail)

Integer, Intent (In)	::	m, n, monlev
Integer, Intent (Inout)	::	isx(m), iuser(*), ifail
Integer, Intent (Out)	::	df
Real (Kind=nag_wp), Intent (In)	::	wmean(m+1), c((m+1)*(m+2)/2), sw, fin, fout, tau
Real (Kind=nag_wp), Intent (Inout)	::	ruser(*)
Real (Kind=nag_wp), Intent (Out)	::	b(m+1), se(m+1), rsq, rms
External	::	monfun

C Header Interface

#include <nag.h>

void

g02eff_ (const Integer *m, const Integer *n, const double wmean[], const double c[], const double *sw, Integer isx[], const double *fin, const double *fout, const double *tau, double b[], double se[], double *rsq, double *rms, Integer *df, const Integer *monlev,
void (NAG_CALL *monfun)(const char *flag, const Integer *var, const double *val, Integer iuser[], double ruser[], const Charlen length_flag),
Integer iuser[], double ruser[], Integer *ifail)

The routine may be called by the names g02eff or nagf_correg_linregm_fit_stepwise.

3 Description

The general multiple linear regression model is defined by

y = β_{0} + X β + ε,

where

$y$ is a vector of $n$ observations on the dependent variable,
$β_{0}$ is an intercept coefficient,
$X$ is an $n \times p$ matrix of $p$ explanatory variables,
$β$ is a vector of $p$ unknown coefficients, and
$ε$ is a vector of length $n$ of unknown, Normally distributed, random errors.

g02eff employs a full stepwise regression to select a subset of explanatory variables from the

p

available variables (the intercept is included in the model) and computes regression coefficients and their standard errors, and various other statistical quantities, by minimizing the sum of squares of residuals. The method applies repeatedly a forward selection step followed by a backward elimination step and halts when neither step updates the current model.

The criterion used to update a current model is the variance ratio of residual sum of squares. Let

s_{1}

and

s_{2}

be the residual sum of squares of the current model and this model after undergoing a single update, with degrees of freedom

q_{1}

and

q_{2}

, respectively. Then the condition:

\frac{(s_{2} - s_{1}) / (q_{2} - q_{1})}{s_{1} / q_{1}} > f_{1},

must be satisfied if a variable

k

will be considered for entry to the current model, and the condition:

\frac{(s_{1} - s_{2}) / (q_{1} - q_{2})}{s_{1} / q_{1}} < f_{2},

must be satisfied if a variable

k

will be considered for removal from the current model, where

f_{1}

and

f_{2}

are user-supplied values and

f_{2} \leq f_{1}

.

In the entry step the entry statistic is computed for each variable not in the current model. If no variable is associated with a test value that exceeds

f_{1}

then this step is terminated; otherwise the variable associated with the largest value for the entry statistic is entered into the model.

In the removal step the removal statistic is computed for each variable in the current model. If no variable is associated with a test value less than

f_{2}

then this step is terminated; otherwise the variable associated with the smallest value for the removal statistic is removed from the model.

The data values

X

and

y

are not provided as input to the routine. Instead, summary statistics of the design and data matrix

Z = (X ∣ y)

are required.

Explanatory variables are entered into and removed from the current model by using sweep operations on the correlation matrix

R

of

Z

, given by:

R = (\begin{matrix} 1 & \dots & r_{1 p} & r_{1 y} \\ ⋮ & ⋱ & ⋮ & ⋮ \\ r_{p 1} & \dots & 1 & r_{p y} \\ r_{y 1} & \dots & r_{y p} & 1 \end{matrix}),

where

r_{i j}

is the correlation between the explanatory variables

i

and

j

, for

i = 1, 2, \dots, p

and

j = 1, 2, \dots, p

, and

r_{y i}

(and

r_{i y}

) is the correlation between the response variable

y

and the

i

th explanatory variable, for

i = 1, 2, \dots, p

.

A sweep operation on the

k

th row and column (

k \leq p

) of

R

replaces:

\begin{array}{l} r_{k k} ​ by ​ −1 / r_{k k}; \\ r_{i k} ​ by ​ r_{i k} / | r_{k k} |, i = 1, 2, \dots, p + 1 ​ ​ (i \neq k); \\ r_{k j} ​ by ​ r_{k j} / | r_{k k} |, j = 1, 2, \dots, p + 1 ​ ​ (j \neq k); \\ r_{i j} ​ by ​ r_{i j} - r_{i k} r_{k j} / | r_{k k} |,  ​ i = 1, 2, \dots, p + 1 ​ ​ (i \neq k); ​ j = 1, 2, \dots, p + 1 ​ ​ (j \neq k) . \end{array}

The

k

th explanatory variable is eligible for entry into the current model if it satisfies the collinearity tests:

r_{k k} > τ

and

(r_{i i} - \frac{r_{i k} r_{k i}}{r_{k k}}) τ \leq 1,

for a user-supplied value (

> 0

) of

τ

and where the index

i

runs over explanatory variables in the current model. The sweep operation is its own inverse, therefore, pivoting on an explanatory variable

k

in the current model has the effect of removing it from the model.

Once the stepwise model selection procedure is finished, the routine calculates:

(a)the least squares estimate for the $i$ th explanatory variable included in the fitted model;
(b)standard error estimates for each coefficient in the final model;
(c)the square root of the mean square of residuals and its degrees of freedom;
(d)the multiple correlation coefficient.

The routine makes use of the symmetry of the sweep operations and correlation matrix which reduces by almost one half the storage and computation required by the sweep algorithm, see Clarke (1981) for details.

4 References

Clarke M R B (1981) Algorithm AS 178: the Gauss–Jordan sweep operator with detection of collinearity Appl. Statist. 31 166–169

Dempster A P (1969) Elements of Continuous Multivariate Analysis Addison–Wesley

Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley

5 Arguments

1: $m$ – Integer Input

On entry: the number of explanatory variables available in the design matrix,

Z

.

Constraint:

m > 1

.

2: $n$ – Integer Input

On entry: the number of observations used in the calculations.

Constraint:

n > 1

.

3: $wmean (m + 1)$ – Real (Kind=nag_wp) array Input

On entry: the mean of the design matrix,

Z

.

4: $c ((m + 1) \times (m + 2) / 2)$ – Real (Kind=nag_wp) array Input

On entry: the upper-triangular variance-covariance matrix packed by column for the design matrix,

Z

. Because the routine computes the correlation matrix

R

from c, the variance-covariance matrix need only be supplied up to a scaling factor.

5: $sw$ – Real (Kind=nag_wp) Input

On entry: if weights were used to calculate c then sw is the sum of positive weight values; otherwise sw is the number of observations used to calculate c.

Constraint:

sw > 1.0

.

6: $isx (m)$ – Integer array Input/Output

On entry: the value of

isx (j)

determines the set of variables used to perform full stepwise model selection, for

j = 1, 2, \dots, m

.

$isx (j) = −1$: To exclude the variable corresponding to the $j$ th column of $X$ from the final model.
$isx (j) = 1$: To consider the variable corresponding to the $j$ th column of $X$ for selection in the final model.
$isx (j) = 2$: To force the inclusion of the variable corresponding to the $j$ th column of $X$ in the final model.

Constraint:

isx (j) = −1, 1 ​ or ​ 2

, for

j = 1, 2, \dots, m

.

On exit: the value of

isx (j)

indicates the status of the

j

th explanatory variable in the model.

$isx (j) = −1$: Forced exclusion.
$isx (j) = 0$: Excluded.
$isx (j) = 1$: Selected.
$isx (j) = 2$: Forced selection.

7: $fin$ – Real (Kind=nag_wp) Input

On entry: the value of the variance ratio which an explanatory variable must exceed to be included in a model.

Suggested value:

fin = 4.0

.

Constraint:

fin > 0.0

.

8: $fout$ – Real (Kind=nag_wp) Input

On entry: the explanatory variable in a model with the lowest variance ratio value is removed from the model if its value is less than fout. fout is usually set equal to the value of fin; a value less than fin is occasionally preferred.

Suggested value:

fout = fin

.

Constraint:

0.0 \leq fout \leq fin

.

9: $tau$ – Real (Kind=nag_wp) Input

On entry: the tolerance,

τ

, for detecting collinearities between variables when adding or removing an explanatory variable from a model. Explanatory variables deemed to be collinear are excluded from the final model.

Suggested value:

tau = 1.0 \times 10^{−6}

.

Constraint:

tau > 0.0

.

10: $b (m + 1)$ – Real (Kind=nag_wp) array Output

On exit:

b (1)

contains the estimate for the intercept term in the fitted model. If

isx (j) \neq 0

, then

b (j + 1)

contains the estimate for the

j

th explanatory variable in the fitted model; otherwise

b (j + 1) = 0

.

11: $se (m + 1)$ – Real (Kind=nag_wp) array Output

On exit:

se (j)

contains the standard error for the estimate of

b (j)

, for

j = 1, 2, \dots, m + 1

.

12: $rsq$ – Real (Kind=nag_wp) Output

On exit: the

R^{2}

-statistic for the fitted regression model.

13: $rms$ – Real (Kind=nag_wp) Output

On exit: the mean square of residuals for the fitted regression model.

14: $df$ – Integer Output

On exit: the number of degrees of freedom for the sum of squares of residuals.

15: $monlev$ – Integer Input

On entry: if a subroutine is provided by you to monitor the model selection process, set monlev to

1

; otherwise set monlev to

0

.

Constraint:

monlev = 0

or

1

.

16: $monfun$ – Subroutine, supplied by the NAG Library or the user. External Procedure

You may define your own function or specify the NAG defined default function g02efh.

If

monlev = 0

, monfun is not referenced; otherwise its specification is:

The specification of monfun is:

Fortran Interface

Subroutine monfun (

flag, var, val, iuser, ruser)

Integer, Intent (In)	::	var
Integer, Intent (Inout)	::	iuser(*)
Real (Kind=nag_wp), Intent (In)	::	val
Real (Kind=nag_wp), Intent (Inout)	::	ruser(*)
Character (1), Intent (In)	::	flag

C Header Interface

void	monfun (const char flag, const Integer var, const double *val, Integer iuser[], double ruser[], const Charlen length_flag)

1: $flag$ – Character(1) Input

On entry: the value of flag indicates the stage of the stepwise selection of explanatory variables.

$flag ='A'$: Variable var was added to the current model.
$flag ='B'$: Beginning the backward elimination step.
$flag ='C'$: Variable var failed the collinearity test and is excluded from the model.
$flag ='D'$: Variable var was dropped from the current model.
$flag ='F'$: Beginning the forward selection step
$flag ='K'$: Backward elimination did not remove any variables from the current model.
$flag ='S'$: Starting stepwise selection procedure.
$flag ='V'$: The variance ratio for variable var takes the value val.
$flag ='X'$: Finished stepwise selection procedure.

2: $var$ – Integer Input

On entry: the index of the explanatory variable in the design matrix

Z

to which flag pertains.

3: $val$ – Real (Kind=nag_wp) Input

On entry: if

flag ='V'

, val is the variance ratio value for the coefficient associated with explanatory variable index var.

4: $iuser (*)$ – Integer array User Workspace

5: $ruser (*)$ – Real (Kind=nag_wp) array User Workspace

monfun is called with the arguments iuser and ruser as supplied to g02eff. You should use the arrays iuser and ruser to supply information to monfun.

monfun must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which g02eff is called. Arguments denoted as Input must not be changed by this procedure.

17: $iuser (*)$ – Integer array User Workspace

18: $ruser (*)$ – Real (Kind=nag_wp) array User Workspace

iuser and ruser are not used by g02eff, but are passed directly to monfun and may be used to pass information to this routine.

19: $ifail$ – Integer Input/Output

On entry: ifail must be set to

0

,

−1

or

1

to set behaviour on detection of an error; these values have no effect when no error is detected.

A value of

0

causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of

−1

means that an error message is printed while a value of

1

means that it is not.

If halting is not appropriate, the value

−1

or

1

is recommended. If message printing is undesirable, then the value

1

is recommended. Otherwise, the value

0

is recommended. When the value $- 1$ or $1$ is used it is essential to test the value of ifail on exit.

On exit:

ifail = 0

unless the routine detects an error or a warning has been flagged (see Section 6).

6 Error Indicators and Warnings

If on entry

ifail = 0

or

−1

, explanatory error messages are output on the current error message unit (as defined by x04aaf).

Errors or warnings detected by the routine:

$ifail = 1$: On entry, $fin = ⟨ value ⟩$ .
Constraint: $fin > 0.0$ .

On entry, $fout = ⟨ value ⟩$ ; $fin = ⟨ value ⟩$ .
Constraint: $0.0 \leq fout \leq fin$ .

On entry, $m = ⟨ value ⟩$ .
Constraint: $m > 1$ .

On entry, $monlev = ⟨ value ⟩$ .
Constraint: $monlev = 0$ or $1$ .

On entry, $n = ⟨ value ⟩$ .
Constraint: $n > 1$ .

On entry, $sw = ⟨ value ⟩$ .
Constraint: $sw > 1.0$ .

On entry, $tau = ⟨ value ⟩$ .
Constraint: $tau > 0.0$ .

$ifail = 2$: On entry, $isx (⟨ value ⟩) = ⟨ value ⟩$ .
Constraint: $isx (j) = −1$ , $1$ or $2$ , for $j = 1, 2, \dots, m$ .

On entry, $isx (i) \neq 1$ , for all $i = 1, 2, \dots, m$ .
Constraint: there must be at least one free variable.

On entry at least one diagonal element of $c \leq 0.0$ .
Constraint: c must be positive definite.

$ifail = 3$: The design and data matrix $Z$ is not positive definite, results may be inaccurate. All output is returned as documented.

$ifail = 4$: All variables are collinear, no model to select.

$ifail = - 99$: An unexpected error has been triggered by this routine. Please contact NAG.
See Section 7 in the Introduction to the NAG Library FL Interface for further information.

$ifail = - 399$: Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library FL Interface for further information.

$ifail = - 999$: Dynamic memory allocation failed.
See Section 9 in the Introduction to the NAG Library FL Interface for further information.

7 Accuracy

g02eff returns a warning if the design and data matrix is not positive definite.

8 Parallelism and Performance

Background information to multithreading can be found in the Multithreading documentation.

g02eff is not threaded in any implementation.

9 Further Comments

Although the condition for removing or adding a variable to the current model is based on a ratio of variances, these values should not be interpreted as

F

-statistics with the usual interpretation of significance unless the probability levels are adjusted to account for correlations between variables under consideration and the number of possible updates (see, e.g., Draper and Smith (1985)).

g02eff allocates internally

O (4 \times m + (m + 1) \times (m + 2) / 2 + 2)

of real storage.

10 Example

This example calculates a full stepwise model selection for the Hald data described in Dempster (1969). Means, the upper-triangular variance-covariance matrix and the sum of weights are calculated by g02buf. The NAG defined default monitor function g02efh is used to print information at each step of the model selection process.

10.1 Program Text

Program Text (g02effe.f90)

10.2 Program Data

Program Data (g02effe.d)

10.3 Program Results

Program Results (g02effe.r)

NAG Library Manual, Mark 28.6

Interfaces: FL CL CPP AD PY MB

NAG FL Interface Introduction

G02 (Correg) Chapter Contents

G02 (Correg) Chapter Introduction

g02ef: FL CL CPP AD PY MB

© The Numerical Algorithms Group Ltd. 2022