NAG Library Function Document

nag_regsn_mult_linear (g02dac)

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Parallelism and Performance

9 Further Comments

+− 10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

1 Purpose

nag_regsn_mult_linear (g02dac) performs a general multiple linear regression when the independent variables may be linearly dependent. Parameter estimates, standard errors, residuals and influence statistics are computed. nag_regsn_mult_linear (g02dac) may be used to perform a weighted regression.

2 Specification

#include <nag.h>

#include <nagg02.h>

void

nag_regsn_mult_linear (Nag_IncludeMean mean, Integer n, const double x[], Integer tdx, Integer m, const Integer sx[], Integer ip, const double y[], const double wt[], double *rss, double *df, double b[], double se[], double cov[], double res[], double h[], double q[], Integer tdq, Nag_Boolean *svd, Integer *rank, double p[], double tol, double com_ar[], NagError *fail)

3 Description

The general linear regression model is defined by

y = X β + ε

where

$y$ is a vector of $n$ observations on the dependent variable,
$X$ is an $n$ by $p$ matrix of the independent variables of column rank $k$ ,
$β$ is a vector of length $p$ of unknown arguments, and
$ε$ is a vector of length $n$ of unknown random errors such that $var ε = V σ^{2}$ , where $V$ is a known diagonal matrix.

Note: the

p

independent variables may be selected from a set of

m

potential independent variables.

V = I

, the identity matrix, then least squares estimation is used.

V \neq I

, then for a given weight matrix

W \propto V^{- 1}

, weighted least squares estimation is used.

The least squares estimates

\hat{β}

of the arguments

β

minimize

{(y - X β)}^{T} (y - X β)

while the weighted least squares estimates minimize

{(y - X β)}^{T} W (y - X β)

nag_regsn_mult_linear (g02dac) finds a

Q R

decomposition of

X

(or

W^{1 / 2} X

in the weighted case), i.e.,

X = {Q R}^{*} (or ​ W^{1 / 2} X = {Q R}^{*})

where

R^{*} = (\begin{matrix} R \\ 0 \end{matrix})

and

R

is a

p

p

upper triangular matrix and

Q

is an

n

n

orthogonal matrix.

R

is of full rank, then

\hat{β}

is the solution to

R \hat{β} = c_{1}

where

c = Q^{T} y

(or

Q^{T} W^{1 / 2} y

) and

c_{1}

is the first

p

elements of

c

R

is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of

R

R = Q_{*} (\begin{matrix} D & 0 \\ 0 & 0 \end{matrix}) P^{T}

where

D

is a

k

k

diagonal matrix with nonzero diagonal elements,

k

being the rank of

R

and

Q_{*}

and

P

are

p

p

orthogonal matrices. This gives the solution

\hat{β} = P_{1} D^{- 1} Q_{*_{1}}^{T} c_{1}

P_{1}

being the first

k

columns of

P

, i.e.,

P = (P_{1} P_{0})

and

Q_{*_{1}}

being the first

k

columns of

Q_{*}

Details of the SVD are made available, in the form of the matrix

P^{*}

P^{*} = (\begin{matrix} D^{- 1} P_{1}^{T} \\ P_{0}^{T} \end{matrix}) .

This will be only one of the possible solutions. Other estimates may be obtained by applying constraints to the arguments. These solutions can be obtained by using nag_regsn_mult_linear_tran_model (g02dkc) after using nag_regsn_mult_linear (g02dac). Only certain linear combinations of the arguments will have unique estimates; these are known as estimable functions.

The fit of the model can be examined by considering the residuals,

r_{i} = y_{i} - \hat{y}

, where

\hat{y} = X \hat{β}

are the fitted values. The fitted values can be written as

H y

for an

n

n

matrix

H

. The

i

th diagonal element of

H

h_{i}

, gives a measure of the influence of the

i

th value of the independent variables on the fitted regression model. The values

h_{i}

are sometimes known as leverages. Both

r_{i}

and

h_{i}

are provided by nag_regsn_mult_linear (g02dac).

The output of nag_regsn_mult_linear (g02dac) also includes

\hat{β}

, the residual sum of squares and associated degrees of freedom,

(n - k)

, the standard errors of the parameter estimates and the variance-covariance matrix of the parameter estimates.

In many linear regression models the first term is taken as a mean term or an intercept, i.e.,

X_{i, 1} = 1

, for

i = 1, 2, \dots, n

. This is provided as an option. Also note that not all the potential independent variables need to be included in a model; a facility to select variables to be included in the model is provided.

Details of the

Q R

decomposition and, if used, the SVD, are made available. These allow the regression to be updated by adding or deleting an observation using nag_regsn_mult_linear_addrem_obs (g02dcc), adding or deleting a variable using nag_regsn_mult_linear_add_var (g02dec) and nag_regsn_mult_linear_delete_var (g02dfc) or estimating and testing an estimable function using nag_regsn_mult_linear_est_func (g02dnc).

4 References

Cook R D and Weisberg S (1982) Residuals and Influence in Regression Chapman and Hall

Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley

Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore

Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl. 20(3) 2–25

McCullagh P and Nelder J A (1983) Generalized Linear Models Chapman and Hall

Searle S R (1971) Linear Models Wiley

5 Arguments

1: mean – Nag_IncludeMeanInput

On entry: indicates if a mean term is to be included.

$mean = Nag_MeanInclude$: A mean term, (intercept), will be included in the model.
$mean = Nag_MeanZero$: The model will pass through the origin, zero point.

Constraint:

mean = Nag_MeanInclude

Nag_MeanZero

2: n – IntegerInput

On entry: the number of observations,

n

Constraint:

n \geq 2

3: x[ $n \times tdx$ ] – const doubleInput

On entry:

x [(i) \times tdx + j]

must contain the

i

th observation for the

j

th potential independent variable, for

i = 0, 1, \dots, n - 1

and

j = 0, 1, \dots, m - 1

4: tdx – IntegerInput

On entry: the stride separating matrix column elements in the array x.

Constraint:

tdx \geq m

5: m – IntegerInput

On entry: the total number of independent variables in the dataset,

m

Constraint:

m \geq 1

6: sx[m] – const IntegerInput

On entry: indicates which of the potential independent variables are to be included in the model. If

sx [j] > 0

, then the variable contained in the corresponding column of x is included in the regression model.

Constraints:

$sx [j] \geq 0$ , for $j = 0, 1, \dots, m - 1$ ;
if $mean = Nag_MeanInclude$ , then exactly $ip - 1$ values of sx must be $> 0$ ;
if $mean = Nag_MeanZero$ , then exactly ip values of sx must be $> 0$ .

7: ip – IntegerInput

On entry: the number

p

of independent variables in the model, including the mean or intercept if present.

Constraints:

if $mean = Nag_MeanInclude$ , $1 \leq ip \leq m + 1$ ;
if $mean = Nag_MeanZero$ , $1 \leq ip \leq m$ .

8: y[n] – const doubleInput

On entry: observations on the dependent variable,

y

9: wt[n] – const doubleInput

On entry: optionally, the weights to be used in the weighted regression.

wt [i - 1] = 0.0

, then the

i

th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights. The values of res and h will be set to zero for observations with zero weights.

If weights are not provided then wt must be set to NULL and the effective number of observations is n.

Constraint: if

wt is not NULL

wt [i - 1] = 0.0

, for

i = 1, 2, \dots, n

10: rss – double *Output

On exit: the residual sum of squares for the regression.

11: df – double *Output

On exit: the degrees of freedom associated with the residual sum of squares.

12: b[ip] – doubleOutput

On exit:

b [i]

, for

i = 0, 1, \dots, ip - 1

, contain the least squares estimates of the arguments of the regression model,

\hat{β}

mean = Nag_MeanInclude

, then

b [0]

will contain the estimate of the mean argument and

b [i]

will contain the coefficient of the variable contained in column

j

of x, where

sx [j]

is the

i

th positive value in the array sx.

mean = Nag_MeanZero

, then

b [i - 1]

will contain the coefficient of the variable contained in column

j

of x, where

sx [j]

is the

i

th positive value in the array sx.

13: se[ip] – doubleOutput

On exit:

se [i]

, for

i = 0, 1, \dots, ip - 1

, contains the standard errors of the ip parameter estimates given in b.

14: cov[ $ip \times (ip + 1) / 2$ ] – doubleOutput

On exit: the first

ip \times (ip + 1) / 2

elements of cov contain the upper triangular part of the variance-covariance matrix of the ip parameter estimates given in b. They are stored packed by column, i.e., the covariance between the parameter estimate given in

b [i]

and the parameter estimate given in

b [j]

j \geq i

, is stored in

cov [j (j + 1) / 2 + i]

, for

i = 0, 1, \dots, ip - 1

and

j = i, \dots, ip - 1

15: res[n] – doubleOutput

On exit: the (weighted) residuals,

r_{i}

16: h[n] – doubleOutput

On exit: the diagonal elements of

H

h_{i}

, the leverages.

17: q[ $n \times tdq$ ] – doubleOutput

Note: the

(i, j)

th element of the matrix

Q

is stored in

q [(i - 1) \times tdq + j - 1]

On exit: the results of the

Q R

decomposition: the first column of q contains

c

, the upper triangular part of columns 2 to

ip + 1

contain the

R

matrix, the strictly lower triangular part of columns 2 to

ip + 1

contain details of the

Q

matrix.

18: tdq – IntegerInput

On entry: the stride separating matrix column elements in the array q.

Constraint:

tdq \geq ip + 1

19: svd – Nag_Boolean *Output

On exit: if a singular value decomposition has been performed then svd will be Nag_TRUE, otherwise svd will be Nag_FALSE.

20: rank – Integer *Output

On exit: the rank of the independent variables.

svd = Nag_FALSE

rank = ip

svd = Nag_TRUE

, rank is an estimate of the rank of the independent variables. rank is calculated as the number of singular values greater than tol (largest singular value). It is possible for the SVD to be carried out but rank to be returned as ip.

21: p[ $2 \times ip + ip \times ip$ ] – doubleOutput

On exit: details of the

Q R

decomposition and SVD if used.

svd = Nag_FALSE

, only the first ip elements of p are used, these will contain details of the Householder vector in the

Q R

decomposition (see Sections 2.2.1 and 3.3.6 in the f08 Chapter Introduction).

svd = Nag_TRUE

, the first ip elements of p will contain details of the Householder vector in the

Q R

decomposition and the next ip elements of p contain singular values. The following ip by ip elements contain the matrix

P^{*}

stored by rows.

22: tol – doubleInput

On entry: the value of tol is used to decide what is the rank of the independent variables. The smaller the value of tol the stricter the criterion for selecting the singular value decomposition. If

tol = 0.0

, then the singular value decomposition will never be used, this may cause run time errors or inaccurate results if the independent variables are not of full rank.

Suggested value:

tol = 0.000001

Constraint:

tol \geq 0.0

23: com_ar[ $\dim$ ] – doubleOutput

Note: the dimension, dim, of the array com_ar must be at least

5 \times (ip - 1) \times ip \times ip

On exit: if

svd = Nag_TRUE

, com_ar contains information which is needed by nag_regsn_mult_linear_newyvar (g02dgc).

24: fail – NagError *Input/Output

The NAG error argument (see Section 3.6 in the Essential Introduction).

6 Error Indicators and Warnings

NE_2_INT_ARG_LT: On entry, $n = ⟨value⟩$ while $ip = ⟨value⟩$ . These arguments must satisfy $n \geq ip$ .
On entry, $tdq = ⟨value⟩$ while $ip + 1 = ⟨value⟩$ . These arguments must satisfy $tdq \geq ip + 1$ .
On entry, $tdx = ⟨value⟩$ while $m = ⟨value⟩$ . These arguments must satisfy $tdx \geq m$ .
NE_ALLOC_FAIL: Dynamic memory allocation failed.
NE_BAD_PARAM: On entry, argument mean had an illegal value.
NE_BAD_SX_OR_IP: Either a value of sx is $< 0$ , or ip is incompatible with mean and sx, or $ip >$ the effective number of observations.
NE_INT_ARG_LT: On entry, $ip = ⟨value⟩$ .
Constraint: $ip \geq 1$ .
On entry, $m = ⟨value⟩$ .
Constraint: $m \geq 1$ .
On entry, $n = ⟨value⟩$ .
Constraint: $n \geq 2$ .
On entry, $sx [⟨value⟩]$ must not be less than 0: $sx [⟨value⟩] = ⟨value⟩$ .
NE_REAL_ARG_LT: On entry, tol must not be less than 0.0: $tol = ⟨value⟩$ .
On entry, $wt [⟨value⟩]$ must not be less than 0.0: $wt [⟨value⟩] = ⟨value⟩$ .
NE_SVD_NOT_CONV: The singular value decomposition has failed to converge.
NE_ZERO_DOF_RESID: The degrees of freedom for the residuals are zero, i.e., the designated number of arguments $=$ the effective number of observations. In this case the parameter estimates will be returned along with the diagonal elements of $H$ , but neither standard errors nor the variance-covariance matrix will be calculated.

7 Accuracy

The accuracy of this function is closely related to the accuracy of the

Q R

decomposition.

8 Parallelism and Performance

Not applicable.

9 Further Comments

Function nag_regsn_std_resid_influence (g02fac) can be used to compute standardized residuals and further measures of influence. nag_regsn_mult_linear (g02dac) requires, in particular, the results stored in res and h.

10 Example

For this function two examples are presented. There is a single example program for nag_regsn_mult_linear (g02dac), with a main program and the code to solve the two example problems is given in the functions ex1 and ex2.

Example 1 (ex1)

Data from an experiment with four treatments and three observations per treatment are read in. The treatments are represented by dummy

(0 - 1)

variables. An unweighted model is fitted with a mean included in the model.

Example 2 (ex2)

This example program uses nag_regsn_mult_linear (g02dac) to find the coefficient of the

n

degree polynomial

p (x) = a_{n} x^{n} + a_{n - 1} x^{n - 1} + \dots a_{1} x + a_{o}

that fits the data,

p (x (i))

y (i)

, in a least squares sense.

In this example nag_regsn_mult_linear (g02dac) is called with both

mean = Nag_MeanInclude

and

mean = Nag_MeanZero

. The polynomial degree, the number of data points and the tolerance can be modified using the example data file.

NAG Library Function Documentnag_regsn_mult_linear (g02dac)

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Parallelism and Performance

9 Further Comments

10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

NAG Library Function Document

nag_regsn_mult_linear (g02dac)