NAG Library Routine Document

G02JAF

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Parameters

6 Error Indicators and Warnings

7 Accuracy

8 Further Comments

+− 9 Example

9.1 Program Text

9.2 Program Data

9.3 Program Results

1 Purpose

G02JAF fits a linear mixed effects regression model using restricted maximum likelihood (REML).

2 Specification

SUBROUTINE G02JAF (

N, NCOL, LDDAT, DAT, LEVELS, YVID, CWID, NFV, FVID, FINT, NRV, RVID, NVPR, VPR, RINT, SVID, GAMMA, NFF, NRF, DF, REML, LB, B, SE, MAXIT, TOL, WARN, IFAIL)

INTEGER	N, NCOL, LDDAT, LEVELS(NCOL), YVID, CWID, NFV, FVID(NFV), FINT, NRV, RVID(NRV), NVPR, VPR(NRV), RINT, SVID, NFF, NRF, DF, LB, MAXIT, WARN, IFAIL
REAL (KIND=nag_wp)	DAT(LDDAT,NCOL), GAMMA(NVPR+2), REML, B(LB), SE(LB), TOL

3 Description

G02JAF fits a model of the form:

y = X β + Z ν + ε

where

$y$ is a vector of $n$ observations on the dependent variable,
$X$ is a known $n$ by $p$ design matrix for the fixed independent variables,
$β$ is a vector of length $p$ of unknown fixed effects,
$Z$ is a known $n$ by $q$ design matrix for the random independent variables,
$ν$ is a vector of length $q$ of unknown random effects,

and

$ε$ is a vector of length $n$ of unknown random errors.

Both

ν

and

ε

are assumed to have a Gaussian distribution with expectation zero and

Var [\begin{matrix} ν \\ ε \end{matrix}] = [\begin{matrix} G & 0 \\ 0 & R \end{matrix}]

where

R = σ_{R}^{2} I

I

is the

n \times n

identity matrix and

G

is a diagonal matrix. It is assumed that the random variables,

Z

, can be subdivided into

g \leq q

groups with each group being identically distributed with expectations zero and variance

σ_{i}^{2}

. The diagonal elements of matrix

G

therefore take one of the values

\{σ_{i}^{2} : i = 1, 2, \dots, g\}

, depending on which group the associated random variable belongs to.

The model therefore contains three sets of unknowns, the fixed effects,

β

, the random effects

ν

and a vector of

g + 1

variance components,

γ

, where

γ = \{σ_{1}^{2}, σ_{2}^{2}, \dots, σ_{g - 1}^{2}, σ_{g}^{2}, σ_{R}^{2}\}

. Rather than working directly with

γ

, G02JAF uses an iterative process to estimate

γ^{*} = \{σ_{1}^{2} / σ_{R}^{2}, σ_{2}^{2} / σ_{R}^{2}, \dots, σ_{g - 1}^{2} / σ_{R}^{2}, σ_{g}^{2} / σ_{R}^{2}, 1\}

. Due to the iterative nature of the estimation a set of initial values,

γ_{0}

, for

γ^{*}

is required. G02JAF allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

G02JAF fits the model using a quasi-Newton algorithm to maximize the restricted log-likelihood function:

- 2 l_{R} = \log (|V|) + (n - p) \log (r^{'} V^{- 1} r) + \log |X^{'} V^{- 1} X| + (n - p) (1 + \log (2 π / (n - p)))

where

V = Z G Z^{'} + R, r = y - X b and b = {(X^{'} V^{- 1} X)}^{- 1} X^{'} V^{- 1} y .

Once the final estimates for

γ^{*}

have been obtained, the value of

σ_{R}^{2}

is given by:

σ_{R}^{2} = (r^{'} V^{- 1} r) / (n - p) .

Case weights,

W_{c}

, can be incorporated into the model by replacing

X^{'} X

and

Z^{'} Z

with

X^{'} W_{c} X

and

Z^{'} W_{c} Z

respectively, for a diagonal weight matrix

W_{c}

The log-likelihood,

l_{R}

, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

4 References

Goodnight J H (1979) A tutorial on the SWEEP operator The American Statistician 33(3) 149–158

Harville D A (1977) Maximum likelihood approaches to variance component estimation and to related problems JASA 72 320–340

Rao C R (1972) Estimation of variance and covariance components in a linear model J. Am. Stat. Assoc. 67 112–115

Stroup W W (1989) Predictable functions and prediction space in the mixed model procedure Applications of Mixed Models in Agriculture and Related Disciplines Southern Cooperative Series Bulletin No. 343 39–48

Wolfinger R, Tobias R and Sall J (1994) Computing Gaussian likelihoods and their derivatives for general linear mixed models SIAM Sci. Statist. Comput. 15 1294–1310

5 Parameters

1: N – INTEGERInput

On entry:

n

, the number of observations.

Constraint:

N \geq 1

2: NCOL – INTEGERInput

On entry: the number of columns in the data matrix, DAT.

Constraint:

NCOL \geq 1

3: LDDAT – INTEGERInput

On entry: the first dimension of the array DAT as declared in the (sub)program from which G02JAF is called.

Constraint:

LDDAT \geq N

4: DAT(LDDAT,NCOL) – REAL (KIND=nag_wp) arrayInput

On entry: array containing all of the data. For the

i

th observation:

$DAT (i, YVID)$ holds the dependent variable, $y$ ;
if $CWID \neq 0$ , $DAT (i, CWID)$ holds the case weights;
if $SVID \neq 0$ , $DAT (i, SVID)$ holds the subject variable.

The remaining columns hold the values of the independent variables.

Constraints:

if $CWID \neq 0$ , $DAT (i, CWID) \geq 0.0$ ;
if $LEVELS (j) \neq 1$ , $1 \leq DAT (i, j) \leq LEVELS (j)$ .

5: LEVELS(NCOL) – INTEGER arrayInput

On entry:

LEVELS (i)

contains the number of levels associated with the

i

th variable of the data matrix DAT. If this variable is continuous or binary (i.e., only takes the values zero or one) then

LEVELS (i)

should be

1

; if the variable is discrete then

LEVELS (i)

is the number of levels associated with it and

DAT (j, i)

is assumed to take the values

1

LEVELS (i)

, for

j = 1, 2, \dots, N

Constraint:

LEVELS (i) \geq 1

, for

i = 1, 2, \dots, NCOL

6: YVID – INTEGERInput

On entry: the column of DAT holding the dependent,

y

, variable.

Constraint:

1 \leq YVID \leq NCOL

7: CWID – INTEGERInput

On entry: the column of DAT holding the case weights.

CWID = 0

, no weights are used.

Constraint:

0 \leq CWID \leq NCOL

8: NFV – INTEGERInput

On entry: the number of independent variables in the model which are to be treated as being fixed.

Constraint:

0 \leq NFV < NCOL

9: FVID(NFV) – INTEGER arrayInput

On entry: the columns of the data matrix DAT holding the fixed independent variables with

FVID (i)

holding the column number corresponding to the

i

th fixed variable.

Constraint:

1 \leq FVID (i) \leq NCOL

, for

i = 1, 2, \dots, NFV

10: FINT – INTEGERInput

On entry: flag indicating whether a fixed intercept is included (

FINT = 1

Constraint:

FINT = 0

1

11: NRV – INTEGERInput

On entry: the number of independent variables in the model which are to be treated as being random.

Constraints:

$0 \leq NRV < NCOL$ ;
$NRV + RINT > 0$ .

12: RVID(NRV) – INTEGER arrayInput

On entry: the columns of the data matrix

DAT

holding the random independent variables with

RVID (i)

holding the column number corresponding to the

i

th random variable.

Constraint:

1 \leq RVID (i) \leq NCOL

, for

i = 1, 2, \dots, NRV

13: NVPR – INTEGERInput

On entry: if

RINT = 1

and

SVID \neq 0

, NVPR is the number of variance components being

estimated - 2

, (

g - 1

), else

NVPR = g

NRV = 0

NVPR

is not referenced.

Constraint: if

NRV \neq 0

1 \leq NVPR \leq NRV

14: VPR(NRV) – INTEGER arrayInput

On entry:

VPR (i)

holds a flag indicating the variance of the

i

th random variable. The variance of the

i

th random variable is

σ_{j}^{2}

, where

j = VPR (i) + 1

RINT = 1

and

SVID \neq 0

and

j = VPR (i)

otherwise. Random variables with the same value of

j

are assumed to be taken from the same distribution.

Constraint:

1 \leq VPR (i) \leq NVPR

, for

i = 1, 2, \dots, NRV

15: RINT – INTEGERInput

On entry: flag indicating whether a random intercept is included (

RINT = 1

SVID = 0

, RINT is not referenced.

Constraint:

RINT = 0

1

16: SVID – INTEGERInput

On entry: the column of DAT holding the subject variable.

SVID = 0

, no subject variable is used.

Specifying a subject variable is equivalent to specifying the interaction between that variable and all of the random-effects. Letting the notation

Z_{1} \times Z_{S}

denote the interaction between variables

Z_{1}

and

Z_{S}

, fitting a model with

RINT = 0

, random-effects

Z_{1} + Z_{2}

and subject variable

Z_{S}

is equivalent to fitting a model with random-effects

Z_{1} \times Z_{S} + Z_{2} \times Z_{S}

and no subject variable. If

RINT = 1

the model is equivalent to fitting

Z_{S} + Z_{1} \times Z_{S} + Z_{2} \times Z_{S}

and no subject variable.

Constraint:

0 \leq SVID \leq NCOL

17: GAMMA( $NVPR + 2$ ) – REAL (KIND=nag_wp) arrayInput/Output

On entry: holds the initial values of the variance components,

γ_{0}

, with

GAMMA (i)

the initial value for

σ_{i}^{2} / σ_{R}^{2}

, for

i = 1, 2, \dots, g

. If

RINT = 1

and

SVID \neq 0

g = NVPR + 1

, else

g = NVPR

GAMMA (1) = - 1.0

, the remaining elements of GAMMA are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

On exit:

GAMMA (i)

, for

i = 1, 2, \dots, g

, holds the final estimate of

σ_{i}^{2}

and

GAMMA (g + 1)

holds the final estimate for

σ_{R}^{2}

Constraint:

GAMMA (1) = - 1.0 ​ or ​ GAMMA (i) \geq 0.0

, for

i = 1, 2, \dots, g

18: NFF – INTEGEROutput

On exit: the number of fixed effects estimated (i.e., the number of columns,

p

, in the design matrix

X

19: NRF – INTEGEROutput

On exit: the number of random effects estimated (i.e., the number of columns,

q

, in the design matrix

Z

20: DF – INTEGEROutput

On exit: the degrees of freedom.

21: REML – REAL (KIND=nag_wp)Output

On exit:

- 2 l_{R} (\hat{γ})

where

l_{R}

is the log of the restricted maximum likelihood calculated at

\hat{γ}

, the estimated variance components returned in GAMMA.

22: LB – INTEGERInput

On entry: the size of the array B.

Constraint:

LB \geq FINT + \sum_{i = 1}^{NFV} \max (LEVELS (FVID (i)) - 1, 1) + L_{S} \times (RINT + \sum_{i = 1}^{NRV} LEVELS (RVID (i)))

where

L_{S} = LEVELS (SVID)

SVID \neq 0

and

1

otherwise.

23: B(LB) – REAL (KIND=nag_wp) arrayOutput

On exit: the parameter estimates,

(β, ν)

, with the first NFF elements of B containing the fixed effect parameter estimates,

β

and the next NRF elements of B containing the random effect parameter estimates,

ν

Fixed effects

FINT = 1

B (1)

contains the estimate of the fixed intercept. Let

L_{i}

denote the number of levels associated with the

i

th fixed variable, that is

L_{i} = LEVELS (FVID (i))

. Define

if $FINT = 1$ , $F_{1} = 2$ else if $FINT = 0$ , $F_{1} = 1$ ;
$F_{i + 1} = F_{i} + \max (L_{i} - 1, 1)$ , $i \geq 1$ .

Then for

i = 1, 2, \dots, NFV

if $L_{i} > 1$ , $B (F_{i} + j - 2)$ contains the parameter estimate for the $j$ th level of the $i$ th fixed variable, for $j = 2, 3, \dots, L_{i}$ ;
if $L_{i} \leq 1$ , $B (F_{i})$ contains the parameter estimate for the $i$ th fixed variable.

Random effects

Redefining

L_{i}

to denote the number of levels associated with the

i

th random variable, that is

L_{i} = LEVELS (RVID (i))

. Define

if $RINT = 1$ , $R_{1} = 2$ else if $RINT = 0$ , $R_{1} = 1$ ;
$R_{i + 1} = R_{i} + L_{i}$ , $i \geq 1$ .

Then for

i = 1, 2, \dots, NRV

if SVID=0,
- if $L_{i} > 1$ , $B (NFF + R_{i} + j - 1)$ contains the parameter estimate for the $j$ th level of the $i$ th random variable, for $j = 1, 2, \dots, L_{i}$ ;
- if $L_{i} \leq 1$ , $B (NFF + R_{i})$ contains the parameter estimate for the $i$ th random variable;
if SVID ≠ 0 ,
- let $L_{S}$ denote the number of levels associated with the subject variable, that is $L_{S} = LEVELS (SVID)$ ;
- if $L_{i} > 1$ , $B (NFF + (s - 1) L_{S} + R_{i} + j - 1)$ contains the parameter estimate for the interaction between the $s$ th level of the subject variable and the $j$ th level of the $i$ th random variable, for $s = 1, 2, \dots, L_{S}$ and $j = 1, 2, \dots, L_{i}$ ;
- if $L_{i} \leq 1$ , $B (NFF + (s - 1) L_{S} + R_{i})$ contains the parameter estimate for the interaction between the $s$ th level of the subject variable and the $i$ th random variable, for $s = 1, 2, \dots, L_{S}$ ;
- if $RINT = 1$ , $B (NFF + 1)$ contains the estimate of the random intercept.

24: SE(LB) – REAL (KIND=nag_wp) arrayOutput

On exit: the standard errors of the parameter estimates given in B.

25: MAXIT – INTEGERInput

On entry: the maximum number of iterations.

MAXIT < 0

, the default value of

100

is used.

MAXIT = 0

, the parameter estimates

(β, ν)

and corresponding standard errors are calculated based on the value of

γ_{0}

supplied in GAMMA.

26: TOL – REAL (KIND=nag_wp)Input

On entry: the tolerance used to assess convergence.

TOL \leq 0.0

, the default value of

ε^{0.7}

is used, where

ε

is the machine precision.

27: WARN – INTEGEROutput

On exit: is set to

1

if a variance component was estimated to be a negative value during the fitting process. Otherwise WARN is set to

0

WARN = 1

, the negative estimate is set to zero and the estimation process allowed to continue.

28: IFAIL – INTEGERInput/Output

On entry: IFAIL must be set to

0

- 1 ​ or ​ 1

. If you are unfamiliar with this parameter you should refer to Section 3.3 in the Essential Introduction for details.

For environments where it might be inappropriate to halt program execution when an error is detected, the value

- 1 ​ or ​ 1

is recommended. If the output of error messages is undesirable, then the value

1

is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is

0

. When the value $- 1 or 1$ is used it is essential to test the value of IFAIL on exit.

On exit:

IFAIL = 0

unless the routine detects an error or a warning has been flagged (see Section 6).

6 Error Indicators and Warnings

If on entry

IFAIL = 0

- 1

, explanatory error messages are output on the current error message unit (as defined by X04AAF).

Errors or warnings detected by the routine:

$IFAIL = 1$

On entry,	$N < 2$ ,
or	$NCOL < 1$ ,
or	$LDDAT < N$ ,
or	$YVID < 1$ or $YVID > NCOL$ ,
or	$CWID < 0$ or $CWID > NCOL$ ,
or	$NFV < 0$ or $NFV \geq NCOL$ ,
or	$FINT \neq 0$ and $FINT \neq 1$ ,
or	$NRV < 0$ or $NRV > NCOL$ or $NRV + RINT \leq 0$ ,
or	$NVPR \leq 0$ or $NVPR > NRV$ ,
or	$RINT \neq 0$ and $RINT \neq 1$ ,
or	$SVID < 0$ or $SVID > NCOL$ ,
or	LB is too small.

$IFAIL = 2$

On entry,	$LEVELS (i) < 1$ , for at least one $i$ ,
or	$FVID (i) < 1$ , or $FVID (i) > NCOL$ , for at least one $i$ ,
or	$RVID (i) < 1$ , or $RVID (i) > NCOL$ , for at least one $i$ ,
or	$VPR (i) < 1$ or $VPR (i) > NVPR$ , for at least one $i$ ,
or	at least one discrete variable in array DAT has a value less than $1.0$ or greater than that specified in LEVELS,
or	$GAMMA (i) < 0.0$ , for at least one $i$ , and $GAMMA (1) \neq - 1.0$ .

$IFAIL = 3$: Degrees of freedom $< 1$ . The number of parameters exceed the effective number of observations.

$IFAIL = 4$: The routine failed to converge to the specified tolerance in MAXIT iterations. See Section 8 for advice.

7 Accuracy

The accuracy of the results can be adjusted through the use of the TOL parameter.

8 Further Comments

Wherever possible any block structure present in the design matrix

Z

should be modelled through a subject variable, specified via SVID, rather than being explicitly entered into DAT.

G02JAF uses an iterative process to fit the specified model and for some problems this process may fail to converge (see

IFAIL = 4

). If the routine fails to converge then the maximum number of iterations (see MAXIT) or tolerance (see TOL) may require increasing; try a different starting estimate in GAMMA. Alternatively, the model can be fit using maximum likelihood (see G02JBF) or using the noniterative MIVQUE0.

To fit the model just using MIVQUE0, the first element of GAMMA should be set to

- 1.0

and MAXIT should be set to zero.

Although the quasi-Newton algorithm used in G02JAF tends to require more iterations before converging compared to the Newton–Raphson algorithm recommended by Wolfinger et al. (1994), it does not require the second derivatives of the likelihood function to be calculated and consequentially takes significantly less time per iteration.

9 Example

The following dataset is taken from Stroup (1989) and arises from a balanced split-plot design with the whole plots arranged in a randomized complete block-design.

In this example the full design matrix for the random independent variable,

Z

, is given by:

Z = (\begin{matrix} 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \\ 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \end{matrix})

= (\begin{matrix} A & 0 & 0 & 0 \\ 0 & A & 0 & 0 \\ 0 & 0 & A & 0 \\ 0 & 0 & 0 & A \\ A & 0 & 0 & 0 \\ 0 & A & 0 & 0 \\ 0 & 0 & A & 0 \\ 0 & 0 & 0 & A \end{matrix}),

(1)

where

A = (\begin{matrix} 1 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 1 \end{matrix}) .

The block structure evident in (1) is modelled by specifying a four-level subject variable, taking the values

\{1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4\}

. The first column of

1 s

is added to

A

by setting

RINT = 1

. The remaining columns of

A

are specified by a three level factor, taking the values,

\{1, 2, 3, 1, 2, 3, 1, \dots\}

NAG Library Routine DocumentG02JAF

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Parameters

6 Error Indicators and Warnings

7 Accuracy

8 Further Comments

9 Example

9.1 Program Text

9.2 Program Data

9.3 Program Results

NAG Library Routine Document

G02JAF