NAG Library Function Document

nag_ml_hier_mixed_regsn (g02jec)

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Parallelism and Performance

9 Further Comments

+− 10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

1 Purpose

nag_ml_hier_mixed_regsn (g02jec) fits a multi-level linear mixed effects regression model using maximum likelihood (ML). Prior to calling nag_ml_hier_mixed_regsn (g02jec) the initialization function nag_hier_mixed_init (g02jcc) must be called.

2 Specification

#include <nag.h>

#include <nagg02.h>

void

nag_ml_hier_mixed_regsn (Integer lvpr, const Integer vpr[], Integer nvpr, double gamma[], Integer *effn, Integer *rnkx, Integer *ncov, double *lnlike, Integer lb, Integer id[], Integer pdid, double b[], double se[], double czz[], Integer pdczz, double cxx[], Integer pdcxx, double cxz[], Integer pdcxz, const double rcomm[], const Integer icomm[], const Integer iopt[], Integer liopt, const double ropt[], Integer lropt, NagError *fail)

3 Description

nag_ml_hier_mixed_regsn (g02jec) fits a model of the form:

y = X β + Z ν + ε

where	$y$ is a vector of $n$ observations on the dependent variable,
	$X$ is a known $n$ by $p$ design matrix for the fixed independent variables,
	$β$ is a vector of length $p$ of unknown fixed effects,
	$Z$ is a known $n$ by $q$ design matrix for the random independent variables,
	$ν$ is a vector of length $q$ of unknown random effects,
and	$ε$ is a vector of length $n$ of unknown random errors.

Both

ν

and

ε

are assumed to have a Gaussian distribution with expectation zero and variance/covariance matrix defined by

Var [\begin{matrix} ν \\ ε \end{matrix}] = [\begin{matrix} G & 0 \\ 0 & R \end{matrix}]

where

R = σ_{R}^{2} I

I

is the

n \times n

identity matrix and

G

is a diagonal matrix. It is assumed that the random variables,

Z

, can be subdivided into

g \leq q

groups with each group being identically distributed with expectation zero and variance

σ_{i}^{2}

. The diagonal elements of matrix

G

therefore take one of the values

\{σ_{i}^{2} : i = 1, 2, \dots, g\}

, depending on which group the associated random variable belongs to.

The model therefore contains three sets of unknowns: the fixed effects

β

, the random effects

ν

and a vector of

g + 1

variance components

γ

, where

γ = \{σ_{1}^{2}, σ_{2}^{2}, \dots, σ_{g - 1}^{2}, σ_{g}^{2}, σ_{R}^{2}\}

. Rather than working directly with

γ

, nag_ml_hier_mixed_regsn (g02jec) uses an iterative process to estimate

γ^{*} = \{σ_{1}^{2} / σ_{R}^{2}, σ_{2}^{2} / σ_{R}^{2}, \dots, σ_{g - 1}^{2} / σ_{R}^{2}, σ_{g}^{2} / σ_{R}^{2}, 1\}

. Due to the iterative nature of the estimation a set of initial values,

γ_{0}

, for

γ^{*}

is required. nag_ml_hier_mixed_regsn (g02jec) allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

nag_ml_hier_mixed_regsn (g02jec) fits the model by maximizing the log-likelihood function:

- 2 l_{R} = \log (|V|) + n \log (r^{T} V^{- 1} r) + \log (2 π / n)

where

V = Z G Z^{T} + R, r = y - X b and b = {(X^{T} V^{- 1} X)}^{- 1} X^{T} V^{- 1} y .

Once the final estimates for

γ^{*}

have been obtained, the value of

σ_{R}^{2}

is given by

σ_{R}^{2} = (r^{T} V^{- 1} r) / (n - p) .

Case weights,

W_{c}

, can be incorporated into the model by replacing

X^{T} X

and

Z^{T} Z

with

X^{T} W_{c} X

and

Z^{T} W_{c} Z

respectively, for a diagonal weight matrix

W_{c}

The log-likelihood,

l_{R}

, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

4 References

Goodnight J H (1979) A tutorial on the SWEEP operator The American Statistician 33(3) 149–158

Harville D A (1977) Maximum likelihood approaches to variance component estimation and to related problems JASA 72 320–340

Rao C R (1972) Estimation of variance and covariance components in a linear model J. Am. Stat. Assoc. 67 112–115

Stroup W W (1989) Predictable functions and prediction space in the mixed model procedure Applications of Mixed Models in Agriculture and Related Disciplines Southern Cooperative Series Bulletin No. 343 39–48

Wolfinger R, Tobias R and Sall J (1994) Computing Gaussian likelihoods and their derivatives for general linear mixed models SIAM Sci. Statist. Comput. 15 1294–1310

5 Arguments

Note: prior to calling nag_ml_hier_mixed_regsn (g02jec) the initialization function nag_hier_mixed_init (g02jcc) must be called, therefore this documention should be read in conjunction with the document for nag_hier_mixed_init (g02jcc).

In particular some argument names and conventions described in that document are also relevant here, but their definition has not been repeated. Specifically, RNDM, wt, n, nff, nrf, nlsv, levels, fixed, DAT, licomm and lrcomm should be interpreted identically in both functions.

1: lvpr – IntegerInput

On entry: the sum of the number of random parameters and the random intercept flags specified in the call to nag_hier_mixed_init (g02jcc).

Constraint:

lvpr = \sum_{i} RNDM (1, i) + RNDM (2, i)

2: vpr[lvpr] – const IntegerInput

On entry: a vector of flags indicating the mapping between the random variables specified in rndm and the variance components,

σ_{i}^{2}

. See Section 9 for more details.

Constraint:

1 \leq vpr [i - 1] \leq nvpr

, for

i = 1, 2, \dots, lvpr

3: nvpr – IntegerInput

On entry:

g

, the number of variance components being estimated (excluding the overall variance,

σ_{R}^{2}

Constraint:

1 \leq nvpr \leq lvpr

4: gamma[ $nvpr + 1$ ] – doubleInput/Output

On entry: holds the initial values of the variance components,

γ_{0}

, with

gamma [i - 1]

the initial value for

σ_{i}^{2} / σ_{R}^{2}

, for

i = 1, 2, \dots, nvpr

gamma [0] = - 1.0

, the remaining elements of gamma are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

On exit:

gamma [i - 1]

, for

i = 1, 2, \dots, nvpr

, holds the final estimate of

σ_{i}^{2}

and

gamma [nvpr]

holds the final estimate for

σ_{R}^{2}

Constraint:

gamma [0] = - 1.0 ​ or ​ gamma [i - 1] \geq 0.0

, for

i = 1, 2, \dots, g

5: effn – Integer *Output

On exit: effective number of observations. If there are no weights (i.e.,

wt is NULL

), or all weights are nonzero, then

effn = n

6: rnkx – Integer *Output

On exit: the rank of the design matrix,

X

, for the fixed effects.

7: ncov – Integer *Output

On exit: number of variance components not estimated to be zero. If none of the variance components are estimated to be zero, then

ncov = nvpr

8: lnlike – double *Output

On exit:

- 2 l_{R} (\hat{γ})

where

l_{R}

is the log of the maximum likelihood calculated at

\hat{γ}

, the estimated variance components returned in gamma.

9: lb – IntegerInput

On entry: the dimension of the arrays b and se.

Constraint:

lb \geq nff + nrf \times nlsv

10: id[ $pdid \times lb$ ] – IntegerOutput

Note: where

ID (i, j)

appears in this document, it refers to the array element

id [(j - 1) \times pdid + i - 1]

On exit: an array describing the parameter estimates returned in b. The first

nlsv \times nrf

columns of ID describe the parameter estimates for the random effects and the last nff columns the parameter estimates for the fixed effects.

A print function for decoding the parameter estimates given in b using information from id is supplied with the example program for this function.

For fixed effects:

for l=nrf×nlsv+1 ,…, nrf×nlsv+nff
- if $b [l - 1]$ contains the parameter estimate for the intercept then
  $ID (1, l) = ID (2, l) = ID (3, l) = 0;$
- if $b [l - 1]$ contains the parameter estimate for the $i$ th level of the $j$ th fixed variable, that is the vector of values held in the $k$ th column of DAT when $fixed [j + 1] = k$ then
  $\begin{array}{l} ID (1, l) = 0, \\ ID (2, l) = j, \\ ID (3, l) = i; \end{array}$
- if the $j$ th variable is continuous or binary, that is $levels [fixed [j + 1] - 1] = 1$ , then $ID (3, l) = 0$ ;
- any remaining rows of the $l$ th column of ID are set to $0$ .

For random effects:

let
- $N_{R_{b}}$ denote the number of random variables in the $b$ th random statement, that is $N_{R_{b}} = RNDM (1, b)$ ;
- $R_{j b}$ denote the $j$ th random variable from the $b$ th random statement, that is the vector of values held in the $k$ th column of DAT when $RNDM (2 + j, b) = k$ ;
- $N_{S_{b}}$ denote the number of subject variables in the $b$ th random statement, that is $N_{S_{b}} = RNDM (3 + N_{R_{b}}, b)$ ;
- $S_{j b}$ denote the $j$ th subject variable from the $b$ th random statement, that is the vector of values held in the $k$ th column of DAT when $RNDM (3 + N_{R_{b}} + j, b) = k$ ;
- $L (S_{j b})$ denote the number of levels for $S_{j b}$ , that is $L (S_{j b}) = levels [RNDM (3 + N_{R_{b}} + j, b) - 1]$ ;
then
- for $l = 1, 2, \dots nrf \times nlsv$ , if $b [l - 1]$ contains the parameter estimate for the $i$ th level of $R_{j b}$ when $S_{k b} = s_{k}$ , for $k = 1, 2, \dots, N_{S_{b}}$ and $1 \leq s_{k} \leq L (S_{j b})$ , i.e., $s_{k}$ is a valid value for the $k$ th subject variable, then
  $\begin{array}{l} ID (1, l) = b, \\ ID (2, l) = j, \\ ID (3, l) = i, \\ ID (3 + k, l) = s_{k}, k = 1, 2, \dots, N_{S_{b}}; \end{array}$
- if the parameter being estimated is for the intercept then $ID (2, l) = ID (3, l) = 0$ ;
- if the $j$ th variable is continuous, or binary, that is $L (S_{j b}) = 1$ , then $ID (3, l) = 0$ ;
- the remaining rows of the $l$ th column of ID are set to $0$ .

In some situations, certain combinations of variables are never observed. In such circumstances all elements of the

l

th row of ID are set to

- 999

11: pdid – IntegerInput

On entry: the stride separating matrix row elements in the array id.

Constraint:

pdid \geq 3 + \max_{j} (RNDM (3 + RNDM (1, j), j))

, i.e.,

3 +

maximum number of subject variables (see nag_hier_mixed_init (g02jcc)).

12: b[lb] – doubleOutput

On exit: the parameter estimates, with the first

nrf \times nlsv

elements of

b

containing the parameter estimates for the random effects,

ν

, and the remaining nff elements containing the parameter estimates for the fixed effects,

β

. The order of these estimates are described by the id argument.

13: se[lb] – doubleOutput

On exit: the standard errors of the parameter estimates given in b.

14: czz[ $\dim$ ] – doubleOutput

Note: the dimension, dim, of the array czz must be at least

pdczz \times nrf \times nlsv

Where

CZZ (i, j)

appears in this document, it refers to the array element

czz [(j - 1) \times pdczz + i - 1]

On exit: if

nlsv = 1

, then CZZ holds the lower triangular portion of the matrix

(1 / σ^{2}) (Z^{T} {\hat{R}}^{- 1} Z + {\hat{G}}^{- 1})

, where

\hat{R}

and

\hat{G}

are the estimates of

R

and

G

respectively. If

nlsv > 1

then CZZ holds this matrix in compressed form, with the first nrf columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next nrf columns the part corresponding to the second level of the overall subject variable etc.

15: pdczz – IntegerInput

On entry: the stride separating matrix row elements in the array czz.

Constraint:

pdczz \geq nrf

16: cxx[ $\dim$ ] – doubleOutput

Note: the dimension, dim, of the array cxx must be at least

pdcxx \times nff

Where

CXX (i, j)

appears in this document, it refers to the array element

cxx [(j - 1) \times pdcxx + i - 1]

On exit: CXX holds the lower triangular portion of the matrix

(1 / σ^{2}) X^{T} {\hat{V}}^{- 1} X

, where

\hat{V}

is the estimated value of

V

17: pdcxx – IntegerInput

On entry: the stride separating matrix row elements in the array cxx.

Constraint:

pdcxx \geq nff

18: cxz[ $\dim$ ] – doubleOutput

Note: the dimension, dim, of the array cxz must be at least

pdcxz \times nlsv \times nrf

Where

CXZ (i, j)

appears in this document, it refers to the array element

cxz [(j - 1) \times pdcxz + i - 1]

On exit: if

nlsv = 1

, then CXZ holds the matrix

(1 / σ^{2}) (X^{T} {\hat{V}}^{- 1} Z) \hat{G}

, where

\hat{V}

and

\hat{G}

are the estimates of

V

and

G

respectively. If

nlsv > 1

then CXZ holds this matrix in compressed form, with the first nrf columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next nrf columns the part corresponding to the second level of the overall subject variable etc.

19: pdcxz – IntegerInput

On entry: the stride separating matrix row elements in the array cxz.

Constraint:

pdcxz \geq nff

20: rcomm[ $\dim$ ] – const doubleCommunication Array

Note: the dimension, dim, of the array rcomm must be at least

lrcomm

On entry: communication array initialized by a call to nag_hier_mixed_init (g02jcc).

21: icomm[ $\dim$ ] – const IntegerCommunication Array

Note: the dimension, dim, of the array icomm must be at least

licomm

On entry: communication array initialized by a call to nag_hier_mixed_init (g02jcc).

22: iopt[liopt] – const IntegerInput

On entry: optional arguments passed to the optimization function.

By default nag_ml_hier_mixed_regsn (g02jec) fits the specified model using a modified Newton optimization algorithm as implemented in the NAG Fortran Library routine E04LBF. In some cases, where the calculation of the derivatives is computationally expensive it may be more efficient to use a sequential QP algorithm. The sequential QP algorithm as implemented in the NAG Fortran Library routine E04UCF can be chosen by setting

iopt [4] = 1

. If

liopt < 4

iopt [4] \neq 1

then E04LBF will be used.

Different optional arguments are available depending on the optimization function used. In all cases, using a value of

- 1

will cause the default value to be used. In addition only the first liopt values of iopt are used, so for example, if only the first element of iopt needs changing and default values for all other optional arguments are sufficient liopt can be set to

1

NAG Fortran Library routine E04LBF is being used

$i$	Description	Equivalent E04LBF argument	Default Value
$0$	Number of iterations	MAXCAL	$1000$
$1$	Unit number for monitoring information. See nag_open_file (x04acc) for details on how to assign a file to a unit number.	n/a	Output sent to stdout
$2$	Print optional arguments ( $1 =$ print)	n/a	$- 1$ (no printing performed)
$3$	Frequency that monitoring information is printed	IPRINT	$- 1$
$4$	Optimizer used	n/a	n/a

If requested, monitoring information is displayed in a similar format to that given by E04LBF.

NAG Fortran Library routine E04UCF is being used

$i$	Description	Equivalent E04UCF argument	Default Value
$0$	Number of iterations	Major Iteration Limit	$\max (50, 3 \times nvpr)$
$1$	Unit number for monitoring information. See nag_open_file (x04acc) for details on how to assign a file to a unit number.	n/a	Output sent to stdout
$2$	Print optional arguments ( $1 =$ print, otherwise no print)	List/NoList	$- 1$ (no printing performed)
$3$	Frequency that monitoring information is printed	Major Print Level	$0$
$4$	Optimizer used	n/a	n/a
$5$	Number of minor iterations	Minor Iteration Limit	$\max (50, 3 \times nvpr)$
$6$	Frequency that additional monitoring information is printed	Minor Print Level	$0$

liopt \leq 0

then default values are used for all optional arguments and iopt may be set to NULL.

23: liopt – IntegerInput

On entry: length of the options array iopt.

24: ropt[lropt] – const doubleInput

On entry: optional arguments passed to the optimization function.

Different optional arguments are available depending on the optimization function used. In all cases, using a value of

- 1.0

will cause the default value to be used. In addition only the first lropt values of ropt are used, so for example, if only the first element of ropt needs changing and default values for all other optional arguments are sufficient lropt can be set to

1

NAG Fortran Library routine E04LBF is being used

$i$	Description	Equivalent E04LBF argument	Default Value
$0$	Sweep tolerance	n/a	$\max (\sqrt{eps}, \sqrt{eps} \times \max_{i} ({zz}_{i i}))$
$1$	Accuracy of linear minimizations	ETA	$0.9$
$2$	Accuracy to which solution is required	XTOL	$0.0$
$3$	Initial distance from solution	STEPMX	$100000.0$

NAG Fortran Library routine E04UCF is being used

$i$	Description	Equivalent E04UCF argument	Default Value
$0$	Sweep tolerance	n/a	$\max (\sqrt{eps}, \sqrt{eps} \times \max_{i} ({zz}_{i i}))$
$1$	Lower bound for $γ^{*}$	n/a	$eps / 100$
$2$	Upper bound for $γ^{*}$	n/a	$10^{20}$
$3$	Line search tolerance	Line Search Tolerance	$0.9$
$4$	Optimality tolerance	Optimality Tolerance	${eps}^{0.72}$

where

eps

is the machine precision returned by nag_machine_precision (X02AJC) and

{zz}_{i i}

denotes the

i

diagonal element of

Z^{T} Z

lropt \leq 0

then default values are used for all optional arguments and ropt and may be set to NULL.

25: lropt – IntegerInput

On entry: length of the options array ropt.

26: fail – NagError *Input/Output

The NAG error argument (see Section 3.6 in the Essential Introduction).

6 Error Indicators and Warnings

NE_ALLOC_FAIL: Dynamic memory allocation failed.
NE_BAD_PARAM: On entry, argument $⟨value⟩$ had an illegal value.
NE_INT: On entry, $lb = ⟨value⟩$ .
Constraint: $lb \geq ⟨value⟩$ .
On entry, $lvpr = ⟨value⟩$ .
Constraint: $lvpr \geq ⟨value⟩$ .
On entry, $nvpr = ⟨value⟩$ .
Constraint: $1 \leq nvpr \leq ⟨value⟩$ .
On entry, $pdcxx = ⟨value⟩$ .
Constraint: $pdcxx \geq ⟨value⟩$ .
On entry, $pdcxz = ⟨value⟩$ .
Constraint: $pdcxz \geq ⟨value⟩$ .
On entry, $pdczz = ⟨value⟩$ .
Constraint: $pdczz \geq ⟨value⟩$ .
On entry, $pdid = ⟨value⟩$ .
Constraint: $pdid \geq ⟨value⟩$ .
NE_INT_ARRAY: On entry, at least one value of i, for $i = 1, 2, \dots, nvpr$ , does not appear in vpr.
On entry, icomm has not been initialized correctly.
On entry, $vpr [⟨value⟩] = ⟨value⟩$ and $nvpr = ⟨value⟩$ .
Constraint: $1 \leq vpr [i - 1] \leq nvpr$ .
NE_INTERNAL_ERROR: An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
NE_NEG_ELEMENT: At least one negative estimate for gamma was obtained. All negative estimates have been set to zero.
NE_REAL_ARRAY: On entry, $gamma [⟨value⟩] = ⟨value⟩$ .
Constraint: $gamma [i - 1] \geq 0$ .
NW_KT_CONDITIONS: Current point cannot be improved upon.
NW_NOT_CONVERGED: Optimal solution found, but requested accuracy not achieved.
NW_TOO_MANY_ITER: Too many major iterations.

7 Accuracy

Not applicable.

8 Parallelism and Performance

nag_ml_hier_mixed_regsn (g02jec) is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.

nag_ml_hier_mixed_regsn (g02jec) makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.

Please consult the Users' Note for your implementation for any additional implementation-specific information.

9 Further Comments

The argument vpr gives the mapping between the random variables and the variance components. In most cases

vpr [i - 1] = i

, for

i = 1, 2, \dots, \sum_{i} RNDM (1, i) + RNDM (2, i)

. However, in some cases it might be necessary to associate more than one random variable with a single variance component, for example, when the columns of DAT hold dummy variables.

Consider a dataset with three variables:

DAT = (\begin{matrix} 1 & 1 & 3.6 \\ 2 & 1 & 4.5 \\ 3 & 1 & 1.1 \\ 1 & 2 & 8.3 \\ 2 & 2 & 7.2 \\ 3 & 2 & 6.1 \end{matrix})

where the first column corresponds to a categorical variable with three levels, the next to a categorical variable with two levels and the last column to a continuous variable. So in a call to nag_hier_mixed_init (g02jcc)

levels = (\begin{matrix} 3 & 2 & 1 \end{matrix})

also assume a model with no fixed effects, no random intercept, no nesting and all three variables being included as random effects, then

\begin{matrix} fixed = (\begin{matrix} 0 & 0 \end{matrix}); \\ RNDM = {(\begin{matrix} 3 & 0 & 1 & 2 & 3 \end{matrix})}^{T} . \end{matrix}

Each of the three columns in DAT therefore correspond to a single variable and hence there are three variance components, one for each random variable included in the model, so

vpr = (\begin{matrix} 1 & 2 & 3 \end{matrix}) .

This is the recommended way of supplying the data to nag_ml_hier_mixed_regsn (g02jec), however it is possible to reformat the above dataset by replacing each of the categorical variables with a series of dummy variables, one for each level. The dataset then becomes

DAT = (\begin{matrix} 1 & 0 & 0 & 1 & 0 & 3.6 \\ 0 & 1 & 0 & 1 & 0 & 4.5 \\ 0 & 0 & 1 & 1 & 0 & 1.1 \\ 1 & 0 & 0 & 0 & 1 & 8.3 \\ 0 & 1 & 0 & 0 & 1 & 7.2 \\ 0 & 0 & 1 & 0 & 1 & 6.1 \end{matrix})

where each column only has one level

levels = (\begin{matrix} 1 & 1 & 1 & 1 & 1 & 1 \end{matrix}) .

Again a model with no fixed effects, no random intercept, no nesting and all variables being included as random effects is required, so

\begin{matrix} fixed = (\begin{matrix} 0 & 0 \end{matrix}); \\ RNDM = {(\begin{matrix} 6 & 0 & 1 & 2 & 3 & 4 & 5 & 6 \end{matrix})}^{T} . \end{matrix}

With the data entered in this manner, the first three columns of DAT correspond to a single variable (the first column of the original dataset) as do the next two columns (the second column of the original dataset). Therefore vpr must reflect this

vpr = (\begin{matrix} 1 & 1 & 1 & 2 & 2 & 3 \end{matrix}) .

In most situations it is more efficient to supply the data to nag_hier_mixed_init (g02jcc) in terms of categorical variables rather than transform them into dummy variables.

10 Example

This example fits a random effects model with three levels of nesting to a simulated dataset with

90

observations and

12

variables.

NAG Library Function Documentnag_ml_hier_mixed_regsn (g02jec)

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Parallelism and Performance

9 Further Comments

10 Example

10.1 Program Text

10.2 Program Data

10.3 Program Results

NAG Library Function Document

nag_ml_hier_mixed_regsn (g02jec)