NAG Library Routine Document

G01WAF

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Parameters

6 Error Indicators and Warnings

7 Accuracy

8 Further Comments

+− 9 Example

9.1 Program Text

9.2 Program Data

9.3 Program Results

1 Purpose

G01WAF calculates the mean and, optionally, the standard deviation using a rolling window for an arbitrary-sized data stream.

2 Specification

SUBROUTINE G01WAF (

M, NB, X, IWT, WT, PN, RMEAN, RSD, LRSD, RCOMM, LRCOMM, IFAIL)

INTEGER	M, NB, IWT, PN, LRSD, LRCOMM, IFAIL
REAL (KIND=nag_wp)	X(NB), WT(*), RMEAN(max(0,NB+min(0,PN-M+1))), RSD(LRSD), RCOMM(LRCOMM)

3 Description

Given a sample of

n

observations, denoted by

x = \{x_{i} : i = 1, 2, \dots, n\}

and a set of weights,

w = \{w_{j} : j = 1, 2, \dots, m\}

, G01WAF calculates the mean and, optionally, the standard deviation, in a rolling window of length

m

The mean is defined as

μ_{i} = \frac{\sum_{j = 1}^{m} w_{j} x_{i + j - 1}}{W}

(1)

and the standard deviation as

σ_{i} = \sqrt{\frac{\sum_{j = 1}^{m} w_{j} {(x_{i + j - 1} - μ_{i})}^{2}}{W - \frac{\sum_{j = 1}^{m} w_{j}^{2}}{W}}}

(2)

with

W = \sum_{j = 1}^{m} w_{j}

Four different types of weighting are possible:

(i)

No weights ( $w_{j} = 1$ )

When no weights are required both the mean and standard deviations can be calculated in an iterative manner, with

\begin{matrix} μ_{i + 1} = & μ_{i} + \frac{(x_{i + m} - x_{i})}{m} \\ σ_{i + 1} = & σ_{i} + {(x_{i + m} - μ_{i})}^{2} - {(x_{i} - μ_{i})}^{2} - \frac{{(x_{i + m} - x_{i})}^{2}}{m} \end{matrix}

where the initial values

μ_{1}

and

σ_{1}

are obtained using the one pass algorithm of West (1979).

(ii)

Each observation has its own weight

In this case, rather than supplying a vector of

m

weights a vector of

n

weights is supplied instead,

v = \{v_{j} : j = 1, 2, \dots, n\}

and

w_{j} = v_{i + j - 1}

in (1) and (2).

If the standard deviations are not required then the mean is calculated using the iterative formula:

\begin{matrix} W_{i + 1} = & W_{i} + (v_{i + m} - v_{i}) \\ μ_{i + 1} = & W_{i} μ_{i} + (v_{i + m} x_{i + m} - v_{i} x_{i}) \end{matrix}

where

W_{1} = \sum_{i = 1}^{m} v_{i}

and

μ_{1} = W_{1}^{- 1} \sum_{i = 1}^{m} v_{i} x_{i}

If both the mean and standard deviation are required then the one pass algorithm of West is applied multiple times.

(iii)

Each position in the window has its own weight

This is the case as described in (1) and (2), where the weight given to each observation differs depending on which summary is being produced. When these types of weights are specified both the mean and standard deviation are calculated by applying the one pass algorithm of West multiple times.

(iv)

Each position in the window has a weight equal to its position number ( $w_{j} = j$ )

This is a special case of (iii).

If the standard deviations are not required then the mean is calculated using the iterative formula:

\begin{matrix} S_{i + 1} = & S_{i} + (x_{i + m} - x_{i}) \\ μ_{i + 1} = & μ_{i} + \frac{2 (m x_{i + m} - S_{i})}{m (m + 1)} \end{matrix}

where

S_{1} = \sum_{i = 1}^{m} x_{i}

and

μ_{1} = 2 {(m^{2} + m)}^{- 1} S_{1}

If both the mean and standard deviation are required then the one pass algorithm of West is applied multiple times.

For large datasets, or where all the data is not available at the same time,

x

(and if each observation has its own weight,

v

) can be split into arbitrary sized blocks and G01WAF called multiple times.

4 References

West D H D (1979) Updating mean and variance estimates: An improved method Comm. ACM 22 532–555

5 Parameters

1: M – INTEGERInput

On entry:

m

, the length of the rolling window.

PN \neq 0

, M must be unchanged since the last call to G01WAF.

Constraint:

M \geq 1

2: NB – INTEGERInput

On entry:

b

, the number of observations in the current block of data. The size of the block of data supplied in X (and when

IWT = 1

, WT) can vary; therefore NB can change between calls to G01WAF.

Constraints:

$NB \geq 0$ ;
if $LRCOMM = 0$ , $NB \geq M$ .

3: X(NB) – REAL (KIND=nag_wp) arrayInput

On entry: the current block of observations, corresponding to

x_{i}

, for

i = k + 1, \dots, k + b

, where

k

is the number of observations processed so far and

b

is the size of the current block of data.

4: IWT – INTEGERInput

On entry: the type of weighting to use.

$IWT = 0$: No weights are used.
$IWT = 1$: Each observation has its own weight.
$IWT = 2$: Each position in the window has its own weight.
$IWT = 3$: Each position in the window has a weight equal to its position number.

PN \neq 0

, IWT must be unchanged since the last call to G01WAF.

Constraint:

IWT = 0

1

2

3

5: WT( $*$ ) – REAL (KIND=nag_wp) arrayInput

Note: the dimension of the array WT must be at least

NB

IWT = 1

and at least

M

IWT = 2

On entry: the user-supplied weights.

IWT = 1

WT (i) = v_{i}

, for

i = 1, 2, \dots, n

IWT = 2

WT (j) = w_{j}

, for

j = 1, 2, \dots, m

Otherwise, WT is not referenced.

Constraints:

if $IWT = 1$ , $WT (i) \geq 0$ , for $i = 1, 2, \dots, NB$ ;
if $IWT = 2$ , $WT (1) \neq 0$ and $\sum_{i = 1}^{m} WT (i) > 0$ ;
if $IWT = 2$ and $LRSD \neq 0$ , $WT (i) \geq 0$ , for $i = 1, 2, \dots, M$ .

6: PN – INTEGERInput/Output

On entry:

k

, the number of observations processed so far. On the first call to G01WAF, or when starting to summarise a new dataset, PN must be set to

0

PN \neq 0

, it must be the same value as returned by the last call to G01WAF.

On exit:

k + b

, the updated number of observations processed so far.

Constraint:

PN \geq 0

7: RMEAN( $\max (0, NB + \min (0, PN - M + 1))$ ) – REAL (KIND=nag_wp) arrayOutput

On exit:

μ_{l}

, the (weighted) moving averages, for

l = 1, 2, \dots, b + \min (0, k - m + 1)

. Where

μ_{l}

is the summary to the window that ends on

X (l + m - \min (k, m - 1) - 1)

. Therefore, if, on entry,

PN \geq M - 1

RMEAN (l)

is the summary corresponding to the window that ends on

X (l)

and if, on entry,

PN = 0

RMEAN (l)

is the summary corresponding to the window that ends on

X (M + l - 1)

(or, equivalently, starts on

X (l)

8: RSD(LRSD) – REAL (KIND=nag_wp) arrayOutput

On exit: if

LRSD \neq 0

then

σ_{l}

, the (weighted) standard deviation. The ordering of RSD is the same as the ordering of RMEAN.

LRSD = 0

, RSD is not referenced.

9: LRSD – INTEGERInput

On entry: the dimension of the array RSD as declared in the (sub)program from which G01WAF is called. If the standard deviations are not required then LRSD should be set to zero.

Constraint:

LRSD = 0

LRSD \geq \max (0, NB + \min (0, PN - M + 1))

10: RCOMM(LRCOMM) – REAL (KIND=nag_wp) arrayCommunication Array

On entry: communication array, used to store information between calls to G01WAF. If

LRCOMM = 0

, RCOMM is not referenced and all the data must be supplied in one go.

11: LRCOMM – INTEGERInput

On entry: the dimension of the array RCOMM as declared in the (sub)program from which G01WAF is called.

Constraint:

LRCOMM = 0

LRCOMM \geq 2 M + 20

12: IFAIL – INTEGERInput/Output

On entry: IFAIL must be set to

0

- 1 ​ or ​ 1

. If you are unfamiliar with this parameter you should refer to Section 3.3 in the Essential Introduction for details.

For environments where it might be inappropriate to halt program execution when an error is detected, the value

- 1 ​ or ​ 1

is recommended. If the output of error messages is undesirable, then the value

1

is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is

0

. When the value $- 1 or 1$ is used it is essential to test the value of IFAIL on exit.

On exit:

IFAIL = 0

unless the routine detects an error or a warning has been flagged (see Section 6).

6 Error Indicators and Warnings

If on entry

IFAIL = 0

- 1

, explanatory error messages are output on the current error message unit (as defined by X04AAF).

Errors or warnings detected by the routine:

$IFAIL = 11$: On entry, $M = ⟨value⟩$ .
Constraint: $M \geq 1$ .

$IFAIL = 12$: On entry, $M = ⟨value⟩$ .
On entry at previous call, $M = ⟨value⟩$ .
Constraint: if $PN > 0$ , M must be unchanged since previous call.

$IFAIL = 21$: On entry, $NB = ⟨value⟩$ .
Constraint: $NB \geq 0$ .

$IFAIL = 22$: On entry, $NB = ⟨value⟩$ , $M = ⟨value⟩$ .
Constraint: if $LRCOMM = 0$ , $NB \geq M$ .

$IFAIL = 41$: On entry, $IWT = ⟨value⟩$ .
Constraint: $IWT = 0$ , $1$ , $2$ or $3$ .

$IFAIL = 42$: On entry, $IWT = ⟨value⟩$ .
On entry at previous call, $IWT = ⟨value⟩$ .
Constraint: if $PN > 0$ , IWT must be unchanged since previous call.

$IFAIL = 51$: On entry, $WT (⟨value⟩) = ⟨value⟩$ .
Constraint: $WT (i) \geq 0$ .

$IFAIL = 52$: On entry, $WT (1) = ⟨value⟩$ .
Constraint: if $IWT = 2$ , $WT (1) > 0$ .

$IFAIL = 53$: On entry, at least one window had all zero weights.

$IFAIL = 54$: On entry, unable to calculate at least one standard deviation due to the weights supplied.

$IFAIL = 55$: On entry, sum of weights supplied in WT is $⟨value⟩$ .
Constraint: if $IWT = 2$ , the sum of the weights $> 0$ .

$IFAIL = 61$: On entry, $PN = ⟨value⟩$ .
Constraint: $PN \geq 0$ .

$IFAIL = 62$: On entry, $PN = ⟨value⟩$ .
On exit from previous call, $PN = ⟨value⟩$ .
Constraint: if $PN > 0$ , PN must be unchanged since previous call.

$IFAIL = 91$: On entry, $LRSD = ⟨value⟩$ .
Constraint: $LRSD = 0$ or $LRSD \geq ⟨value⟩$ .

$IFAIL = 101$: RCOMM has been corrupted between calls.

$IFAIL = 111$: On entry, $LRCOMM = ⟨value⟩$ .
Constraint: $LRCOMM \geq ⟨value⟩$ . On entry, $LRCOMM = ⟨value⟩$ .
Constraint: $LRCOMM \geq ⟨value⟩$ .

$IFAIL = - 999$: Dynamic memory allocation failed.

7 Accuracy

Not applicable.

8 Further Comments

The more data that is supplied to G01WAF in one call, i.e., the larger NB is, the more efficient the routine will be. In addition, where possible, the input parameters should be chosen so that G01WAF can use the iterative formula as described in Section 3.

9 Example

This example calculates Spencer's

15

-point moving average for the change in rate of the Earth's rotation between

1821

and

1850

. The data is supplied in three chunks, the first consisting of five observations, the second

10

observations and the last

15

observations.

This example plot shows the smoothing effect of using different length rolling windows on the mean and standard deviation. Two different window lengths,

m = 5

and

10

, are used to produce the unweighted rolling mean and standard deviations for the change in rate of the Earth's rotation between

1821

and

1850

G01WAF (PDF version)

G01 Chapter Contents

G01 Chapter Introduction

NAG Library Manual

NAG Library Routine DocumentG01WAF

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Parameters

6 Error Indicators and Warnings

7 Accuracy

8 Further Comments

9 Example

9.1 Program Text

9.2 Program Data

9.3 Program Results

NAG Library Routine Document

G01WAF