naginterfaces.library.stat.moving_​average

naginterfaces.library.stat.moving_average(m, x, iwt=0, wt=None, pn=0, wantsd=False, comm=None)

moving_average calculates the mean and, optionally, the standard deviation using a rolling window for an arbitrary sized data stream.

For full information please refer to the NAG Library document for g01wa

https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/g01/g01waf.html

Parameters

mint

$m$, the length of the rolling window.

If $pn\ne 0$, $m$ must be unchanged since the last call to moving_average.

xfloat, array-like, shape $\left(\text{nb}\right)$

The current block of observations, corresponding to $x_{\text{i}}$, for $\text{i}=k+1,\dots ,k+b$, where $k$ is the number of observations processed so far and $b$ is the size of the current block of data.

iwtint, optional

The type of weighting to use.

$iwt=0$

No weights are used.

$iwt=1$

Each observation has its own weight.

$iwt=2$

Each position in the window has its own weight.

$iwt=3$

Each position in the window has a weight equal to its position number.

If $pn\ne 0$, $iwt$ must be unchanged since the last call to moving_average.

wtNone or float, array-like, shape $(:)$, optional

Note: the required length for this argument is determined as follows: if $iwt=1$: $\text{nb}$; if $iwt=2$: $m$; otherwise: $0$.

The user-supplied weights.

If $iwt=1$, $wt[\text{i}-1]={\nu }_{\text{i}+k}$, for $\text{i}=1,2,\dots ,b$.

If $iwt=2$, $wt[\text{j}-1]=w_{\text{j}}$, for $\text{j}=1,2,\dots ,m$.

pnint, optional

$k$, the number of observations processed so far. On the first call to moving_average, or when starting to summarise a new dataset, $pn$ must be set to $0$.

If $pn\ne 0$, it must be the same value as returned by the last call to moving_average.

wantsdbool, optional

If the standard deviations are required then $wantsd$ should be set to $True$.

commNone or dict, communication object, optional, modified in place

Communication structure.

If $None$ all the data must be supplied in one go, otherwise need not be set.

Returns

pnint

$k+b$, the updated number of observations processed so far.

rmeanfloat, ndarray, shape $(max(0,\text{nb}+min(0,pn-m+1)))$

${\mu }_{\text{l}}$, the (weighted) moving averages, for $\text{l}=1,2,\dots ,b+min(0,k-m+1)$. Therefore, ${\mu }_l$ is the mean of the data in the window that ends on $x[l+m-min(k,m-1)-2]$.

If, on entry, $pn\ge m-1$, i.e., at least one windows worth of data has been previously processed, then $rmean[l-1]$ is the summary corresponding to the window that ends on $x[l-1]$.

On the other hand, if, on entry, $pn=0$, i.e., no data has been previously processed, then $rmean[l-1]$ is the summary corresponding to the window that ends on $x[m+l-2]$ (or, equivalently, starts on $x[l-1]$).

rsdNone or float, ndarray, shape $(:)$

If $wantsd=True$ then ${\sigma }_l$, the (weighted) standard deviation. The ordering of $rsd$ is the same as the ordering of $rmean$.

Raises

NagValueError

(errno $11$)

On entry, $m=⟨value⟩$.

Constraint: $m\ge 1$.

(errno $12$)

On entry, $m=⟨value⟩$.

On entry at previous call, $m=⟨value⟩$.

Constraint: if $pn>0$, $m$ must be unchanged since previous call.

(errno $21$)

On entry, $\text{nb}=⟨value⟩$.

Constraint: $\text{nb}\ge 0$.

(errno $22$)

On entry, $\text{nb}=⟨value⟩$, $m=⟨value⟩$.

Constraint: if $comm$ is None, $\text{nb}\ge m$.

(errno $41$)

On entry, $iwt=⟨value⟩$.

Constraint: $iwt=0$, $1$, $2$ or $3$.

(errno $42$)

On entry, $iwt=⟨value⟩$.

On entry at previous call, $iwt=⟨value⟩$.

Constraint: if $pn>0$, $iwt$ must be unchanged since previous call.

(errno $51$)

On entry, $wt[⟨value⟩]=⟨value⟩$.

Constraint: $wt[i-1]\ge 0$.

(errno $52$)

On entry, $wt\left[0\right]=⟨value⟩$.

Constraint: if $iwt=2$, $wt\left[0\right]>0$.

(errno $55$)

On entry, sum of weights supplied in $wt$ is $⟨value⟩$.

Constraint: if $iwt=2$, the sum of the weights $>0$.

(errno $61$)

On entry, $pn=⟨value⟩$.

Constraint: $pn\ge 0$.

(errno $62$)

On entry, $pn=⟨value⟩$.

On exit from previous call, $pn=⟨value⟩$.

Constraint: if $pn>0$, $pn$ must be unchanged since previous call.

(errno $101$)

$comm$[‘rcomm’] has been corrupted between calls.

Warns

NagAlgorithmicWarning

(errno $53$)

On entry, at least one window had all zero weights.

(errno $54$)

On entry, unable to calculate at least one standard deviation due to the weights supplied.

Notes

Given a sample of $n$ observations, denoted by $x=\{x_i:i=1,2,\dots ,n\}$ and a set of weights, $w=\{w_j:j=1,2,\dots ,m\}$, moving_average calculates the mean and, optionally, the standard deviation, in a rolling window of length $m$.

For the $i$th window the mean is defined as

$${\mu }_i=\frac{{\sum }_{j=1}^m{w}_j{x}_{i+j-1}}{W}$$

and the standard deviation as

$${\sigma }_i=\sqrt{\frac{{\sum }_{j=1}^m{w}_j{(x_{i+j-1}-{\mu }_i)}^2}{W-\frac{{\sum }_{j=1}^m{w}_j^2}{W}}}$$

with $W={\sum }_{j=1}^m{w}_j$.

Four different types of weighting are possible:

1. No weights ( $w_j=1$ )

   When no weights are required both the mean and standard deviations can be calculated in an iterative manner, with

   $$\begin{array}{c}\begin{array}{cc}{\mu }_{i+1}=& {\mu }_i+\frac{(x_{i+m}-x_i)}{m}\\ {\sigma }_{i+1}^2=& (m-1){\sigma }_i^2+{(x_{i+m}-{\mu }_i)}^2-{(x_i-{\mu }_i)}^2-\frac{{(x_{i+m}-x_i)}^2}{m}\end{array}\end{array}$$

   where the initial values ${\mu }_1$ and ${\sigma }_1$ are obtained using the one pass algorithm of West (1979).

2. Each observation has its own weight

   In this case, rather than supplying a vector of $m$ weights a vector of $n$ weights is supplied instead, $v=\{v_j:j=1,2,\dots ,n\}$ and $w_j=v_{i+j-1}$ in [equation] and [equation].

   If the standard deviations are not required then the mean is calculated using the iterative formula:

   $$\begin{array}{c}\begin{array}{cc}{W}_{i+1}=& W_i+(v_{i+m}-v_i)\\ {\mu }_{i+1}=& {\mu }_i+W_i^{-1}(v_{i+m}{x}_{i+m}-v_i{x}_i)\end{array}\end{array}$$

   where $W_1={\sum }_{i=1}^m{v}_i$ and ${\mu }_1=W_1^{-1}{\sum }_{i=1}^m{v}_i{x}_i$.

   If both the mean and standard deviation are required then the one pass algorithm of West (1979) is used in each window.

3. Each position in the window has its own weight

   This is the case as described in [equation] and [equation], where the weight given to each observation differs depending on which summary is being produced. When these types of weights are specified both the mean and standard deviation are calculated by applying the one pass algorithm of West (1979) multiple times.

4. Each position in the window has a weight equal to its position number ( $w_j=j$ )

   This is a special case of (iii).

   If the standard deviations are not required then the mean is calculated using the iterative formula:

   $$\begin{array}{c}\begin{array}{cc}{S}_{i+1}=& S_i+(x_{i+m}-x_i)\\ {\mu }_{i+1}=& {\mu }_i+\frac{2(m{x}_{i+m}-S_i)}{m(m+1)}\end{array}\end{array}$$

   where $S_1={\sum }_{i=1}^m{x}_i$ and ${\mu }_1=2{(m^2+m)}^{-1}{S}_1$.

   If both the mean and standard deviation are required then the one pass algorithm of West is applied multiple times.

For large datasets, or where all the data is not available at the same time, $x$ (and if each observation has its own weight, $v$) can be split into arbitrary sized blocks and moving_average called multiple times.

References

Chan, T F, Golub, G H and Leveque, R J, 1982, Updating Formulae and a Pairwise Algorithm for Computing Sample Variances, Compstat, Physica-Verlag

West, D H D, 1979, Updating mean and variance estimates: An improved method, Comm. ACM (22), 532–555

See also

naginterfaces.library.examples.stat.moving_average_ex.main()