naginterfaces.library.stat.quantiles_stream_fixed¶

naginterfaces.library.stat.quantiles_stream_fixed(ind, n, rv, nb, eps, q, nq, rcomm, icomm)[source]¶

quantiles_stream_fixed finds approximate quantiles from a data stream of known size using an out-of-core algorithm.

For full information please refer to the NAG Library document for g01an

https://support.nag.com/numeric/nl/nagdoc_30.3/flhtml/g01/g01anf.html

Parameters

indint

Indicates the action required in the current call to quantiles_stream_fixed.

$i n d = 0$

Return the required length of $r c o m m$ and $i c o m m$ in $i c o m m [0]$ and $i c o m m [1]$ respectively. $n$ and $e p s$ must be set and $licomm$ must be at least $2$ .

$i n d = 1$

Initialise the communication arrays and process the first $n b$ values from the data stream as supplied in $r v$ .

$i n d = 2$

Process the next block of $n b$ values from the data stream. The calling program must update $r v$ and (if required) $n b$ , and re-enter quantiles_stream_fixed with all other parameters unchanged.

$i n d = 3$

Calculate the $n q$ $ϵ$ -approximate quantiles specified in $q$ . The calling program must set $q$ and $n q$ and re-enter quantiles_stream_fixed with all other parameters unchanged. This option can be chosen only when $n p \geq ⌈ e x p (1.0) / e p s ⌉$ .

nint

$n$ , the total number of values in the data stream.

rvfloat, array-like, shape $(:)$

Note: the required length for this argument is determined as follows: if $i n d in (1, 2)$ : $n b$ ; otherwise: $0$ .

If $i n d = 1$ or $2$ , the vector containing the current block of data, otherwise $r v$ is not referenced.

nbint

If $i n d = 1$ or $2$ , the size of the current block of data. The size of blocks of data in array $r v$ can vary;, therefore, $n b$ can change between calls to quantiles_stream_fixed.

epsfloat

Approximation factor $ϵ$ .

qfloat, array-like, shape $(:)$

Note: the required length for this argument is determined as follows: if $i n d = 3$ : $n q$ ; otherwise: $0$ .

If $i n d = 3$ , the quantiles to be calculated, otherwise $q$ is not referenced. Note that $q [i] = 0.0$ , corresponds to the minimum value and $q [i] = 1.0$ to the maximum value.

nqint

If $i n d = 3$ , the number of quantiles requested, otherwise $n q$ is not referenced.

rcommfloat, ndarray, shape $(:)$ , modified in place

Note: the required length for this argument is determined as follows: if $i n d \neq 0$ : the value returned in $i c o m m [0]$ by a call to quantiles_stream_fixed with $i n d = 0$ ; otherwise: $0$ .

Communication array.

icommint, ndarray, shape $(:)$ , modified in place

Note: the required length for this argument is determined as follows: if $i n d \neq 0$ : the value returned in $i c o m m [1]$ by a call to quantiles_stream_fixed with $i n d = 0$ ; otherwise: $2$ .

Communication array.

Returns

indint

Indicates output from a successful call.

$i n d = 1$

Lengths of $r c o m m$ and $i c o m m$ have been returned in $i c o m m [0]$ and $i c o m m [1]$ respectively.

$i n d = 2$

quantiles_stream_fixed has processed $n p$ data points and expects to be called again with additional data (i.e., $n p < n$ ).

$i n d = 3$

quantiles_stream_fixed has returned the requested $ϵ$ -approximate quantiles in $q v$ . These quantiles are based on $n p$ data points.

$i n d = 4$

Routine has processed all $n$ data points (i.e., $n p = n$ ).

npint

The number of elements processed so far.

qvfloat, ndarray, shape $(:)$

If $i n d = 3$ , $q v [i]$ contains the $ϵ$ -approximate quantiles specified by the value provided in $q [i]$ .

Raises

NagValueError

(errno $1$ )

On entry, $i n d = ⟨ v a l u e ⟩$ .

Constraint: $i n d = 0$ , $1$ , $2$ or $3$ .

(errno $2$ )

On entry, $n = ⟨ v a l u e ⟩$ .

Constraint: $n > 0$ .

(errno $3$ )

On entry, $e p s = ⟨ v a l u e ⟩$ .

Constraint: $e x p (1.0) / n \leq e p s \leq 1.0$ .

(errno $4$ )

On entry, $i n d = 1$ or $2$ and $n b = ⟨ v a l u e ⟩$ .

Constraint: if $i n d = 1$ or $2$ then $n b > 0$ .

(errno $7$ )

Number of data elements streamed, $⟨ v a l u e ⟩$ is not sufficient for a quantile query when $e p s = ⟨ v a l u e ⟩$ .

Supply more data or reprocess the data with a higher $e p s$ value.

(errno $8$ )

On entry, $i n d = 3$ and $n q = ⟨ v a l u e ⟩$ .

Constraint: if $i n d = 3$ then $n q > 0$ .

(errno $9$ )

On entry, $i n d = 3$ and $q [⟨ v a l u e ⟩] = ⟨ v a l u e ⟩$ .

Constraint: if $i n d = 3$ then $0.0 \leq q [i] \leq 1.0$ for all $i$ .

Notes

A quantile is a value which divides a frequency distribution such that there is a given proportion of data values below the quantile. For example, the median of a dataset is the $0.5$ quantile because half the values are less than or equal to it.

quantiles_stream_fixed uses a slightly modified version of an algorithm described in a paper by Zhang and Wang (2007) to determine $ϵ$ -approximate quantiles of a data stream of $n$ real values, where $n$ is known. Given any quantile $q \in [0.0, 1.0]$ , an $ϵ$ -approximate quantile is defined as an element in the data stream whose rank falls within $[(q - ϵ) n, (q + ϵ) n]$ . In case of more than one $ϵ$ -approximate quantile being available, the one closest to $q n$ is returned.

References: Zhang, Q and Wang, W, 2007, A fast algorithm for approximate quantiles in high speed data streams, Proceedings of the 19th International Conference on Scientific and Statistical Database Management, IEEE Computer Society, 29

NAG and Python

Return to Front

naginterfaces.library.stat.quantiles_stream_fixed¶

naginterfaces.library.stat.quantiles_​stream_​fixed¶

naginterfaces.library.stat.quantiles_stream_fixed¶