A quantile is a value which divides a frequency distribution such that there is a given proportion of data values below the quantile. For example, the median of a dataset is the quantile because half the values are less than or equal to it.
g01apc uses a slightly modified version of an algorithm described in a paper by
Zhang and Wang (2007) to determine
-approximate quantiles of a large arbitrary-sized data stream of real values, where
is a user-defined approximation factor. Let
denote the number of data elements processed so far then, given any quantile
, an
-approximate quantile is defined as an element in the data stream whose rank falls within
. In case of more than one
-approximate quantile being available, the one closest to
is used.
Zhang Q and Wang W (2007) A fast algorithm for approximate quantiles in high speed data streams Proceedings of the 19th International Conference on Scientific and Statistical Database Management IEEE Computer Society 29
-
1:
– Integer *
Input/Output
-
On initial entry: must be set to .
On entry: indicates the action required in the current call to
g01apc.
- Initialize the communication arrays and attempt to process the first nb values from the data stream. eps, rv and nb must be set and licomm must be at least .
- Attempt to process the next block of nb values from the data stream. The calling program must update rv and (if required) nb, and re-enter g01apc with all other parameters unchanged.
- Continue calculation following the reallocation of either or both of the communication arrays rcomm and icomm.
- Calculate the nq -approximate quantiles specified in q. The calling program must set q and nq and re-enter g01apc with all other parameters unchanged. This option can be chosen only when .
On exit: indicates output from the call.
- g01apc has processed np data points and expects to be called again with additional data.
- Either one or more of the communication arrays rcomm and icomm is too small. The new minimum lengths of rcomm and icomm have been returned in and respectively. If the new minimum length is greater than the current length then the corresponding communication array needs to be reallocated, its contents preserved and g01apc called again with all other parameters unchanged.
If there is more data to be processed, it is recommended that
lrcomm and
licomm are made significantly bigger than the minimum to limit the number of reallocations.
- g01apc has returned the requested -approximate quantiles in qv. These quantiles are based on np data points.
Constraint:
, , or .
-
2:
– const double
Input
-
Note: the dimension,
dim, of the array
rv
must be at least
On entry: if
,
or
, the vector containing the current block of data, otherwise
rv is not referenced.
-
3:
– Integer
Input
-
On entry: if
,
or
, the size of the current block of data. The size of blocks of data in array
rv can vary;, therefore,
nb can change between calls to
g01apc.
Constraint:
if , or , .
-
4:
– double
Input
-
On entry: approximation factor .
Constraint:
.
-
5:
– Integer *
Output
-
On exit: , the number of elements processed so far.
-
6:
– const double
Input
-
Note: the dimension,
dim, of the array
q
must be at least
On entry: if
, the quantiles to be calculated, otherwise
q is not referenced. Note that
, corresponds to the minimum value and
to the maximum value.
Constraint:
if ,
, for .
-
7:
– double
Output
-
Note: the dimension,
dim, of the array
qv
must be at least
On exit: if , contains the -approximate quantiles specified by the value provided in .
-
8:
– Integer
Input
-
On entry: if
, the number of quantiles requested, otherwise
nq is not referenced.
Constraint:
if , .
-
9:
– double
Communication Array
-
On entry: if
or
then the first
elements of
rcomm as supplied to
g01apc must be identical to the first
elements of
rcomm returned from the last call to
g01apc, where
is the value of
lrcomm used in the last call. In other words, the contents of
rcomm must not be altered between calls to this function. If
rcomm needs to be reallocated then its contents must be preserved. If
then
rcomm need not be set.
On exit:
rcomm holds information required by subsequent calls to
g01apc.
-
10:
– Integer
Input
-
On entry: the dimension of the array
rcomm.
Constraints:
- if , ;
- otherwise .
-
11:
– Integer
Communication Array
-
On entry: if
or
then the first
elements of
icomm as supplied to
g01apc must be identical to the first
elements of
icomm returned from the last call to
g01apc, where
is the value of
licomm used in the last call. In other words, the contents of
icomm must not be altered between calls to this function. If
icomm needs to be reallocated then its contents must be preserved. If
then
icomm need not be set.
On exit:
holds the minimum required length for
rcomm and
holds the minimum required length for
icomm. The remaining elements of
icomm are used for communication between subsequent calls to
g01apc.
-
12:
– Integer
Input
-
On entry: the dimension of the array
icomm.
Constraints:
- if , ;
- otherwise .
-
13:
– NagError *
Input/Output
-
The
NAG error argument (see
Section 7 in the Introduction to the
NAG Library CL Interface).
- NE_ALLOC_FAIL
-
Dynamic memory allocation failed.
See
Section 3.1.2 in the Introduction to the
NAG Library CL Interface for further information.
- NE_ARRAY_SIZE
-
On entry, .
Constraint: .
On entry, .
Constraint: .
- NE_BAD_PARAM
-
On entry, argument had an illegal value.
- NE_ILLEGAL_COMM
-
The contents of
icomm have been altered between calls to this function.
The contents of
rcomm have been altered between calls to this function.
- NE_INT
-
On entry, , or and .
Constraint: if , or then .
On entry, and .
Constraint: if then .
On entry, .
Constraint: , , or .
- NE_INTERNAL_ERROR
-
An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact
NAG for assistance.
See
Section 7.5 in the Introduction to the
NAG Library CL Interface for further information.
- NE_NO_LICENCE
-
Your licence key may have expired or may not have been installed correctly.
See
Section 8 in the Introduction to the
NAG Library CL Interface for further information.
- NE_Q_OUT_OF_RANGE
-
On entry, and .
Constraint: if then for all .
- NE_REAL
-
On entry, .
Constraint: .
- NE_TOO_SMALL
-
Number of data elements streamed,
is not sufficient for a quantile query when
.
Supply more data or reprocess the data with a higher
eps value.
Not applicable.
Background information to multithreading can be found in the
Multithreading documentation.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this function. Please also consult the
Users' Note for your implementation for any additional implementation-specific information.
It is not possible to determine in advance the final size of the communication arrays
rcomm and
icomm without knowing the size of the dataset. However, if a rough size (
) is known, the speed of the computation can be increased if the sizes of the communication arrays are not smaller than
where