The routine may be called by the names f01saf or nagf_matop_real_nmf.
3Description
The matrix $A$ is factorized into the product of an $m\times k$ matrix $W$ and a $k\times n$ matrix $H$, both with non-negative elements. The factorization is approximate, $A\approx WH$, with $W$ and $H$ chosen to minimize the functional
$$f(W,H)={\Vert A-WH\Vert}_{F}^{2}\text{.}$$
You are free to choose any value for $k$, provided $k<\mathrm{min}\phantom{\rule{0.125em}{0ex}}(m,n)$. The product $WH$ will then be a low-rank approximation to $A$, with rank at most $k$.
f01saf finds $W$ and $H$ using an iterative method known as the Hierarchical Alternating Least Squares algorithm. You may specify initial values for $W$ and $H$, or you may provide a seed value for f01saf to generate the initial values using a random number generator.
4References
Cichocki A and Phan A–H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations IEICE Transactions on Fundamentals of Electronics, Communications and Computer SciencesE92–A 708–721
Cichocki A, Zdunek R and Amari S–I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization Lecture Notes in Computer Science4666 Springer 169–176
Ho N–D (2008) Nonnegative matrix factorization algorithms and applications PhD Thesis Univ. Catholique de Louvain
5Arguments
1: $\mathbf{m}$ – IntegerInput
On entry: $m$, the number of rows of the matrix $A$. Also the number of rows of the matrix $W$.
Constraint:
${\mathbf{m}}\ge 2$.
2: $\mathbf{n}$ – IntegerInput
On entry: $n$, the number of columns of the matrix $A$. Also the number of columns of the matrix $H$.
Constraint:
${\mathbf{n}}\ge 2$.
3: $\mathbf{k}$ – IntegerInput
On entry: $k$, the number of columns of the matrix $W$; the number of rows of the matrix $H$. See Section 9.2 for further details.
4: $\mathbf{a}({\mathbf{lda}},*)$ – Real (Kind=nag_wp) arrayInput
Note: the second dimension of the array a
must be at least
${\mathbf{n}}$.
On entry: the $m\times n$ non-negative matrix $A$.
5: $\mathbf{lda}$ – IntegerInput
On entry: the first dimension of the array a as declared in the (sub)program from which f01saf is called.
Constraint:
${\mathbf{lda}}\ge {\mathbf{m}}$.
6: $\mathbf{w}({\mathbf{ldw}},*)$ – Real (Kind=nag_wp) arrayInput/Output
Note: the second dimension of the array w
must be at least
${\mathbf{k}}$.
On entry:
if ${\mathbf{seed}}\le 0$, w should be set to an initial iterate for the non-negative matrix factor, $W$.
If ${\mathbf{seed}}\ge 1$, w need not be set. f01saf will generate a random initial iterate.
On exit: the non-negative matrix factor, $W$.
7: $\mathbf{ldw}$ – IntegerInput
On entry: the first dimension of the array w as declared in the (sub)program from which f01saf is called.
Constraint:
${\mathbf{ldw}}\ge {\mathbf{m}}$.
8: $\mathbf{h}({\mathbf{ldh}},*)$ – Real (Kind=nag_wp) arrayInput/Output
Note: the second dimension of the array h
must be at least
${\mathbf{n}}$.
On entry:
if ${\mathbf{seed}}\le 0$, h should be set to an initial iterate for the non-negative matrix factor, $H$.
If ${\mathbf{seed}}\ge 1$, h need not be set. f01saf will generate a random initial iterate.
On exit: the non-negative matrix factor, $H$.
9: $\mathbf{ldh}$ – IntegerInput
On entry: the first dimension of the array h as declared in the (sub)program from which f01saf is called.
Constraint:
${\mathbf{ldh}}\ge {\mathbf{k}}$.
10: $\mathbf{seed}$ – IntegerInput
On entry:
if ${\mathbf{seed}}\le 0$, the supplied values of $W$ and $H$ are used for the initial iterate.
If ${\mathbf{seed}}\ge 1$, the value of seed is used to seed a random number generator for the initial iterates $W$ and $H$. See Section 9.3 for further details.
11: $\mathbf{errtol}$ – Real (Kind=nag_wp)Input
On entry: the convergence tolerance for when the Hierarchical Alternating Least Squares iteration has reached a stationary point. If ${\mathbf{errtol}}\le 0.0$, $\mathrm{max}\phantom{\rule{0.125em}{0ex}}({\mathbf{m}},{\mathbf{n}})\times \sqrt{\mathit{machineprecision}}$ is used.
12: $\mathbf{maxit}$ – IntegerInput
On entry: specifies the maximum number of iterations to be used. If ${\mathbf{maxit}}\le 0$, $200$ is used.
13: $\mathbf{ifail}$ – IntegerInput/Output
On entry: ifail must be set to $0$, $\mathrm{-1}$ or $1$ to set behaviour on detection of an error; these values have no effect when no error is detected.
A value of $0$ causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of $\mathrm{-1}$ means that an error message is printed while a value of $1$ means that it is not.
If halting is not appropriate, the value $\mathrm{-1}$ or $1$ is recommended. If message printing is undesirable, then the value $1$ is recommended. Otherwise, the value $0$ is recommended. When the value $-\mathbf{1}$ or $\mathbf{1}$ is used it is essential to test the value of ifail on exit.
On exit: ${\mathbf{ifail}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see Section 6).
6Error Indicators and Warnings
If on entry ${\mathbf{ifail}}=0$ or $\mathrm{-1}$, explanatory error messages are output on the current error message unit (as defined by x04aaf).
Errors or warnings detected by the routine:
${\mathbf{ifail}}=1$
On entry, ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{m}}\ge 2$.
${\mathbf{ifail}}=2$
On entry, ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{n}}\ge 2$.
${\mathbf{ifail}}=3$
On entry, ${\mathbf{k}}=\u27e8\mathit{\text{value}}\u27e9$, ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: $1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}({\mathbf{m}},{\mathbf{n}})$.
${\mathbf{ifail}}=4$
On entry, ${\mathbf{lda}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{lda}}\ge {\mathbf{m}}$.
${\mathbf{ifail}}=5$
On entry, ${\mathbf{ldw}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{ldw}}\ge {\mathbf{m}}$.
${\mathbf{ifail}}=6$
On entry, ${\mathbf{ldh}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{k}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{ldh}}\ge {\mathbf{k}}$.
${\mathbf{ifail}}=7$
The routine has failed to converge after $\u27e8\mathit{\text{value}}\u27e9$ iterations. The factorization given by w and h may still be a good enough approximation to be useful. Alternatively an improved factorization may be obtained by increasing maxit or using different initial choices of w and h.
${\mathbf{ifail}}=8$
An internal error occurred when generating initial values for w and h. Please contact NAG.
${\mathbf{ifail}}=9$
On entry, one of more of the elements of a, w or h were negative.
${\mathbf{ifail}}=-99$
An unexpected error has been triggered by this routine. Please
contact NAG.
See Section 7 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-399$
Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-999$
Dynamic memory allocation failed.
See Section 9 in the Introduction to the NAG Library FL Interface for further information.
7Accuracy
The Hierarchical Alternating Least Squares algorithm used by f01saf is locally convergent; it is guaranteed to converge to a stationary point of $f(W,H)$, but this may not be the global minimum. The iteration is deemed to have converged if the gradient of $f(W,H)$ is less than errtol times the gradient at the initial values of $W$ and $H$.
Due to the local convergence property, you may wish to run f01saf multiple times with different starting iterates. This can be done by explicitly providing the starting values of $W$ and $H$ each time, or by choosing a different random seed for each routine call.
Note that even if f01saf exits with ${\mathbf{ifail}}={\mathbf{7}}$, the factorization given by $W$ and $H$ may still be a good enough approximation to be useful.
8Parallelism and Performance
f01saf is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
f01saf makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the Users' Note for your implementation for any additional implementation-specific information.
9Further Comments
Each iteration of the Hierarchical Alternating Least Squares algorithm requires $\mathit{O}\left(mnk\right)$ floating-point operations.
The real allocatable memory required is $m\times n+k(m+n)$.
If $A$ is large and sparse, then f01sbf should be used to compute a non-negative matrix factorization.
9.1Uniqueness
Note that non-negative matrix factorization is not unique. For a factorization given by the matrices $W$ and $H$, an equally good solution is given by $WD$ and ${D}^{\mathrm{-1}}H$, where $D$ is any real non-negative $k\times k$ matrix whose inverse is also non-negative. In f01saf, $W$ and $H$ are normalized so that the columns of $W$ have unit length.
9.2Choice of $\mathit{k}$
The most appropriate choice of the factorization rank, $k$, is often problem dependent. Details of your particular application may help in guiding your choice of $k$, for example, it may be known a priori that the data in $A$ naturally falls into a certain number of categories.
Alternatively, trial and error can be used. Compute non-negative matrix factorizations for several different values of $k$ (typically with $k\ll \mathrm{min}\phantom{\rule{0.125em}{0ex}}(m,n)$) and select the one that performs the best.
Finally, it is also possible to use a singular value decomposition of $A$ to guide your choice of $k$, by looking for an abrupt decay in the size of the singular values of $A$. The singular value decomposition can be computed using f08kbf.
9.3Generating Random Initial Iterates
If ${\mathbf{seed}}\ge 1$ on entry, then f01saf uses the routines g05kffandg05saf, with the NAG basic generator, to populate w and h. For further information on this random number generator see Section 2.1.1 in the G05 Chapter Introduction.
Note that this generator gives a repeatable sequence of random numbers, so if the value of seed is not changed between routine calls, then the same initial iterates will be generated.
10Example
This example finds a non-negative matrix factorization for the matrix