NAG FL Interface
f01sbf (real_nmf_rcomm)
1
Purpose
f01sbf computes a nonnegative matrix factorization for a real nonnegative $m$ by $n$ matrix $A$. It uses reverse communication for evaluating matrix products, so that the matrix $A$ is not accessed explicitly.
2
Specification
Fortran Interface
Subroutine f01sbf ( 
irevcm, m, n, k, w, ldw, h, ldh, ht, ldht, seed, errtol, comm, icomm, ifail) 
Integer, Intent (In) 
:: 
m, n, k, ldw, ldh, ldht, seed 
Integer, Intent (Inout) 
:: 
irevcm, icomm(9), ifail 
Real (Kind=nag_wp), Intent (In) 
:: 
errtol 
Real (Kind=nag_wp), Intent (Inout) 
:: 
w(ldw,*), h(ldh,*), ht(ldht,*), comm((2*m+n)*k+3) 

C Header Interface
#include <nag.h>
void 
f01sbf_ (Integer *irevcm, const Integer *m, const Integer *n, const Integer *k, double w[], const Integer *ldw, double h[], const Integer *ldh, double ht[], const Integer *ldht, const Integer *seed, const double *errtol, double comm[], Integer icomm[], Integer *ifail) 

C++ Header Interface
#include <nag.h> extern "C" {
void 
f01sbf_ (Integer &irevcm, const Integer &m, const Integer &n, const Integer &k, double w[], const Integer &ldw, double h[], const Integer &ldh, double ht[], const Integer &ldht, const Integer &seed, const double &errtol, double comm[], Integer icomm[], Integer &ifail) 
}

The routine may be called by the names f01sbf or nagf_matop_real_nmf_rcomm.
3
Description
The matrix
$A$ is factorized into the product of an
$m$ by
$k$ matrix
$W$ and a
$k$ by
$n$ matrix
$H$, both with nonnegative elements. The factorization is approximate,
$A\approx WH$, with
$W$ and
$H$ chosen to minimize the functional
You are free to choose any value for $k$, provided $k<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left(m,n\right)$. The product $WH$ will then be a lowrank approximation to $A$, with rank at most $k$.
f01sbf finds $W$ and $H$ using an iterative method known as the Hierarchical Alternating Least Squares algorithm. You may specify initial values for $W$ and $H$, or you may provide a seed value for f01sbf to generate the initial values using a random number generator.
f01sbf does not explicitly need to access the elements of $A$; it only requires the result of matrix multiplications of the form $AX$ or ${A}^{\mathrm{T}}Y$. A reverse communication interface is used, in which control is returned to the calling program whenever a matrix product is required.
4
References
Cichocki A and Phan A–H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E92–A 708–721
Cichocki A, Zdunek R and Amari S–I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization Lecture Notes in Computer Science 4666 Springer 169–176
Ho N–D (2008) Nonnegative matrix factorization algorithms and applications PhD Thesis Univ. Catholique de Louvain
5
Arguments
Note: this routine uses
reverse communication. Its use involves an initial entry, intermediate exits and reentries, and a final exit, as indicated by the argument
irevcm. Between intermediate exits and reentries,
all arguments other than w and
ht must remain unchanged.

1:
$\mathbf{irevcm}$ – Integer
Input/Output

On initial entry: must be set to $0$.
On intermediate exit:
specifies what action you must take before reentering
f01sbf with
irevcm unchanged. The value of
irevcm should be interpreted as follows:
 ${\mathbf{irevcm}}=1$
 Indicates the start of a new iteration. No action is required by you, but w and h are available for printing, and a limit on the number of iterations can be applied.
 ${\mathbf{irevcm}}=2$
 Indicates that before reentry to f01sbf, the product ${A}^{\mathrm{T}}W$ must be computed and stored in ht.
 ${\mathbf{irevcm}}=3$
 Indicates that before reentry to f01sbf, the product $A{H}^{\mathrm{T}}$ must be computed and stored in w.
On final exit: ${\mathbf{irevcm}}=0$.
Note: any values you return to f01sbf as part of the reverse communication procedure should not include floatingpoint NaN (Not a Number) or infinity values, since these are not handled by f01sbf. If your code does inadvertently return any NaNs or infinities, f01sbf is likely to produce unexpected results.

2:
$\mathbf{m}$ – Integer
Input

On entry: $m$, the number of rows of the matrix $A$. Also the number of rows of the matrix $W$.
Constraint:
${\mathbf{m}}\ge 2$.

3:
$\mathbf{n}$ – Integer
Input

On entry: $n$, the number of columns of the matrix $A$. Also the number of columns of the matrix $H$.
Constraint:
${\mathbf{n}}\ge 2$.

4:
$\mathbf{k}$ – Integer
Input

On entry:
$k$, the number of columns of the matrix
$W$. Also, the number of rows of the matrix
$H$. See
Section 9.2 for further details.
Constraint:
$1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)$.

5:
$\mathbf{w}\left({\mathbf{ldw}},*\right)$ – Real (Kind=nag_wp) array
Input/Output

Note: the second dimension of the array
w
must be at least
${\mathbf{k}}$.
On initial entry:
 if ${\mathbf{seed}}\le 0$, w should be set to an initial iterate for the nonnegative matrix factor, $W$.
 If ${\mathbf{seed}}\ge 1$, w need not be set. f01sbf will generate a random initial iterate.
On intermediate exit:
if
${\mathbf{irevcm}}=1$ or
$2$,
w contains the current iterate of the
$m\times k$ nonnegative matrix
$W$.
On intermediate reentry:
 if ${\mathbf{irevcm}}=3$, w must contain $A{H}^{\mathrm{T}}$, where ${H}^{\mathrm{T}}$ is stored in $\mathit{ht}$.
 If ${\mathbf{irevcm}}=0$, $1$ or $2$, w must not be changed.
On final exit:
w contains the
$m\times k$ nonnegative matrix
$W$.

6:
$\mathbf{ldw}$ – Integer
Input

On entry: the first dimension of the array
w as declared in the (sub)program from which
f01sbf is called.
Constraint:
${\mathbf{ldw}}\ge {\mathbf{m}}$.

7:
$\mathbf{h}\left({\mathbf{ldh}},*\right)$ – Real (Kind=nag_wp) array
Input/Output

Note: the second dimension of the array
h
must be at least
${\mathbf{n}}$.
On initial entry:
 if ${\mathbf{seed}}\le 0$, h should be set to an initial iterate for the nonnegative matrix factor, $H$.
 If ${\mathbf{seed}}\ge 1$, h need not be set. f01sbf will generate a random initial iterate.
On intermediate exit:
if
${\mathbf{irevcm}}=1$,
h contains the current iterate of the
$k\times n$ nonnegative matrix
$H$.
On intermediate reentry:
h must not be changed.
On final exit:
h contains the
$k\times n$ nonnegative matrix
$H$.

8:
$\mathbf{ldh}$ – Integer
Input

On entry: the first dimension of the array
h as declared in the (sub)program from which
f01sbf is called.
Constraint:
${\mathbf{ldh}}\ge {\mathbf{k}}$.

9:
$\mathbf{ht}\left({\mathbf{ldht}},*\right)$ – Real (Kind=nag_wp) array
Input/Output

Note: the second dimension of the array
ht
must be at least
${\mathbf{k}}$.
On initial entry:
ht need not be set.
On intermediate exit:
if
${\mathbf{irevcm}}=3$,
ht contains the
$n\times k$ nonnegative matrix
${H}^{\mathrm{T}}$, which is required in order to form
$A{H}^{\mathrm{T}}$.
On intermediate reentry: if
${\mathbf{irevcm}}=2$,
ht must contain
${A}^{\mathrm{T}}W$.
If
${\mathbf{irevcm}}=0$,
$1$ or
$3$,
ht must not be changed.
On final exit:
ht is undefined.

10:
$\mathbf{ldht}$ – Integer
Input

On entry: the first dimension of the array
ht as declared in the (sub)program from which
f01sbf is called.
Constraint:
${\mathbf{ldht}}\ge {\mathbf{n}}$.

11:
$\mathbf{seed}$ – Integer
Input

On initial entry:
 if ${\mathbf{seed}}\le 0$, the supplied values of $W$ and $H$ are used for the initial iterate.
 If ${\mathbf{seed}}\ge 1$, the value of seed is used to seed a random number generator for the initial iterates $W$ and $H$. See Section 9.3 for further details.

12:
$\mathbf{errtol}$ – Real (Kind=nag_wp)
Input

On entry: the convergence tolerance for when the Hierarchical Alternating Least Squares iteration has reached a stationary point. If ${\mathbf{errtol}}\le 0.0$, $\mathrm{max}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)\times \sqrt{\mathit{machineprecision}}$ is used.

13:
$\mathbf{comm}\left(\left(2\times {\mathbf{m}}+{\mathbf{n}}\right)\times {\mathbf{k}}+3\right)$ – Real (Kind=nag_wp) array
Communication Array


14:
$\mathbf{icomm}\left(9\right)$ – Integer array
Communication Array


15:
$\mathbf{ifail}$ – Integer
Input/Output

On initial entry:
ifail must be set to
$0$,
$1$ or
$1$ to set behaviour on detection of an error; these values have no effect when no error is detected.
A value of $0$ causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of $1$ means that an error message is printed while a value of $1$ means that it is not.
If halting is not appropriate, the value
$1$ or
$1$ is recommended. If message printing is undesirable, then the value
$1$ is recommended. Otherwise, the value
$1$ is recommended since useful values can be provided in some output arguments even when
${\mathbf{ifail}}\ne {\mathbf{0}}$ on exit.
When the value $\mathbf{1}$ or $\mathbf{1}$ is used it is essential to test the value of ifail on exit.
On final exit:
${\mathbf{ifail}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see
Section 6).
6
Error Indicators and Warnings
If on entry
${\mathbf{ifail}}=0$ or
$1$, explanatory error messages are output on the current error message unit (as defined by
x04aaf).
Errors or warnings detected by the routine:
 ${\mathbf{ifail}}=1$

On initial entry, ${\mathbf{irevcm}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{irevcm}}=0$.
On intermediate reentry, ${\mathbf{irevcm}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{irevcm}}=1$, $2$ or $3$.
 ${\mathbf{ifail}}=2$

On entry, ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{m}}\ge 2$.
 ${\mathbf{ifail}}=3$

On entry, ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{n}}\ge 2$.
 ${\mathbf{ifail}}=4$

On entry, ${\mathbf{k}}=\u2329\mathit{\text{value}}\u232a$, ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: $1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)$.
 ${\mathbf{ifail}}=5$

On entry, ${\mathbf{ldw}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{ldw}}\ge {\mathbf{m}}$.
 ${\mathbf{ifail}}=6$

On entry, ${\mathbf{ldh}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{k}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{ldh}}\ge {\mathbf{k}}$.
 ${\mathbf{ifail}}=7$

On entry, ${\mathbf{ldht}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{ldht}}\ge {\mathbf{n}}$.
 ${\mathbf{ifail}}=8$

An internal error occurred when generating initial values for
w and
h. Please contact
NAG.
 ${\mathbf{ifail}}=9$

On entry, one of more of the elements of
w or
h were negative.
 ${\mathbf{ifail}}=99$
An unexpected error has been triggered by this routine. Please
contact
NAG.
See
Section 7 in the Introduction to the NAG Library FL Interface for further information.
 ${\mathbf{ifail}}=399$
Your licence key may have expired or may not have been installed correctly.
See
Section 8 in the Introduction to the NAG Library FL Interface for further information.
 ${\mathbf{ifail}}=999$
Dynamic memory allocation failed.
See
Section 9 in the Introduction to the NAG Library FL Interface for further information.
7
Accuracy
The Hierarchical Alternating Least Squares algorithm used by
f01sbf is locally convergent; it is guaranteed to converge to a stationary point of
$f\left(W,H\right)$, but this may not be the global minimum. The iteration is deemed to have converged if the gradient of
$f\left(W,H\right)$ is less than
errtol times the gradient at the initial values of
$W$ and
$H$.
Due to the local convergence property, you may wish to run f01sbf multiple times with different starting iterates. This can be done by explicitly providing the starting values of $W$ and $H$ each time, or by choosing a different random seed each time.
Note that even if f01sbf exits with ${\mathbf{ifail}}={\mathbf{7}}$, the factorization given by $W$ and $H$ may still be a good enough approximation to be useful.
8
Parallelism and Performance
f01sbf is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
f01sbf makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the
Users' Note for your implementation for any additional implementationspecific information.
f01sbf is designed to be used when $A$ is large and sparse. Whenever a matrix multiplication is required, the routine will return control to the calling program so that the multiplication can be done in the most efficient way possible. Note that $W$ and $H$ will not, in general, be sparse even if $A$ is sparse.
If
$A$ is small and dense, then
f01saf can be used to compute
$W$ and
$H$ without the use of a reverse communication interface.
9.1
Uniqueness
Note that nonnegative matrix factorization is not unique. For a factorization given by the matrices $W$ and $H$, an equally good solution is given by $WD$ and ${D}^{1}H$, where $D$ is any real nonnegative $k\times k$ matrix whose inverse is also nonnegative. In f01sbf, $W$ and $H$ are normalized so that the columns of $W$ have unit length.
9.2
Choice of $k$
The most appropriate choice of the factorization rank, $k$, is often problem dependent. Details of your particular application may help in guiding your choice of $k$, for example, it may be known a priori that the data in $A$ naturally falls into a certain number of categories.
Alternatively, trial and error can be used. Compute nonnegative matrix factorizations for several different values of $k$ (typically with $k\ll \mathrm{min}\phantom{\rule{0.125em}{0ex}}\left(m,n\right)$) and select the one that performs the best.
Finally, it is also possible to use a singular value decomposition of
$A$ to guide your choice of
$k$, by looking for an abrupt decay in the size of the singular values of
$A$. The singular value decomposition can be computed using
f12fbf.
9.3
Generating Random Initial Iterates
If
${\mathbf{seed}}\ge 1$ on entry, then
f01sbf uses the routines
g05kff and
g05saf, with the NAG basic generator, to populate
w and
h. For further information on this random number generator see
Section 2.1.1 in the
G05 Chapter Introduction.
Note that this generator gives a repeatable sequence of random numbers, so if the value of
seed is not changed between routine calls, then the same initial iterates will be generated.
9.4
Use in Conjunction with NAG Library Routines
To compute the nonnegative matrix factorization, the following skeleton code can normally be used:
revcm: Do
Call f01sbf(irevcm,m,n,k,w,ldw,h,ldh,ht,ldht, &
seed,errtol,comm,icomm,ifail)
If (irevcm == 0) Then
Exit revcm
Else If (irevcm == 1) Then
.. Print W and H if required and check number of iterations ..
Else If (irevcm == 2) Then
.. Compute A^TW and store in ht..
Else If (irevcm == 3) Then
.. Compute AH^T and store in w ..
End If
End Do revcm
The code used to compute the matrix products will vary depending on the way
$A$ is stored. If all the elements of
$A$ are stored explicitly, then
f06yaf) can be used. If
$A$ is triangular, then
f06yff should be used. If
$A$ is symmetric, then
f06ycf should be used. For sparse
$A$ stored in coordinate storage format
f11xaf and
f11xef can be used. Alternatively, if
$A$ is stored in compressed column format
f11mkf can be used.
10
Example
This example finds a nonnegative matrix factorization for the matrix
10.1
Program Text
10.2
Program Data
10.3
Program Results