naginterfaces.library.mv.factor

naginterfaces.library.mv.factor(matrix, n, x, nvar, isx, nfac, iop, wt=None, io_manager=None)

factor computes the maximum likelihood estimates of the parameters of a factor analysis model. Either the data matrix or a correlation/covariance matrix may be input. Factor loadings, communalities and residual correlations are returned.

For full information please refer to the NAG Library document for g03ca

https://support.nag.com/numeric/nl/nagdoc_30.2/flhtml/g03/g03caf.html

Parameters

matrixstr, length 1

Selects the type of matrix on which factor analysis is to be performed.

$matrix=\text{'D'}$

The data matrix will be input in $x$ and factor analysis will be computed for the correlation matrix.

$matrix=\text{'S'}$

The data matrix will be input in $x$ and factor analysis will be computed for the covariance matrix, i.e., the results are scaled as described in Further Comments.

$matrix=\text{'C'}$

The correlation/variance-covariance matrix will be input in $x$ and factor analysis computed for this matrix.

See Further Comments.

nint

If $matrix=\text{'D'}$ or $\text{'S'}$ the number of observations in the data array $x$.

If $matrix=\text{'C'}$ the (effective) number of observations used in computing the (possibly weighted) correlation/variance-covariance matrix input in $x$.

xfloat, array-like, shape $(:,m)$

Note: the required extent for this argument in dimension 1 is determined as follows: if $matrix\text{in}(\text{'D'},\text{'S'})$: $n$; if $matrix=\text{'C'}$: $m$; otherwise: $0$.

The input matrix.

If $matrix=\text{'D'}$ or $\text{'S'}$, $x$ must contain the data matrix, i.e., $x[\text{i}-1,\text{j}-1]$ must contain the $\text{i}$th observation for the $\text{j}$th variable, for $\text{j}=1,2,\dots ,m$, for $\text{i}=1,2,\dots ,n$.

If $matrix=\text{'C'}$, $x$ must contain the correlation or variance-covariance matrix.

Only the upper triangular part is required.

nvarint

$p$, the number of variables in the factor analysis.

isxint, array-like, shape $\left(m\right)$

$isx[\text{j}-1]$ indicates whether or not the $\text{j}$th variable is included in the factor analysis. If $isx[\text{j}-1]\ge 1$, the variable represented by the $\text{j}$th column of $x$ is included in the analysis; otherwise it is excluded, for $\text{j}=1,2,\dots ,m$.

nfacint

$k$, the number of factors.

iopint, array-like, shape $\left(5\right)$

Options for the optimization. There are four options to be set:

$\text{iprint}$	controls iteration monitoring;
	if $\text{iprint}\le 0$, there is no printing of information else if $\text{iprint}>0$, information is printed at every iprint iterations. The information printed consists of the value of $F\left(\Psi \right)$ at that iteration, the number of evaluations of $F\left(\Psi \right)$, the current estimates of the communalities and an indication of whether or not they are at the boundary.
$\text{maxfun}$	the maximum number of function evaluations.
$\text{acc}$	the required accuracy for the estimates of ${\psi }_i$.
$\text{eps}$	a lower bound for the values of $\psi$, see Notes.

Let $ϵ=\text{machine precision}$ then if $iop\left[0\right]=0$, the following default values are used:

$\text{iprint}=-1$

$\text{maxfun}=100p$

$\text{acc}=10\sqrt{ϵ}$

$\text{eps}=ϵ$

If $iop\left[0\right]\ne 0$, then

$\text{iprint}=iop\left[1\right]$

$\text{maxfun}=iop\left[2\right]$

$\text{acc}={10}^{-l}$ where $l=iop\left[3\right]$

$\text{eps}={10}^{-l}$ where $l=iop\left[4\right]$

wtNone or float, array-like, shape $(:)$, optional

Note: the required length for this argument is determined as follows: if $\text{weight}=\text{'W'}\text{and}matrix\text{in}(\text{'D'},\text{'S'})$: $n$; otherwise: $1$.

If $wt\text{is not}None$ and $matrix=\text{'D'}$ or $\text{'S'}$, $wt$ must contain the weights to be used in the factor analysis. The effective number of observations in the analysis will then be the sum of weights. If $wt[i-1]=0.0$, the $i$th observation is not included in the analysis.

If $\text{weight}=\text{'U'}$ or $matrix=\text{'C'}$, $wt$ is not referenced and the effective number of observations is $n$.

io_managerFileObjManager, optional

Manager for I/O in this routine.

Returns

efloat, ndarray, shape $\left(nvar\right)$

The eigenvalues ${\theta }_{\text{i}}$, for $\text{i}=1,2,\dots ,p$.

statfloat, ndarray, shape $\left(4\right)$

The test statistics.

$stat\left[0\right]$

Contains the value $F\left(\hat{\Psi }\right)$.

$stat\left[1\right]$

Contains the test statistic, ${\chi }^2$.

$stat\left[2\right]$

Contains the degrees of freedom associated with the test statistic.

$stat\left[3\right]$

Contains the significance level.

comfloat, ndarray, shape $\left(nvar\right)$

The communalities.

psifloat, ndarray, shape $\left(nvar\right)$

The estimates of ${\psi }_{\text{i}}$, for $\text{i}=1,2,\dots ,p$.

resfloat, ndarray, shape $(nvar\times (nvar-1)/2)$

The residual correlations. The residual correlation for the $i$th and $j$th variables is stored in $res[(j-1)(j-2)/2+i-1]$, $i<j$.

flfloat, ndarray, shape $(nvar,nfac)$

The factor loadings. $fl[\text{i}-1,\text{j}-1]$ contains ${\lambda }_{\text{i}\text{j}}$, for $\text{j}=1,2,\dots ,k$, for $\text{i}=1,2,\dots ,p$.

Raises

NagValueError

(errno $1$)

On entry, $iop\left[0\right]\ne 1$ and $iop\left[4\right]=⟨value⟩$.

Constraint: $1\le \text{eps}\le \text{machine precision}$.

(errno $1$)

On entry, $iop\left[0\right]\ne 1$ and $iop\left[3\right]=⟨value⟩$.

Constraint: $1\le \text{acc}\le \text{machine precision}$.

(errno $1$)

On entry, $iop\left[0\right]\ne 1$ and $iop\left[2\right]=⟨value⟩$.

Constraint: $\text{maxfun}\ge 1$.

(errno $1$)

On entry, $nfac=⟨value⟩$ and $nvar=⟨value⟩$.

Constraint: $nfac\le nvar$.

(errno $1$)

On entry, $nfac=⟨value⟩$.

Constraint: $nfac\ge 1$.

(errno $1$)

On entry, $m=⟨value⟩$ and $nvar=⟨value⟩$.

Constraint: $m\ge nvar$.

(errno $1$)

On entry, $matrix=⟨value⟩$.

Constraint: $matrix=\text{'D'}$, $\text{'S'}$ or $\text{'C'}$.

(errno $1$)

On entry, $\text{weight}=⟨value⟩$.

Constraint: when $matrix=\text{'D'}$ or $\text{'S'}$, $\text{weight}=\text{'U'}$ or $\text{'W'}$.

(errno $1$)

On entry, $n=⟨value⟩$ and $nvar=⟨value⟩$.

Constraint: $n>nvar$.

(errno $1$)

On entry, $nvar=⟨value⟩$.

Constraint: $nvar>1$.

(errno $2$)

On entry, $i=⟨value⟩$ and $wt[i-1]<0.0$.

Constraint: $wt[i-1]\ge 0.0$.

(errno $3$)

The effective number of observations $\le 1$.

(errno $3$)

The number of variables $\ge$ number of included observations.

(errno $3$)

On entry, $nvar=⟨value⟩$ and $⟨value⟩$ values of $isx>0$

Constraint: exactly $nvar$ elements of $isx>0$.

(errno $4$)

Two eigenvalues of $S^{\ast }$ are equal.

(errno $4$)

On entry, the data matrix is not of full column rank or the input correlation/covariance matrix is not positive definite.

(errno $5$)

The singular value decomposition has failed to converge.

(errno $6$)

The estimation procedure has failed to converge in $⟨value⟩$ iterations.

Warns

NagAlgorithmicWarning

(errno $7$)

The convergence is not certain but a lower point could not be found.

Notes

In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

Let $p$ variables, $x_1,x_2,\dots ,x_p$, with variance-covariance matrix $\Sigma$ be observed. The aim of factor analysis is to account for the covariances in these $p$ variables in terms of a smaller number, $k$, of hypothetical variables, or factors, $f_1,f_2,\dots ,f_k$. These are assumed to be independent and to have unit variance. The relationship between the observed variables and the factors is given by the model:

$$x_i=\sum _{j=1}^k{\lambda }_{ij}{f}_j+e_i\text{,}i=1,2,\dots ,p$$

where ${\lambda }_{\text{i}\text{j}}$, for $\text{j}=1,2,\dots ,k$, for $\text{i}=1,2,\dots ,p$, are the factor loadings and $e_{\text{i}}$, for $\text{i}=1,2,\dots ,p$, are independent random variables with variances ${\psi }_{\text{i}}$, for $\text{i}=1,2,\dots ,p$. The ${\psi }_i$ represent the unique component of the variation of each observed variable. The proportion of variation for each variable accounted for by the factors is known as the communality. For this function it is assumed that both the $k$ factors and the $e_i$’s follow independent Normal distributions.

The model for the variance-covariance matrix, $\Sigma$, can be written as:

$$\Sigma =\Lambda {\Lambda }^T+\Psi$$

where $\Lambda$ is the matrix of the factor loadings, ${\lambda }_{ij}$, and $\Psi$ is a diagonal matrix of unique variances, ${\psi }_{\text{i}}$, for $\text{i}=1,2,\dots ,p$.

The estimation of the parameters of the model, $\Lambda$ and $\Psi$, by maximum likelihood is described by Lawley and Maxwell (1971). The log-likelihood is:

$$-\frac{1}{2}(n-1)\log \left(\left|\Sigma \right|\right)-\frac{1}{2}(n-1)trace(S,{\Sigma }^{-1})+\text{constant,}$$

where $n$ is the number of observations, $S$ is the sample variance-covariance matrix or, if weights are used, $S$ is the weighted sample variance-covariance matrix and $n$ is the effective number of observations, that is, the sum of the weights. The constant is independent of the parameters of the model. A two stage maximization is employed. It makes use of the function $F\left(\Psi \right)$, which is, up to a constant, $-2/(n-1)$ times the log-likelihood maximized over $\Lambda$. This is then minimized with respect to $\Psi$ to give the estimates, $\hat{\Psi }$, of $\Psi$. The function $F\left(\Psi \right)$ can be written as:

$$F\left(\Psi \right)=\sum _{j=k+1}^p({\theta }_j-\log \left({\theta }_j\right))-(p-k)$$

where values ${\theta }_{\text{j}}$, for $\text{j}=1,2,\dots ,p$ are the eigenvalues of the matrix:

$$S^{\ast }={\Psi }^{-1/2}S{\Psi }^{-1/2}\text{.}$$

The estimates $\hat{\Lambda }$, of $\Lambda$, are then given by scaling the eigenvectors of $S^{\ast }$, which are denoted by $V$:

$$\hat{\Lambda }={\Psi }^{1/2}V{(\Theta -I)}^{1/2}\text{.}$$

where $\Theta$ is the diagonal matrix with elements ${\theta }_i$, and $I$ is the identity matrix.

The minimization of $F\left(\Psi \right)$ is performed using opt.bounds_mod_deriv2_comp which uses a modified Newton algorithm. The computation of the Hessian matrix is described by Clark (1970). However, instead of using the eigenvalue decomposition of the matrix $S^{\ast }$ as described above, the singular value decomposition of the matrix $R{\Psi }^{-1/2}$ is used, where $R$ is obtained either from the $QR$ decomposition of the (scaled) mean centred data matrix or from the Cholesky decomposition of the correlation/covariance matrix. The function opt.bounds_mod_deriv2_comp ensures that the values of ${\psi }_i$ are greater than a given small positive quantity, $\delta$, so that the communality is always less than $1$. This avoids the so called Heywood cases.

In addition to the values of $\Lambda$, $\Psi$ and the communalities, factor returns the residual correlations, i.e., the off-diagonal elements of $C-(\Lambda {\Lambda }^T+\Psi )$ where $C$ is the sample correlation matrix. factor also returns the test statistic:

$${\chi }^2=[n-1-(2p+5)/6-2k/3]F\left(\hat{\Psi }\right)$$

which can be used to test the goodness-of-fit of the model (1), see Lawley and Maxwell (1971) and Morrison (1967).

References

Clark, M R B, 1970, A rapidly convergent method for maximum likelihood factor analysis, British J. Math. Statist. Psych.

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2–25

Lawley, D N and Maxwell, A E, 1971, Factor Analysis as a Statistical Method, (2nd Edition), Butterworths

Morrison, D F, 1967, Multivariate Statistical Methods, McGraw–Hill