Independent component analysis
|
|
Independent component analysis (ICA) is a computational method for separating a multivariate signal into additive subcomponents supposing the mutual statistical independence of the non-Gaussian source signals.
| Contents |
Definition
The independence assumption is correct in most cases so the blind ICA separation of a mixed signal gives very good results. It is also used for signals that are not supposed to be generated by a mixing for analysis purposes.
The statistical method finds the independent components (aka factors, latent variables or sources) by maximizing the statistical independence of the estimated components. Non-Gaussianity, motivated by the central limit theorem, is one method for measuring the independence of the components. Non-Gaussianity can be measured, for instance, by kurtosis or approximations of negentropy.
Typical algorithms for ICA use centering, whitening and dimensionality reduction as preprocessing steps in order to simplify and reduce the complexity of the problem for the actual iterative algorithm. Whitening and dimension reduction can be achieved with principal component analysis or singular value decomposition. Algorithms for ICA include infomax, FastICA and JADE, but there are many others also.
The ICA method is not able to extract the actual number of source signals, the order of the source signals, nor the signs or the scales of the sources.
The method is important to blind signal separation and has many practical applications.
Mathematical definitions
Linear independent component analysis can be divided into noiseless and noisy cases, where noiseless ICA is a special case of noisy ICA. Nonlinear ICA should be considered as a separate case.
General definition
The data is represented by the random vector <math>x=(x_1,\ldots,x_m)<math> and the components as the random vector <math>s=(s_1,\ldots,s_n)<math>. The task is to transform the observed data <math>x<math>, using a linear static transformation <math>s=Wx<math>, into maximally independent components <math>s<math> measured by some function <math>F(s_1,\ldots,s_n)<math> of independence.
Generative model
Linear noiseless ICA
The components <math>x_i<math> of the observed random vector <math>x=(x_1,\ldots,x_m)^T<math> are generated as a sum of the independent components <math>s_k<math>, <math>k=1,\ldots,n<math>:
<math>x_i = a_{i,1} s_1 + \ldots + a_{i,k} s_k + \ldots + a_{i,n} s_n<math>
weighted by the mixing weights <math>a_{i,k}<math>.
The same generative model can be written in vectorial form as <math>x=\sum_{k=1}^{n} a_k s_k<math>, where the observed random vector <math>x<math> is represented by the basis vectors <math>a_k=(a_{1,k},\ldots,a_{m,k})^T<math>. The basis vectors <math>a_k<math> form the columns of the mixing matrix <math>A=(a_1,\ldots,a_n)<math> and the generative formula can be written as <math>x=As<math>, where <math>s=(s_1,\ldots,s_n)^T<math>.
Given the model and realizations (samples) <math>x_1,\ldots,x_N<math> of the random vector <math>x<math>, the task is to estimate both the mixing matrix <math>A<math> and the sources <math>s<math>.
The original sources <math>s<math> can be recovered by multiplying the the observed signals <math>x<math> with the inverse of the mixing matrix <math>W=A^{-1}<math>, also known as the unmixing matrix. Here it is assumed that the mixing matrix is square (<math>n=m<math>).
Linear noisy ICA
With the added assumption of zero-mean and uncorrelated Gaussian noise <math>n\sim N(0,diag(\Sigma))<math>, the ICA model takes the form <math>x=As+n<math>.
Nonlinear ICA
The mixing of the sources does not need to be linear. Using a nonlinear mixing function <math>f(\cdot|\theta)<math> with parameters <math>\theta<math> the nonlinear ICA model is <math>x=f(s|\theta)+n<math>.
Identifiability
The identifiability of independent component analysis requires that:
- Only one of the sources <math>s_k<math> can be Gaussian,
- The number of observed mixtures, <math>m<math>, must be at least as large as the number of estimated components <math>n<math>: <math>m \ge n<math>,
- The mixing matrix <math>A<math> must be of full rank.
History and background
The problem of source separation is an old one in electrical engineering and has been well studied; many algorithms exist depending on the nature of the mixed signals. The problem of BSS is more difficult since without knowledge of the signals that have been mixed, it is not possible to design appropriate preprocessing to optimally separate them. The general framework for independent component analysis was introduced by Jeanny Herault and Christian Jutten in 1986 and was most clearly stated by Pierre Comon in 1994.
In 1995 Tony Bell and Terry Sejnowski introduced a fast and efficient ICA algorithm based on infomax, a principle introduced by Ralph Linsker in 1992. In 1997 Shun-ichi Amari realized that the infomax ICA algorithm could be improved by using the natural gradient, which was independently discovered by Jean-Francois Cardoso. However, the original infomax ICA algorithm with sigmoidal nonlinearities was only suitable for super-Gaussian sources. Te-Won Lee, in collaboration with Mark Girolami, developed an efficient extended version of the infomax ICA algorithm that is suitable for general non-Gaussian signals.
Several different approaches have been taken to blind source separation, which include maximum likelihood, Bussgang methods based on cumulants, projection pursuit and negentropy methods. All of these are all closely related to the infomax framework. Thus, a large number of researchers who have attacked ICA from a variety of different directions have converged on a common set of principles and, ultimately, a well understood class of algorithms.
Applications
An important application of extended infomax ICA is to EEG recordings of scalp potentials in humans. The electrical signals originating from the brain are quite weak at the scalp, in the microvolt range, and there are larger artifactual components arising from eye movements and muscles. It has been a difficult challenge to eliminate these artifacts without altering the brain signals. ICA is ideally suited to this task, since the brain and the scalp are good volume conductors and to a good approximation, the recordings are different linear mixtures of the brain signals and the artifacts. The extended infomax ICA algorithm has proven to be an effective method for separating out these artifacts. This algorithm has many other biomedical applications, including the analysis of extremely large datsets from functional Magnetic Resonance Imaging (fMRI) experiments.
ICA can be extended to analyze non-physical signals. For instance, ICA has been applied to discover discussion topics on a bag of news list archives.
Related to
- Blind signal separation (BSS)
- Principal component analysis (PCA)
- Varimax rotation
- Projection pursuit
- Image processing
- Signal processing
- Blind deconvolution
- Factor analysis
- Nonlinear PCA
- Redundancy reduction
Links
- What is independent component analysis? (http://www.cs.helsinki.fi/u/ahyvarin/whatisica.shtml) by Aapo Hyvärinen
- FastICA package for Matlab (http://www.cis.hut.fi/projects/ica/fastica/)