Paris' Independent Component Analysis & Blind Source Separation page
Independent Component Analysis (ICA) and Blind Source Separation (BSS)
have received quite a lot of attention lately so that I decided to compile
a list of online resources for whoever is interested. By no means is this
page complete and if you have any additions do send me mail at paris at
media dot mit dot edu.
In the papers section I do not list all of the papers of every author (that's
why you should check their homepages) but the really good ones are here.
Also not all ICA & BSS people have home pages so if you discover any
or if I missed yours tell me about it and I'll add them. Also note that this is a pretty ancient page, so expect many links to not work!
People working on ICA & BSS
Alex Westner a
brave man who scoffs at the complexity of real-world mixtures
Other ICA Pages
Daniël Schobben, Kari Torkkola and me maintain these:
Code and Software ...
package by Aapo Hyvärinen (very cool, get it!)
By Jean-François Cardoso:
Cichocki and Barros.
Mostly old instantaneous ICA code (in
Online Demos of BSS
Barak Pearlmutter's Demo
on Contextual ICA
My little frequency domain algorithm:
(This is actually a
static mix, I just put it up cause it sounds cool, but the algorithm can deal with
convolved mixtures too)
Hans van Hateren's demo
Online Papers on ICA & BSS
(since I don't really sit around all day playing with this page, there some links that are
extinct by now. Rather than giving up, check out the home page of the corresponding author in
the top of the page. You are most likely to find their papers there. You are also most
likely to find their newer papers there too).
Yes, I actually finished my dissertation!
- Smaragdis, P. 2001. Redundancy reduction for
audition, a unifying approach. Ph.D. dissertation, MAS
department, Massachusetts Institute of Technology.
Smaragdis, P. 1997. Information
Theoretic Approaches to Source Separation, Masters Thesis, MAS
Department, Massachusetts Institute of Technology.
I realize that the name of the thesis is not terribly enlightening.
What happens in there, apart from the obligatory background stuff, is the
development of a new frequency domain separation algorithm to deal with
convolved mixtures. The idea is to move to a space where the separation
parameters are orthogonal, to assist convergence, and to be able to implement
at the same time faster convolution schemes. In addition to this the algorithm
is on-line and real-time so that you can actually use it. Results are nice
Smaragdis, P. 1997. Efficient
Blind Separation of Convolved Sound Mixtures,IEEE ASSP Workshop on
Applications of Signal Processing to Audio and Acoustics. New Paltz NY,
Pretty much the same material, geared towards DSP-heads. Written before
my thesis so it is a little outdated.
Smaragdis, P. 1998. Blind
Separation of Convolved Mixtures in the Frequency Domain. International
Workshop on Independence & Artificial Neural Networks University of
La Laguna, Tenerife, Spain, February 9 - 10, 1998.
Condenced version of my thesis. Most up to date compared to my other
Tony Bell has some neat papers on Blind Source Separation:
And a couple of papers on ICA alone:
Kari Torkkola has some practical papers on simultaneous Blind Source
Separation and Deconvolution:
Separation of Delayed Sources Based on Information Maximization. Proceedings
of the IEEE Conference on Acoustics, Speech and Signal Processing,
May 7-10 1996, Atlanta, GA, USA.
Separation of Convolved Sources Based on Information Maximization.
Workshop on Neural Networks for Signal Processing, Sept 4-6 1996, Kyoto,
Filters for Blind Deconvolution Using Information Maximizationrm. NIPS96
Workshop: Blind Signal Processing and Their Applications, Snowmaas
Barak Pearlmutter has a paper on context sensitive ICA:
Erkki Oja has papers on PCA, nonlinear PCA and ICA:
Oja, E., Karhunen, J., Wang, L., and Vigario, R.:Principal
and independent components in neural networks - recent developments.
VII Italian Workshop on Neural Nets WIRN'95, May 18 - 20, 1995, Vietri
sul Mare, Italy (1995).
Oja, E.:The nonlinear
PCA learning rule and signal separation - mathematical analysis. Helsinki
University of Technology, Laboratory of Computer and Information Science,
Oja, E. and Taipale, O.:Applications
of learning and intelligent systems - the Finnish technology programme.
Int. Conf. on Artificial Neural Networks ICANN-95, Industrial Conference,
Oct. 9 - 13, 1995, Paris, France (1995).
Oja, E.: PCA, ICA,
and nonlinear Hebbian learning. Proc. Int. Conf. on Artificial Neural
Networks ICANN-95, Oct. 9 - 13, 1995, Paris, France, pp. 89 - 94 (1995).
Oja, E. and Karhunen, J.:Signal
separation by nonlinear Hebbian learning. In M. Palaniswami, Y. Attikiouzel,
R. Marks II, D. Fogel, and T. Fukuda (Eds.), Computational Intelligence
- a Dynamic System Perspective. New York: IEEE Press, pp. 83 - 97 (1995).
Juha Karhunen has written ICA & BSS papers with Oja (right above)
and some on his own:
Approaches to Independent Component Analysis and Source Separation.
appear in Proc. 4th European Symposium on Artificial Neural Networks (ESANN'96),
April 24 - 26, 1996, Bruges, Belgium (invited paper).
Karhunen, J., Wang, L., and Vigario, R.,Nonlinear
PCA Type Approaches for Source Separation and Independent Component AnalysisProc.
of the 1995 IEEE Int. Conf. on Neural Networks (ICNN'95), Perth, Australia,
November 27 - December 1, 1995, pp. 995-1000.
Karhunen, J., Wang, L., and Joutsensalo, J.,Neural
Estimation of Basis Vectors in Independent Component Analysis Proc.
of the Int. Conf. on Artificial Neural Networks (ICANN'95), Paris,
France, October 9-13, 1995, pp. 317-322.
Andrzej Cichocki organized a special invited session on BSS in Nolta
'95 and has a nice list of papers on the subject:
(another apparently defunct set of links ...)
Shun-ichi Amari, Andrzej Cichocki and Howard Hua Yang, "Recurrent
Neural Networks for Blind Separation of Sources", , pp.37-42.
Anthony J. Bell and Terrence J. Sejnowski, "Fast
Blind Separation based on Information Theory", pp. 43-47.
Adel Belouchrani and Jean-Francois Cardoso, "Maximum
Likelihood Source Separation by the Expectation-Maximization Technique:
Deterministic and Stochastic Implementation", pp.49-53.
Jean-Francois Cardoso, "The
Invariant Approach to Source Separation", pp. 55-60.
Andrzej Cichocki, Wlodzimierz Kasprzak and Shun-ichi Amari, "Multi-Layer
Neural Networks with Local Adaptive Learning Rules for Blind Separation
of Source Signals", , pp.61-65.
Yannick Deville and Laurence Andry, "Application
of Blind Source Separation Techniques to Multi-Tag Contactless Identification
Systems", , pp. 73-78.
Jie Huang , Noboru Ohnishi and Naboru Sugie "Sound
SeparatioN Based on Perceptual Grouping of Sound Segments", , pp.67-72.
Christian Jutten and Jean-Francois Cardoso, "Separation
of Sources: Really Blind ?" , pp. 79-84.
Kiyotoshi Matsuoka and Mitsuru Kawamoto, "Blind Signal Separation Based
on a Mutual Information Criterion", pp. 85-91.
Lieven De Lathauwer, Pierre Comon, Bart De Moor and Joos Vandewalle,
Power Method - Application in Independent Component Analysis" ,
Jie Zhu, Xi-Ren Cao, and Ruey-Wen Liu, "Blind
Source Separation Based on Output Independence - Theory and Implementation"
Papers are included in Proceedings 1995 International Symposium
on Nonlinear Theory and Applications NOLTA'95, Vol.1, NTA Research
Society of IEICE, Tokyo, Japan, 1995.
Shun-ichi Amari wrote some excelent papers with the RIKEN people on
BSS and the math behind it:
S. Amari, A. Cichocki and H. H. Yang, A
New Learning Algorithm for Blind Signal Separation (128K), In: Advances
in Neural Information Processing Systems 8, Editors D. Touretzky, M. Mozer,
and M. Hasselmo, pp.?-?(to appear), MIT Press, Cambridge MA, 1996.
Shun-ichi Amari, Neural
Learning in Structured Parameter Spaces , NIPS'96
Shun-ichi Amari, Information
Geometry of Neural Networks - New Bayesian Duality Theory - , ICONIP'96
Shun-ichi Amari, Gradient
Learning in Structured Parameter Spaces: Adaptive Blind Separation of Signal
Sources , WCNN'96
Shun-ichi Amari and Jean-Francois Cardoso, Blind
Source Separation - Semiparametric Statistical Approach, sumitted to
IEEE Tr. on Signal Processing.
Shun-ichi Amari, Natural
Gradient Works Efficiently in Learning, sumitted to Neural Computation.
Howard Hua Yang and Shun-ichi Amari, Adaptive
On-Line Learning Algorithms for Blind Separation - Maximum Entropy and
Minimum Mutual Information , accepted for Neural Computation.
Shun-ichi Amari, Tian-Ping CHEN, Andrzej CICHOCKI, Stability
Analysis of Adaptive Blind Source Separation , accepted for Neural
Shun-ichi Amari, Superefficiency
in Blind Source Separation , sumitted to IEEE Tr. on Signal Processing.
Shun-ichi Amari and Noboru Murata, Statistical
Analysis of Regularization Constant - From Bayes, MDL and NIC Points of
View, International Work-Donf. on Artificial and Natural Neural Networks
Geometry of Semiparametric Models and Applications , ISI'97
Jean-François Cardoso has lots (and lots, and lots, ...) of papers on
Jean-François Cardoso and Beate Laheld. Equivariant
adaptive source separation. To appear in IEEE Trans. on S.P.
Jean-François Cardoso. Performance
and implementation of invariant source separation algorithms In Proc.
Jean-François Cardoso, Sandip Bose, and Benjamin Friedlander. On
optimal source separation based on second and fourth order cumulants
In Proc. IEEE Workshop on SSAP, Corfou, Greece, 1996.
Jean-François Cardoso. The
equivariant approach to source separation In Proc. NOLTA, pages
Jean-François Cardoso. Séparation
de sources dans l'espace signal In Proc. GRETSI, Juan les Pins,France,
Jean-François Cardoso. A
tetradic decomposition of 4th-order tensors: application to the source
separation problem In M. Moonen and B. de Moor, editors, Algorithms,
architectures and applications, volume III of SVD and signal processing,
pages 375-382. Elsevier, 1995.
Jean-François Cardoso, Sandip Bose, and Benjamin Friedlander. Output
cumulant matching for source separation In Proc. IEEE SP Workshop
on Higher-Order Stat., Aiguablava, Spain, pages 44-48, 1995.
Adel Belouchrani and Jean-François Cardoso. Maximum
likelihood source separation for discrete sources In Proc. EUSIPCO,
pages 768-771, Edinburgh, September 1994.
Jean-François Cardoso. On
the performance of source separation algorithms In Proc. EUSIPCO,
pages 776-779, Edinburgh, September 1994.
Jean-François Cardoso, Adel Belouchrani, and Beate Laheld. A
new composite criterion for adaptive and iterative blind source separation
In Proc. ICASSP, volume 4, pages 273-276, April 1994.
Beate Laheld and Jean-François Cardoso. Adaptive
source separation with uniform performance In Proc. EUSIPCO,
pages 183-186, Edinburgh, September 1994.
Jean-François Cardoso and Antoine Souloumiac. An
efficient technique for blind separation of complex sources In Proc.
IEEE SP Workshop on Higher-Order Stat., Lake Tahoe, USA, pages 275-279,
Jean-François Cardoso. Iterative
techniques for blind source separation using only fourth order cumulants
In Proc. EUSIPCO, pages 739-742, 1992.
Jean-François Cardoso and Beate Laheld. Adaptive blind source separation
for channel spatial equalization In Proc. of COST 229 workshop on adaptive
signal processing, pages 19-26, 1992.
Jean-François Cardoso. Eigen-structure
of the fourth-order cumulant tensor with application to the blind source
separation problem In Proc. ICASSP, pages 2655-2658, 1990.
Jean-François Cardoso. Source
separation using higher order moments In Proc. ICASSP, pages
Jean-François Cardoso and Pierre Comon. Independent
component analysis, a survey of some algebraic methods In Proc.
ISCAS'96, vol.2, pp. 93-96, 1996.
Jean-François Cardoso, Infomax
and maximum likelihood for source separation,. To appear in IEEE Letters
on Signal Processing, April, 1997.
Cichocki & Kasprzak have a nice collection of papers:
Cichocki A., Kasprzak W.: Nonlinear
Learning Algorithms for Blind Separation of Natural Images , Neural
Network World, vol.6, 1996, No.4, IDG Co., Prague, 515-523.
Cichocki A., Kasprzak W., Amari S.-I.: Neural
Network Approach to Blind Separation And Enhancement of Images ,
, (Trieste, Italy, September 1996).
Kasprzak W., Cichocki A.: Hidden
Image Separation From Incomplete Image Mixtures by Independent Component
Analysis , ICPR'96 , Vienna, August 1996.
Cichocki A., Amari S., Adachi M., Kasprzak W.: Self-Adaptive
Neural Networks for Blind Separation of Sources , 1996 IEEE
International Symposium on Circuits and Systems, ISCAS'96, Vol. 2,
IEEE, Piscataway, NJ, 1996, 157-160.
Mark Girolami at University of Paisley has some papers too:
Girolami, M and Fyfe, C.
Blind Separation of Sources Using Exploratory Projection Pursuit Networks.
and Signal Processing, International Conference on the Engineering Applications
of Neural Networks, ISBN 952-90-7517-0, London, pp249-252, 1996.
Girolami, M and Fyfe, C.
Higher Order Cumulant Maximisation Using Nonlinear Hebbian and Anti-Hebbian
Learning for Adaptive Blind Separation of Source Signals, Proc IWSIP-96,
IEEE/IEE International Workshop on Signal and Image Processing, Advances
in Computational Intelligence, Elsevier Science, pp141 - 144, Manchester,
4-7 November 1996.
Girolami, M and Fyfe, C.
Multivariate Density Factorisation for Independent Component Analysis :
An Unsupervised Artificial Neural Network Approach, AISTATS-97,
3'rd International Workshop on Artificial Intelligence and Statistics,
Fort Lauderdale, Florida, Jan 1997.
Girolami, M and Fyfe, C.
Negentropy and Kurtosis as Projection Pursuit Indices Provide Generalised
ICA Algorithms, NIPS-96 Blind Signal Separation Workshop, (Org A.
Cichocki & A.Back), Aspen, Colorado, 7 Dec, 1996.
Girolami, M and Fyfe, C.
A Temporal Model of Linear Anti-Hebbian Learning, Neural Processing
Letters, In Press Vol 4, Issue 3, Jan 1997.
Te-Won Lee at the Salk Institute has some interesting papers:
separation of delayed and convolved sources, T.W. Lee and A.J. Bell
and R. Lambert, accepted for publication in "Advances in Neural
Information Processing Systems", MIT Press, Cambridge MA, 1996
Source Separation of Real World Signals, T.W. Lee, A.J. Bell and
R. Orglmeister, to appear in "IEEE International Conference Neural
Networks ", Houston, 1997
Juergen Schmidhuber at IDSIA has related papers:
Henrik Sahlin's papers on separation and second order statistics:
H. Broman, U. Lindgren, H. Sahlin and P. Stoica
Separation: A TITO system identification approach'', Tech. rep. CTH-TE-33,
Department of Applied Electronics, Chalmers University of Technology, Sept.
1995, Submitted to IEEE Trans. SP.
H. Sahlin and U. Lindgren "The
Asymptotic Cramer-Rao Lower Bound for Blind Signal Separation". "The
Proceedings of the 8th IEEE Signal Processing Workshop on Statistical Signal
and Array Processing", Corfu, Greece, 1996
H. Sahlin "Asymptotic
parameter variance analysis for Blind Signal Separation" "The proceedings
of RVK96", Lulea, Sweden, 1996
H. Sahlin and H. Broman "Blind
Separation of Images" "The Proceedings of Asilomar Conference on Signals,
Systems, and Computers", Pacific Grove, CA, USA, 1996.
U. Lindgren, H. Sahlin and H. Broman
separation using second order statistics" "The Proceedings of EUSIPCO-96",
Trieste, Italy, 1996.
H. Sahlin, U. Lindgren and H. Broman
Input Multi Output Blind Signal Separation Using Second Order Statistics"
Tech. rep. CTH-TE-54, Department of Applied Electronics, Chalmers University
of Technology, Dec., 1996.
Signal Separation by Second Order Statistics" Licentiate Thesis, Technical
Report No. 251L, Department of Applied Electronics, Chalmers University
of Technology, 1997.
H. Sahlin and H. Broman
Separation Applied to Real World Signals" Proceedings of International
Workshop on Accoustic Echo and Noise Control, London, UK, September,1997.
Gustavo Deco and Dragan Obradovic have an excellent book on ICA and
Kevin Knuth on ICA:
source separation and localization. To be published in: SPIE'98
Proceedings: Bayesian Inference for Inverse Problems, San Diego, July 1998.
Knuth K.H. 1999. A Bayesian approach to source separation. In: J.-F.
Cardoso, C. Jutten and P. Loubaton (eds.), Proceedings of the First
International Workshop on Independent Component Analysis and Signal
Separation: ICA'99, Aussios, France, Jan. 1999, pp. 283-288.
- Knuth K.H. and Vaughan H.G., Jr. 1998.
Convergent Bayesian formulations of
blind source separation and electromagnetic source estimation. Presented
at the MaxEnt98 workshop in Munich, July 1998.
Alex Westner has worked on real world experiments:
- Roberto Manduchi on ICA and textures:
- Lucas Parra:
- Lucas Parra, Clay Spence, "On-line convolutive source separation
of non-stationary signals", Journal of VLSI Signal Processing,
Special issue on the 1998 IEEE Neural Networks and Signal Processing
Workshop, to appear in summer 2000, (.ps.gz
249K, .pdf 482K)
- Lucas Parra, Klaus-Robert Mueller, Clay Spence, Andreas Ziehe,
Paul Sajda, "Unmixing Hyperspectral Data", Advances in Neural Information Processing Systems 12, MIT
Press, to appear 1999. (.ps.gz 76K)
- Simone Fiori has some papers on ICA by the adaptive activation
function networks and learning on Stiefel-Grassman manifold:
- S. Fiori,
"Entropy Optimization by the PFANN Network: Application to
Independent Component Analysis", Network: Computation in Neural Systems,
Vol. 10, No. 2, pp. 171 - 186, May 1999
- S. Fiori,
Learning for Blind Source Separation",
Electronics Letters, Vol. 35, No. 22, pp. 1963 - 1964, Oct. 1999
- S. Fiori, "Blind Separation of
Distributed Source Signals
by the Neural Extended APEX Algorithm", Neurocomputing, Vol. 34, No. 1-4,
pp. 239 - 252, August 2000
- S. Fiori, "Blind Signal Processing by
the Adaptive Activation Function
Neurons", Neural Networks, Vol. 13, No. 6, pp. 597 - 611, August 2000
- S. Fiori, "A Theory for Learning by
Weight Flow on Stiefel-Grassman
Manifold", Neural Computation. Accepted for publication: To appear.
MIT Media Lab
20 Ames Street, Rm E15-401
Cambridge, MA 02139