Machine Learning manuscript No. (will be inserted by the editor) Temporal Kernel CCA and its Application in Multimodal Neuronal Data Analysis Felix Bießmann · Frank C. Meinecke · Arthur Gretton · Alexander Rauch · Gregor Rainer · Nikos K. Logothetis · Klaus-Robert Muller¨ the date of receipt and acceptance should be inserted later Felix Bießmann TU Berlin, Machine Learning Group Franklinstr 28/29 10587 Berlin E-mail:
[email protected] Frank C. Meinecke TU Berlin, Machine Learning Group Franklinstr 28/29 10587 Berlin E-mail:
[email protected] Arthur Gretton MPI Biological Cybernetics, Dept. Empirical Inference Spemannstr. 38 72076 Tubingen¨ E-mail:
[email protected] Alexander Rauch MPI Biological Cybernetics, Dept. Physiology of Cognitive Processes Spemannstr. 38 72076 Tubingen¨ E-mail:
[email protected] Gregor Rainer University of Fribourg, Visual Cognition Laboratory Chemin du Musee 5 CH-1700 Fribourg E-mail:
[email protected] Nikos K. Logothetis MPI Biological Cybernetics, Dept. Physiology of Cognitive Processes Spemannstr. 38 72076 Tubingen¨ E-mail:
[email protected] Klaus-Robert Muller¨ TU Berlin, Machine Learning Group Franklinstr 28/29 10587 Berlin E-mail:
[email protected] 2 Abstract Data recorded from multiple sources sometimes exhibit non-instanteneous cou- plings. For simple data sets, cross-correlograms may reveal the coupling dynamics. But when dealing with high-dimensional multivariate data there is no such measure as the cross- correlogram. We propose a simple algorithm based on Kernel Canonical Correlation Anal- ysis (kCCA) that computes a multivariate temporal filter which links one data modality to another one. The filters can be used to compute a multivariate extension of the cross- correlogram, the canonical correlogram, between data sources that have different dimen- sionalities and temporal resolutions.