Multimodal motion processing in area V5/MT

Evidence from an artificial class of audio-visual events

authored by: Lukas Scheef, Henning Boecker, Marcel Daamen, Ursula Fehse, Martin W. Landsberg, Dirk Oliver Granath, Heinz Mechling, Alfred O. Effenberg
Abstract: Audio-visual integration in the human brain influences perception and precision of motor tasks. We tested audio-visual integration during height estimation when presenting video clips of counter movement jumps (CMJ), using sparse sampling fMRI at 3T. Employing the technique of "sonification", we created artificial auditory-visual motion events by transforming the ground reaction force of the CMJs into the auditory domain, modulating frequency and amplitude of the standard pitch "A" (440 Hz). We combined these "sonificated" movements with either concordant or discordant visual movement displays. We hypothesized that processing of concordant audio-visual stimuli would enhance neural activity in audio-visual integration areas. Therefore, four conditions were compared: 1. unimodal visual, 2. unimodal auditory, 3. auditory + visual concordant, and 4. auditory +visual discordant. The unimodal conditions, when compared against each other, resulted in expected activation maxima in primary visual and auditory cortex, respectively. Enhanced activation was found in area V5/MT bilaterally for the concordant multimodal, as compared to both unimodal, conditions. This effect was specific for the concordant bimodal condition, as evidenced by a direct comparison between concordant and discordant bimodal conditions. Using "sonification", we provide evidence that area V5/MT is modulated by concordant auditory input, albeit the artificial nature of the stimuli, which argues for a role of this region in multimodal motion integration, beyond the pure visual domain. This may explain previous behavioral evidence of facilitatory effects exerted by auditory motion stimuli on the perception of visual motion, and may provide the basis for future applications in motor learning and rehabilitation.
External Organisation(s): University of Bonn
Type: Article
Journal: Brain research
Volume: 1252
Pages: 94-104
No. of pages: 11
ISSN: 0006-8993
Publication date: 03.02.2009
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Neuroscience(all), Molecular Biology, Clinical Neurology, Developmental Biology
Electronic version(s): https://doi.org/10.1016/j.brainres.2008.10.067 (Access: Unknown)

BibTeX

@article{03fed45e594a47058d811adefd5c65cb,
title = "Multimodal motion processing in area V5/MT: Evidence from an artificial class of audio-visual events",
abstract = "Audio-visual integration in the human brain influences perception and precision of motor tasks. We tested audio-visual integration during height estimation when presenting video clips of counter movement jumps (CMJ), using sparse sampling fMRI at 3T. Employing the technique of {"}sonification{"}, we created artificial auditory-visual motion events by transforming the ground reaction force of the CMJs into the auditory domain, modulating frequency and amplitude of the standard pitch {"}A{"} (440 Hz). We combined these {"}sonificated{"} movements with either concordant or discordant visual movement displays. We hypothesized that processing of concordant audio-visual stimuli would enhance neural activity in audio-visual integration areas. Therefore, four conditions were compared: 1. unimodal visual, 2. unimodal auditory, 3. auditory + visual concordant, and 4. auditory +visual discordant. The unimodal conditions, when compared against each other, resulted in expected activation maxima in primary visual and auditory cortex, respectively. Enhanced activation was found in area V5/MT bilaterally for the concordant multimodal, as compared to both unimodal, conditions. This effect was specific for the concordant bimodal condition, as evidenced by a direct comparison between concordant and discordant bimodal conditions. Using {"}sonification{"}, we provide evidence that area V5/MT is modulated by concordant auditory input, albeit the artificial nature of the stimuli, which argues for a role of this region in multimodal motion integration, beyond the pure visual domain. This may explain previous behavioral evidence of facilitatory effects exerted by auditory motion stimuli on the perception of visual motion, and may provide the basis for future applications in motor learning and rehabilitation.",
keywords = "Area V5/MT, Audio-visual, fMRI, Multisensory integration, Sonification, Sparse sampling",
author = "Lukas Scheef and Henning Boecker and Marcel Daamen and Ursula Fehse and Landsberg, {Martin W.} and Granath, {Dirk Oliver} and Heinz Mechling and Effenberg, {Alfred O.}",
year = "2009",
month = feb,
day = "3",
doi = "10.1016/j.brainres.2008.10.067",
language = "English",
volume = "1252",
pages = "94--104",
journal = "Brain research",
issn = "0006-8993",
publisher = "Elsevier",
}