Multisensory Perception & Action
(Marc Ernst)
We used all our senses to construct a reliable percept representing the world with which we interact. The view we took in the independent Max-Planck Research Group for Human Multisensory Perception and Action was that in many aspects of behavior, motor actions and multisensory processing are inseparably linked and therefore have to be studied in a closed action/perception loop. We believe that human perception and action is tailored to the statistics of the natural environment and when the environment changes our perceptions will follow these changes through the process of adaptation minimizing potential costs during interaction.
In the neural processing such statistics will represent itself in probability distributions. We followed Hermann von Helmholtz in our belief that human perception is a problem of inference, for which the sensory data are often not sufficient to uniquely determine the percept. Thus, prior knowledge has to be used to constrain the process of inference from ambiguous sensory signals. A principled way to describe the combination of prior knowledge with sensory data in a probabilistic way is the Bayesian Framework. Therefore, we regularly use this Bayesian Framework to construct “ideal observer” models—models that use the available information in the most optimal way, provided some task and cost function. These models can then be used as a benchmark against which human performance can be tested. To do so in the Multisensory Perception and Action Group we used quantitative psychophysical and neuropsychological methods together with Virtual Reality techniques. Quantitative psychophysical methods are important to best determine the relevant perceptual parameters minimizing uncertainty and unknowns. Virtual Reality is important, because it provides us with a tool to precisely control the perceptual situation that are investigated, while at the same time it allows for a degree of interaction, which is necessary for studying the action/perception loop. Often, however, today’s Virtual Reality techniques and human-computer interaction devices are not sufficiently developed to be readily used in the study of human perception and action. Therefore, some of our work concentrated on the development of human-machine interfaces. This was mostly done in the framework of European projects. For example, the European Projects Touch-Hapsys and ImmerSence focuses on the development of haptic interaction devices whereas the European project CyberWalk had the goal to develop an omni-directional treadmill for enabling near-natural locomotion in Virtual Reality.