Paolo Pretto

Project Leader

I am leading the research group together with .

In our group, we conduct both fundamental and applied research on self-motion. The aim of our fundamental research is a comprehensive understanding of the perceptual and cognitive processes underlying self-motion. We perform experimental studies to measure perceptual thresholds, to untangle the mechanisms behind motion sickness, and to develop and validate models of the processes by which the brain constructs percepts of self-motion. Also, we apply knowledge of self-motion perception in the development of motion cueing algorithms, we assess the fidelity of motion cueing solutions, we investigate potential side-effects of autonomous driving on passengers, and we develop solutions to these problems.

Background

The genuine passion for cars and the human brain has led my professional growth. I graduated cum laude in Psychology at the University of Padova, Italy, with a thesis on driver’s perception in virtual environments. After a short experience in the automotive industry, I started a bi-national Phd (University of Padova, Italy & University of Tübingen, Germany) on Perception, Psychophysics and Behavioral Sciences. During the PhD and my early Post-doc years I studied the perception and control of self-motion in realistically simulated environment, with focus on driving scenarios. Since 2011 I am leading the Motion Perception and Simulation research group. In this role, beside my primary research activity, I have initiated and managed several collaborations with automotive OEMs, I have been scientific leader and project manager for the project, and I have organized several international scientific events. My current research interests include visual and multisensory self-motion perception, virtual reality and simulation, user experience, psychophysics and behavioural research methods.

Selected publications

• Foggy perception slows us down eLife (2012)
• Perception-Based Motion Cueing: A Cybernetics Approach to Motion Simulation Recent Progress in Brain and Cognitive Engineering (2015)
• Roll rate perceptual thresholds in active and passive curve driving simulation Simulation: Transactions of the Society for Modeling and Simulation International (2016)

Continuous subjective rating of the perceived quality of driving simulation

We have compared two motion cueing algorithms (MCA), a filter-based and an optimization-based MCA, using a newly developed motion cueing quality rating method. The goal of the comparison was to investigate whether optimization-based MCAs have, compared to filter-based approaches, the potential to improve the quality of motion simulations. The driving simulation experiment was executed on the Daimler Driving Simulator (Sindelfingen, Germany). Eighteen participants rated the perceived motion incongruence, i.e., the perceived mismatch between the motion felt in the simulator and the motion one would expect from a drive in a real car. The results show that the quality of the motion cueing was rated better for the optimization-based MCA than for the filter-based MCA, indicating that there exists a potential to improve the quality of the motion simulation with optimization-based methods. Furthermore, it was shown that the continuous rating method provides reliable and repeatable results within and between participants.

Cleij D, Venrooij J, Pretto P, Katliar M, Bülthoff HH, Steffen B, Hoffmeyer FW and Schöner H-P (2017) Comparison between filter- and optimization-based motion cueing algorithms for driving simulation Transportation Research Part F: Traffic Psychology and Behaviour Epub ahead.

Visually induced motion sickness

It is well known that visual motion can produce a compelling illusion of self-motion in stationary observers. Indeed, we are all familiar with the so-called ‘train illusion’, which occurs whenever we feel that our train is departing while, in fact, we see the train at the neighboring platform leaving the station. This phenomenon is exploited in many fixed-base motion simulators. A negative side effect is that prolonged exposure to visually induced self-motion (Vection) can give rise to motion sickness. In this project we look at motion perception and motion sickness when the entire visual surround is rotated around the observer’s vertical (yaw) axis. This specific situation is interesting, because it allows us to disentangle different theories on motion sickness, all calling upon different causal factors. By adopting an integrative approach to this problem, and obtaining continuous measures of motion sickness, perceived self-motion, eye and head movements, we aim at a better understanding of factors contributing to visually induced motion sickness.

Nooij SAE, Pretto P, Oberfeld D, Hecht H and Bülthoff HH (2017) Vection is the main contributor to motion sickness induced by visual yaw rotation: Implications for conflict and eye movement theories PLoS ONE 12(4): e0175305.

Motion perception in curves

When in darkness, humans can perceive the direction and magnitude of rotations and of linear translations in the horizontal plane. This project has addressed the integrated perception of combined translational and rotational motion, as it occurs when moving along a curved trajectory. We questioned whether the perceived motion through the environment follows the predictions of a self-motion perception models, which assume linear addition of rotational and translational components. For curved motion in darkness, such models predict a non-veridical motion percept, consisting of an underestimation of the perceived rotation, a distortion of the perceived travelled path, and a bias in the perceived heading (i.e., the perceived instantaneous direction of motion with respect to the body). These model predictions were evaluated in two experiments. In Experiment 1, seven participants were moved along a circular trajectory in darkness while facing the motion direction. They indicated perceived yaw rotation using an online tracking task, and perceived travelled path by drawings. In Experiment 2, the heading was systematically varied, and six participants indicated whether they perceived facing inward or outward of the circular path. Overall, we found that the perceived heading was quite veridical, and no evidence for the heading bias predicted by the model. This suggests that the sum of the perceived rotational and translational components alone cannot adequately explain the overall perceived motion through the environment. Possibly, knowledge about motion dynamics and familiar stimuli combinations may play an important additional role in shaping the percept.

Nooij SAE, Nesti A, Bülthoff HH and Pretto P (2016) Perception of rotation, path, and heading in circular trajectories Experimental Brain Research 234(8) 2323–2337.

Motion sensitivity in driving simulation

In driving simulation, simulator tilt is used to reproduce sustained linear acceleration. In order to feel realistic, this tilt is performed at a rate below the human tilt rate detection threshold, which is usually assumed constant. However, it is known that many factors affect the threshold, such as visual information, simulator motion in additional directions, or the driver’s active effort required for controlling the vehicle. Here we investigated the effect of these factors on the roll rate detection threshold during simulated curve driving. Ten participants reported whether they detected roll motion in multiple trials during simulated curve driving, while roll rate was varied over trials. Roll rate detection thresholds were measured under four conditions. In the first three conditions, participants were moved passively through a curve with the following: (i) roll only in darkness; (ii) combined roll/sway in darkness; (iii) combined roll/sway and visual information. In the fourth (iv) condition participants actively drove through the curve. The results showed that roll rate thresholds in simulated curve driving increase, that is, sensitivity decreases, when the roll tilt is combined with sway motion. Moreover, an active control task seemed to further increase the detection threshold, that is, impair motion sensitivity, but with large individual differences. We hypothesize that this is related to the level of immersion during the task.

Nesti A, Nooij SAE, Losert M, Bülthoff HH and Pretto P (2016) Roll rate perceptual thresholds in active and passive curve driving simulation Simulation: Transactions of the Society for Modeling and Simulation International 92(5) 417-426.

Inertial feedback in teleoperation of unmanned aerial vehicle

The main purpose of this project wass to augment the information available to the operator of a remote unmanned aerial vehicle (UAV), typically limited to video stream from onboard cameras, by providing the operator with additional information on the UAV state using inertial feedback, i.e. by moving a platform on which the control station is located.  The motion of the platform informs then the operator about the current UAV state with regards to, e.g., wind disturbances, without increasing the information load of the visual channel with additional gauges. This constitutes one of the first attempts to create a multisensory teleoperation control station that exploits the characteristics of human inertial motion sensors. Experimental results have shown that adding inertial information to visual feedback about the UAV state improves control performance in a hover task (higher precision is attained). Moreover, we have also implemented the possibility of substituting a motion channel like, e.g., roll, with information that are not related to the vehicle, but rather to the task, like, e.g., offset between actual and target vehicle position. This way, the platform motion informs the operator about the accomplishment of a specific task, regardless of UAV state. We have shown that with task-related inertial feedback participants are able to fly faster while maintaining safer distances to the obstacles in a collision avoidance task, with performance that compares well with the one obtained when haptic feedback is provided in addition to the visual.

Lächele J, Venrooij J, Pretto P and Bülthoff HH (2016) Effects of vehicle- and task-related motion feedback on operator performance in teleoperation In: Leveraging Emerging Technologies for Future Capabilities, , 72nd American Helicopter Society International Annual Forum (AHS 2016), Curran, Red Hook, NY, USA, 3310-3316.

Human discrimination of visual-inertial self-motion

To successfully perform daily activities such as maintaining posture or running, humans need to be sensitive to self-motion over a large range of motion intensities. Recent studies have shown that the human ability to discriminate self-motion in the presence of either inertial only motion cues or visual-only motion cues is not constant but rather decreases with motion intensity. However, these results do not yet allow for a quantitative description of how self-motion is discriminated in the presence of combined visual and inertial cues, since little is known about visual–inertial perceptual integration and the resulting self-motion perception over a wide range of motion intensity. Here we investigate these two questions for head-centred yaw rotations presented either in darkness or combined with visual cues. Participants discriminated a reference motion, repeated unchanged for every trial, from a comparison motion, iteratively adjusted in peak velocity so as to measure the participants’ differential threshold, i.e. the smallest perceivable change in stimulus intensity. A total of six participants were tested at four reference velocities. Overall, differential thresholds increase with stimulus intensity following a trend well described power functions. Moreover, combining visual and inertial stimuli does not lead to improved discrimination performance over the investigated range of yaw rotations. We further observed that the time necessary for a visual-only stimulus to evoke a compelling self-motion illusion is independent from stimulus intensity and facilitated by recent exposure.

Nesti A, Beykirch KA, Pretto P and Bülthoff HH (2015) Human discrimination of head-centred visual–inertial yaw rotations Experimental Brain Research 233(12) 3553-3564.

Development and validation of perception-based motion cueing

The goal of vehicle motion simulation is the realistic reproduction of the perception a human observer would have inside the moving vehicle by providing realistic motion cues inside a motion simulator. Motion cueing algorithms (MCA) play a central role in this process by converting the desired vehicle motion into simulator input commands with maximal perceptual fidelity, while remaining within the limited workspace of the motion simulator. In this project, we have developed a perception-based motion cueing algorithm that relies on knowledge about human self-motion perception and uses it to calculate the vehicle motion percept, i.e. how the motion of a vehicle is perceived by a human observer. The goal of the perception-based algorithm is then to reproduce the simulator motion that minimizes the difference between the vehicle’s desired percept and the actual simulator percept, i.e. the “perceptual error”. Also, we have designed and executed the first experimental validation of the new  perception-based MCA. The goal of such an evaluation was to show the potential benefits and improvements with respect to the current motion cueing approaches. In order to make this comparison possible, a novel methodology was developed, based on the psychophysical method of magnitude estimation with cross modality matching, which is typical of human perception research. The results of this first validation experiment have shown that an improvement in the current standards of motion cueing is possible. Moreover, the psychophysical methodology allowed for the collection of quantitative data and the use of inferential statistics, which was used to find differences on the quality score of MCAs.

Pretto P, Venrooij J, Nesti A and Bülthoff HH (2015) Perception-Based Motion Cueing: A Cybernetics Approach to Motion Simulation In: Recent Progress in Brain and Cognitive Engineering, (Ed) S.-W. Lee, Springer, Dordrecht, The Netherlands, 131-152.

WABS Wahrnehmungsbasierte Bewegungssimulation

Within the WABS project we developed an alternative approach to motion cueing. Motion cueing is the process of converting a desired physical motion (e.g. from a vehicle model) into commands that are sent to the motion simulator. This conversion is done by a so-called Motion Cueing Algorithm (MCA).The WABS project is based on the idea that motion cueing can be improved by including novel insights in human self-motion perception, obtained from fundamental motion perception studies, in the MCAs. This ‘perception-based motion cueing’ approach allows for exploiting the limitations and ambiguities of the human perceptual system.

This project was funded by the German Federal Ministry for Education and Research through the VIP (Validation of the Innovation Potential of scientific research) funding program for the period 2011-2014. More about