Andre Mastmeyer

Personalized Virtual Reality Simulations Andre Mastmeyer

In the context of individualized medicine and in the age of 4D imaging, current research covers the plausible personalized, visuo-haptic Virtual Reality (VR) immersion for training and planning of puncture interventions. In this book, this subject is first overviewed and presented regarding the technical details by selected English scientific publications. The patient model results from a mostly automatic procedure for the segmentation of puncture-relevant organs and structures. For increased realism, the influence of simulated respiratory movement extracted and modeled from 4D-CT images was also taken into account. Based on these modeling steps, innovative, efficient GPU algorithms for volume-based visuo-haptic simulation and the direct volume rendering of physiological processes of the patient model were proposed. For a time-invariant patient model, the local consideration of space bending mathematically introduced by an inverted movement model leads to efficient visuo-haptic rendering methods: Simulated imaging of ultrasound, fluoroscopy, the stereoscopic two-eye perspective and haptics are elaborated. This approach eventually allowed the immersive interaction of the ultrasound probe and puncture needle with breathing virtual patients. The methods were developed in cooperation with clinical partners and successfully applied and evaluated in a user study. The medical potential of the methods presented in this work lies in the future in the pre-operative and personalized preparation of interventions. Specifically, for the described application of liver punctures, the number of needle repositionings could be significantly reduced, relieving the patient and saving treatment costs.