Opportunities for Novel Interactions with Virtual Realities
Virtual Reality (VR) is entering a wide range of professional and leisure contexts, such as remote collaboration and work meetings, training and gaming simulations, well-being applications, and social gatherings. Although VR devices have increased in technical fidelity and nowadays allow for rich visual and auditory output, making people perceive VR similar to physical reality remains challenging. Current VR systems mainly rely on controllers as the primary input channel. While this works for many scenarios, it does not resemble the interaction that users expect from their experience in physical reality. Further, considering output modalities, controllers cannot render the complex haptics of virtual objects with which VR users may interact while immersed in VR.
To make VR truly immersive and increase the presence of VR users within the underlying Virtual Environment (VE), VR systems should enable natural interaction known from reality. For example, a virtual door should be opened using a door handle just like a physical one. At the same time, VEs should react accordingly to the user's behavior by providing plausible feedback (e.g., creating a feeling of stiffness when the user reaches towards a virtual wall). To venture towards this form of VR, we must consider aspects that impact the interaction opportunities of VR users.
In this thesis, we address conflicts between the real world and VR as such conflicts can impede interaction within VEs. Next, we research ways to integrate the real world into VR to allow familiar and natural interaction with objects of interest. Finally, we introduce enhancements for haptics that enrich VR beyond what is possible with state-of-the-art controllers. Thus, we present research structured along three research themes.
In the first theme – “Avoiding Conflicts with the Real World” - we enhance redirected walking through Electrical Muscle Stimulation (EMS) and thereby minimize the physical space needed for natural locomotion in VR. Further, to use the available space more efficiently, we employ non-Euclidean virtual architectures that allow one to create an illusion of a VE that exceeds the available physical space. We then evaluate ways to utilize distractions to guide attention in VR, thereby making these illusions less conspicuous.
In the second theme – “Integrating the Real World” - we integrate real-world objects into the VR experience. We investigate ways to include objects from the real world in VR as well as what advantages the VE can provide compared to reality by manipulating virtual physics. In addition to integrating physical objects, we continue integrating neurological responses of VR users to control the virtual narrative via Brain Computer Interfaces (BCIs). In particular, we improve the sensing capabilities of VR devices by including neurological responses. Here, we design corresponding stimuli in such a way that they naturally blend into the VE.
In our third theme – “Enriching the Virtual World” - we focus on the haptic enhancements of virtual experiences. First, we enrich experiences that occur in more than one location. In particular, we enhance remote VR collaboration using haptic props by investigating different manipulation strategies that enable shared ownership of physically separate objects. Next, we investigate the deployment of flying User Interfaces (UIs) in VR. Here, we use drones equipped with 3D-printed input devices as haptic end effectors. We developed a system that autonomously positions the drones around VR users, thereby providing haptic feedback at the physical position at which users would expect to feel a virtual object when reaching out for it.
In conclusion, we introduce a wide array of interaction enhancements for VR. With that, we contribute insights that can help to shape future VR experiences, thereby bringing VR closer to becoming a ubiquitously employable technology. We complete this thesis by outlining our ideas of promising future research endeavors that can drive us toward the ultimate form of VR – a simulation indistinguishable from reality.