Enter the Rift
Where we are…
The 2nd developer iteration (DK2) of Oculus’s Rift technology is amazing, but still not consumer ready. Most of the reason for that is the software, with the VR & HCI communities still learning how to use the interface in meaningful ways, but the hardware has some of its limitations too. In any case, it’s not there just yet.
Starting with the hardware side (shorter list), the headset is still too heavy/bulky (though well distributed across the straps), and long-term use (more than a half hour or so) starts to get really toasty. These things will be worked out tough, and new features will continue to increase the sense of immersion (I can’t wait to see what Oculus puts together with galvanic vestibular stimulation [balance hacking]!). Increases in resolution would also be nice, but at this point aren’t a hard requirement for making the experience effective.
On the software side, there are 2 major classes of issues. The first is mostly best-practices related – taking advantage of the tools and techniques from the latest SDK (0.4+), and really optimizing the rendering pipeline for low latency. Probably the coolest trick of the newest release is the ‘timewarp’ functionality, which analyzes head tracking data (fused from several different sensor inputs across the camera tracking, magnetometry, accelerometry, and gyroscopy) and predicts where the wearer’s head and gaze will be in the near future. This is combined with repeated sampling of frame render timing to see how long it takes to draw a frame in-context, and suggests that the next frame be drawn from that near-future position, so by the time the picture makes it onto the display it and the wearer’s position are well aligned and apparently highly synchronous. Hopefully more applications will be making use of this soon, and even extending it into the rest of the rendered world by creating near-term temporal projections about the location of objects in motion (implementing one or the other will result in perceptual lag of a timely world, or world-lag as perceived in a timely manner). We’re talking tiny fractions of a second here – 13-20ms, or 50-75hz – but that matters when it comes to how we visually process the world.
As an aside, our own visual processing system uses a lot of shortcuts similar to ‘timewarp’ for piecing together the world in a coherent narrative without having to do complete processing at every step. That’s what allows the low-persistence mode to work for sharpening the DK2 display, and also why we’re susceptible to optical illusions. When stimulus and expectations (projected stimulus, or filled-in-blanks) don’t match the result is higher-energy consumption in processing and re-assessing the visual field, and frequent nausea (assumptions by our survival instincts that we’ve consumed a neurotoxin, and should probably get rid of it – or at the very least, have a lie-down). Simulator sickness is a high-potency form of motion sickness, and needs special care on the software side to eliminate it.
On to the second facet of issues on the software side: there’s only so long one can use a VR headset before coming up against an applicability barrier, a kind of, “So what’s it good for?”, or “Now what?” VR sure is nifty, but it’s not an end to itself – just like any really good display (“Hey, check out my high-def edge-lit LCD!”), what really matters is what you choose to display, and then whether or not how it’s displayed enhances or detracts from that delivery.
What’s next…
So, what’s the killer app (or class of apps) for VR? Where do 3-dimensional perception and “sense of presence” matter? Training of all kinds will get a huge boost, as the realism of the training environment can be shifted from physical analogs to virtual ones (emergency responders, surgery, military, etc.), and for craft-piloting of all types (cars, boats, trains, planes, spacecraft, you name it): increased presence, decreased implementation and maintenance costs over traditional simulators, and far more dynamic environments.
My first thoughts from the DK1 still hold as well – every architect anywhere is going to want to make use of these, either for themselves or for collaboration with clients and partners who don’t have the same knack for looking at schematics and seeing a fully realized space in their heads. Firms are already adapting and/or popping up for exactly this.
Games will of course be a big market, although the kind of games for which this is good and where it really makes sense still need to be figured out – putting a VR-toting player up against desktop counterparts in an FPS is going to be a really bad experience for the poor guy in the (frequently virtually perforated) headset, where the established mechanics don’t translate effectively and everybody else walks all over them. Those mechanics will need to be re-worked, and/or new environments figured out. I’m hoping networked flight-sims get a boost, and penetrate further into the general gamer consciousness.
But what else? We’re still talking niche use case here. Content creation, and/or explorable spaces, especially those not usually accessible to the general public, will be nice; accessibility tools (more immersive telepresence for the disabled); remote craft control improvements… There are lots of new venues where a very eager community will find some way of trying it out, and eventually I’m hoping will produce some prime examples to solidly answer the question, “Why VR?”
For my part I have 3 different projects in mind, which I’ll be exploring in more detail here as I play around with them. The first two are all about exploring the limits and capabilities of perception in VR, looking at self and then environment:
The Rift, and particularly the DK2 enhancements, are all about “presence” – making it feel like You are There. I want to explore the proprioceptive side of that – how much of you are we talking about, and how you is it? There have been some fun experiments along these lines using video mapping in the past, mostly based off of the work done at the Neuroscience department of the Karolinska Institutet in Stockholm (I specifically love their 2008 paper at PLOS One). Some folks have even dabbled into using the Rift as the display device in such a scenario. I’m planning on going a little further: using the DK2 together with the Leap Motion controller to interact directly with a rigged model in the VR space, what are the dimensions and cues responsible for establishing proprioception and to what degree? We know already that humanoid form is important, as is synchronicity of stimuli, but how humanoid, and how synchronous? Can one self-establish the tactile cues necessary for calibration (e.g., “using this finger, touch these points on the opposite arm/wrist/hand…”), and how can the depth of translation be measured? (prior techniques used galvanic sensors during simulated threat, and threat simulation may not be a good tool here). Once these parameters have been defined I’m hoping to also identify any impediments and hurdles to be overcome and propose some solutions thereto, and to share the research and tools with the rest of the community (and various university neuroscience departments) to see what they want to make of if – therapeutic applications alone may be pretty significant.
The next part of exploratory development is around information representation: how to use available modes of visual communication to explore (navigate, consume, and manipulate) data or systems of varying complexity when one has spatial degrees of freedom. Which is to say, when combining spatial (absolute [zones] and relative [as compared to others] positions), visual (shape, color, texture, orientation, scale), aural, and temporal (change over time, such as movement, animation, or some shift in any of the available axes) degrees, how much information can reasonably be identified and communicated? How subtle is too subtle? How much is too much when it comes to visual busy-ness, and at what point does it make sense to use such space to represent data, systems, or both (the locus and mechanism by which data is consumed or created within a system)? Smarter people than me will be spending more time on this stuff, but I’m also interested in just tinkering around since I’m a strong kinesthetic thinker myself – I’d love this to be intuitive and useful, but chances are the learning curve is too high, and data modelers and programmers will eventually spend time in school on The Language of Virtual Infomechanics 101.
My last project is about using archetypes and quasi-subconscious cues to navigate a responsive environment as an analog for dream-like space and situations. More on that one later (since I want to get this posted sooner than later and have other stuff to chase down at the moment).
All that, and play: lots and lots of playing with good work being done by lots of folks. Worst case scenario, this is a really fun toy. 🙂
Latest Comments:
Comments are closed for this post.
[…] Enter the Rift […]