Jump to content
  • Advertisement
Sign in to follow this  
  • entries
    68
  • comments
    177
  • views
    82389

VR Lessons Learned So Far

Sign in to follow this  
capn_midnight

3384 views

[color=rgb(77,79,81)][font=Helvetica]


This is a loosely organized list of things I've noticed while using and developing virtual reality applications for the smartphone-in-headset form factor. It is specific to my experience and may not reflect anyone else's personal preference, such that VR is apparently quite dependent on preference. But I think that steadfast rules of design are necessary for the impending VR bubble, to convey an aesthetic and unified design such that users may expect certain, common idioms and adapt to VR software quickly. Thus, this is list a roadmap of my current aesthetic for VR. It is a living document, in the sense that future experiences may invalidate assumptions I have made and force me to recognize more universal truths. Proceed with caution.

[/font][/color]

  • Presence is the ability to feel like you are in the scene, not just viewing a special screen. You'll hear a lot of people talk about it, and It is important, but ultimately I believe it to be a descriptor of an end result, a combination of elements done well. There is no one thing that makes "presence", just as there is no one thing that makes an application "intuitive", "user friendly", "elegant", or "beautiful". They either are or they are not, and it's up to the individual experiences of the users to determine it.
  • Presence is a double-edged sword. [font=inherit]

    I've found that, once I feel "present" in the application, I also feel alone, almost a "ghost town" feeling. Even if the app has a single-user purpose, it seems like it would be better in an "arcade" sort of setting. To be able to see other people may help with presence.

    [/font]
  • The hardware is not yet ready for the mass market. That's good, actually, because the software and design side of things are a lot worse off. Now is the time to get into VR development. I'll say nothing more about the hardware issues from a performance side. They are well known, and being worked on [font=inherit]

    fervently

    [/font]by people with far more resources than I.
  • Mixing 2D and 3D elements is a no-go. Others have talked about not placing fixed-screen-space 2D heads-up-display elements in the view for video game applications, but it extends much further than that. The problem is two-fold: we currently have to take off the display to do simple things involving any sort of user input, and there is no way to manage separate application windows. We're a long way off from getting this one right. For now, we'll have to settle for being consistent in a single app on its own. A good start would be to build a form API that uses three-space objects to represent its controls.
  • Give the user an avatar. This may be a personal preference, but when I look down, I want to see a body. It doesn't have to be my body, it just needs something there. Floating in the air gives me no sense of how tall I stand, which in turn gives me no sense of how far away everything is.
  • Match the avatar to the UI, and vice versa. If your application involves a character running around, then encourage the user to stand and design around gamepads. If you must have a user sit at a keyboard, then create a didactic explanation for the restriction of their movement: put them in a vehicle.
  • Gesture control may finally be useful. I'm still researching this issue, but the experiments I've done so far have indicated that the ability to move the view freely and see depth make gestures significantly easier to execute than they have been with 2D displays. I am anxious to finish soldering together a device for performing arm gestures and test this more thoroughly. This demo makes it clear that this is at least an extremely lucrative path of study.
  • Use all of the depth cues. Binocular vision is not the only one. Place familiar objects with well-known sizes in the scene. Use fog/haze and a hue shift towards blue at further distances. But most importantly, do not give the user long view distances. Restrict it with blind corners instead. Binocular vision is only good for a few feet before the other depth cues become more important, and we are not yet capable of making a convincing experience without the binocular cue.
  • Object believability has more to do with textures and shading than polygon count. Save on polygon count in favor of more detailed textures and smooth shading.
  • Frame rate is important. I remember being perfectly happy with 30FPS on games 10 years ago. That's not going to cut it anymore. You have to hit 60FPS, at least. Oculus Rift is targeting 75FPS. I'm sure that is a good goal. Make sure you're designing your content and algorithms to maintain this benchmark.
  • Use lots of non-repetitive textures. Flat colors give nothing for your eyes to "catch" on to make the stereo image. The design of these viewer devices is such that the eyes must actually fight their natural focus angle to see things in the display correctly. It will be easier for the user if you make it as hard as possible to not focus on object surfaces. Repetitive textures are only slightly better than flat colors, as they provide a chance to focus at the wrong angle, yet still achieve what is known as the "wallpaper effect". And do not place smaller objects in any sort of pattern with regular spacing.
  • Support as many different application interactions as possible. If the user has a keyboard hooked up, let them use the keyboard. If they have a gamepad, let them use the gamepad. If the user wants to use the app on their desktop with a regular 2D display, let them. Do not presume to know how the user will interact with the application. This early in development, not everyone will have all of the same hardware. Even into the future, it will be unlikely that an app will be successfully monetizable with a user base solely centered on those who have all of the requisite hardware to have a full VR experience. [font=inherit]

    Be maximally accessible.

    [/font]
  • [font=inherit]

    Make the application useful. This seems like it shouldn't be said, but ask yourself what would happen if you were to rip out the "VR" aspect of the application and have people use it with traditional IO elements. Treat the VR aspect of it as tertiary. Presence by its very definition means forgetting about the artifice of the experience. If the experience is defined by its VR nature, then it is actively destroying presence by reveling in artifice.

    [/font]
  • [font=inherit]

    Much research needs to be done on user input especially for large amounts of text. Typing on a keyboard is still the gold standard of text entry, but tying the user to the keyboard does not make for the best experience, and reaquiring a spatial reference to the keyboard after putting the headset on and moving away from the keyboard is nearly impossible. Too often, I find myself reaching completely behind in the wrong direction.

    [/font]
  • [font=inherit]

    3D Audio is essential. We could mostly get away without audio in 2D application development, but in VR it is a significant component to sensing orientation and achieving presence. I believe it works by giving us a reference to fixed points in space that can always be sensed, even if they are not in view. Because you always hear the audio, you never lose the frame of reference.

    [/font]
    [color=rgb(77,79,81)][font=Helvetica]


    I may add to this later.

    [/font][/color]
Sign in to follow this  


5 Comments


Recommended Comments

Interesting notes. With regard to presence creating a feeling of isolation, that's not necessarily a bad thing. I agree that it's worth looking at ways that it can be managed (try very low, continuous white noise, especially using indistinct voices) for cases when you don't want it, but in cases where VR is simply being used as an interface, the feeling of being around other people may be distracting. For instance, if I'm visiting a virtual mall then I want to feel like there are other people around me, but if I'm using a virtual store interface I want to have that solo experience. It may be that this is also a matter of preference. Some people panic when they feel isolated, some people relax when they feel isolated. It may be that allowing the user to tune compensatory behavior is the best route for now.

Share this comment


Link to comment

Testing all of these assumptions out is why I'm building a framework for building VR applications in modern web browsers: https://www.github.com/capnmidnight/VR

 

It's been fascinating to try things that I thought would be awesome and see them not work out at all, while also trying things that I thought were not going to be a big deal but turned out to be hugely important.

 

For example, I thought speech commands were going to be a really big deal, but they turned out to be almost completely useless. It's really difficult to get a speech engine to recognize very short phrases, and longer phrases are cumbersome. It just doesn't make for anywhere approaching a good experience.

 

And then there are the weird things, like combining actions from one hand over the Leap Motion with mouse motions in the other hand. I think it's because the mouse allows me to make very broad movements and the Leap allows me to make small ones, and the two combined cover areas that the other doesn't do well.

 

Anyway, really fascinating stuff.

Share this comment


Link to comment

From my brief VR experience I noticed many commonalities (and some differences) with your experience. I did a rollercoaster (no, not THAT one). Having a seated position matching the real-world seated position was hugely helpful for combating simulator sickness (as compared to some of the awful FPS attempts). The frame rate was also hugely important. Graphics features that can be turned on/off were vastly context dependent. I managed to turn off shadow receiving for everything except the coaster cars without any drama. But turning those last shadows off drained it of life - the flickering superstructure shadows are what a rollercoaster is all about. I agree that distant vistas are curiously dull. I get that binocular vision doesn't help much in this case, but sight-seers everywhere love distant mountains and oceans. I found that particle effects were good to provide a spatial sense, e.g. floating dandelion seeds. I'd love to incorporate avatars more. Do you think fake avatars would work at all if the player lacks a kinect/hydra/leap? For example infer the position of arms, show them holding a controller, when a button is pressed show the fingers pressing the buttons? It may be super-weird. Such an experimental field right now!

Share this comment


Link to comment
Interesting thoughts, keep them updated if you want. What device do you use? (just out of curiosity)

Share this comment


Link to comment

@jefferytitan: I don't think the 75fps framerate target that Carmack talks about is strictly necessary, but it definitely needs to be at least 60fps. But what is surprising is that scenes don't need to be extremely detailed to achieve presence. I would say it's better to achieve high framerate than to achieve high polygon count, high res textures, etc. We used to play 3D games that were nothing more than billboarded sprites and were fine with it. Something like that would still work in VR, as long as the framerate and interactions were smooth. 

 

On avatars, I've been using a model of a fat, cartoon bear that I threw together with a simple running animation. The arms are not rigged to my Leap Motion yet, but you mostly can't see the arms. Having a body and feet to see, though, has been nice. I like it, at least. YMMV.

 

However, with the Leap, I've noticed that it absolutely *is* necessary to have a fully rigged hand model. Simple spheres for each of the knuckles works fine, but a single sphere for the entire hand is not good enough. However, I'm having some network performance issues with the Smartphone HMD sync in Psychologist.js, so I might have to degrade the visual in that case (i.e. transmit only the hand's location and not all of the individual finger joints).

 

But without direct, kinematic sync with body motion, it's best to show nothing. I've found any sort of user-oriented movement that was not triggered by the user's actual movement to be the most likely to cause motion sickness. This is where I refer to presence being a double-edged sword. If you haven't achieved presence, the user (or rather, me) still feels like they are looking at a screen and non-didactic movement isn't disorienting. When you get the user feeling like they are in the scene, they need to be in complete control of what is attributed to them.

 

So in short, if you do use some sort of fake hand model, I'd try to keep it abstract. Show the model of the tool held in the hand but do not show the hand itself, and keep movement to a minimum. 

 

 

@cozzie: I have the Samsung Galaxy Note 3 for doing the Smartphone-style HMDs, and a cardboard box that I built myself for holding it, with a strap I cribbed from a pair if head-wearable magnifying lenses (but not the lenses, I bought lenses specifically on Amazon.com). It's not Google Cardboard, because Google Cardboard was announced the day I ordered the lenses.

 

I also just acquired an Oculus Rift DK2. I spent a little time last night and most of this morning getting it to work with Psychologist.js (okay, I officially hate that name now, it's too long). It's up and running now, but requires specially development builds of Chromium or Firefox. 

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!