Leap Motion + Vive

I finally had a chance to sit down and get back into VR development today. I've got an idea for a project, and a direction to take it, but nothing concrete enough to make too many permanent statements about it. Here are a couple though:

  1. I want it to be fairly stationary, in a meditation-like environment
  2. I want sensory input to be fairly customizable (i.e. play your own music, see your own pictures). [No progress here after a day, but a central theme to incorporate]
  3. I want to try hand-only interaction (via a Leap Motion Controller [LMC])

In regards to point 1, I imagine it more as a "centered experience", but you can still move around (as opposed to a seated style game).

This is mostly about getting started with point 3. Here is a quick video demo of what I managed to work through today: Combining Vive + LMC Rigs, hand interactions with objects, visual selection with ray-casting, and hand gesture recognition.

Leap Motion Controller + Vive Test from Alteration x10 on Vimeo.

I found it easier to integrate the LMC rig into the SteamVR Camera Rig, than the other way around. Of the available hands in their Hands Module, I prefer the PolyHand3 model. I think it kind of reminds me of Hexadecimal from ReBoot. Toss it all in an Empty GameObject with an InteractionManager and you've got yourself a handy-dandy prefab.

Physical interaction is fairly simple with the Interaction Engine. By adding a couple of invisible "Physics Hands" to to HandPool and attaching an Interaction Behavior script to the GameObject you want interact with, you're basically good to go!

Basic Hand Gesture Detection is also fairly easy to get started with. There are a couple examples of how to do a thumbs up detection, as well as a away-facing palm out gesture. They are quite adaptable for complex detection, via a detection logic gate. I managed to get a palm-out detector for the left hand, and a palm-out / thumbs up detector for the right hand up without too much fuss.

I used the thumbs-up to reset an interactable cube to it's starting position, and the left-hand palm-out to freeze it in place once it was moving (via toggling it's kinematic property). I wanted to be able to move the cube back towards me with a right-hand palm-out gesture, but thought it would be cooler if it would only work while I was looking at the object. By adding a Physics.Raycast to the Vive camera GameObject, I could select the cube by looking at it (turn it green and toggle an boolean "is being looked at" property). While it's being looked at, the right-hand open-palm gesture will call MoveTowards, and bring it back!

Cool stuff!