• Advertisement
Sign in to follow this  

Unity Real time motion capture

This topic is 2900 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi everyone! My name's Cash and I'm part of a team working on a project at the University of Sussex here in the UK called eMove. It's a real-time motion capture system for use in gaming. We're far enough along now that we think it's a good time to start getting developer feedback from people like yourselves :) We've got a few demonstration videos which should help give a feel for what we're doing: Suit run down (bit of a commercial video, sorry about that!) Motorcycle tech demo (created using the Unity engine) UDK tech demo UDK development progress 1 (over the shoulder iphone video, please excuse the quality!) UDK development progress 2 Basically what we're doing at the moment is controlling the rig of our robot character using the data from the eMove suit. The suit we're using is an early prototype, so there are still a few software issues to iron out, but we're getting there and I think you'll get the idea. The first production suits are in the final design stages and will look a lot like the suit you see in-game during the first UDK tech demo. We first started the eMove project a few months back using the Panda3D engine, then moved to the Unity engine for a while, where we put together the motorcycle demo. We were over the moon when we discovered the Unreal 3 engine was now free for non-commercials though.. it's been great working with it :) We'de love your feedback, positive or negative. Ideas, suggestions, whatever :) Myself and my colleague Jake (who you see using the suit in most of the videos) are the guys working on the games side of things, so it's only a small team, and the more brain-picking we can do the better! Thanks for reading! - Cash & the eMove team

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by eMove

It's a real-time motion capture system

Real time is such misused term.

What is input latency of your project? What is the sampling resolution? What is sampling jitter?

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Real time is such misused term.

What is input latency of your project? What is the sampling resolution? What is sampling jitter?


The underlying software captures 120 samples per second, at <5ms latency (using a serial connection to the suit). This software is currently run separately to the game itself, in the background. Local TCP is used to communicate the mocap data between game and background software, this ups the latency a bit, to approximately 50ms.

Over the coming months, the background software will be collapsed into a DLL and used directly by the game, bringing the latency back down to <5ms give or take.

Basically game framerate is our primary bottleneck, and the last development video uses a spring driven ragdoll to follow the mocap. That currently suffers from high latency (100ms or more) due to the heavy physics, we're working hard to bring that down. Non-physical animation is quick though, and in most cases this is what's used.

Edit: Regarding sample jitter, with a properly calibrated suit (which is something we do manually at the moment, but will be automatic in future) this is very low. The suit's joint angles use high accuracy rotation sensors, and hasn't given us any problem with jitter (unless we drop the suit on the floor, it doesn't like that!)

Share this post


Link to post
Share on other sites
A little bulky looking rig, but in general looks good. What about non gaming apps? Have you prototyped anything with fine motor control? What is the degree of accuracy? And why no data gloves? Is this because of the gaming focus?

In general what are your thoughts on the different engines used?

I am not sure the complaint on 'real time'? He is not doing capturing and offline processing.. ie real time.

Share this post


Link to post
Share on other sites
Quote:
Original post by stake


I am not sure the complaint on 'real time'? He is not doing capturing and offline processing.. ie real time.


That is on-the-fly processing.

Real-time means that each sample will be provided in 5 ms and never more than 5 ms. Imagine controlling a scalpel with such a tool, or a crane. And you get lag while making a precision move, causing to cut an artery, or swing the load hanging from the crane for 150ms longer, causing positive feedback to create a pendulum.

Share this post


Link to post
Share on other sites
Quote:
Original post by stake
A little bulky looking rig, but in general looks good. What about non gaming apps? Have you prototyped anything with fine motor control? What is the degree of accuracy? And why no data gloves? Is this because of the gaming focus?

In general what are your thoughts on the different engines used?

I am not sure the complaint on 'real time'? He is not doing capturing and offline processing.. ie real time.


The prototype suit is indeed bulky, but the production suit (which we'll be moving to in the next 2-3 months) should be much lighter and more pleasant to use.

There are certainly many non-gaming application for the suit, but gaming is something we're focusing on for this project.

The joysticks on the suit are designed to be interchangable on the production suit, so there is potential to swap them out for data gloves or other controllers in future. We use joysticks because they offer a nice solid grip and work well when controlling player movement (with 2 analog sticks) as well as have triggers for firing weapons and what not.

In terms of engines, I found Unity very nice to use. It has an extremely rapid asset pipeline, a choice of scripting languages (C# and javascript) and a very intuitive way of using those scripts to control game objects. It is however lacking in the really high end side of things: advanced shader/post-processing control and low-level animation (which was what really hurt us).

UDK (the free Unreal 3 engine) has a steep learning curve, what with it's purely UnrealScript codebase, and a at time painfully slow asset pipeline. But it's in an incredible engine, and can't recommend it enough. The animation system in particular is superb, and offers very low level bone control (which has made our lives much much easier). The Kismet visual scripting language also makes it really easy to expose your lower level code to quick changes by non-coder types in the editor.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
That is on-the-fly processing.

Real-time means that each sample will be provided in 5 ms and never more than 5 ms. Imagine controlling a scalpel with such a tool, or a crane. And you get lag while making a precision move, causing to cut an artery, or swing the load hanging from the crane for 150ms longer, causing positive feedback to create a pendulum.


There is very minimal cleanup or post-processing done to the mocap data as it comes in, since it's in effect just a pile of joint angles. In the simplest case, you can literally just dump these angles onto your character rig and you're away.

Any latency in the system is purely at a communication or physics level.

Share this post


Link to post
Share on other sites
50 ms is quite high latency! Are you running these on loopback or across a LAN? You should be able to get much lower latency across a LAN. Are you using VRPN or straight up TCP? Maybe is you could easily switch to UDP and you could bring the latency down. Depends if you want to try that out.

Also with a high amount of jitter the sampling rate goes way down unless you plan on doing a lot of buffering.

Share this post


Link to post
Share on other sites
Quote:
Original post by stake
50 ms is quite high latency! Are you running these on loopback or across a LAN? You should be able to get much lower latency across a LAN. Are you using VRPN or straight up TCP? Maybe is you could easily switch to UDP and you could bring the latency down. Depends if you want to try that out.

Also with a high amount of jitter the sampling rate goes way down unless you plan on doing a lot of buffering.


We're just doing vanilla TCP localhost comms, no network involved.

50ms is a conservative figure, and the main reason it's not much lower is UnrealScript's sockets aren't that hot (Epic have actually recommended against their use for any real-time applications).

We could pull this TCP code out into a compiled library to speed it right up, but this need for TCP communication is only a temporary measure until everything lives in a library and the serial comms to the suit is done directly by the game process. At this stage our latency becomes negligible.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
  • Advertisement
  • Popular Now

  • Advertisement
  • Similar Content

    • By RoKabium Games
      Custom coffee mugs have arrived... More caffeine!
      Have a great weekend everyone! 
      #gamedev #indiedev #sama #caffeine
    • By Atwo Studios
       
      Hey guys,

      Anthony here from Atwo Studios bringing you some new updates for the new year!
      In this video I go over our game ROY, the new games and some general updates to the company!

      If you have not checked out ROY feel free to give it a try! Many people have said they enjoyed the game thus far!
      ROY: https://goo.gl/o6JJ5P
       
    • By Affgoo
      https://play.google.com/store/apps/details?id=com.NE.Alien
      still a lot of work to do, but its pretty stable  please let me know what you think <3
      Atlas Sentry is a game of destroy everything. Using your turret, simply swivel and shoot your way to victory, upgrading your weapons to unleash destruction on the variety of spaceships. The bigger your combo’s the more score you get! Earn silver as you play and then purchase new weapons and abilities to better deal with your enemy. Different enemies use different tactics and weapons, work out your own priorities in their destruction order. 

      Features: 
      **2 different game modes 
      **A level select mode with 20 difficult levels including a final boss, can you defeat it? **Arcade mode of endless destruction, how long will you last? 
      **High scores to compete against others, see who can take the top spot. 
       
    • By Chamferbox
      Chamferbox, a mini game asset store has just opened with some nice game assets, 
      Here you can find a free greek statue asset 

      Also check their dragon, zombie dragon and scorpion monster out:



      They're running the Grand Opening Sale, it's 30% off for all items, but for gamedev member, you can use this coupon code:
      GRANDOPEN
      to get 50% off prices What are you waiting for, go to
      http://chamferbox.com
      and get those models now!

      View full story
    • By Dafu
      FES Retro Game Framework is now available on the Unity Asset Store for your kind consideration!
      FES was born when I set out to start a retro pixel game project. I was looking around for an engine to try next. I tried a number of things, from GameMaker, to Fantasy Consoles, to MonoGame and Godot and then ended up back at Unity. Unity is just unbeatable in it's cross-platform support, and ease of deployment, but it sure as heck gets in the way of proper retro pixel games!
      So I poured over the Unity pipeline and found the lowest levels I could tie into and bring up a new retro game engine inside of Unity, but with a completely different source-code-only, classic game-loop retro blitting and bleeping API. Months of polishing and tweaking later I ended up with FES.
      Some FES features:
      Pixel perfect rendering RGB and Indexed color mode, with palette swapping support Primitive shape rendering, lines, rectangles, ellipses, pixels Multi-layered tilemaps with TMX file support Offscreen rendering Text rendering, with text alignment, overflow settings, and custom pixel font support Clipping Sound and Music APIs Simplified Input handling Wide pixel support (think Atari 2600) Post processing and transition effects, such as scanlines, screen wipes, screen shake, fade, pixelate and more Deploy to all Unity supported platforms I've put in lots of hours into a very detail documentation, you can flip through it here to get an better glimpse at the features and general overview: http://www.pixeltrollgames.com/fes/docs/index.html
      FES is carefully designed and well optimized (see live stress test demo below). Internally it uses batching, it chunks tilemaps, is careful about memory allocations, and tries to be smart about any heavy operations.
      Please have a quick look at the screenshots and live demos below and let me know what you think! I'd love to hear some opinions, feedback and questions!
      I hope I've tickled your retro feels!



      More images at: https://imgur.com/a/LFMAc
      Live demo feature reel: https://simmer.io/@Dafu/fes
      Live blitting stress test: https://simmer.io/@Dafu/fes-drawstress
      Unity Asset Store: https://www.assetstore.unity3d.com/#!/content/102064

      View full story
  • Advertisement