Jump to content
  • Advertisement
Sign in to follow this  
Portugal Stew

Voxel video

This topic is 3294 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm toying with a mystery game concept and I have been lately toying with the idea of using live actors like were used in Myst. What I want is beauty over dynamics (infact, with my concept dynamic characters would detract from the vision), but I want them to work in a 3D polygonal environment. While I'm still a freshmen Computer Science major with plenty of time to contemplate this, do any of you know if it would be possible to create voxel models out of videos? And assuming I had the technology necessary to build voxel models of every frame of the video viewable from every angle (or at least every necessary angle), would it be possible to play the models like a video? I did some light research (i.e. checked Wikipedia) and I understand that voxel rendering is very computationally expensive, but my understanding is that this is mainly because voxel renders use various shading techniques and smoothing, neither of which would be particularly necessary with video footage which is by nature pre-rendered. I am actually fairly confident that even relatively high-resolution voxel videos can be done on normal gamer hardware, but I'm a bit worried about a few potential problems I foresee. One, motion blurs, an nearly unavoidable artifact of film, could make producing voxel models very tricky (although there are ways to limit them to be nearly nonexistant). Two, I am concerned that placing voxel videos in a 3D environment, either voxel or polygonal, especially a dynamic environment, would be too computationally expensive and that it would be difficult to run voxel and polygonal graphics together in parallel. Does anyone have any insights on this? Has similar technology been used before with any success? Would I ultimately be better off doing something less interesting with the BS degree I'm working on and just use motion capture?

Share this post


Link to post
Share on other sites
Advertisement
Forget playing the video. You suggested reconstructing a 3D image out of the 2D image? 3D camera scanners that construct the 3D object out of several 2D shots exist, but they in itself are pretty big undertakings. And they also usually superimpose some pattern over the object so the algorithms can recognize the shape.

As far as I know, nothing like it has ever been done with several cameras. Forget the game, just being able to record 3D video would be huge, massive even.

Share this post


Link to post
Share on other sites
If you somehow had volumetric data, I am certain you could come up with a way to display it at interactive framerates. But as Radan said, getting the data to begin with would be at the very cutting edge of what we can do with markerless motion capture (as far as I know).

There are some papers on the subject. It's not my field, but a quick google turns up e.g., this (which seems really very simple). Since the models demonstrated are incredibly simple (a box, and a coffee cup with a relatively thin handle), my guess is that this particular method runs into problems with more complicated geometry that creates more self-occlusions. It would seem that you'd need to exploit temporal coherence somehow (e.g. by estimating velocities at voxels under a rigid-body constraint) to get this, and they don't do that at all.

Personally, my gut impulse -- especially if I didn't need my models to interact with the world -- would be to use an image-space approach if I insisted on attempting something like this. I.e., I'd avoid trying to construct an intermediate 3d representation from video and instead just display data from an appropriate camera based on the direction from which the player is looking at the model -- perhaps with some sort of (homography-based?) interpolation in-between; think Quicktime VR. This would require that cameras be calibrated well and that they have the same focal lengths as the perspective transformation used in the game; this would be a finicky thing to set up but probably possible.

Note that the equipment cost for anything like this, however -- lots of cameras set up on tripods surrounding an actor -- would be in the high tens of thousands of dollars at the absolute minimum, and probably in the hundreds. (That's what a "real" mocap setup costs, at least).

[Edited by - Emergent on September 14, 2009 1:30:03 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Radan
Forget the game, just being able to record 3D video would be huge, massive even.
Not really - the technology has existed for several years. As Emergent suggested, multiple cameras surrounding the subject are necessary, but he missed one important part of the puzzle - depth cameras.

A normal camera records an image, but you then have to attempt an image-based reconstruction of 3D form through edge detection of multiple offset images (much as the human eye does). However, several groups have developed methods to reliably reconstruct depth from a standard camera as well (see the MIT solution). The real ace here is a (relatively) cheap commercial solution (see a brief description here).

Given multiple cameras around a subject recording depth as well as colour, voxel reconstruction is a fairly trivial software issue. In short, the technical legwork has been done long ago [wink]

Share this post


Link to post
Share on other sites
I have to disagree, I was researching the subject of Virtual Sets, but in the process I saw some advanced sets with plugins, basically a set means a virtual studio where everything is 3D and interactive only the actor is a video.
Now the interesting part was aplugin for a sports studiom where they showed
a normal soccer match that can be paused at anytime (the data is a normal video), and the image is processed and a 3D world is constructed.
Here is the video in question, this is an Ad. for the program
but the part you want is when they start talking about Sports Studio (should be somewhere in the middle).
http://www.orad.tv/upload/downloads/IBC07.wmv

Share this post


Link to post
Share on other sites
Wow swiftcoder; that's very, very cool. I wonder how reliable all this is. It would be a real boon to robotics to have reliable "cameras with z-buffers."

Share this post


Link to post
Share on other sites
I would like to correct myself somewhat to say that what I have in mind is not necessarily to create a model that can be viewed in accurate detail from every angle, but can be viewed with 360 degrees of rotation at eye level. This no doubt has much simpler solutions, but the concept of generating full-scale models intrigues me.

Assuming 3D video can be captured, does anyone have any idea how it can be played back? How would it be compressed? I'm curious to pre-render CGI scenes as voxel videos, which should open some interesting possibilities (even if polygonal scenes are significantly cheaper to use, voxel videos would be susceptible to traditional video filters with slight modifications, amongst other things).

Much of this is simply novelty for the sake of novelty, but a lot of it is just my fascination with the possibilities. Theoretically, if the technology is perfected, post-production visual effects could be revolutionized, where by just using one or two extra cameras green-screen footage can be edited to correct aperture length, camera angle, and more.

Share this post


Link to post
Share on other sites
Quote:
Original post by Emergent
Wow swiftcoder; that's very, very cool. I wonder how reliable all this is. It would be a real boon to robotics to have reliable "cameras with z-buffers."
Also worth mentioning that the ZCam appears to be the camera used in Microsoft's Project Natal (Microsoft acquired the ZCam technology shortly before). I assume the depth sensing is the key to their (very good) free-form interaction.

Share this post


Link to post
Share on other sites
Quote:
Original post by Portugal Stew
I would like to correct myself somewhat to say that what I have in mind is not necessarily to create a model that can be viewed in accurate detail from every angle, but can be viewed with 360 degrees of rotation at eye level.

Are you talking about rotating the view point without moving the camera, or about orbiting the camera around the subject?

Share this post


Link to post
Share on other sites
Quote:
Original post by Emergent
Note that the equipment cost for anything like this, however -- lots of cameras set up on tripods surrounding an actor -- would be in the high tens of thousands of dollars at the absolute minimum, and probably in the hundreds.


Nope...

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!