Hi gamedev.net people, I wasn't thinking along the line of games, but a google search led me to a forum here, and it struck me that you folk might give me some good feedback / criticism. Recently I've written up some work on a "top-down" approach to volume data storage and direct volume rendering (like pyramid encoding or mipmapping) using an upsampling / downsampling kernel borrowed from 2d image processing that is lightning fast but produces amazing results. The write-up is here: http://assassinationscience.com/johncostella/volume 3d imaging people are enthusiastic and would like me to implement it in various volume rendering systems. I'm currently trying to figure out where to best expend my limited time. (This is a personal hobby research project, not my day job.) Comments, criticisms, etc. gratefully received. Thanks in advance, John Costella

Putting aside volume rendering for a moment, this kernel sounds like a superior way to generate mipmaps. I'll have to compare the results to the traditional box filter downsampling.

Wow, the idea to store volume data as a set of diffs against upsamples of a low-res representation is brilliant. I'm not very familiar with volume rendering, is that a common technique, or was that your idea? Sure, the "magic" kernel you're using makes the diffs small, but just that idea alone is great.

Hi venzon,

I only found out in recent days from Steve Pieper at Harvard that the idea was originally proposed as "Pyramid Encoding" way back in 1983: see

http://web.mit.edu/persci/people/adelson/pub_pdfs/pyramid83.pdf
http://web.mit.edu/persci/people/adelson/pub_pdfs/RCA84.pdf

(I talked to Professor Adelson by email yesterday; he's a cool guy.)

I didn't know this when I came to the same methods in recent years for 2d images. If you look at my page at

http://assassinationscience.com/johncostella/jpegclear

you'll see that their 1983 and 1984 papers cover everything -- the only difference being that they used a five-tap filter which they tuned to give the best results, not the "magic kernel". (My previous life as a theoretical physicist taught me that you only know that you've discovered something worthwhile when you find out that someone else already discovered it decades earlier ... :)

Dr Pieper suggested that I call the data storage format I recommend a "magic pyramid". I'm working on the libraries to convert conventional "slice by slice" volume data (typically medical imaging data) into "magic pyramid" format (including the easier-to-digest diced cubes I propose for higher-resolution scales). I'll have a C# version done in a few more days. The most interest seems to be to get it into VTK, so I might chew that challenge off next (not trivial).

As for mipmaps, the magic kernel should give equivalently good results for the least possible machine cycles (a handful of integer additions and bit-shifts is all that is needed). My original implementation did a 2d image in a single pass, rather than an x- and then a y-downsampling, which can shave off further machine cycles if necessary. But I "factorized" it out into x and y last week so that I could encapsulate the filter in the Magic class and then trivially generalize it to 3d.

On the topic of 2d images, the upsampling filter also provides what I would argue is an optimal way of rendering a low-resolution approximation at higher resolution. If you look at the images on that page above (or try it yourself) you'll see that you can "zoom in" as much as you want and the upsampled image "looks the same" as it did when it was only a few pixels wide. (Ignore the JPEG noise!) (The magic kernel gets you to the closest factor of 2; any other method -- even bilinear -- allows adjustment to the precise factor needed; I have this built into the Jpc class in the code.)

Of course, the tricky question is how much of this is already built into the GPU, so to some extent these are methods for the good folk at NVIDIA etc. to look at, rather than developers using existing hardware.

The final comment I'll make is that it might not be worth the extra effort to store the diffs (called "Laplacian Encoding" in the original Pyramid Encoding work) instead of simply storing the downsamples themselves (as with mipmaps). In 2d the mipmaps only take up an extra 1/3 of the storage required for the full-resolution image. In 3d the factor is a measly 1/7. (But if lossy compression were used, then it would most definitely be worthwhile compressing the diffs rather than the downsamples.)

John