ever seen geometry images?
Author
I don't know what I was looking at, but I heard about Geometry Images the other day. The concept is that you compress a 3D mesh into a 2D image through cut paths on the model. There are some limitations stated that I don't have the knowledge to interpret, but it looks interesting.
It seems that smooth meshes compress pretty well using wavelet compression of the image. The geometry images are all 2^n + 1 (257x257, 129x129); I would be grateful if somebody could explain why.
Has anyone ever seen these outside an academic setting (or even outside the paper)? What kinds of things do you think they could be useful, especially related to 3D game graphics?
Link to the paper [PDF]
The reason that they compress so well is that the deltas of properties for adjacent vertices are usually smooth as compared to those of vertices situated in opposite side of the mesh.
Consider, for example, that the distance between two adjacent vertices is - with considerable probability - very small as compared to the maximum distance of any two arbitrary vertices. Since topological adjacency is converted to adjacency in the texture, the less distance the vertices are apart the less their rate of change between their adjacent texel representations is.
The smoother the signal (the resulting texture) is, the less wavelet frequencies are needed to represent that signal.
The technique in question introduces geometric distortion, magnitude of which is determined by the ratio of the number of original vertices and the generated parameter grid (geometry texture) resolution. This distortion, combined with the fact that sampling from a texture in a vertex processing stage used to be quite expensive and that legacy graphics hardware cannot create new geometry based on texture data makes this a bit unfeasible now.
D3D10 could provide means of hardware implementation for this technique, as two of the cons mentioned above could be eliminated (generating new geometry based on texture fetch, and being fast about it).
[Edited by - Nik02 on July 11, 2007 6:40:43 AM]
Consider, for example, that the distance between two adjacent vertices is - with considerable probability - very small as compared to the maximum distance of any two arbitrary vertices. Since topological adjacency is converted to adjacency in the texture, the less distance the vertices are apart the less their rate of change between their adjacent texel representations is.
The smoother the signal (the resulting texture) is, the less wavelet frequencies are needed to represent that signal.
The technique in question introduces geometric distortion, magnitude of which is determined by the ratio of the number of original vertices and the generated parameter grid (geometry texture) resolution. This distortion, combined with the fact that sampling from a texture in a vertex processing stage used to be quite expensive and that legacy graphics hardware cannot create new geometry based on texture data makes this a bit unfeasible now.
D3D10 could provide means of hardware implementation for this technique, as two of the cons mentioned above could be eliminated (generating new geometry based on texture fetch, and being fast about it).
[Edited by - Nik02 on July 11, 2007 6:40:43 AM]
Geometry Images have been a hot topic in the computer graphics research community over the past few years. For those who are interested, I have compiled a listing of academic articles on Geometry Images on my website.
The goal of creating a Geometry Image (GIM) is two fold. Firstly, we are trying to convert a polygon model with irregular connectivity (variable number of edges connected to vertices), to one with regular connectivity (constant number of connected edges for each vertex). These have various advantages for surface processing over irregular meshes. Secondly, we are trying to represent the model as an image so we can apply well-known image processing techniques to the surface (including compression).
The advantages of GIM reach much further than just compression:
- The regularity of vertices both on the mesh and in memory have advantages for vertex caching in graphics hardware through straight-forward render sequence selection (triangle strips)
- The 2D grid allows the application of mesh level-of-detail using simple image down-sampling (i.e. geometric mip-mapping)
- Texture co-ordinates are unnecessary as the geometry and texture samples share common co-ordinates in texture space (they have the same 2D array indices)
- The storage of geometry as a texture opens up a broad range of possibilities for performing surface processing in pixel shaders, as the 2D array naturally maps to texture memory, and any nearby vertices on the surface can be accessed at neighboring texture addresses
On the other hand, they are not as flexible as irregular polygon meshes. To start with, the surface represented by the Geometry Image (GIM) must be a closed manifold. e.g Plant models, such as those produced by Xfrog can not be turned into a GIM with much success. You probably would not use them to represent simple models, such as a plain house, because this can be done with very few polygons. Models with high topology introduce high distortion into the GIM creation process, briefly mentioned by Nik02. The advantages provided by GIMs may be negligible when we talk about the small models that are used in games etc. Most of the models "played with" in graphics research are N00,000 - N,000,000+ polygons.
The process used to create a GIM is called parameterization, and is widely used in computer graphics to define texture mapping, surface morphing, mesh editing, and many other things. Its is a mathematical concept that maps a function's range to a specific domain, or more simply, it maps one surface onto another. For Geometry Images, we are parameterizing the 3D surface of the model to a square on the 2D plane. The cutting you talked about is an integral part of this process. Imagine trying to flatten the surface of an orange onto a table, the only way we can do it is to first cut/tear the orange peel so it can be flattened.
The novelty in the "Geometry Image" SIGGRAPH paper by Gu et. al. is that the parameterization is forced to fill the whole square texture space and be continuous across the image boundaries. That is, there is a duplication of texture samples along the boundary so that when we use them as mesh vertices the mesh does not have cracks in it. The observation that they could then use that texture as a regular remeshing of the surface was the idea that made it widely applicable.
There are many academic papers that feature GIMs, but I dont think they are being used much outside the research community at the moment. Nik02 said that DX10 might have support for them, but you can take advantage of GIMs with current hardware also if you know how.
The author of the website you got that paper from psykr is a very well-known researcher in computer graphics and has invented many things that actually get used in everyday computer graphics. You can safely assume that if he is interested in a topic, it will eventually become available to the computing public.
Hope this was informative.
Luke Domanski.
[Edited by - Luke Domanski on July 19, 2007 7:30:34 PM]
The goal of creating a Geometry Image (GIM) is two fold. Firstly, we are trying to convert a polygon model with irregular connectivity (variable number of edges connected to vertices), to one with regular connectivity (constant number of connected edges for each vertex). These have various advantages for surface processing over irregular meshes. Secondly, we are trying to represent the model as an image so we can apply well-known image processing techniques to the surface (including compression).
The advantages of GIM reach much further than just compression:
- The regularity of vertices both on the mesh and in memory have advantages for vertex caching in graphics hardware through straight-forward render sequence selection (triangle strips)
- The 2D grid allows the application of mesh level-of-detail using simple image down-sampling (i.e. geometric mip-mapping)
- Texture co-ordinates are unnecessary as the geometry and texture samples share common co-ordinates in texture space (they have the same 2D array indices)
- The storage of geometry as a texture opens up a broad range of possibilities for performing surface processing in pixel shaders, as the 2D array naturally maps to texture memory, and any nearby vertices on the surface can be accessed at neighboring texture addresses
On the other hand, they are not as flexible as irregular polygon meshes. To start with, the surface represented by the Geometry Image (GIM) must be a closed manifold. e.g Plant models, such as those produced by Xfrog can not be turned into a GIM with much success. You probably would not use them to represent simple models, such as a plain house, because this can be done with very few polygons. Models with high topology introduce high distortion into the GIM creation process, briefly mentioned by Nik02. The advantages provided by GIMs may be negligible when we talk about the small models that are used in games etc. Most of the models "played with" in graphics research are N00,000 - N,000,000+ polygons.
The process used to create a GIM is called parameterization, and is widely used in computer graphics to define texture mapping, surface morphing, mesh editing, and many other things. Its is a mathematical concept that maps a function's range to a specific domain, or more simply, it maps one surface onto another. For Geometry Images, we are parameterizing the 3D surface of the model to a square on the 2D plane. The cutting you talked about is an integral part of this process. Imagine trying to flatten the surface of an orange onto a table, the only way we can do it is to first cut/tear the orange peel so it can be flattened.
The novelty in the "Geometry Image" SIGGRAPH paper by Gu et. al. is that the parameterization is forced to fill the whole square texture space and be continuous across the image boundaries. That is, there is a duplication of texture samples along the boundary so that when we use them as mesh vertices the mesh does not have cracks in it. The observation that they could then use that texture as a regular remeshing of the surface was the idea that made it widely applicable.
There are many academic papers that feature GIMs, but I dont think they are being used much outside the research community at the moment. Nik02 said that DX10 might have support for them, but you can take advantage of GIMs with current hardware also if you know how.
The author of the website you got that paper from psykr is a very well-known researcher in computer graphics and has invented many things that actually get used in everyday computer graphics. You can safely assume that if he is interested in a topic, it will eventually become available to the computing public.
Hope this was informative.
Luke Domanski.
[Edited by - Luke Domanski on July 19, 2007 7:30:34 PM]
Any hardware that can sample a texture in the vertex shader could be used to implement the rendering phase of the geometry texture.
However, you can't (or it is extremely difficult to) effectively perform the cutting or parametrization in graphics hardware without taking advantage of the geometry shader - and the fact that the GS is able to refer to face adjacency info. Not to say that this is easy to implement, but it is possible and feasible.
The last row and column likely represent the correction vectors to compensate for the compression distortion in the edges of the parameter space. The last single pixel could store a bias value; for example, the center or the scale of the mesh data.
In graphics hardware-based implementations, it would be wise to use power of two textures and store the geometry data in 2^n-1 dimensions with the correction vectors in 2^n row and column.
However, you can't (or it is extremely difficult to) effectively perform the cutting or parametrization in graphics hardware without taking advantage of the geometry shader - and the fact that the GS is able to refer to face adjacency info. Not to say that this is easy to implement, but it is possible and feasible.
Quote:
The geometry images are all 2^n + 1 (257x257, 129x129); I would be grateful if somebody could explain why.
The last row and column likely represent the correction vectors to compensate for the compression distortion in the edges of the parameter space. The last single pixel could store a bias value; for example, the center or the scale of the mesh data.
In graphics hardware-based implementations, it would be wise to use power of two textures and store the geometry data in 2^n-1 dimensions with the correction vectors in 2^n row and column.
Hello, I'm using Geometry Images as a mean to produce a hardware accelerated real-time Ray Tracer... actually this one: http://graphics.cs.uiuc.edu/geomrt/
I'm using CGAL for the parameterization step; however I have some problems with my GI (the edges seem to be too sharp) and I believe it's because of my triangle rasterization routine which re-orders vertices!
Anyway, this could be taken to the consoles world (let's say to the Xbox, taking advantage of what Niko said about DX10). It could be interesting for ray traced games.
I'm using CGAL for the parameterization step; however I have some problems with my GI (the edges seem to be too sharp) and I believe it's because of my triangle rasterization routine which re-orders vertices!
Anyway, this could be taken to the consoles world (let's say to the Xbox, taking advantage of what Niko said about DX10). It could be interesting for ray traced games.
Quote:Original post by VerMan
Hello, I'm using Geometry Images as a mean to produce a hardware accelerated real-time Ray Tracer... actually this one: http://graphics.cs.uiuc.edu/geomrt/
I'm using CGAL for the parameterization step; however I have some problems with my GI (the edges seem to be too sharp) and I believe it's because of my triangle rasterization routine which re-orders vertices!
Anyway, this could be taken to the consoles world (let's say to the Xbox, taking advantage of what Niko said about DX10). It could be interesting for ray traced games.
Interesting indeed. I thought about implementing a geometry texture raytracer in D3D10 just recently, but I didn't get to the hierarchy optimization part yet.
I'm going to study your technique and see what I can make of it :)
Quote:Original post by Nik02
However, you can't (or it is extremely difficult to) effectively perform the cutting or parametrization in graphics hardware without taking advantage of the geometry shader - and the fact that the GS is able to refer to face adjacency info. Not to say that this is easy to implement, but it is possible and feasible.
You are correct. The advantage of GIMs is thier application to rendering and surface processing, which is what I was refering to. The cutting and parameterization is not something you would be doing on the graphics hardware itself, as far as I know. You might be able to come up with some General Purpose GPU (GPGPU) style algorithm to achieve it, but it is probably better to do it as a pre-process on the CPU using a cutting algorithm + sparse linear system solver for parameterization. The CGAL library suggested by VerMan would be the best freely available library for doing this. It does everything except decide on the actual cut, you will need to perform this task yourself then pass the cut to the parameterization library as an edge list.
The cut algorithm described in the Geometry Image paper is fairly straight-forward, but requires re-parameterisation over and over to get a low distortion cut. Maybe instead use the cutting technique based on morse theory http://graphics.cs.uiuc.edu/~jch/papers/morsecut.pdf
Quote:Quote:
The geometry images are all 2^n + 1 (257x257, 129x129); I would be grateful if somebody could explain why.
The last row and column likely represent the correction vectors to compensate for the compression distortion in the edges of the parameter space. The last single pixel could store a bias value; for example, the center or the scale of the mesh data.
In graphics hardware-based implementations, it would be wise to use power of two textures and store the geometry data in 2^n-1 dimensions with the correction vectors in 2^n row and column.
This might be the reason in the case of lossy compression. But also, the other reason cited in the paper is to achieve crack-free image downsampling for level-of-detail using mip-mapping. To cut a long story short, each boundary sample of the original GIM represents a vertex on the mesh cut, and it has a duplicate partner on the boundary (so that the mesh is crack-free). The downsampled GIM must also exhibit the same properties. So when downsampling, we use a 3x3 avaraging box filter on matching boundary samples. We make the boundary sample the center point of the box and consider neighbouring samples on both sides of the cut when averaging. This creates a boundary in the downsampled image with matching samples on the boundary. Of course we use the same filter on interior samples as well, but the cut is not an issue there. This is not possible with a 2x2 filter. The 3x3 filter scheme requires a 2^n+1 image to ensure that matching boundary samples become the centers of these filters.
- Luke
even if you would have solved the problem of storing a mesh in a 2D texture, you would not be able to use a DXT compression for this texture, because DXT is supposed to be used with colors and introduce a high error level, that would make it impossible to use this stuff in a game.
Quote:Original post by wolf
even if you would have solved the problem of storing a mesh in a 2D texture, you would not be able to use a DXT compression for this texture, because DXT is supposed to be used with colors and introduce a high error level, that would make it impossible to use this stuff in a game.
I dunno about that... even if DXTC is inapplicable (which it may be, although as 3Dc has shown, it can often be adapted), it's an "advantage", but certainly not a requirement. There are tons of uncompressed textures being processed in a typical game frame and the GPU handles them just fine. Only if bandwidth or total memory usage is a bottleneck is the compression important and as has been noted earlier in the thread, there are efficient ways to compress geometry images other than typical DXTC that would produce the same advantages (perhaps even more-so).
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
