17 replies to this topic

[mrbastard]

I'm working on a demo which computes the mandelbrot set in an HLSL pixel shader. I've been getting some great effects by using an 'orbit trap' and using the resulting values as texture coordinates.

The problem is that where the texture coordinates change very quickly between pixels I get horrid artifacts. See the thin, broken purple lines in the image below.


Ignore the texture magnification blur in the middle bottom of the image - it's just a low res texture!

Here's an image of the UVs, ( u in red, v in green, blue=0 ) - hopefully that points out where the artifacts would occur, even if you can't immediately see them in the first image



Now, some of you are probably already reaching for a link to docs on sampleGrad. I've tried playing with it, but so far have only managed to bias the problem one way or the other - one side or other of the round features is still 'bad'. I may not fully understand what I'm doing though, and I'd be very grateful for an explanation if anyone thinks this is the right route to go down.

Another possible solution which I'm working on is to render the UVs to a texture and smooth them out with a box filter. I'll lose detail in the UV map, but I don't think that will make much difference to the final image.

I've also tried just painting over the artefacts - as they occur at the very edge of the 'trapping circle' I guessed they would fall between ~0.99 and 1.0. It turns out that the artefacts are actually on pixels next to (but sufficiently different in UV value from) the pixels with values 0.9...1.0. This is what led me to believe I couldn't solve this with knowledge of only one pixel - I need to use ddx/ddy/sampleGrad or another pass.

Any suggestions?

[MJP]

It looks like it's just a filtering artifact. If you sample with linear filtering then the texture units will end fetching adjacent texels and using them in the filtering, which can be bad for cases like yours where the UV's have discontinuities. However in your case it appears as though you only have artifacts when sampling at the very edges of your texture (which makes sense), which means you should be able to use CLAMP or BORDER filtering mode to avoid bringing in texels from an unrelated part of the texture.

[Hodgman]

When texture coordinates change very quickly from one pixel to the next, the hardware will assume that it needs to sample a low-resolution mip-map. The quickest way to hack around this behaviour is to disable mip-mapping, either on your texture (by not supplying mips), by setting max-lod to 0 on your sampler, or using SampleLevel to force it to use LOD 0.

The same artefacts occur in deferred shading when using projected light textures, or "cookies"/"gobos", but I can't remember the common work-arounds off the top of my head, besides disabling mipmapping altogether as above...

That effect looks awesome BTW, especially with the Alex Grey artwork.

[mrbastard]

Thanks MJP.

I'd already wondered if it was the texture addressing mode, but to be sure I tested again. Both CLAMP and BORDER give much the same result. Here's BORDER:



Just in case though, here's my setup code:

sampDesc.Filter = D3D11_FILTER_MIN_MAG_MIP_LINEAR;
sampDesc.AddressU = D3D11_TEXTURE_ADDRESS_BORDER;
sampDesc.AddressV = D3D11_TEXTURE_ADDRESS_BORDER;
sampDesc.AddressW = D3D11_TEXTURE_ADDRESS_BORDER;
sampDesc.ComparisonFunc = D3D11_COMPARISON_NEVER;
sampDesc.MinLOD = 0; 
sampDesc.MaxLOD = D3D11_FLOAT32_MAX;

There are other parts of the image, outside of the area in the shots, where the UVs become > 1.0 and wrap without artifacts. The artifacts only appear when neighbouring pixels have wildly different UVs. I've tried detecting these big shifts using ddx and ddy, but so far I either miss a lot of the artifacts or get a lot of false positives.

[mrbastard]

When texture coordinates change very quickly from one pixel to the next, the hardware will assume that it needs to sample a low-resolution mip-map. The quickest way to hack around this behaviour is to either disable mip-mapping on your texture, or use SampleLevel to force it to use LOD 0.

Brilliant, thanks. SampleLevel 0 clears it up a treat. I really do need the mipmapping though - lots of magnification and minification zooming around a fractal. See the minification sparklies:




I'll see if I can get any further using ddx and ddy to select pixels where I want to set the mip level. I started reading a few articles on how this kind of stuff is handled in deferred rendering when I first found out about SampleGrad - I'll have to do some more!

That effect looks awesome BTW, especially with the Alex Grey artwork.

Thanks! It looks even better in motion. Funnily enough, when I went looking for textures to try out with fractal trapping, Alex Grey was my first thought. Well, maybe after M.C. Escher I'd like to have a go at reproducing some of Grey's designs procedurally - maybe the flame and ring in the images above. Hopefully he'd approve!

I'm not sure whether to leave the images and music in the finished demo and just stick it on youtube, or to attempt something of my own so I can let people download the whole thing and interact with it.

[MJP]

The same artefacts occur in deferred shading when using projected light textures, or "cookies"/"gobos", but I can't remember the common work-arounds off the top of my head, besides disabling mipmapping altogether as above...

The usual way to do it is to just derive a mip level using some metric other than UV gradients (for instance, something based on the depth of the pixel).

[LorenzoGatti]

I don't know if you can do it easily and cheaply, but maybe you can examine the texture coordinates of adjacent pixels and choose a mip level according to how close or far to each other they are.
If you compute the maximum difference between the U or V coordinate of your fragment and the corresponding coordinate of each adjacent pixel, it translates directly to the ideal texture size for that pixel; then you can use trilinear interpolation to mix the two closest mip levels.

[mrbastard]

I don't know if you can do it easily and cheaply, but maybe you can examine the texture coordinates of adjacent pixels and choose a mip level according to how close or far to each other they are.
If you compute the maximum difference between the U or V coordinate of your fragment and the corresponding coordinate of each adjacent pixel, it translates directly to the ideal texture size for that pixel; then you can use trilinear interpolation to mix the two closest mip levels.

Thanks. I could certainly do that in an extra pass - I'd rather avoid doing the actual mandelbrot function more than once per pixel.

I'm not sure I follow your reasoning about the ideal texture size, but I'm in a rush in my lunch break. I'll have some time this evening and will give it a try.

Thanks again

[MJP]

You don't need an extra pass, the derivatives of any value can be calculated in a pixel shader using ddx and ddy. You could use those to detect discontinuities, and clamp your gradients to a smaller value to prevent going down to a smaller mip level.

[Jason Z]

I agree with Hodgman - the effect looks fantastic. Lateralus is one of my favorite albums.... I would love to see a video of this in action! Great work!

[mrbastard]

You don't need an extra pass, the derivatives of any value can be calculated in a pixel shader using ddx and ddy. You could use those to detect discontinuities, and clamp your gradients to a smaller value to prevent going down to a smaller mip level.

Excellent, thanks again. I kind of thought that was the case but wasn't confident enough in my understanding to be sure.

I've been getting very useful results doing this. I'm going to have to spend some time working out how best to apply it to the different parts of the image, but I'm definitely getting close now. I'll post another screenshot when I'm happy with the results - hopefully this weekend.

[mrbastard]

I agree with Hodgman - the effect looks fantastic. Lateralus is one of my favorite albums.... I would love to see a video of this in action! Great work!

Thanks! Likewise, I love Lateralus. I saw Tool in Berlin on the 10,000 Days tour, and hearing (feeling in the case of the bass drum) Danny Carey's drumming on Laterlaus was incredible.

I'm not sure how much more time I'll put into the demo once this issue is fixed. I had plans for a 'narrative' - keyframes of constant buffers for different parts of the song, interpolated as needed. I'd also very much like to make the central flame billow with a bit of sin().

This is the only problem with fractals - the possibilities are endless!

I should point out, my original inspiration for this technique was an image by Inigo Quilez. I spent a lot of time looking at the bunny image, trying to see what he was doing, and if there was a way to do something similar in real time. Where Inigo actually traps against the alpha channel to get the perfect outlines, I'm just trapping against a circle. The alternative would need a texture sample per iteration of the mandelbrot algorithm - up to 128 in my implementation.

[MJP]

I'm jealous, I've always wanted to see Tool live.

[Jason Z]

What do you think about contributing a sample application to Hieroglyph 3 that implements this algorithm? If you don't have time to translate from your code to the Hieroglyph basis, then I could volunteer to do the conversion for you. It seems like a cool enough application that it should live on somewhere And of course you will be credited with your work.

About Tool - I haven't seen them live in quite a while now, not since they were touring in the U.S. for 10,000 days. I can't wait to see them again, but I'm pretty sure that won't happen until the next album comes around (which could take a while given their past history...).

[mrbastard]

Who'd have thought there'd be so many graphics programmer Tool fans!

What do you think about contributing a sample application to Hieroglyph 3 that implements this algorithm? If you don't have time to translate from your code to the Hieroglyph basis, then I could volunteer to do the conversion for you. It seems like a cool enough application that it should live on somewhere And of course you will be credited with your work.

Great idea, I'd love to. I've been meaning to spend some time with Hieroglyph since I bought the book last month. I'm guessing it'll be fairly easy to convert as it's pretty much all in the pixel shader. The only real exception to that is use of FMOD to play back the sound and do FFT to give me the current sound spectrum. The spectrum goes in an array in my constant struct and is used to distort the trapping shape, UVs and colours in time to the music. Not necessarily a big loss though, as I haven't yet found a way to use it that I'm satisfied with.

Anyway - I'm hoping to work on the texturing problem over the weekend, and probably finish up the demo over the next couple of weeks. Then I'll have a look at translating it to Hieroglyph.

[Jason Z]

Yes, I see a pattern forming about graphics programming and Tool...

The rest of the application sounds cool - I don't have any bindings to FMOD, but that is primarily just to minimize the number of dependencies that the library has. Perhaps I can check out some of the audio tools that are already in the DXSDK, and see if something can fit in there in the place of FMOD...

[Hodgman]

Who'd have thought there'd be so many graphics programmer Tool fans!

I never would've known if not for this thread

P.S. it would be cool if you found a way to incorporate a Fibonacci spiral to go with the lyrical composition of Lateralus ;)

[mrbastard]

The rest of the application sounds cool - I don't have any bindings to FMOD, but that is primarily just to minimize the number of dependencies that the library has. Perhaps I can check out some of the audio tools that are already in the DXSDK, and see if something can fit in there in the place of FMOD...

Great, thanks.

P.S. it would be cool if you found a way to incorporate a Fibonacci spiral to go with the lyrical composition of Lateralus ;)

Funnily enough the Fibonacci sequence (arguably) crops up in the Mandelbrot set in a few places - the bulb size/periodicity for example. I say arguably because there's a whiff of numerology to it I think - if you go looking for numbers and relationships in the set you're bound to find something!


I was going to vary the power used in the mandelbrot function according to the vocal rhythm, e.g.:
"black" : z=pow(z,1)+c : (the set is a circle)
"and" : z=pow(z,1)+c : (the set is a circle)
"white are" : z=pow(z,2)+c : (the set is a normal mandelbrot set)
"all I see" : z=pow(z,3)+c : (cubic mandelbrot set)
"in my in-fan-cy" : z=pow(z,5)+c : (quintic mandelbrot set)
"red and yel-low then came to be" : z=pow(z,8)+c : (octic(?) mandelbrot set)
"reach-ing out to me" : z=pow(z,5)+c : (quintic mandelbrot set)
"lets me see" : z=pow(z,5)+c : (cubic mandelbrot set)

Definitely would be cool to have an actual spiral for the "swing on a spiral" bits though I'll put my mind to it.