I'd like to get the game I'm working on to look like it uses an old software renderer, and one of the artifacts I'd like to reproduce is how in many old games (specifically on the PS1, but also many computer games) the vertices would shake slightly. From what I can find this was because the coordinates were passed from the math hardware to the rasterization hardware in integers. I tried to produce a similar effect by flooring the calculated positions in my vertex shader:

gl_Position.xy=floor(gl_Position.xy*256)/256;

but it doesn't seem to produce any noticable effect unless I lower the 256 down to less than 10, and then it is EXTREMELY wobbly to the point of being unusable. (I chose 256 because my window is 1024 pixels wide, and the position ranges from -1 to 1 (unless I'm remembering this incorrectly?))

Any ideas?

in what space is gl_Position.xy? if it's after you've applied the view projection matrix, then you'd need to divide it by w to get the screenspace positions.

then you could do your magic and then multiply by w again.

kind of like

gl_Position.xy = fractgl_Position.xy/gl_Position..w*(1./4.))*4.*gl_Position.w;

(untested ;) )

btw. you shall also be able to simulate that texture interpolation bugs by multiplying the texturecoordinates in the vertexshader by gl_Position.w and dividing it per pixel again (you'd obviously also need to pass .w to the pixelshader.

you'll still miss the lovely gabs between triangles that those oldschool rasterizer had, that would be some more effort.

glPosition was post all transformations. that line was the last line in the vertex shader. I'm not sure quite what you're going for wit that code.. I assume that you meant to type this:

gl_Position.xy = fract(gl_Position.xy/gl_Position.w*(1./4.))*4.*gl_Position.w;

Using fract doesn't make sense to me in this case. floor makes more sense, but I get similar problems anyway. Everything shows up strangely stretched in one quarter of the screen.

The vertex positions in world space need to be rounded to integer coordinates. I don't think the system had any kind of floating point support.

It didn't even have any 3D support, the GPU only saw 2D coordinates. All projection was done in the CPU using fixed point. Also clipping - this one is important because a lot of issues arised from inaccurate clipping, but in modern hardware clipping is done with the GPU which of course will try to apply as much accuracy as it can. You'll have to deal with this as well.

Also: at what resolution are you rendering? The PS1 had 256×240 and 320×240 as the available resolutions. At lot of the problems (like the shakiness) became exaggerated because pixels were so big. Make sure you're rendering to a low-resolution buffer and then scale it to fill the screen, otherwise all your effort will be moot.

You're talking about texture artifacts (which is caused exclusively by lack of perspective correction in the texture, nothing else), he's talking about vertices moving out of place (it was common for vertices to get off by 1 pixel as the camera moved, resulting in seemingly shaking polygons and such), which was indeed caused by inaccuracies in the computations, and also were quite noticeable because of the low resolution (at a high resolution they aren't as noticeable).

EDIT: typo.

glPosition was post all transformations. that line was the last line in the vertex shader. I'm not sure quite what you're going for wit that code.. I assume that you meant to type this:

gl_Position.xy = fract(gl_Position.xy/gl_Position.w*(1./4.))*4.*gl_Position.w;

Using fract doesn't make sense to me in this case. floor makes more sense, but I get similar problems anyway. Everything shows up strangely stretched in one quarter of the screen.

sorry, that was a typo, was a bit in a hurry. floor instead of fract is correct.

but my point was, that you need the division by .w to be in screenspace.

maybe your issue is a different bug, post a screenshot maybe :)