Values like 0xffffffff and 0xff are used as write/read masks in stenciling and blending but I dont understand them. Can someone explain me? Thanks.

0xFF is equivalent to a binary 11111111 (0xFFFFFF is a 32bit value)
So its effectivly a mask with all bits set to 1.

most color values will be a 32 bit (0xRRGGBBAA) number (Where RR is red, GG is green , BB is blue and AA is alpha)

If you use a 0xFF00FFFF mask with such a color it effectivly removes the green component of the color for example. (Most of the time you will use a mask with all or most bits set)

As the stencil buffer uses a single byte per pixel you quite often use a 0xFF read/write mask for it (this basically allows all reads/writes), a 0x00 mask allows no reads/writes
using for example a 0x0F write mask would allow writes of values 0-15 but would cut off writing of any higher values (attempting to write the number 16 would result in the writing of a 0 and 17 would become 1), a 0x1F write mask would allow 0-31 and 0xFE would allow any even number between 0 and 254(inclusive). (odd numbers would become the nearest lower even number)

As there are compartivly few situations where you want to restrict writes or reads to a limited set of values the most common masks are 0xFF and 0x00.

Where can I find some reference about all kinds of character that can be used? Like 0xFE, etc

Get your mask (just which bits are on/off)

say 01011100 thats binary, just a number completely made up in my head. Convert it to hexadecimal

first 4 bits 0101 is 5 in decimal and 5 in hexadecimal, second 4 bits 1100 is 12 in decimal, C in hexadecimal. Stick them together 0x5C.

Hexadecimal can represent numbers 0-15: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F. 4 bits can represent numbers 0-15.

I'd suggest doing a little search on binary/hexadecimal and how to convert between the two. You can work out your masks then, no need for any reference each time.

Binary: 01011100
Decimal: 92
Hexadecimal: 0x5C

The hexadecimal one says there are 5, 16s and "C" (which is 12) 1s. 5*16 + 12*1 = 92. The binary one is doing similar but its saying how many 128s, 64s, 32s, 16s, 8s, 4s, 2s, 1s. Ones powers of 16, binary is powers of 2, decimal is powers of 10 (9*10^1 + 2*10^0 = 92).

Thank you for you answer

[quote name='SimonForsman' timestamp='1305585488' post='4811677']
If you use a 0xFF00FFFF mask with such a color it effectivly removes the green component of the color for example. (Most of the time you will use a mask with all or most bits set)

So if I used a mask like 0xFFF0FFFF or 0xFF0FFFFF what will happen?

As the stencil buffer uses a single byte per pixel you quite often use a 0xFF read/write mask for it (this basically allows all reads/writes), a 0x00 mask allows no reads/writes
using for example a 0x0F write mask would allow writes of values 0-15 but would block writing of any higher values. (This is naturally not as common as allowing or blocking everything)

Why values from 0-15? Is that because each pixel stores 32 bits? But shouldn't 0xF0 write values from 0-15 and 0x0F from 16-31?


Where can I find some reference about all kinds of character that can be used? Like 0xFE, etc
[/quote]

basically its hexadecimal values http://en.wikipedia....iki/Hexadecimal

so 0x00 (hex) = 0000 0000 (Binary) = 0 (Decimal)
0x01 = 0000 0001 (binary) = 1 (Decimal)
.....
0x09 = 0000 1001 = 9
0x0A = 0000 1010 = 10
0x0F = 0000 1111 = 15
0x10 = 16 0001 0000 = 16
0x 11 = 17 0001 0001 = 17
0x 19 = 25 0001 1001 = 25

As you can see the first symbol in the hexadecimal number matches the first 4 in the binary equivalent

when you apply a mask you normally use a bitwise and operator (& in c++)

the bitwise and operator basically sets the result bit to 1 if the equivalent bit is set in both operands

so 9 & 17 becomes
0000 1001 &
0001 0001
--------------
0000 0001 (last bit is the only bit set in both operands)

the bitmask 0x0F looks like 0000 1111 in binary which when applied to a number effectivly sets the first 4 bits to 0

Thus 16 (0001 0000) & 0x0F becomes

0001 0000 &
0000 1111
--------------
0000 0000 = 0

as the & operator will always set the first 4 bits to 0 (When used with 0x0F) the only possible results are:

0000 0000 = 0
0000 0001 = 1
0000 0010 = 2
0000 0011 = 3
0000 0100 = 4
0000 0101 = 5
0000 0110 = 6
0000 0111= 7
0000 1000 = 8
0000 1001 = 9
0000 1010 = 10
0000 1011 = 11
0000 1100 = 12
0000 1101 = 13
0000 1110 = 14
0000 1111 = 15

using a color mask like 0xFF F0 FF FF
would basically keep the red, blue and alpha components as they are and cut off the last 4 bits of the green component (assuming you're using RGBA colors), this basically rounds the green value down to the nearest multiple of 16
(each color is a value between 0 and 255, having a green component of 71 and using that mask will set the green component to 64).

I don't think D3D or OpenGL supports color write bitmasks though (for OpenGL it seems like color write masks used to be per channel , not per bit) (With shaders you can use proper bitmasks for colors aswell though).

bitmasking is primarily used for the stencil buffer to be able to keep multiple values stored in it.

When performing a stencil test you basically want to check if the value in the stencil buffer is nonzero, if it is nonzero you render that pixel, if its zero you don't, by using bitmasks you can store for example the lit(or unlit) areas for 8 different lights.
the first light would then use a 0x01 bitmask, the second light a 0x02 bitmask, third light a 0x04 bitmask, fourth 0x08 , 5th 0x10 , 6th 0x20 7th 0x40 and 8th a 0x80 mask and using a stenciltest with the appropriate mask would then let you render only the parts lit by a specific light (a simple way to render shadows then was to render the entire scene with only ambient lighting and then render the parts lit by each light using additive blending)

(Color masking is far less useful but could be used to create some effects, i'm not creative enough to think of anything useful right now though)