So I'm making a fire demo and I have to average a bunch of pixels(the core algorithm is a per pixel averaging function for some nice fire modeling). The problem I'm getting, is two difference answers based on the code. Why is this happening?
The following code is giving me two different results:
BYTE Average = (current + left + right + bottom + bottomleft + bottomright + upperleft + upperright)/8;
BYTE Average2 = (current + left + right + bottom + bottomleft + bottomright + upperleft + upperright);
Average2 = Average2/8;
Current = 255, while all other variables are equal to zero. Current refers to the value of the current pixel (x, y), and right is (x + 1, y), left is (x - 1, y), etc..
Average ends up with a value of ~31, which is 255/8 = ~31, this makes sense. Average2 gives me a value of 3. Why is this, how can I prevent this? Why does separating the division by 8 from the initial expression give me two completely different answers? C++ is so fucking subtle!!!
c++ prob with casting and arithmetic
"~" is and operator in C and C++ (bitwise not), so saying the result is ~31 is simply wrong: it should be 31. I can't see how Average2 could end up being 3. What is Average in your second example?
Author
"~" is and operator in C and C++ (bitwise not), so saying the result is ~31 is simply wrong: it should be 31. I can't see how Average2 could end up being 3. What is Average in your second example?
By "~", i meant approximately, so "~31" equals the approximately 31, where the exact value is like 31.548.... Average is 31, where as Average2 is 3; Anyway, I solved my problem. I changed the type from BYTE to DWORD, and the math works out. I made a mistake by using BYTES, as the the math was overloading and resetting to zero when the values went above 255. By using DWORDS, all the math works out. Thank you for attempting to answer my question.
As you discovered, the problem is overflow.
In the first version, the compiler internally does the math as machine-word size integers (DWORDs in Microsoft parlance) and then converts to a BYTE after the division. In the second, the addition is coerced into a BYTE first, before the division, which means if the addition sums to more than 255 you get overflow and basically your answer will be gibberish.
In the first version, the compiler internally does the math as machine-word size integers (DWORDs in Microsoft parlance) and then converts to a BYTE after the division. In the second, the addition is coerced into a BYTE first, before the division, which means if the addition sums to more than 255 you get overflow and basically your answer will be gibberish.
It still doesn't explain where he got a 3 from when he plugged in 255,0,0,0,0,0,0,0. My guess is that he didn't plug in the values he said he did. Wait, or he wrote "Average2=Average/8;", so he was dividing by 8 a second time. That's the code I remember reading, which is why I asked about the value of "Average" in the second example. Did the post get edited?
Author
It still doesn't explain where he got a 3 from when he plugged in 255,0,0,0,0,0,0,0. My guess is that he didn't plug in the values he said he did. Wait, or he wrote "Average2=Average/8;", so he was dividing by 8 a second time. That's the code I remember reading, which is why I asked about the value of "Average" in the second example. Did the post get edited?
Sorry, i wrote the wrong code and changed it in an edit. Even with the right code, my average function was outputting values that were much too small. By changing to DWORDS everything worked out. Thanks again alvaro. Just asking this question yesterday, helped myself solve it. So these forums and users roxor.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
