Sign in to follow this  

c++ prob with casting and arithmetic

This topic is 2333 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

So I'm making a fire demo and I have to average a bunch of pixels(the core algorithm is a per pixel averaging function for some nice fire modeling). The problem I'm getting, is two difference answers based on the code. Why is this happening?

The following code is giving me two different results:

BYTE Average = (current + left + right + bottom + bottomleft + bottomright + upperleft + upperright)/8;

BYTE Average2 = (current + left + right + bottom + bottomleft + bottomright + upperleft + upperright);
Average2 = Average2/8;

Current = 255, while all other variables are equal to zero. Current refers to the value of the current pixel (x, y), and right is (x + 1, y), left is (x - 1, y), etc..

Average ends up with a value of ~31, which is 255/8 = ~31, this makes sense. Average2 gives me a value of 3. Why is this, how can I prevent this? Why does separating the division by 8 from the initial expression give me two completely different answers? C++ is so fucking subtle!!!

Share this post


Link to post
Share on other sites
[quote name='alvaro' timestamp='1311382253' post='4839170']
"~" is and operator in C and C++ (bitwise not), so saying the result is ~31 is simply wrong: it should be 31. I can't see how Average2 could end up being 3. What is Average in your second example?
[/quote]


By "~", i meant approximately, so "~31" equals the approximately 31, where the exact value is like 31.548.... Average is 31, where as Average2 is 3; Anyway, I solved my problem. I changed the type from BYTE to DWORD, and the math works out. I made a mistake by using BYTES, as the the math was overloading and resetting to zero when the values went above 255. By using DWORDS, all the math works out. Thank you for attempting to answer my question.

Share this post


Link to post
Share on other sites
As you discovered, the problem is overflow.

In the first version, the compiler internally does the math as machine-word size integers (DWORDs in Microsoft parlance) and [i]then[/i] converts to a BYTE after the division. In the second, the addition is coerced into a BYTE [i]first[/i], before the division, which means if the addition sums to more than 255 you get overflow and basically your answer will be gibberish.

Share this post


Link to post
Share on other sites
It still doesn't explain where he got a 3 from when he plugged in 255,0,0,0,0,0,0,0. My guess is that he didn't plug in the values he said he did. Wait, or he wrote "Average2=Average/8;", so he was dividing by 8 a second time. That's the code I remember reading, which is why I asked about the value of "Average" in the second example. Did the post get edited?

Share this post


Link to post
Share on other sites
[quote name='alvaro' timestamp='1311420013' post='4839253']
It still doesn't explain where he got a 3 from when he plugged in 255,0,0,0,0,0,0,0. My guess is that he didn't plug in the values he said he did. Wait, or he wrote "Average2=Average/8;", so he was dividing by 8 a second time. That's the code I remember reading, which is why I asked about the value of "Average" in the second example. Did the post get edited?
[/quote]

Sorry, i wrote the wrong code and changed it in an edit. Even with the right code, my average function was outputting values that were much too small. By changing to DWORDS everything worked out. Thanks again alvaro. Just asking this question yesterday, helped myself solve it. So these forums and users roxor.

Share this post


Link to post
Share on other sites

This topic is 2333 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this