Jump to content

  • Log In with Google      Sign In   
  • Create Account

#ActualKaptein

Posted 09 October 2012 - 01:50 PM

?255*409
104295

int16_t is signed 16bits, and thus it ranges from -32768 to 32767
as you can see, part of your equation is out of range, and will overflow the bits
-104 is correct, as far as the computer is concerned Posted Image

i also believe (cant remember off the top of my head) that multiplication has presedence over division
which means that the multiplication happens first

#2Kaptein

Posted 09 October 2012 - 01:49 PM

?255*409
104295

int16_t is signed 16bits, and thus it ranges from -32768 to 32767
as you can see, part of your equation is out of range, and will overflow the bits
-104 is correct, as far as the computer is concerned :)

#1Kaptein

Posted 09 October 2012 - 01:46 PM

integers don't have precision, which in this case means decimals, or "parts"
for example: 3 \ 2 = 1, 4 \ 2 = 2, 5 \ 2 = 2, 6 \ 2 = 3 and so on
and, modulus returns the remainder of a division: 0 mod 3 = 0, 1 mod 3 = 1, 2 mod 3 = 2, 3 mod 3 = 0, 4 mod 3 = 1 and so on (as you can see it goes 0, 1, 2, 0, 1, 2, 0, 1, 2)

in any case, what you have to do is force the compiler to use floating point precision primitives for the operands
like this: int x = (int) ( (float)x / (float)y )
what i did here is cast the integer x to a float (32bits decimal number-thing), same with y, and divide them
then i cast the result back to an int (which also means i lose any decimals, even if the result was .9999)

so here are some pitfalls:
(int)0.9999 = 0
float x = fabsf(-85.3) gives x = 85 !

if fabsf is undefined (by for example not including math.h), then it returns an int!

PARTNERS