Jump to content

  • Log In with Google      Sign In   
  • Create Account

Bismuth

Member Since 28 Jun 2009
Offline Last Active Dec 11 2012 07:36 AM

Posts I've Made

In Topic: Integer multiplication, division

09 October 2012 - 02:44 PM

Your variable a holds a negative value and you're casting it to an unsigned integer. Unsigned variables cannot hold negative values. And yes, it is technically enough to cast just a or b, because the other operands will be promoted automatically.

Eh, sorry it was a typo. I wonder why I typed uint anyway.

int16_t cc = (int32_t)aa * bb / (pe->fade_out); // This works.

In Topic: Integer multiplication, division

09 October 2012 - 02:30 PM

Thank you for explaining the situation, this clears up my doubts. I am programming an application in AVR Studio 5.1 using gcc for an Atmega328p microcontroller. Due to limited RAM on a microcontroller (2 KB) I tend to stick to 8-bit and 16-bit variables. I did some tests and it seems that increasing at least one of the vars in the equation will produce the correct result.

This does not seem to work though.
int16_t result = (uint32_t)a * b / c; // does not work

I'll do some more tests.

Regards,
Bismuth

In Topic: Integer multiplication, division

09 October 2012 - 02:07 PM

Actually, you're running out of precision in that operation. -255 * 409 is -104,295, but int16_t only has 16 bits of storage, which means it can only store values in the range −32,768 to 32,767. So what happens? Your intermediate result gets chomped down to 16 bits *before* the divide happens, and the end result is a weird value due to overflow.

What should you do about it? Use a bigger data type.


Could you please elaborate on that? What exactly is the factor that causes the calculation to be processed in 16-bit? Shouldn't the program detect that its running out of precision and automatically use a bigger (i.e. 32-bit) temporary buffer?

Which variables exactly should be increase in size?
EDIT: Is there a way to increase the temporary buffer size without changing the variable data types?

In Topic: Integer multiplication, division

09 October 2012 - 01:51 PM

But the final result -104 does not overflow the output buffer. It's clearly in the range, so it should be assigned no problem. I'm not sure I understand what's going on here.

In Topic: Opinion: What should I name the generic handle type?

02 August 2011 - 09:48 AM

I vote for handle.

PARTNERS