32-bit fixed point overflow

Question

I'm doing some 'early computing' on a 32-bit Windows PC, and looking at the limits.

Now, 2**32 is 4,294,967,296, and I find that

4294967290 + 5

is perfectly OK, and

    4294967290 + 6

quite properly overflows.

What puzzles me is that

   429496729 * 10

overflows, although the product, 4294967290, is in range.

Anyone interested?

what environment are you using? Your compiler / interpreter may be doing some inexact bounds checking. — ThomasMcLeod
– ThomasMcLeod, Commented Mar 2, 2011 at 14:00
That's not fixed point by the way, it's integer. Fixed point is integer with scaling (for example, 100 = 1.00, 4323 = 43.23, and so on). — paxdiablo
– paxdiablo, Commented Mar 2, 2011 at 14:05

GSerg · Accepted Answer · 2011-03-02 14:02:56Z

4

In the absense of any code, I'd guess 429496729 gets implicitly typed as signed integer, for which 4294967290 is too much.

answered Mar 2, 2011 at 14:02

GSerg

78.8k18 gold badges173 silver badges377 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

phkahler Over a year ago

You beat me to it. The result is probably negative to prove it.

John White Over a year ago

Thanks fellows, quite right. Actually there was no code, but the Visual C 6 debugger watch window (used for my 'probing') evaluates numeric integers as signed. Quite reasonably actually. All is well with unsigned int.

Collectives™ on Stack Overflow

32-bit fixed point overflow

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related