Strange results while using numpy arrays

Question

I am getting two different results for some inputs but not others. Let me explain using the concrete example. I have the following function:

In [86]: def f(x, p):
    ...:     n = len(p)
    ...:     tot = 0
    ...:     for i in range(n):
    ...:         tot += p[i] * x**(n-i-1)
    ...:     return tot

p is an array with very small values:

In [87]: p
Out[87]: 
array([ -3.93107522e-45,   9.17048746e-40,  -8.11593366e-35,
         3.05584286e-30,  -1.06065846e-26,  -3.03946945e-21,
         1.05944707e-16,  -1.56986924e-12,   1.07293061e-08,
        -3.22670121e-05,   1.12072912e-01])

Now consider the outputs:

In [90]: [f(i, p) for i in range(11, 20)]
Out[90]: 
[0.11171927108787173,
 0.1116872502272328,
 0.1116552507123586,
 0.11162327253386167,
 0.11159131568235707,
 0.11155938014846242,
 0.1115274659227979,
 0.11149557299598616,
 0.11146370135865244]

In [88]: [f(i, p) for i in np.array(range(11, 20))]
Out[88]: 
[0.11171927108787173,
 0.1116872502272328,
 0.1116552507123586,
 0.11162327253386167,
 0.11159131568235707,
 0.11155938014846242,
 0.1115274659227979,
 0.11149557299598616,
 0.11146370135865244]

As you can see, these outputs are exactly same as they should be. The only difference is that in one case I am using range(a, b) while in the other case I am converting that range to a numpy array.

But now, let us change the values inside the range:

In [91]: [f(i, p) for i in range(50001, 50010)]
Out[91]: 
[-0.011943965521167818,
 -0.011967640114171604,
 -0.011991315947644229,
 -0.012014993019120554,
 -0.012038671327427961,
 -0.012062350870605351,
 -0.012086031644648818,
 -0.012109713648648865,
 -0.012133396879791744]

In [92]: [f(i, p) for i in np.array(range(50001, 50010))]
Out[92]: 
[491.26519430165808,
 491.32457916465478,
 491.38395932037008,
 491.38726606180143,
 491.44663641006275,
 491.50600185375316,
 491.56536239249812,
 491.56864971072332,
 491.6280006336612]

And they are not even close! Am I missing something ridiculously simple?

That looks like an interger range issue. on newer python versions ints have unlimited range, you can check that by typing something ridiculous like 10**100-1. numpy ints are C longs so they do not provide that luxury. — Paul Panzer
– Paul Panzer, Commented Feb 19, 2017 at 5:26

user2357112 · Accepted Answer · 2017-02-19 05:22:13Z

2

You're missing the fact that ordinary Python integers are arbitrary-precision, while NumPy integers are fixed-size.

This:

x**(n-i-1)

overflows with the NumPy inputs.

answered Feb 19, 2017 at 5:22

user2357112

286k32 gold badges490 silver badges569 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Peaceful Over a year ago

I see. What is the remedy then because I have numpy array in the actual code. In fact, it is a Pandas Series.

Stephen Rauch · Accepted Answer · 2017-02-19 06:19:23Z

1

The values in f(x, p) for x in the error case, are type numpy.int32. They can overflow. The fix in this case is relatively straight forward, convert the values to int:

tot += p[i] * np.asarray(x).astype(int) ** (n - i - 1)

edited Feb 19, 2017 at 6:19

answered Feb 19, 2017 at 5:34

Stephen Rauch♦

50.1k32 gold badges118 silver badges143 bronze badges

2 Comments

Peaceful Over a year ago

That may not work when x is passed as a numpy array. Right?

Peaceful Over a year ago

Hmm. However, it throws an error in my case: TypeError: only length-1 arrays can be converted to Python scalars

Collectives™ on Stack Overflow

Strange results while using numpy arrays

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related