Converting empty strings to 0 using Numpy

Question

I have a numpy array where each element looks something like this:

['3' '1' '35' '0' '0' '8.05' '2']
['3' '1' '' '0' '0' '8.4583' '0']
['1' '1' '54' '0' '0' '51.8625' '2']

I would like to replace all empty strings like the ones in the second row above, with some default value like 0. How can I do this with numpy?

The ultimate goal is to be able to run this: S.astype(np.float), but I suspect the empty strings are causing problems in the conversion.

so these are numpy string arrays?

Bitwise
– Bitwise

2013-04-10 21:28:50 +00:00
Commented Apr 10, 2013 at 21:28 — Bitwise
– Bitwise, Commented Apr 10, 2013 at 21:28
Yes. It's created using np.array

Oleksi
– Oleksi

2013-04-10 21:29:48 +00:00
Commented Apr 10, 2013 at 21:29 — Oleksi
– Oleksi, Commented Apr 10, 2013 at 21:29
Can't the numpy arrays be used with list comprehensions?

user1778770
– user1778770

2013-04-10 21:36:50 +00:00
Commented Apr 10, 2013 at 21:36 — user1778770
– user1778770, Commented Apr 10, 2013 at 21:36

Bitwise · Accepted Answer · 2015-12-10 14:25:28Z

19

If your array is t:

t[t=='']='0'

and then convert it.

Explanation:

t=='' creates a boolean array with the same shape as t that has a True value where the corresponding t value is an empty space. This boolean array is then used to assign '0' only to the appropriate indices in the original t.

edited Dec 10, 2015 at 14:25

answered Apr 10, 2013 at 21:30

Bitwise

7,8357 gold badges39 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user1778770 Over a year ago

Cool, I didn't remember this construct, I was just about to post an answer with list comprehensions, but this is quite shorter!

Bitwise Over a year ago

@user1778770 use list comprehension only if you have a list. as long as you have a numpy array, you should take advantage of that.

user1778770 Over a year ago

I think you could make your answer even better by adding a link to a page where the construct you've exposed is documented. That's for those beginners who will wonder what's going on. I haven't found one

Jason Sperske · Accepted Answer · 2022-02-16 23:30:33Z

6

Here is an approach that uses map not that is does not produce the same data type as calling .astype():

def FloatOrZero(value):
    try:
        return float(value)
    except:
        return 0.0


print map(FloatOrZero, ['3', '1', '', '0', '0', '8.4583', '0'])

Outputs:

[3.0, 1.0, 0.0, 0.0, 0.0, 8.4583, 0.0]

It's possible that this approach will give you more flexibility to cleanup data but it could also be harder to reason about if you are wanting to work with a numpy.array.

edited Feb 16, 2022 at 23:30

answered Apr 10, 2013 at 21:32

Jason Sperske

30.5k8 gold badges77 silver badges127 bronze badges

1 Comment

mLstudent33 Over a year ago

are map, reduce faster than @askewchan's answer?

askewchan · Accepted Answer · 2013-04-10 21:31:06Z

5

Just do this first:

s = np.array(['1', '0', ''])
s[s==''] = '0'

s.astype(float)
#array([ 1.,  0.,  0.])

answered Apr 10, 2013 at 21:31

askewchan

46.7k18 gold badges125 silver badges135 bronze badges

2 Comments

mLstudent33 Over a year ago

is this faster than map?

Jason Sperske Over a year ago

I was curious so I wrote a toy example using cProfile and they are not exactly the same. I suspect the overall work is roughly the same but calling map creates a new array of float values while calling astype(float) keeps the original np.array and just lets you see each value as a float. I now think this answer is better and just as readable so I updated my answer so people don't blindly copy bad information.

Collectives™ on Stack Overflow

Converting empty strings to 0 using Numpy

3 Answers 3

3 Comments

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related