numpy array of strings indexing behavior

Question

I have an array of strings

>>> lines
array(['RL5\\Stark_223', 'RL5\\Stark_223', 'RL5\\Stark_223', ...,
       'RL5\\Stark_238', 'RL5\\Stark_238', 'RL5\\Stark_238'], 
      dtype='|S27')

Why can I index into a string for the first array element

>>> lines[0][0:3]
'RL5'

But not into the same place for all array elements

>>> lines[:][0:3]
array(['RL5\\Stark_223', 'RL5\\Stark_223', 'RL5\\Stark_223'], 
      dtype='|S27')

Can anyone suggest a method to get the following result:

array(['RL5', 'RL5', 'RL5', ...'RL5', 'RL5')

Jaime · Accepted Answer · 2013-09-12 20:49:23Z

5

To extract the first n characters of every string you can abuse .astype:

>>> s = np.array(['RL5\\Stark_223', 'RL5\\Stark_223', 'RL5\\Stark_223'])
>>> s
array(['RL5\\Stark_223', 'RL5\\Stark_223', 'RL5\\Stark_223'], 
      dtype='|S13')
>>> s.astype('|S3')
array(['RL5', 'RL5', 'RL5'], 
      dtype='|S3')

answered Sep 12, 2013 at 20:49

Jaime

67.7k19 gold badges128 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

isosceleswheel Over a year ago

Don't know why this answer was so hard to find! Simple and to the point -- thanks!

Daniel · Accepted Answer · 2013-09-12 22:03:54Z

3

Dont forget chararrays!

lines.view(np.chararray).ljust(3)
chararray(['RL5', 'RL5', 'RL5', 'RL5', 'RL5', 'RL5'], 
      dtype='|S3')

Although its strangely slower:

#Extend lines to 600000 elements

%timeit lines.view(np.chararray).ljust(3)
1 loops, best of 3: 542 ms per loop

%timeit np.vectorize(lambda x: x[:3])(lines)
1 loops, best of 3: 239 ms per loop

%timeit map(lambda s: s[0:3], lines)
1 loops, best of 3: 243 ms per loop

%timeit arr.astype('|S3')
100 loops, best of 3: 4.72 ms per loop

Could be because its duplicating the data, the benefit of this is the dtype of the output array is minimized: S3 vs S64.

edited Sep 12, 2013 at 22:03

answered Sep 12, 2013 at 18:14

Daniel

19.6k7 gold badges64 silver badges74 bronze badges

Comments

DonaldAnderson · Accepted Answer · 2013-09-12 18:06:34Z

1

try this

map(lambda s:s[0:3],lines)

answered Sep 12, 2013 at 18:06

DonaldAnderson

1685 bronze badges

Comments

Andy Hayden · Accepted Answer · 2013-09-12 18:06:47Z

0

You can use numpy's vectorize:

In [11]: np.vectorize(lambda x: x[:3])(lines)
Out[11]: 
array(['RL5', 'RL5', 'RL5', 'RL5', 'RL5', 'RL5'], 
      dtype='|S64')

answered Sep 12, 2013 at 18:06

Andy Hayden

378k110 gold badges640 silver badges546 bronze badges

Comments

user2561747 · Accepted Answer · 2016-11-03 21:00:06Z

If you're looking for fast and (somewhat more) flexible, try:

lines.view('|S1').reshape(-1, lines.dtype.itemsize)[:, :3].reshape(-1).view('|S3')

Which can be used for more arbitrary slicing and dicing.

Timing info:

import numpy as np
lines = np.array(['RL5\\Stark_223', 'RL5\\Stark_223', 'RL5\\Stark_223', 
'RL5\\Stark_238', 'RL5\\Stark_238', 'RL5\\Stark_238'], dtype='|S27').repeat(100000)

%timeit lines.view(np.chararray).ljust(3)
1 loop, best of 3: 231 ms per loop

%timeit np.vectorize(lambda x: x[:3])(lines)
1 loop, best of 3: 226 ms per loop

%timeit map(lambda s: s[0:3], lines)
1 loop, best of 3: 171 ms per loop

%timeit lines.astype('|S3')
100 loops, best of 3: 3.58 ms per loop

%timeit lines.view('|S1').reshape(-1, lines.dtype.itemsize)[:, :3].reshape(-1).view('|S3')
100 loops, best of 3: 5.16 ms per loop

Collectives™ on Stack Overflow

numpy array of strings indexing behavior

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related