how do i import from a unicode (utf-8) csv file into a numpy array

Question

im not trying to do this smart or fast, just trying to do it at all.

i have a file looks like this :

$ cat all_user_token_counts.csv  
@5raphaels,in,15
@5raphaels,for,15
@5raphaels,unless,11
@5raphaels,you,11

i know its uncode utf-8 encoded because i created it, like this

    debug('opening ' + ALL_USER_TOKEN_COUNTS_FILE)
    file = codecs.open(ALL_USER_TOKEN_COUNTS_FILE, encoding="utf-8",mode= "w")
    for (user, token) in tokenizer.get_tokens_from_all_files():
        #... count tokens ..
        file.write(unicode(username +","+ token +","+ str(count) +"\r\n"))

i want to read it in to a numpy array so it looks like this, or something..

   array([[u'@5raphaels', u'in', 15],
          [u'@5raphaels', u'for', 11],
          [u'@5raphaels', u'unless', 11]], 
          dtype=('<U10', '<U10', int))

As i experiment in process of writing this question it comes to me that it may not even be possible? If so, I'd love to know!

Thanks in advance!

realised my real problem was that i was trying to use different data types in the one array - and I dont think Numpy likes that, ultimately — utunga
– utunga, Commented May 8, 2012 at 5:00

unutbu · Accepted Answer · 2011-08-02 11:21:31Z

2

This can be done easily with np.loadtxt:

import numpy as np
arr=np.loadtxt('all_user_token_counts.csv',delimiter=',',
                  dtype = '|U10,<U10,int')
print(arr)

# [(u'@5raphaels', u'in', 15) (u'@5raphaels', u'for', 15)
#  (u'@5raphaels', u'unless', 11) (u'@5raphaels', u'you', 11)]

answered Aug 2, 2011 at 11:21

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

gdrt Over a year ago

This, unfortunately, throws UnicodeDecodeError: 'charmap' codec can't decode byte...

Collectives™ on Stack Overflow

how do i import from a unicode (utf-8) csv file into a numpy array

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related