How to export array in csv or txt in Python

Question

I'm trying to export array to txt or csv file. I've been trying with numpy but i always get some error like TypeError: Mismatch between array dtype ('<U14') and format specifier ('%.18e')

Here is my code without numpy that works great but I need help with part how to export it.

peoples = []
for content in driver.find_elements_by_class_name('x234'):
    people = content.find_element_by_xpath('.//div[@class="zstrim"]').text
    if people != "Django" and people != "Rooky" :
        pass
        peoples.append([people, 1, datetime.now().strftime("%d/%m/%y %H:%M")])
print(peoples)

Really need some help with this.

why you trying to use numpy (linear algebra library) instead of python standart library for reading and writing csv? — Arnial
– Arnial, Commented Jun 22, 2016 at 21:42
@Arnial how? do you have code suggestion how to do that on your way? — PythonMan
– PythonMan, Commented Jun 22, 2016 at 21:48

hpaulj · Accepted Answer · 2016-06-22 22:21:41Z

1

Looks like you are doing something like:

In [1339]: peoples=[]

In [1340]: for _ in range(3):
   ......:     peoples.append([234, datetime.datetime.now().strftime("%d/%m/%y %H:%M")])
   ......:     

In [1341]: peoples
Out[1341]: [[234, '22/06/16 14:57'], [234, '22/06/16 14:57'], [234, '22/06/16 14:57']]

peoples is an array (or here a list of lists), that contains, among other things formatted dates.

In [1342]: np.savetxt('test.txt',peoples)
...    
TypeError: Mismatch between array dtype ('<U14') and format specifier ('%.18e %.18e')

Since I didn't specify fmt it constructed a default one, consisting of two %.18e fields. That's great for general formatting of numbers. But the data includes 14 characters strings ('U14' - unicode in Python3).

If I tell it to use %s, the generic string format, I get:

In [1346]: np.savetxt('test.txt',peoples, fmt='%s', delimiter=',')

In [1347]: cat test.txt
234,22/06/16 14:57
234,22/06/16 14:57
234,22/06/16 14:57

Not ideal, but still it works. fmt='%20s' would be better.

I glossed over a another nuance. peoples is a list of lists. np.savetxt works with arrays, so it first turns that into an array with:

In [1360]: np.array(peoples)
Out[1360]: 
array([['234', '22/06/16 14:57'],
       ['234', '22/06/16 14:57'],
       ['234', '22/06/16 14:57']], 
      dtype='<U14')

But this turns both columns into U14 strings. So I have to format both columns with %s. I can't use a numeric format on the first. What I need to do first is make a structured array with a numeric field(s) and a string field. I know how to do that, but I won't get into the details now.

As per comments, it could be simpler to format each peoples line as a complete string, and write that to a file.

In [1378]: with open('test.txt','w') as f:
    for _ in range(3):
        f.write('%10d,%20s\n'%(234, datetime.datetime.now().strftime("%d/%m/%y %H:%M")))
   ......:         

In [1379]: cat test.txt
       234,      22/06/16 15:18
       234,      22/06/16 15:18
       234,      22/06/16 15:18

edited Jun 22, 2016 at 22:21

answered Jun 22, 2016 at 22:01

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

PythonMan Over a year ago

Ok this helps a lot, because now I understand where is the problem. I used your last part of code, surf more a bit and find solution thanks to you. So here is the final result that will append all every time it runs I added this after pass ....

mybook = open('archive.csv', 'a')                     mybook.write('%20s,%20s\n'%(people, datetime.now().strftime("%d/%m/%y %H:%M")))                     mybook.close()

Community · Accepted Answer · 2017-05-23 12:14:44Z

1

hpauj's answer explains you why your code error but using the csv lib and writing as you go is probably a lot easier:

import csv

with open("out.csv", "w") as f:
    wr = csv.writer(f)
    for content in driver.find_elements_by_class_name('x234'):
        people = content.find_element_by_xpath('.//div[@class="zstrim"]').text
        if people != "Django" and people != "Rooky":
            wr.writerow([people, 1, datetime.now().strftime("%d/%m/%y %H:%M")])

edited May 23, 2017 at 12:14

CommunityBot

11 silver badge

answered Jun 22, 2016 at 22:17

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

1 Comment

PythonMan Over a year ago

yeah csv is much simpler. Thanks for this code it helped a lot! But I must admit hpauj's answer helped me to understand how it works.

Collectives™ on Stack Overflow

How to export array in csv or txt in Python

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related