Python reading data file into lists or arrays

Question

For some reason, I decided to save my list of data as a string. If I use

f = open('tarfile.txt')
for line in f.readlines():
    print line
f.close()

my output looks like:

[  53.7775   13.4375   26.525    48.63    125.      185.      653.    ]    
[  53.7775    13.33625   26.73375   48.68375  125.       185.       653.     ]    
[  53.7775    13.325     27.11375   48.8875   126.       187.       653.     ]    
[  53.7775    13.43625   27.3175    48.92875  126.       187.       653.     ]    
[  53.7775    14.4825    33.07375   51.7325   141.       202.       595.     ]

I would like to read this data in to 2D array. I have searched and tried various methods such as pickle, eval, json, etc but nothing worked

Alok Singhal · Accepted Answer · 2012-10-23 13:53:13Z

7

If you are using numpy (or don't mind using it), you can do numpy.loadtxt('tarfile.txt', usecols=range(1,8)). It is particularly nice if you are going to want your data in a 2-d numpy array anyway.

answered Oct 23, 2012 at 13:53

Alok Singhal

96.8k21 gold badges130 silver badges158 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1532056 Over a year ago

Thank you very much. Exactly what I needed. Worked flawlessly.

mgilson · Accepted Answer · 2012-10-23 13:47:17Z

2

First, you need to translate the [] to something python can handle:

import string
table = string.maketrans('[]','  ')

Now you can iterate through your file, translating, splitting and floating:

for line in f:
    print [float(x) for x in line.translate(table).split()]

If you're guaranteed that [ is the first character and ] is the last character on the line, you can do it with slicing:

print [float(x) for x in line[1:-2].split()]  #-2 accounts for the newline too.

answered Oct 23, 2012 at 13:47

mgilson

312k70 gold badges656 silver badges722 bronze badges

Comments

score 2 · Accepted Answer · 2012-10-23 13:49:21Z

2

My guess is that your input file contains floats separated by whitespace.

To read such a file, strip the [ and ], split each line, and map each field to a float instance.

records = []
for line in f:
    record = [float(field) for field in line.strip().lstrip('[').rstrip(']').split()]
    records.append(record)

edited Oct 23, 2012 at 13:49

answered Oct 23, 2012 at 13:44

user647772

Comments

Emmanuel · Accepted Answer · 2012-10-23 13:52:56Z

1

I think regexp is the best way to parse your data:

import re
pattern = r'\d+.\d*'
array = []
for line in open('tarfile.txt'):
    array.append(re.findall(pattern, line))

answered Oct 23, 2012 at 13:52

Emmanuel

14.2k12 gold badges53 silver badges73 bronze badges

1 Comment

Carl Christian Over a year ago

This was clear and helpful and answered my array question as well.

Collectives™ on Stack Overflow

Python reading data file into lists or arrays

4 Answers 4

1 Comment

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related