Convert a text file to a numpy array

Question

I am trying to convert a list of strings into an array. The list is really an array of numbers that is n rows long by 4 columns that I took from a text file. I need to convert this list to an array that is n rows by 4 columns and is floating type. Below is my code so far:

#Calculate the average velocity through a tidal cycle

from pylab import *
import numpy as np

#Open profile1.ele and get the data
lookup = '##'
data = []
eleline = []

with open('profile1.ele') as f:
for line in f:
    if not line.startswith(lookup):   #disclude lines with '##'
        data.append(line.rstrip("\r\n"))  
    if 'Elements' in line:
        eleline.append(line)
s = ''.join(eleline)   #Convert list to string
numele = s.rsplit()[-2] #Grab # of elements
numele = int(numele) #convert to integer

#Convert data list object into an array
elements = np.asarray(data)

This is where I am having problems. The resultant array is a 1D array with all the information from each line in the list jumbled together.

Here is an example of the input file format.

## =============================================================================
## TIME STEP      1         Duration:  6.0000E+02 sec      Time:  3.4712E+09 sec
## =============================================================================
##      X origin       Y origin     X velocity     Y velocity
   3.1225530E-01 -9.5153722E+00  4.8239441E-09 -1.1614215E-08
   4.0205122E-01 -8.5404981E+00  1.7396887E-09 -1.8665899E-08
   4.3224251E-01 -7.5565436E+00  2.0985602E-09 -2.5349955E-08
   4.3234870E-01 -6.5693932E+00  1.7166213E-09 -3.1156361E-08
   4.2276193E-01 -5.5905580E+00  1.9627062E-09 -3.7317066E-08
   4.0245047E-01 -4.6585868E+00  1.7305504E-09 -4.3153198E-08
   3.6284562E-01 -3.8494609E+00  1.7422198E-09 -4.8249619E-08
   3.1234937E-01 -3.1767707E+00  1.9901861E-09 -5.3221055E-08
   2.6726067E-01 -2.5743939E+00  1.9799420E-09 -5.8343627E-08
   2.2791616E-01 -2.0380240E+00  1.7150138E-09 -6.3250542E-08
   1.8285348E-01 -1.5997592E+00  9.9428594E-10 -6.7249257E-08

I want an array (11 x 4) of this data as a floating point. Example:

   3.1225530E-01 -9.5153722E+00  4.8239441E-09 -1.1614215E-08
   4.0205122E-01 -8.5404981E+00  1.7396887E-09 -1.8665899E-08
   4.3224251E-01 -7.5565436E+00  2.0985602E-09 -2.5349955E-08
   4.3234870E-01 -6.5693932E+00  1.7166213E-09 -3.1156361E-08
   4.2276193E-01 -5.5905580E+00  1.9627062E-09 -3.7317066E-08
   4.0245047E-01 -4.6585868E+00  1.7305504E-09 -4.3153198E-08
   3.6284562E-01 -3.8494609E+00  1.7422198E-09 -4.8249619E-08
   3.1234937E-01 -3.1767707E+00  1.9901861E-09 -5.3221055E-08
   2.6726067E-01 -2.5743939E+00  1.9799420E-09 -5.8343627E-08
   2.2791616E-01 -2.0380240E+00  1.7150138E-09 -6.3250542E-08
   1.8285348E-01 -1.5997592E+00  9.9428594E-10 -6.7249257E-08

Please post the content of file and expected output. Thanks. — Ashwini Chaudhary
– Ashwini Chaudhary, Commented Apr 10, 2014 at 17:40
Sure thing. I added an example of what the input file looks like and what I need to get. — Tbevans88
– Tbevans88, Commented Apr 10, 2014 at 17:46
I don't see how your "Elements"-related code connects to the file you've shown. — DSM
– DSM, Commented Apr 10, 2014 at 17:50
its higher up in the file. It just gets me the # of elements. That part works fine. — Tbevans88
– Tbevans88, Commented Apr 10, 2014 at 17:50

logc · Accepted Answer · 2014-04-10 17:55:09Z

9

Probably you just need the numpy.loadtxt function: http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html#numpy.loadtxt

It worked for me on the input you posted:

In [1]: import numpy as np

In [2]: a = np.loadtxt('example.csv')

In [3]: a
Out[3]:
array([[  3.12255300e-01,  -9.51537220e+00,   4.82394410e-09,
         -1.16142150e-08],
       [  4.02051220e-01,  -8.54049810e+00,   1.73968870e-09,
         -1.86658990e-08],
       [  4.32242510e-01,  -7.55654360e+00,   2.09856020e-09,
         -2.53499550e-08],
       [  4.32348700e-01,  -6.56939320e+00,   1.71662130e-09,
         -3.11563610e-08],
       [  4.22761930e-01,  -5.59055800e+00,   1.96270620e-09,
         -3.73170660e-08],
       [  4.02450470e-01,  -4.65858680e+00,   1.73055040e-09,
         -4.31531980e-08],
       [  3.62845620e-01,  -3.84946090e+00,   1.74221980e-09,
         -4.82496190e-08],
       [  3.12349370e-01,  -3.17677070e+00,   1.99018610e-09,
         -5.32210550e-08],
       [  2.67260670e-01,  -2.57439390e+00,   1.97994200e-09,
         -5.83436270e-08],
       [  2.27916160e-01,  -2.03802400e+00,   1.71501380e-09,
         -6.32505420e-08],
       [  1.82853480e-01,  -1.59975920e+00,   9.94285940e-10,
         -6.72492570e-08]])

edited Apr 10, 2014 at 17:55

answered Apr 10, 2014 at 17:47

logc

3,9331 gold badge20 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tbevans88 Over a year ago

The eleline part is not the problem. That works fine. The problem is with the data list object.

Tbevans88 Over a year ago

Yes, I bet that will work. Trying asap. EDIT: Yep this worked! Thanks, numpy is so smart!

Collectives™ on Stack Overflow

Convert a text file to a numpy array

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related