csv empty strings handling and values appending

Question

With a csv of ~50 rows (stars) and ~30 columns (name, magnitudes and distance), that has some empty string values (''), I am trying to do two things in which all the help so far hasn't been useful. (1) I need to parse empty strings as 0.0, so I can (2) append each row in a list of lists (what I called s).

In other words: - s is a list of stars (each one has all its parameters) - d is a particular parameter for all the stars (distance), which I obtain correctly.

Big issue is with s. My try:

with open('stars.csv', 'r') as mycsv:
    csv_stars = csv.reader(mycsv)
    next(csv_stars) #skip header
    stars = list(csv_stars)
    s = [] # star
    d = [] # distances
    for row in stars:
        row[row==''] = '0'
        s.append(float(row)) #stars
        d.append(arcsec*AU*float(row[30]))

I can't think of a better syntax, and so I get the error

s.append(float(row)) # stars TypeError: float() argument must be a string or a number

From s I would obtain later the magnitudes for all the stars, separately. But first things first...

Hi @cwasdwa, it looks like you are trying to convert array/row object to float. what is the result you get if you do print (type(row)) just before calling s.append(float(row)) #stars — just10minutes
– just10minutes, Commented Jul 9, 2017 at 0:43
@just10minutes I am getting <type 'list'>. With that syntax it does not convert the ' 'into 0.0, but it does convert the first element in each star (which is a name, a string like 'J869423') into 0.0, which is not even empty. To do this syntax I got some help from stackoverflow.com/questions/15936732/… — cwasdwa
– cwasdwa, Commented Jul 9, 2017 at 1:03
Since it is a list you cannot convert to float. I hope we are on same page. Are you expecting "s" to have all 30 column values? — just10minutes
– just10minutes, Commented Jul 9, 2017 at 1:11
Yes, I do get them just fine, a list of lists as expected. I think the problem is with row[row==''] = '0', and I should specify each element of row instead of just row. — cwasdwa
– cwasdwa, Commented Jul 9, 2017 at 1:14

just10minutes · Accepted Answer · 2017-07-09 01:44:53Z

0

@cwasdwa Please look at below code. it will give you an idea. I am sure there might be better way. This solution is based on what I have understood from your code.

with open('stars.csv', 'r') as mycsv:
csv_stars = csv.reader(mycsv)
next(csv_stars) #skip header
stars = list(csv_stars)
s = [] # star
d = [] # distances
for row in stars:        
    newRow = [] #create new row array to convert all '' to 0.0
    for x in row:
        if x =='':
            newRow.append(0.0)
        else:
            newRow.append(x)
    s.append(newRow) #stars

    if row[30] == '':
        value = 0.0
    else:
        value = row[30]
    d.append(arcsec*AU*float(value))

answered Jul 9, 2017 at 1:44

just10minutes

61113 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

cwasdwa Over a year ago

Thank you, I have just tested it, works fine except that s is a list of 1 element (the last star). I am working on solving this from here. Btw, I managed to solve (2) in my question with m = [list() for _ in range(len(stars))] ... m[n].append(float(s[n][i])).

just10minutes Over a year ago

Can you Post a content of your stars.csv? may be it will be helpful. When I tested I do get all the rows under s

cwasdwa Over a year ago

Yes Sir, I copy here the first 2 lines (because of space limitation), you can copy a few times the second row:

#J,Name,m1,m2,m3,m4,m5,m6,m7,m8,m9,m10,m11,m12,m13,m14,m15,m16,m17,m18,m19,m20,m21,m22,m23,m24,m25,m26,m27,m28,d_pc,SpT,SpT_num,Class J1,Name1,1.18,4.09,6.45,3.00,1.00,5.00,1.55,7.91,,8.91,,7.36,8.55,7.00,5.18,7.45,1.09,1.91,0.91,6.91,6.64,7.82,8.64, ,4.00,4.55,3.398,0.023,6.30,M0.0 V,0,a

cwasdwa Over a year ago

It was the indentation, I had in the 5th and it was 4th...:). Your code above saved my day, ty!

just10minutes Over a year ago

I guessed so :), Was about to test once again. Thanks for accepting the answer

Collectives™ on Stack Overflow

csv empty strings handling and values appending

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related