Python eval() SyntaxError when reading string with punctuation

Question

I am trying to read a file with lines that look like this:

1A11:A,GSEKMSTAISVLLAQAVFLLLTSQR,0.724,0,0,1,0,0,1.000,225.000,0.004,0.915

With this code:

for line in datafile:
    line = line.split(',')
    for element in line:
        line[count] = eval(element)

And I get the following error:

    "/Users/Auyer/CODING/PycharmProjects/INSERTGenerator/InsertGen.py", line 10, in fileSplitter
line[count] = eval(element)
File "<string>", line 1
1A11:A
   ^
SyntaxError: invalid syntax

The program works perfectly when it evaluates a number, but won't work when it does with a Punctuated String.

EDIT 1: The desired output was to have the numeric strings stored as numbers, and the strings as strings. Example: the list 'Line' would now look like this:

['1AFP:A', 'KYKKDNIKQSGKICKYVKKPRDAKFSYKGKYC', -1.494, 0, 1, 0, 0, 0, 9.000, 105.000, 0.021, 0.242]

What are you trying to do? What is the desired output? Where is count variable coming from? — alecxe
– alecxe, Commented Jun 1, 2016 at 2:45
Before you go on coding, STOP AND READ THIS. There is almost never a good time to use eval(). There are much safer alternatives available, like ast.literal_eval(). — MattDMo
– MattDMo, Commented Jun 1, 2016 at 3:04

MattDMo · Accepted Answer · 2016-06-01 04:10:03Z

2

As I mentioned in my comment above, using eval() is a pretty awful idea, and lots of people much smarter and more experienced than me agree, so I go with it, and you should too.

First, we are assuming that each line has the same structure and the same number of elements with the same transformation requirements (string, int, float). How much work you put into parsing these lines depends on what you need to do with the results. If you don't mind values like 0 and 1 being floats, the parsing is easy:

for line in inputfile:
    data = line.split(",") # make list
    newdata = data[0:2] + [float(item) for item in data[2:]] 
    # pull in 1st two elements, which are strings
    # then call float() on the rest of them, and add them to new list

You are now free to use newdata as you wish.

If the 0, 0, 1, 0, 0 values in the middle of the list need to remain integers, the parsing takes two more lines, but is still straightforward:

for line in inputfile:
    data = line.split(",")
    newdata = data[0:2] + [float(data[2])]
    newdata += [int(item) for item in data[3:8]]
    newdata += [float(item) for item in data[8:]]

Here we're using list comprehensions to tackle slices of the data list - add the first two strings, then call float() on the 3rd item, then call int() on the 4th through 7th, then float() again on the rest.

answered Jun 1, 2016 at 4:10

MattDMo

103k21 gold badges251 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Auyer Over a year ago

This is really good, but I want to be able to read other files as well, with different structures. Like [String, float, string, string, int]. Thats why I wanted the program to find out what data type it was looking at.

MattDMo Over a year ago

@RafaelAuyer You can't do that. Python can't take an arbitrary string and automatically determine what type it should be cast to, if any. "1" is a perfect example - in your case, it could be cast to int or float successfully. Without a structure, a program can't just guess and know which one is required for this particular circumstance. You need additional logic someplace.

Janarth K · Accepted Answer · 2016-06-01 02:48:42Z

0

eval(string) evaluates an expression. So the string that you pass in has to evaluate to a value. When Python tries to evaluate 1A11:A it hits an error as expected. To both Python and human readers that string looks like junk. An acceptable input string would be something like x+1 where x is already defined previously in the script.

An example using eval:

>>> x = 1
>>> eval('x + 1')
2

answered Jun 1, 2016 at 2:48

Janarth K

1526 bronze badges

Comments

Ani Menon · Accepted Answer · 2016-06-01 02:58:38Z

0

eval() returns the evaluated result of whatever string is passed to it.

In your case "1A11:A" can't be evaluated to a values. Even if you remove that element what about the second element is it a variable? else that also would return an error.

answered Jun 1, 2016 at 2:58

Ani Menon

28.4k17 gold badges111 silver badges128 bronze badges

Comments

ralphie boy · Accepted Answer · 2017-07-03 04:42:20Z

0

I used eval in a unit test program. I wanted to pass a tuple (96,6,6,x) to an object. When x was set equal to '\"'+'9:30'+'\"', the equivalent to a hard-coded (96,6,6,"9:30"), it would never pass that value through. But without changing anything else, passing (96,6,6,(eval x)) worked. All I'm trying to do is run my unit test to exercise the object that will actually be used in a production program. I didn't want to spend a long time figuring out why the unit test itself was broken, although the object is working fine for its actual real-world scenario. And so is the unit test, if I code it that way.

I think if you consider eval as a "rhs" tool, you will have no problem. Meaning, you cannot write eval("x=7") and expect eval to create the variable x and set it to 7. Any characters like that '=' could stray outside what it can do.

P.S. To other newbies. Think before you post on Stack Overflow. Check what eval is supposed to do before you think you are stumped. In this case, a quick direct python interpreter session, i.e.:

eval("1A11:A") Traceback (most recent call last): File "", line 1, in File "", line 1 1A11:A ^ SyntaxError: invalid syntax

edited Jul 3, 2017 at 4:42

answered Jul 3, 2017 at 4:24

ralphie boy

696 bronze badges

1 Comment

ralphie boy Over a year ago

Also remember you can use try: except: blocks if the data are in a sane domain. If it's numbers with the occasional 1A11:A, you can handle the exception. Be rigorous and put the exact exception type, and check if it's always in the first delimited group of characters on the line, etc.

Collectives™ on Stack Overflow

Python eval() SyntaxError when reading string with punctuation

4 Answers 4

2 Comments

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related