Convert 2-d array of strings to float in python_removing scientific notation

Question

How can I convert scientific notations to the original format when I use numpy.astype in x array? Here's my code:

with open ('outfile.csv', 'r') as infile:
    reader = csv.reader(infile)
    reader_list = list(reader)
    reader_array = array(reader_list)
    x = reader_array[:,5].astype(np.float)

    #original array:
    print reader_array[:,5]

    #converted to float
    print x

#original array:
['-0.00041955436132607246' '-0.00036612800229292086'  '0.00022313364860991641' ..., '73.418371245304215' '73.417384428365267'  '73.416718169781149'] 

#converted to float
[ -4.19554361e-04  -3.66128002e-04   2.23133649e-04 ...,   7.34183712e+01    7.34173844e+01   7.34167182e+01]

To be more specific, I want to convert array of strings to floats, but keep the same format as the original array, and do some analysis on it:

#find row number of max value in column 1: (This piece works fine)
max_index = where(reader_array[:,1] == max(reader_array[:,1]))

#take last element in column 5: (This one is also fine)
total_ = (reader_array[(len(reader_array[:,5])-1),5])

#find row number where element in column 5 is equal to 0.1*total_: (here's the problem!)
0.1_index = where((reader_array[:,5]) == (total_)*0.1)

So I think changing the strings to floats but with the same format as the original array allows multiplying array members by another float (0.1 here).

Please note that the value (0.1*total_) may not match any of the rows values in column 5, which I have to think how to solve. But I cannot progress without being able to compare rows with (0.1*total_).

I appreciate if someone can give a hint how to approach please.

This is more a print problem, your question is probably answered here: How to force a ndarray show in normal way instead of scientific notation? — Saullo G. P. Castro
– Saullo G. P. Castro, Commented Nov 29, 2013 at 9:02
Numpy is not enough because float64 only has ~15 significant digits, and your numbers have 17 significant digits. You can use Decimal, or SymPy — Gustav Bertram
– Gustav Bertram, Commented Nov 29, 2013 at 16:02
This isn't a duplicate, the OP has issues like type differences. The question has nothing to do with printing per se (though it does sound like that, the printing issue is just a manifestation of a misunderstanding). — Henry Gomersall
– Henry Gomersall, Commented Nov 29, 2013 at 16:16
@SaulloCastro, Well, not really, I had already tried printing options but that was not my question here. As Henry Gomersall pointed, my problem was with types. I may need to add more clarification for my later questions. Thanks. — PyLearner
– PyLearner, Commented Dec 1, 2013 at 20:20

Henry Gomersall · Accepted Answer · 2013-11-29 15:43:08Z

1

You're intrinsically limited by the fact that floating point numbers are stored using IEEE 754. You can't have arbitrary precision floating points, so in your case, you can't expect them to necessarily be exactly the same as a string representation.

However, in your case, the more pressing issue is that want to compare a string to a float, so of course they are going to be different. Python is dynamically, but strongly typed.

Given both the above points, you need to better define your problem. Why do you need to compare with an array of strings? (what does this even mean!?)

Can you test for closeness rather than equality once you have your data types sorted out (e.g. using numpy.close)?

edited Nov 29, 2013 at 15:43

answered Nov 29, 2013 at 15:36

Henry Gomersall

8,7423 gold badges37 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

PyLearner Over a year ago

Thanks a lot for your cm. Hinting 'closeness' made me think about that and finally found the solution using defining a function to find the nearest point.

Collectives™ on Stack Overflow

Convert 2-d array of strings to float in python_removing scientific notation

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related