Reading multiple arrays in a text file in Python

Question

I am trying to read multiple arrays saved in .txt file. I present the data in Test.txt file as well as the current and expected outputs.

import re
import numpy as np
import ast

with open('Test.txt') as f:
    s = f.readlines()
    #print(s)
    s = ' '.join(s)
    s  = re.findall("\((\[[\w\W]*\])\)",s)
    s=ast.literal_eval(s[0])
    s=np.array(s)
    print([s])

The data in Test.txt is

[array([[1.7],
       [2.8],
       [3.9],
       [5.2]])]
[array([[2.1],
       [8.7],
       [6.9],
       [4.9]])]

The current output is

line 4
    [5.2]])]
          ^
SyntaxError: unmatched ')'

The expected output is

[array([[1.7],
       [2.8],
       [3.9],
       [5.2]])]
[array([[2.1],
       [8.7],
       [6.9],
       [4.9]])]

eval(' '.join(f.readlines()).replace('array', 'np.array').replace(')]', ')],')) should do the job while not being very robust. Why not using npy files or hdf5 for that which are specifically design for such use case and avoid such AST/eval ugly inefficient tricks? — Jérôme Richard
– Jérôme Richard, Commented Sep 10, 2022 at 9:58
This yields ([array([[1.7], [2.8], [3.9], [5.2]])],) which is not the expected output. — user19862793
– user19862793, Commented Sep 10, 2022 at 10:04
Id did not test sorry. A replace('\n', '') was missing. Still, it create a big array and not many one because np.array cannot create the expected result in one call. You need to parse each array separately and call np.array on each and then add each array to a list with append. Regarding the output, it looks like you want an array of shape (n, 1) containing object arrays of shape (m,1). Object-based array are discouraged as they are inefficiently supported (and are not really useful compared to lists here). Please clarify the output type. — Jérôme Richard
– Jérôme Richard, Commented Sep 10, 2022 at 10:27

Shahab Rahnama · Accepted Answer · 2022-09-10 12:28:05Z

1

You can use this:

import re
import ast

s = '''
    [array([[1.7],[2.8],[3.9],
       [5.2]])]
[array([[2.1],
       [8.7],
       [6.9],
       [4.9]])]
'''

s = s.replace('\n', '')
s = s.replace(' ', '')
s = s[1:-1]
s = re.findall("\((\[.*?\])\)",s)

result= []

for i in s:
    result.append(ast.literal_eval(i))
    
print(result)

Output:

[[[1.7], [2.8], [3.9], [5.2]], [[2.1], [8.7], [6.9], [4.9]]]

answered Sep 10, 2022 at 12:28

Shahab Rahnama

1,0321 gold badge8 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reading multiple arrays in a text file in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related