1

I'm fairly new to python and I would like to know if I could get some assistance with the problem I'm trying to solve:

I would like to design a loop to iterate over every file in the directory and place the data into a 2D array for every file. I have a large directory of .txt files that contain 22 lines of 2 numbers per line.

An example of how the files contents will be organized is:

# Start of file_1.txt
1 2
3 4
5 6
7 8

# Start of file 2.txt
6 7
8 9
3 4
5 5

I would like to read the data separated by whitespace into the first two reference locations in the array(i.e. array = [x0][y0]) , and at the newline, write the following data into the next location of the array (i.e. array=[x1][y2]). I see a lot of people saying to use numpy, scipy, and other methods but that is confusing me further.

The output that I am seeking is:

[[1,2],[3,4],[5,6],[7,8], ...]

I'm bit stuck on how to iterate through the files in a directory and simultaneously place them into a 2D array. The code I have so far is:

import os
trainDir = 'training/'
testDir = 'testing/'
array2D = []

for filename in os.listdir(trainDir,testDir):
    if filename.endswith('.txt'):
        array2D.append(str(filename))

print(array2D)

At the moment the above code does not work for two directories, but it does for one. Any help would be appreciated.

Improve this question
5
  • 1
    Have you made any attempt to code yet? Commented Jul 14, 2018 at 18:15
  • I have, I'm looking though a few things online and trying to get a better understanding. Commented Jul 14, 2018 at 18:15
  • You should post your code here so we can fix it. Commented Jul 14, 2018 at 18:17
  • Please excuse my formatting Commented Jul 14, 2018 at 18:22
  • Edit your question and paste your code there. Dont forget to format your code (with Ctrl+K for example)! Commented Jul 14, 2018 at 18:24

3 Answers 3

Reset to default
4

You are defining your array2D wrong at the beginning, that's not valid Python syntax. The following code should work:

import os

d = 'HERE YOU WRITE YOUR DIRECTORY'

array2D = []

for filename in os.listdir(d):
    if not filename.endswith('.pts'):
        continue

    with open(filename, 'r') as f:
        for line in f.readlines():
            array2D.append(line.split(' '))

print(array2D)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

@user3600460 make sure you have paths set correct, you can debug them with print function for example
There may be something wrong with the above code segment: with open(filename, 'r') as f: for line in f.readlines(): array2D.append(line.split(' ')) As I get a filename not found error.
1

I would advise running the python script in the same directory as the files you wish to read, for the sake of simplicity. Otherwise you will have to define a path to the directory that contains the files.

Also, I'm not sure if only you will be using this program, but it might be a good idea to define a try-except block around the FileIO portion of the code to prevent the program from crashing if it is unable to read from a file for any reason.

The below code reads all files in the directory that houses the python script and creates a 2D list of the files' contents (also, it explicitly rebuilds the 1D lists in a manner that ensures you have a list of integers rather than strings): 

import os

output_2d_list = []
current_working_directory = os.path.abspath('.')

# Iterates over all files contained within the same folder as the python script.
for filename in os.listdir(current_working_directory):
    if filename.endswith('.pts'):

        # Ensuring that if something goes wrong during read, that the program exits gracefully.
        try:
            with open(filename, 'r') as current_file:

                # Reads each line of the file, and creates a 1d list of each point: e.g. [1,2].
                for line in current_file.readlines():
                    point = line.split(' ')
                    x = int(point[0])
                    y = int(point[1])
                    point_as_array = [x, y]
                    output_2d_list.append(point_as_array)
        except IOError:
            print "Something went wrong when attempting to read file."

# For testing purposes to see the output of the script.
# In production, you would be working with output_2d_list as a variable instead.
print output_2d_list
Improve this answer

1 Comment

Thank you for the input! I will test this code with my data. I like the aspect of the added try-except for the file I/O.
-3

Simply do the following:

f = open("file_1.txt", "r")

x = [line.split() for line in f]    
f.close()
Improve this answer

3 Comments

The question was "I would like to design a loop to iterate over every file in the directory and place the data into a 2D array for every file."
Note to reviewers: Blatantly wrong answers are not Very Low Quality. Even answers which misunderstood the question or ignore its explicit constraints are not Very Low Quality. The correct review on this is "Looks OK".
@EJoshuaS-StandwithUkraine but what if an answer is both? This manages to not only be blatantly wrong, but it also lacks any form of context or explanation. And the only question it does answer is already answered elsewhere

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.