2

I was wondering if anybody knew how I could change a script in Python so it goes through a folder containing csv files and takes them in groups of three. The script is working when I type the file names in the command line, but I've got lots of files, so that would take forever. It looks like this now:

resultsdir = "blah"

#filename1=sys.argv[1]
#filename2=sys.argv[2]
#filename3=sys.argv[3]

file1 = open(resultsdir+"/"+filename1+".csv")
file2 = open(resultsdir+"/"+filename2+".csv") 
file3 = open(resultsdir+"/"+filename3+".csv")

I'm a complete beginner, I hope I've been able to explain what I want. Cheers for any help!

Improve this question
6
  • 1
    Can you elaborate on your question? I see 3 files being opened, that's all. If you want to loop through the csv file and open them, why not do them one by one? Why three at a time? Commented Apr 29, 2013 at 18:17
  • I have a folder with all the csv files and instead of just taking 3 I want to use the script on all of them. I don't mean looping through the csv files themselves, and I want to use groups of three, but not only once. Commented Apr 29, 2013 at 18:21
  • Does the folder contain only the csv files? And do you select those 3 files arbitrarily? Commented Apr 29, 2013 at 18:21
  • Yes, it only contains the csv files. I would like to select the first three together, then the second three and so on, if that is possible. Cheers! Commented Apr 29, 2013 at 18:23
  • How do you distinguish the first three from the second three? Are they named such that sorting the list of filenames will put them in the right order? [What I'm getting at is that the filenames are likely to be returned in an arbitrary order, so we'll need a way to group them to get the order you want.] Commented Apr 29, 2013 at 18:27

2 Answers 2

Reset to default
7

You could use the glob module (http://docs.python.org/3.3/library/glob.html) to get all .csv files in a directory and open them then.

Example:

import glob
resultsdir = "blah"

files = sorted(glob.glob(resultsdir+'/*.csv'))
while len(files) >= 3:
     file1 = open(files.pop(0))
     file2 = open(files.pop(0))
     file3 = open(files.pop(0))
     # Do something
# if the number of files can't be divided by 3 do something
# with the 1 or 2 files which are left

Edit: Changed files.pop() to files.pop(0) to get the files from the first to the last and not from the last to the first file.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I've already made sure that the directory only contains the csv files, I really only need to be sure it takes the first three files together and so on. It's probably very simple... Thank you for your help though!
They are sorted, first three go together and so on.
Using glob seems to be the simplest solution. I have now added sorted(), so that the list of files will be sorted. When using glob it also make no difference if there are some non csv files in the directory. For other ways you could look at os.listdir or os.walk, but they look more complicated.
1

If all you want is to group by three the elements of a list, here is an example of code that does it:

import itertools


def groupby_three(iterable):
    # x[0] is the index of the scanned element in the input list
    for _, values in itertools.groupby(enumerate(iterable),
                                       lambda x: x[0] / 3):
        yield([y[1] for y in values])

# Group by 3 the integers from 10 to 19
for x in groupby_three(xrange(10, 20)):
    print x

Output:

[10, 11, 12]
[13, 14, 15]
[16, 17, 18]
[19]
Improve this answer

1 Comment

Basically looks like a workable idea, but doesn't address the issue of going through a folder containing csv files and taking them in groups of three.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.