Python: add column to CSV file based on existing column

Question

I already have written what I need for identifying and parsing the value I am seeking, I need help writing a column to the csv file (or a new csv file) with the parsed value. Here's some pseudocode / somewhat realistic Python code for what I am trying to do:

# Given a CSV file, this function creates a new CSV file with all values parsed
def handleCSVfile(csvfile):
  with open(csvfile, 'rb') as file:
    reader = csv.reader(file, delimiter=',', lineterminator='\n')
    for row in reader:
        for field in row:
          if isWhatIWant(field):
            parsedValue = parse(field)
            # write new column to row containing parsed value

I've already written the isWhatIWant and parse functions. If I need to write a completely new csv file, then I am not sure how to have both open simultaneously and read and write from one into the other.

Michael Hoffman · Accepted Answer · 2011-10-05 04:01:01Z

3

I'd do it like this. I'm guessing that isWhatIWant() is something that is supposed to replace a field in-place.

import csv

def handleCSVfile(infilename, outfilename):
    with open(infilename, 'rb') as infile:
        with open(outfilename, 'wb') as outfile:
            reader = csv.reader(infile, lineterminator='\n')
            writer = csv.writer(outfile, lineterminator='\n')

            for row in reader:
                for field_index, field in enumerate(row):
                    if isWhatIWant(field):
                        row[field_index] = parse(field)
                writer.writerow(row)

This sort of pattern occurs a lot and results in really long lines. It can sometimes be helpful to break out the logic from opening and files into a different function, like this:

import csv

def load_save_csvfile(infilename, outfilename):
    with open(infilename, 'rb') as infile:
        with open(outfilename, 'wb') as outfile:
            reader = csv.reader(infile, lineterminator='\n')
            writer = csv.writer(outfile, lineterminator='\n')

            read_write_csvfile(reader, writer)

def read_write_csvfile(reader, writer)
    for row in reader:
        for field_index, field in enumerate(row):
            if isWhatIWant(field):
                row[field_index] = parse(field)
        writer.writerow(row)

This modularizes the code, making it easier for you to change the way the files and formats are handled from the logic independently from each other.

Additional hints:

Don't name variables file as that is a built-in function. Shadowing those names will bite you when you least expect it.
delimiter=',' is the default so you don't need to specify it explicitly.

edited Oct 5, 2011 at 4:01

answered Oct 5, 2011 at 1:30

Michael Hoffman

34.8k7 gold badges68 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dan Over a year ago

This makes sense. I'm only confused on one part. My function parse only receives the value it is looking for and returns the parsed value. It doesn't do any file handling. isWhatIWant just returns true or false depending on if the field is what I'm looking for. I'm not sure how to make this work with this model....

Michael Hoffman Over a year ago

This works better if you can give a complete minimal example, including a definition of these functions (a stub definition is fine). The code I wrote assumes that isWhatIWant() returns something that can be interpreted as True or False and that parse() will take one field value and return another. What part of this can we help you understand?

Dan Over a year ago

Never mind, this works great. The only problem is that the outfilename needed w+b permissions, rwb was invalid, that was the only issue. Otherwise this works perfectly. Thanks,

Michael Hoffman Over a year ago

My mistake there. "wb" should be fine too. I'm going to edit the answer.

Collectives™ on Stack Overflow

Python: add column to CSV file based on existing column

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related