Parsing CSV files using Python 2.7

Question

I'm trying to write a script that will open a CSV file and write rows from that file to a new CSV file based on the match criteria of a unique telephone number in column 4 of csv.csv. The phone numbers are always in column 4, and are often duplicated in the file, however the other columns are often unique, thus each row is inherently unique.

A row from the csv file I'm reading looks like this: (the TN is 9259991234)

2,PPS,2015-09-17T15:44,9259991234,9DF51758-A2BD-4F65-AAA2

I hit an error with the code below saying that '_csv.writer' is not iterable and I'm not sure how to modify my code to solve the problem.

import csv
import sys
import os

os.chdir(r'C:\pTest')

with open(r'csv.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=',')
    with open (r'new_csv.csv', 'ab') as new_f:
        writer = csv.writer(new_f, delimiter=',')
        for row in reader:
            if row[3] not in writer:
                writer.writerow(new_f)

What do you expect row[3] not in writer to do? You cannot test for membership in a writer; that data is already written. If you want to track phone numbers already processed, you need to keep a separate set() object and test against that. — Martijn Pieters
– Martijn Pieters, Commented Oct 1, 2015 at 18:54

Martijn Pieters · Accepted Answer · 2015-10-01 18:57:04Z

2

Your error stems from this expression:

row[3] not in writer

You cannot test for membership against a csv.writer() object. If you wanted to track if you already have processed a phone number, use a separate set() object to track those:

with open(r'csv.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=',')
    with open (r'new_csv.csv', 'ab') as new_f:
        writer = csv.writer(new_f, delimiter=',')
        seen = set()
        for row in reader:
            if row[3] not in seen:
                seen.add(row[3])
                writer.writerow(row)

Note that I also changed your writer.writerow() call; you want to write the row, not the file object.

answered Oct 1, 2015 at 18:57

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

erns Over a year ago

That works perfectly, thank you! This is quite literally my first attempt at writing anything in Python, so the feedback is much appreciated.

Collectives™ on Stack Overflow

Parsing CSV files using Python 2.7

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related