Can't skip header row in csv file with python

Question

I'm using the CSV module in python to read a CSV into memory and I need to skip the header row.

I am using the next command to skip the headers, but it isn't working.

import csv
with open(aws_env_list) as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    next(csv_reader)

The headers are still being produced, crashing my script. My script produces the following line:

Working in AWS Account:  companyAccountName,AWSAccountName,Description,LOB,AWSAccountNumber,CIDRBlock,ConnectedtoMontvale,PeninsulaorIsland,URL,Owner,EngagementCode,CloudOpsAccessType

In the original CSV the headers are only on the first line.

The first few lines of my csv file look like this.

What's wrong with the above and why is this not skipping the headers? Is there a better way?

You need to be clear on what "it's not working" means. That code will skip the first row of the csv file, maybe the headers are over two rows. We don't know, because we can't see anything about the file. — roganjosh
– roganjosh, Commented Aug 7, 2019 at 15:55
Your pasted csv has tab delimiters. You might want to update your code to csv_reader = csv.reader(csv_file, delimiter='\t'). You wouldn't by any chance be typing this at the interactive prompt, would you? — Steven Rumbalski
– Steven Rumbalski, Commented Aug 7, 2019 at 16:25

ctyler9 · Accepted Answer · 2020-05-05 17:52:06Z

3

I don't think you are using the next() function correctly.

Here's the an example from the documentation:

import csv

with open('eggs.csv', newline='') as csvfile:
    spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
    for row in spamreader:
        print(', '.join(row))

When you use csv.reader, it takes the csv file and for each row creates an iterable object. So, if you want to skip the first row (the header row) simply make this change.

with open(aws_env_list) as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    for row in list(csv_reader)[1:]:
        """ do whatever action """

When you add [1:] to the end of csv_reader, it tells it to select only the 2nd object (since 0 is the first). You create in essence a subset of the object which does not contain the first element.

edited May 5, 2020 at 17:52

answered Aug 7, 2019 at 15:59

ctyler9

887 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

roganjosh Over a year ago

The use of next() is correct because it doesn't expand out the iterator, which your code would do. For large CSV files, this will cause a lot of unnecessary memory use. I suspect the issue is that the OP simply didn't start iterating after yielding the first line and discarding it

Steven Rumbalski Over a year ago

csv_reader[1:] will not work as a the reader object is not subscriptable because it's an iterator. Technically this could be solved by doing list(csv_reader)[1:] but that defeats the purpose of having an iterator. The OP was doing the correct thing with next(csv_reader).

Steven Rumbalski Over a year ago

@roganjosh: The slicing simply does not work. (itertools.islice(csv_reader, 1, None) on the otherhand...) The OP does have the wrong delimiter (comma when it should be tab), but that wouldn't cause this problem. One thing that could make it look like the line isn't skipped is running the code in an interactive prompt. The results of unassigned expressions are echoed back to the terminal in interactive mode.

Collectives™ on Stack Overflow

Can't skip header row in csv file with python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related