delete rows in csv based on specific column value - python without using pandas

Question

I have a large csv with the following header columns id, type, state, location, number of students

and the following values:

124, preschool, Pennsylvania, Pittsburgh, 1242
421, secondary school, Ohio, Cleveland, 1244
213, primary school, California, Los Angeles, 3213
155, secondary school, Pennsylvania, Pittsburgh, 2141
etc...

The file is not ordered and I want a new csv file that contains all the schools with the number of students above 2000.

The answers that I found were regarding to ordered csv files, or splitting them after a specific number of rows.

Are you looking to use standard library, or open to 3rd party pandas? — jpp
– jpp, Commented May 31, 2018 at 11:21
What have you tried that didn't work ? (hint: filtering an iterable based on a condition is really trivial). — bruno desthuilliers
– bruno desthuilliers, Commented May 31, 2018 at 11:35

jpp · Accepted Answer · 2018-05-31 11:30:48Z

2

Here's a solution using csv module:

import csv

with open('fin.csv', 'r') as fin, open('fout.csv', 'w', newline='') as fout:

    # define reader and writer objects
    reader = csv.reader(fin, skipinitialspace=True)
    writer = csv.writer(fout, delimiter=',')

    # write headers
    writer.writerow(next(reader))

    # iterate and write rows based on condition
    for i in reader:
        if int(i[-1]) > 2000:
            writer.writerow(i)

Result:

id,type,state,location,number of students
213,primary school,California,Los Angeles,3213
155,secondary school,Pennsylvania,Pittsburgh,2141

answered May 31, 2018 at 11:30

jpp

166k37 gold badges301 silver badges362 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aritesh · Accepted Answer · 2018-05-31 11:58:04Z

2

In case you just want to read file and avoid any other processing, you can use regex - (assuming this is the last column, and value are positive integers) -

import re
f1 = open('Test1.txt','wb')
with open("Test.txt") as f:
    for line in f:
        match = re.search(r'[2-9][0-9]{3,}$', line)
        if (match):
            f1.write(line)

f1.close()

Same thing will be much faster if you do it on bash -

while read line; do
  K='[2-9][0-9]{3,}$'
  if [[ $line =~ $K ]] ; then echo $line; fi
done <Test.txt

answered May 31, 2018 at 11:58

Aritesh

2,1031 gold badge17 silver badges18 bronze badges

Collectives™ on Stack Overflow

delete rows in csv based on specific column value - python without using pandas

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related