How to get rid of "Unnamed: 0" column in a pandas DataFrame read in from CSV file?

This is usually caused by your CSV having been saved along with an (unnamed) index (RangeIndex).

(The fix would actually need to be done when saving the DataFrame, but this isn't always an option.)

Workaround: read_csv with index_col=[0] argument

IMO, the simplest solution would be to read the unnamed column as the index. Specify an index_col=[0] argument to pd.read_csv, this reads in the first column as the index. (Note the square brackets).

df = pd.DataFrame('x', index=range(5), columns=list('abc'))
df

   a  b  c
0  x  x  x
1  x  x  x
2  x  x  x
3  x  x  x
4  x  x  x

# Save DataFrame to CSV.
df.to_csv('file.csv')

pd.read_csv('file.csv')

   Unnamed: 0  a  b  c
0           0  x  x  x
1           1  x  x  x
2           2  x  x  x
3           3  x  x  x
4           4  x  x  x

# Now try this again, with the extra argument.
pd.read_csv('file.csv', index_col=[0])

   a  b  c
0  x  x  x
1  x  x  x
2  x  x  x
3  x  x  x
4  x  x  x

Note
You could have avoided this in the first place by using index=False if the output CSV was created in pandas, if your DataFrame does not have an index to begin with:

df.to_csv('file.csv', index=False)

But as mentioned above, this isn't always an option.

Stopgap Solution: Filtering with str.match

If you cannot modify the code to read/write the CSV file, you can just remove the column by filtering with str.match:

df 

   Unnamed: 0  a  b  c
0           0  x  x  x
1           1  x  x  x
2           2  x  x  x
3           3  x  x  x
4           4  x  x  x

df.columns
# Index(['Unnamed: 0', 'a', 'b', 'c'], dtype='object')

df.columns.str.match('Unnamed')
# array([ True, False, False, False])

df.loc[:, ~df.columns.str.match('Unnamed')]
 
   a  b  c
0  x  x  x
1  x  x  x
2  x  x  x
3  x  x  x
4  x  x  x

To get ride of all Unnamed columns, you can also use regex such as df.drop(df.filter(regex="Unname"),axis=1, inplace=True)

You can do either of the following with 'Unnamed' Columns:

1. Delete unnamed columns
2. Rename them (if you want to use them)

Method 1: Delete Unnamed Columns

# delete one by one like column is 'Unnamed: 0' so use it's name
df.drop('Unnamed: 0', axis=1, inplace=True)

#delete all Unnamed Columns in a single code of line using regex
df.drop(df.filter(regex="Unnamed"),axis=1, inplace=True)

Method 2: Rename Unnamed Columns

df.rename(columns = {'Unnamed: 0':'Name'}, inplace = True)

If you want to write out with a blank header as in the input file, just choose 'Name' above to be ''.

where the OP's input data 'file.csv' was:

,A,B,C
0,1,2,3
1,4,5,6
2,7,8,9

#read file df = pd.read_csv('file.csv')

Another case that this might be happening is if your data was improperly written to your csv to have each row end with a comma. This will leave you with an unnamed column Unnamed: x at the end of your data when you try to read it into a df.

Simply delete that column using: del df['column_name']

Simple do this:

df = df.loc[:, ~df.columns.str.contains('^Unnamed')]

Alternatively:

df = df.drop(columns=['Unnamed: 0'])

from IPython.display import display
import pandas as pd
import io


df = pd.read_csv('file.csv',index_col=[0])
df = pd.read_csv(io.StringIO(df.to_csv(index=False)))
display(df.head(5))

In my experience, there are many reasons you might not want to set that column as index_col =[0] as so many people suggest above. For example it might contain jumbled index values because data were saved to csv after being indexed or sorted without df.reset_index(drop=True) leading to instant confusion.

So if you know the file has this column and you don't want it, as per the original question, the simplest 1-line solutions are:

df = pd.read_csv('file.csv').drop(columns=['Unnamed: 0'])

or

df = pd.read_csv('file.csv',index_col=[0]).reset_index(drop=True)

I think this is the case where the column you are referring is the index. Anyway I would suggest this more general solution:

df = pd.read_csv("file.csv", usecols=lambda col: not col.startswith("Unnamed"))

A solution that is agnostic to whether the index has been written or not when utilizing df.to_csv() is shown below:

df = pd.read_csv(file_name)
if 'Unnamed: 0' in df.columns:
    df.drop('Unnamed: 0', axis=1, inplace=True)

If an index was not written, then index_col=[0] will utilize the first column as the index which is behavior that one would not want.

My issue was slightly different. The first header was unnamed, which caused all my data afterwards to shift left, making it misaligned, and made the last column all NaN.

I resolved it using the following:

df = pd.read_csv('filename.csv', nrows=1)
columns = list(df.columns)[1:]
df = pd.read_csv('filename.csv', skiprows=1, header=None, names=columns)