Error in Python Pandas when Reading CSV File

Question

I am trying to read a CSV File, but its throwing an error. I am not able to understand whats the problem with my syntax or do I need to add more attributes to my read_csv.

I tried the solution on

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte as well. But its not working

import pandas as pd


#Assign file_path variable
file_path = 'rawdump_24th_Sep.csv'

#assign dataframe
df1 = pd.read_csv(file_path,index_col=0)

df.head()

[ERROR]

UnicodeDecodeError Traceback (most recent call last) pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte

During handling of the above exception, another exception occurred:

UnicodeDecodeError Traceback (most recent call last) in 6 7 #assign dataframe ----> 8 df1 = pd.read_csv(file_path,index_col=0) 9 10 df.head()

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision) 700 skip_blank_lines=skip_blank_lines) 701 --> 702 return _read(filepath_or_buffer, kwds) 703 704 parser_f.name = name

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds) 433 434 try: --> 435 data = parser.read(nrows) 436 finally: 437 parser.close()

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1137 def read(self, nrows=None): 1138 nrows = _validate_integer('nrows', nrows) -> 1139 ret = self._engine.read(nrows) 1140 1141 # May alter columns / col_dict

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1993 def read(self, nrows=None): 1994 try: -> 1995 data = self._reader.read(nrows) 1996 except StopIteration: 1997 if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_column_data()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte`

UPDATED

import pandas as pd


#Assign file_path variable
file_path = 'rawdump_24th_Sep.csv'

#assign dataframe
df1 = pd.read_csv(file_path,index_col=0)

df1.head()

UnicodeDecodeError Traceback (most recent call last) pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte

During handling of the above exception, another exception occurred:

UnicodeDecodeError Traceback (most recent call last) in 6 7 #assign dataframe ----> 8 df1 = pd.read_csv(file_path,index_col=0) 9 10 df1.head()

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision) 700 skip_blank_lines=skip_blank_lines) 701 --> 702 return _read(filepath_or_buffer, kwds) 703 704 parser_f.name = name

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds) 433 434 try: --> 435 data = parser.read(nrows) 436 finally: 437 parser.close()

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1137 def read(self, nrows=None): 1138 nrows = _validate_integer('nrows', nrows) -> 1139 ret = self._engine.read(nrows) 1140 1141 # May alter columns / col_dict

~\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1993 def read(self, nrows=None): 1994 try: -> 1995 data = self._reader.read(nrows) 1996 except StopIteration: 1997 if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_column_data()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte

Thanks Michael It worked. It can now be used by adding encoding = "ISO-8859-1" to the read_csv — Chetan Parmar
– Chetan Parmar, Commented Oct 4, 2019 at 9:35

SNG · Accepted Answer · 2023-07-03 09:17:36Z

1

Use specific encoding. pd.read_csv('data.csv', encoding='utf-8')

answered Jul 3, 2023 at 9:17

SNG

3554 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sai Sharan · Accepted Answer · 2019-09-30 03:52:16Z

0

'rawdump_24th_Sep.csv' should be there in the same folder where .py file is saved

import pandas as pd
df1 = pd.read_csv('rawdump_24th_Sep.csv')
df1

answered Sep 30, 2019 at 3:52

Sai Sharan

265 bronze badges

2 Comments

Chetan Parmar Over a year ago

Changed df to df1. The file is in the same folder as the location is mentioned. Updated my question above by changing df to df1

Chetan Parmar Over a year ago

tried same error 'utf-8' codec can't decode byte 0x96 in position 21: invalid start byte

Azra YILDIRIM · Accepted Answer · 2020-04-14 13:23:14Z

0

if your csv file not suit then you can get this error. You should try another dataset.

answered Apr 14, 2020 at 13:23

Azra YILDIRIM

1

Collectives™ on Stack Overflow

Error in Python Pandas when Reading CSV File

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related