Postgres - ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d

Question

I'm trying to import a massive txt file into the postgres. When I typed the following command:

\COPY denton_2018_rawdata FROM 'C:\Users\testu\Downloads\denton_county\2018-website-all-property\2018-08-28_005183_APPRAISAL_INFO.txt' delimiter E'\x01'

I got the following error:

ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d CONTEXT: COPY denton_2018_rawdata, line 22769: "000000027205R 02018000000000000 ..."

So I tried the following command (adding ENCODING 'WINDOWS-1252') :

\COPY denton_2018_rawdata FROM 'C:\Users\testu\Downloads\denton_county\2018-website-all-property\2018-08-28_005183_APPRAISAL_INFO.txt' delimiter E'\x01' ENCODING 'WINDOWS-1252';

But still got the same error. Could anyone help please?

Pavel Stehule · Accepted Answer · 2018-11-05 21:15:52Z

1

PostgreSQL is very strict about UTF8 encoding. It is due possible SQL injection attacks based on invalid UTF8 characters. First you have to know, what is source encoding. Second you should to eliminate all broken chars before import to Postgres.

There are some application that can do this work - like iconv

answered Nov 5, 2018 at 21:15

Pavel Stehule

46.6k6 gold badges103 silver badges102 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Postgres - ERROR: invalid byte sequence for encoding "UTF8": 0xca 0x2d

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related