Requests POST encoding with python

Question

I'm posting this character "ç" that is usually sent through a form, to a website I don't control. I'm having trouble with the encoding of the data. Trying to send data without encoding it to utf-8 produces this error (correction: this error was for another special character, mistake!):

File "D:\Python34\lib\http\client.py", line 1181, in _send_request
body = body.encode('iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0192' in position 512: ordinal not in range(256)

I'm reading the data from a .csv file saved with utf-8 encoding (is this part of the problem?)

After using .encode('utf-8') and a bit of trial and error:

Posting this: ç Displays on their site as: Ã§

Posting this: Ã§ Displays on their site as: ÃƒÂ§

Sample of shortened code:

headers = {
'Accept' : 'application/json, text/javascript, */*; q=0.01',
'Content-Type' : 'application/x-www-form-urlencoded; charset=UTF-8'
}

data = {
"info" : "ç"
}

r = requests.post('www.urlgoeshere.com', headers=headers, data=json.dumps(data,ensure_ascii=False).encode('utf-8'))

Anyone got any tips on how I can send these special characters and have them display correctly? Or should I just give up and edit them all out. Their website form manages it anyway so I would like to as well.

Edit for more complete .csv code:

with open('data.csv', newline='') as dataFile:
dataReader = csv.reader(dataFile)
for row in dataReader:
    data = {
    "values":{
    "title_id":row[1],
    "title":row[0],
    "other stuff":[{"foo":"bar",
    "too":"foobar"}]}
    }

It's in that loop that i do the request as well. Data.csv is a comma-delimited file saved as utf-8.

Apparently you are sending a ISO-8859-1 character. Try "info" : u"ç". Also look if the file containing your code is UTF-8 encoded — Goufalite
– Goufalite, Commented Jun 24, 2016 at 9:51
The code and the datafile are both utf-8. Have to leave for a while though. Be back later (and thanks for the help sofar). — raecer
– raecer, Commented Jun 24, 2016 at 10:15
The line body = body.encode('iso-8859-1') is from httplib, that is trying to encode the body of the requests a iso-8859-1, add the actual code — Padraic Cunningham
– Padraic Cunningham, Commented Jun 24, 2016 at 10:46
It dos if you pass a str type, what you want to pass is bytes, I was also asking about your code that reads froma csv not httplib — Padraic Cunningham
– Padraic Cunningham, Commented Jun 24, 2016 at 13:27
@raecer, '\u0192' is a latin-1 encoded small f with a hook, what happens if you open the file adding encoding="utf-8"? Also can you share the url? — Padraic Cunningham
– Padraic Cunningham, Commented Jun 26, 2016 at 11:17

raecer · Accepted Answer · 2016-06-26 13:29:31Z

1

The simplest answer to this question was that i was opening the datafile without specifying the encoding. Had i added the encoding="utf-8" to the open function, and encoded the request as utf-8 this would have been solved rather quickly.

answered Jun 26, 2016 at 13:29

raecer

1954 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Requests POST encoding with python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related