Python requests.request returning garbage

Question

Calling an api. For almost all calls the response comes clean but for a few there is garbage in the returned json. Executing the garbage one in the Firefox browser with the same parameters returns clean json. Working code for testing:

import requests
headers = {
   'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0',
   'Accept': 'application/json, text/javascript, */*; q=0.01',
   'Accept-Language': 'en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3',
   'Accept-Encoding': 'gzip, deflate, br, zstd',
   'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
   'X-Requested-With': 'XMLHttpRequest',
   'Origin': 'https://veiculos.fipe.org.br',
   'DNT': '1',
   'Sec-GPC': '1',
   'Connection': 'keep-alive',
   'Referer': 'https://veiculos.fipe.org.br/',
   'Cookie': 'ROUTEID=.5',
   'Sec-Fetch-Dest': 'empty',
   'Sec-Fetch-Mode': 'cors',
   'Sec-Fetch-Site': 'same-origin',
   'Priority': 'u=0',
   'TE': 'trailers',
}

def api_call(url, headers, parameters):
    try:
        r = requests.request('post', url, headers=headers, data=parameters)
        print (r.content)
        r_json = r.json()
        r.close()
        return r_json
    except Exception as e:
        print(repr(e))

url = 'https://veiculos.fipe.org.br/api/veiculos/ConsultarAnoModelo'

# First call returning clean json
parameters = [('codigoModelo', 6906), ('codigoTabelaReferencia', 327), ('codigoTipoVeiculo', 1), ('CodigoMarca', 189)]
r_json = api_call(url, headers, parameters)

# Second call returning garbage
parameters = [('codigoModelo', 6340), ('codigoTabelaReferencia', 327), ('codigoTipoVeiculo', 1), ('CodigoMarca', 189)]
r_json = api_call(url, headers, parameters)

The second call always raises a JSONDecodeError because of the initial sequence \x8b\x15\x80 and the final sequence \x03

b'[{"Label":"2016 Gasolina","Value":"2016-1"},{"Label":"2014 Gasolina","Value":"2014-1"}]'
b'\x8b\x15\x80[{"Label":"2011 Gasolina","Value":"2011-1"}]\x03'
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')

The buggy one executed in the browser:

Any ideas?

Thanks to a comment now I know it only errors when run in a Python virtual environment (venv).

When I run this from a command prompt I do not get an exception. How are you executing the posted code in a browser? Firefox does not understand Python, so I don't see how. — Booboo
– Booboo, Commented Nov 12 at 20:39
Does it return the same garbage characters at the start/end if you run the same request twice in a row? — charlotte
– charlotte, Commented Nov 12 at 21:05
@Booboo You pointed me in the right direction. When I run it in a virtual environment (venv) it errors. But in a system host terminal it does ok. — Clodoaldo Neto
– Clodoaldo Neto, Commented Nov 12 at 21:22
that is weird. maybe you could try re-installing requests or setting up your venv from scratch? — charlotte
– charlotte, Commented Nov 12 at 21:25

Gabriel Genellina · Accepted Answer · 2025-11-12 23:16:48Z

4

The response is encoded using Brotli compression:

>>> r.headers['content-encoding']
'br'

If the brotli package is available, requests imports it and uses it to decode the response. Otherwise, it does anything (no error reported either). This explains the different behavior in different environments.

How to fix it: ensure the brotli package is installed. (Alternatively: do not include 'br' with the 'Accept-Encoding' header)

answered Nov 12 at 23:16

Gabriel Genellina

1311 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

jsbueno Nov 13 at 13:24

That. It was quite a surprise for me - and that this default Brotli Python package is a bit huge at a few megabytes, and will prevent "free-threading" on Python 3.14t

Gabriel Genellina Nov 13 at 18:08

Just omit 'br' from the Accept-Encoding header - an API call like this does not benefit from compression.

Ajeet Verma · Accepted Answer · 2025-11-13 01:51:23Z

The problem occcurs in the second api_call(). Because the response r contains �� at start (r.text)

��[{"Label":"2011 Gasolina","Value":"2011-1"}]

and that's why r.json() throws the JSONDecodeError

JSONDecodeError('Expecting value: line 1 column 1 (char 0)')

One way to fix is to simply remove the br encoding from the Accept-Encoding header.

'Accept-Encoding': 'gzip, deflate, zstd'

code:

import requests
headers = {
   'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0',
   'Accept': 'application/json, text/javascript, */*; q=0.01',
   'Accept-Language': 'en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3',
   'Accept-Encoding': 'gzip, deflate, zstd',
   'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
   'X-Requested-With': 'XMLHttpRequest',
   'Origin': 'https://veiculos.fipe.org.br',
   'DNT': '1',
   'Sec-GPC': '1',
   'Connection': 'keep-alive',
   'Referer': 'https://veiculos.fipe.org.br/',
   'Cookie': 'ROUTEID=.5',
   'Sec-Fetch-Dest': 'empty',
   'Sec-Fetch-Mode': 'cors',
   'Sec-Fetch-Site': 'same-origin',
   'Priority': 'u=0',
   'TE': 'trailers',
}


def api_call(url, headers, parameters):
    try:
        r = requests.request('post', url, headers=headers, data=parameters)
        r_json = r.json()
        print(r_json)
        r.close()
        return r_json
    except Exception as e:
        print(repr(e))

url = 'https://veiculos.fipe.org.br/api/veiculos/ConsultarAnoModelo'

# First call returning clean json
parameters = [('codigoModelo', 6906), ('codigoTabelaReferencia', 327), ('codigoTipoVeiculo', 1), ('CodigoMarca', 189)]
r_json = api_call(url, headers, parameters)

# Second call returning garbage
parameters = [('codigoModelo', 6340), ('codigoTabelaReferencia', 327), ('codigoTipoVeiculo', 1), ('CodigoMarca', 189)]

r_json = api_call(url, headers, parameters)

output:

[{'Label': '2016 Gasolina', 'Value': '2016-1'}, {'Label': '2014 Gasolina', 'Value': '2014-1'}]
[{'Label': '2011 Gasolina', 'Value': '2011-1'}]

jackal · Accepted Answer · 2025-11-13 08:48:22Z

1

Your code is unnecessarily complicated. The API only needs a User-Agent to work properly.

You can simplify it as follows:

from requests import post as POST

URL = "https://veiculos.fipe.org.br/api/veiculos/ConsultarAnoModelo"
UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.0 Safari/605.1.15"
DATA = [
    ("codigoModelo", 6906),
    ("codigoTabelaReferencia", 327),
    ("codigoTipoVeiculo", 1),
    ("CodigoMarca", 189),
]
HEADERS = {"User-Agent": UA}

with POST(URL, data=DATA, headers=HEADERS) as response:
    response.raise_for_status()
    print(response.json())

Output:

[{'Label': '2016 Gasolina', 'Value': '2016-1'}, {'Label': '2014 Gasolina', 'Value': '2014-1'}]

edited Nov 13 at 8:48

answered Nov 13 at 8:42

jackal

29.1k3 gold badges9 silver badges27 bronze badges

Collectives™ on Stack Overflow

Python requests.request returning garbage

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related