22,023 questions
0
votes
0
answers
28
views
Loading images in HTML – Differentiation between no response and error response
I'm building a kind of gallery app. Users are adding images by inserting URLs into the database and the HTML page is then hotlinking those. Obviously, because of link rotting, the gallery is full of ...
- 131
Best practices
1
vote
3
replies
85
views
Best Practice for Certificate Authentication
If I need to authenticate with a certificate, I have a few options available:
Add the certificate to the bundle packaged with requests/certifi
Add the certificate to the system store and override ...
- 478
3
votes
3
answers
113
views
Python requests.request returning garbage
Calling an api. For almost all calls the response comes clean but for a few there is garbage in the returned json. Executing the garbage one in the Firefox browser with the same parameters returns ...
- 127k
-1
votes
0
answers
53
views
Error message when scraping mutliple movies via bs4 [duplicate]
The Python code:
from bs4 import BeautifulSoup
import requests
#####################################################
# Extracting the links of multiple movie transcripts
##############################...
- 61
1
vote
0
answers
44
views
Connection management while maintaining requests.Session
What I found
While using the requests library for HTTP requests, I noticed that there were many TCP connections in the FIN_WAIT2 state on the server. Through code analysis, I found that these ...
- 11
3
votes
1
answer
90
views
Unable to post large file with python requests library
I try to post large file and read response as streaming with requests python library.
I have to send a large text file (2 million lines of log ) and receive response back as streaming. If I did by ...
- 93
0
votes
0
answers
52
views
One request fails, but others don't; it also succeeds in terminal
I checked a lot of trouble-shoot topics, but didn't find what might be wrong here... (This is a second attempt at fixing this issue, but this time I dug deeper.)
I'm going off this API documentation: ...
- 69
0
votes
1
answer
195
views
Congress API Auth Token Failure
Congress API Down
Over the past few days I've been building along term web app that interfaces with the official Congress.gov API (found here),but as of 20 minutes ago, the entire API seems to have ...
- 25
1
vote
2
answers
292
views
How to download protected PDF (ViewDocument) using Selenium or requests?
I'm trying to download a protected PDF from the New York State Courts NYSCEF website using Python. The URL looks like this:
https://iapps.courts.state.ny.us/nyscef/ViewDocument?docIndex=...
- 655
1
vote
0
answers
64
views
MS Graph request returns 403
To automate processing of my e-mail messages on MS account, I have registered some application called "NameOfMyRegisteredApplication"; gave it all MS Graph Read rights; and then I ran the ...
- 43
3
votes
0
answers
138
views
Python requests library refusing to accept correct verify certificate
The following application performs a basic HTTP GET request against https://google.com, retrieves the peer certificate and saves it in PEM format into a file called cert.pem.
After that it attempts ...
- 3,307
1
vote
1
answer
187
views
Why httpx timeout not working with stream response
I use httpx to make request to my LLM. Here is the code.
import httpx
import logging
import json
from httpx import TimeoutException
try:
content = ''
logging.info(f"request payload:{...
- 783
1
vote
2
answers
57
views
How do I make a variable in the requests payload for python?
I have the following payload for the requests module in python and I want to make MAC and NAME a variable as I iterate through a csv of information? Is there a way to do this?
payload = '''{
...
0
votes
0
answers
72
views
how do i complete the Checkout bot at the payment process that uses tokens? (using python requests)
I'm working on a Python bot that monitors a Shopify webshop, adds a product to the cart, and tries to continue to checkout. I'm using requests.Session() and BeautifulSoup to handle the stock check, ...
- 13
0
votes
0
answers
17
views
FastAPI returns 422 Unprocessable Entity when posting JSON payload, even though data matches Pydantic model [duplicate]
I'm working on a monitoring project where a client script collects system metrics and posts them to a FastAPI server. However, I'm consistently getting a 422 Unprocessable Entity error even though the ...
0
votes
1
answer
61
views
How to parse link extensions out of a script tag in BeautifulSoup with RegEx in Python
I am trying to parse link extensions out of a script tag in a webpage loaded with requests. I'm able to request the page and load the script tag as a tag element in BeautifulSoup, it looks like this:
{...
0
votes
1
answer
30
views
Using Requests to scrape: name, SKU, Price, Discount price, Can´t obtain data
import requests
from lxml import html
from selenium.webdriver.common.by import By
headers = {
"user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) ...
0
votes
0
answers
92
views
Python json.loads() truncating inner data
I'm having issues when parsing JSON data from an API request.
I get all pages from the API request using this code:
try:
limit = 100 # Max limit per page, can be reduced as needed
...
- 1
0
votes
1
answer
115
views
Python API call freezes instead of raising TimeoutError despite short timeout
I have a very sensitive production script that I’ve been running consistently for the past 4 years. This specific function (runAPI) is called about 100 times every day and has never caused any issues —...
- 43
1
vote
1
answer
62
views
Can I be sure of the order requests are sent in in a Python requests session?
I am using Python requests to communicate with an external API. When I do something like:
with requests.Session() as sess:
resp1 = sess.put(
api_url + f"stuff/{id}/endpoint1",
...
- 1,372
0
votes
0
answers
39
views
AIOHTTP and ASYNCIO GRAPHQL request is replacing session_id in Cookie with another value and causing the request to return nothing but 200 status code
So I'm messing around with the leetcode GRAPHQL api and trying to hit up company endpoints, in the follow synchronous code that I wrote, it works perfectly and the cookie is also saved correctly. It ...
- 21
0
votes
0
answers
371
views
"Connection broken: IncompleteRead(50454677 bytes read, 35845891 more expected)" upon fetching large Collection from Postman API
Upon fecthing a large collection item from Postman's API I get the following error:
RuntimeError: Chunked transfer error: ('Connection broken:
IncompleteRead(50454677 bytes read, 35845891 more ...
- 115
2
votes
1
answer
87
views
python-requests-html render inconsistent result
background:
by default the website is only showing few names and there s a "moreBtn" to generate the full list
code idea:
create Html session, render with script clicking the "moreBtn&...
- 31
1
vote
2
answers
85
views
Importing geographic data with WFS works on Chrome but not on Python
I am trying to pull a geojson file from here.
The JSON appears as expected when I paste that link into Chrome or Safari. However, I get the following error every time when I run the following code on ...
- 121
0
votes
1
answer
244
views
Python requests module: SSL: WRONG_VERSION_NUMBER error with specific website on specific machine
I am working on a project using Python's requests module. I am only getting this error when attempting to request data from api.rugcheck.xyz on my local machine; I am able to request data from other ...
- 1