Process requests and validating API Response for correct data in Python

Question

I want to send a bulk of requests using requests.futures library in python.

The problem is some of the requests are getting Server Error in API response with 200 ok status code.

I need to implement a fallback mechanism to resend the failed requests and get the actual data from server.

This is what I have implemented.

Code to send bulk data and getting bulk response using futures library

def sendBulkRequests(requestURLs):
    # this would send data to server
    return responseList

Validate Response

requestURLsToRetry = []
for i in range(len(responseList)):
    if 'sometext' not in responseList[i].text:
        requestURLsToRetry.append(requestURLs[i])

Now I need to resend requestURLsToRetry to sendBulkRequests()

responseListRetried = sendBulkRequests(requestURLsToRetry)

How do I append responseListRetried it to original responseList to be returned so the next code can process hoping the data returned is correct.

I might need to send sendBulkRequests() multiple times until all correct API responses are received and above condition 2 is verified with no issues.

Sven Asmussen · Accepted Answer · 2022-10-16 16:54:33Z

-1

you cannnot change the list while you are iterating through it as it has to be fixed length. You could just iterate over the second list, or once the first process is finished and you have the list of failed requests just set list1 = listOfFailedRequests and iterate over list1 again with the previously defined function

answered Oct 16, 2022 at 16:54

Sven Asmussen

273 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Shomi Over a year ago

and how do i merge it with the initial (first) list? I might have to perform this in loop for 5-6 times and would need to merge all newly created sub lists to main list with the exact index of failed responses.

Collectives™ on Stack Overflow

Process requests and validating API Response for correct data in Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related