Send XML file to REST API in Python3

Question

I am trying to interact with a REST API using the requests library.

I need to pass a large XML file to the API - I can do this easily with curl in Linux but I want a pythonic way to do this.

In curl my syntax looks like this:

curl -k https://10.244.1.2:4444/webconsole/APIController? -F "reqxml=MYxml.xml"

Note: the -k flag just ignores the ssl error

So far here is what I have come up with:

import requests    
url = r'https://10.244.1.2:4444/webconsole/APIController?reqxml='    
x = open('MYxml.xml', 'r').read()

q = requests.post(url, data=x, verify=False)    
print(q.text)

If I change that q variable to this: q = requests.post(url + x, verify=False) then the command will work. However, with longer XML files it will fail.

I have looked at these two related posts: First, Second but the suggestions look just like my syntax unless I am missing something obvious

Any help is appreciated!

I don't see anything wrong with the requests syntax. Can you describe the web server? Taking a stab in the dark, the web server might be expecting a "Content-Type" xml header and curl is automajically adding it. Perform the curl command again with a -v (verbose logging) and look at the request header. — Brennen Sprimont
– Brennen Sprimont, Commented May 11, 2018 at 20:51
The api is the Sophos XG firewall. The documentation for it is VERY sparse so I am not sure about the headers. I will try the curl suggestion. Thanks for that tip — Joe
– Joe, Commented May 11, 2018 at 20:52
@BrennenSprimont So I can see that curl is adding the multipart/form-data as the content type. I tried adding that to the query but no luck. — Joe
– Joe, Commented May 11, 2018 at 20:57
For the heck of it, try adding an "application/xml" form type. Something like: q = requests.post(url, data=x, verify=False, headers={"Content-Type":"application/xml"}) — Brennen Sprimont
– Brennen Sprimont, Commented May 11, 2018 at 21:01
@BrennenSprimont I had tried those headers before I posted. I thought that would be it too. — Joe
– Joe, Commented May 11, 2018 at 21:45

Brennen Sprimont · Accepted Answer · 2018-05-11 21:27:52Z

(Moving comments discussion into an answer)

Interfacing with web servers that don't have good documentation or error reporting services is always a pain.

However, if you have a method that works (the curl command), you have a foot in the door. curl has a -v flag that can be used to see the headers sent with the payload.

Also, looking at your curl command, you are sending the entire file over. Whereas, your python script is reading the data into a text string and sending that over.

If the web server accepts XML data, you might need to just add the XML content type header and see if that works:

q = requests.post(url, data=x, verify=False, headers={"Content-Type":"application/xml"})

Otherwise, if the web server only accepts files/forms, you'll have to open the file as a binary object (this more closely resembles the working curl command):

# Create a dictionary of files to post with your file in it.
x_files = {"file": ("file", open('MYxml.xml', 'rb'))}
...
q = requests.post(url, files=x_files, verify=False, headers={"Content-Type":"multipart/form-data"})

If neither of the above works and you are still stumped, dump all of the headers from the working curl request and edit them into your answer, something else has to be in there.

Good luck

Joe · Accepted Answer · 2018-05-13 14:40:23Z

Ok this took a lot of trial and error to figure out the right pieces here but I have finally figured it out. @Brennen Sprimont's answer was very helpful in understanding what cURL was doing.

As he pointed out - cURL was actually sending a full file and my initial requests were not.

My first clue came from a website I found which converts cURL commands to python - Link

When I put my cURL command into that site it returned this as part of the result:

data = {'reqxml' : (None, open(xml_file, 'rb'))} (note: in my question I used 'x'but here I used 'data' for that variable)

The None parameter was something missing from my original - also I am reading binary now.

My next major issue was they way I was passing the file - When I execute this from a browser the syntax is:

https://<ipaddress><port>/webconsole/APIController?reqxml=<Login></Login>

Here we can see the data defined by the attribute reqxml=

In cURL this looks like the following:

curl -k https://10.244.1.2:4444/webconsole/APIController? -F "reqxml=MYxml.xml"

There we can the attribute isn't actually passed in the url but with the file

The last mistake I was making was trying to pass the reqxml attribute with the '=' sign. Unlike in cURL or through the browser, requests does not want that. In this case it wanted the values passed as a dictionary without the equals symbol.

Below is a small function which I used to test and works with Sophos XG v17

data_file = 'Myxml.xml'
ipaddress = '10.244.1.2'

def api_call(api_ip, xml_file):
    api_url = r'https://{}:4444/webconsole/APIController?'.format(api_ip)
    data = {'reqxml' : (None, open(xml_file, 'rb'))}
    r = requests.post(api_url, files=data, verify=False)   
    print(r.text)

api_call(ipaddress, data_file)

Collectives™ on Stack Overflow

Send XML file to REST API in Python3

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related