Cannot get XML output through cURL

Question

I am using PHP cURL to fetch XML output from a URL. Here is what my code looks like:

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, 'http://www.mydomain.com?querystring'); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($ch, CURLOPT_USERPWD, "username:password");
$store = curl_exec($ch);
echo $store;
curl_close($ch);

But, instead of returning the XML it just shows my 404 error page. If I type the URL http://www.mydomain.com?querystring in the web browser I can see the XML in the browser.

What am I missing here? :(

Thanks.

Is the PHP script running on the same machine as your browser? Just because the url works in one places means absolutely nothing about how well it'll run on another machine. — Marc B
– Marc B, Commented Jun 29, 2011 at 20:54
@Blueboye - In addition to @Marc B's comment, some website owners check for the existence of certain things to make sure the request comes from a web browser and not a bot (or cURL). You should try adding curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)'); and see if that fixes the problem. That will send a user-agent string. The site may also check for the existence of cookies or other things. — Francois Deschenes
– Francois Deschenes, Commented Jun 29, 2011 at 20:55
I am running the PHP on my localhost but the ULR is an external server. — Blueboye
– Blueboye, Commented Jun 29, 2011 at 20:56
@Francois: WOW! That worked, though now I see the XML nodes data on the screen. It is not XML. How can I make it show as an XML. Another question I have is what is the best way to work on such a XML response? Should I store it in an XML file and then perform operations or can I directly perform XML operations on the response? — Blueboye
– Blueboye, Commented Jun 29, 2011 at 21:00
Possible Duplicates: stackoverflow.com/questions/1115638/php-curl-not-returning-xml ; stackoverflow.com/questions/5405424/… — hakre
– hakre, Commented Jun 29, 2011 at 21:01

Community · Accepted Answer · 2023-11-17 19:24:28Z

2

Some website owners check for the existence of certain things to make sure the request comes from a web browser and not a bot (or cURL). You should try adding curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)'); and see if that fixes the problem. That will send a user-agent string. The site may also check for the existence of cookies or other things.

To output the XML in a web-page, you'll need to use htmlentities(). You might want to wrap it inside a HTML <pre> element as well.

edited Nov 17, 2023 at 19:24

CommunityBot

11 silver badge

answered Jun 29, 2011 at 21:05

Francois Deschenes

25.1k4 gold badges67 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Blueboye Over a year ago

Great. Thanks a lot for your quick help. Any thoughts about my second question: Do I need to save this output into XML file to read the XML nodes or can I do it without saving too?

hakre Over a year ago

Isn't it better refer to htmlspecialchars()? And actually the encoding should be reflected before outputting the XML as HTML with a totally different encoding.

hakre Over a year ago

@Blueboy, you can do it without saving. Use DomDocument, works with files and strings.

Francois Deschenes Over a year ago

@Blueboye - If you want to read the XML in PHP, have a look at SimpleXML (like simplexml_load_string). There's no need to save. You'll essentially want to do $xml = simplexml_load_string(curl_exec($ch));. If you want to echo the entire XML to the screen (so you can see it), you'll want to use htmlentities() as I said in my answer because the browser will not show it properly otherwise.

Collectives™ on Stack Overflow

Cannot get XML output through cURL

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related