Curl not saving all Cookies

Question

I've done scraping for lots of sites but one in particular isn't saving it's cookies to my cookie file. Any ideas?

$ch = curl_init($url); 
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_TIMEOUT,8200);
curl_setopt($ch,CURLOPT_TIMEOUT_MS,8200);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT ,8200);
$cookie_file = "cookies/zapper.txt";
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
if ($fields) {curl_setopt($ch,CURLOPT_POST, count($fields)); }
if ($fields) {curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string); }

This is the first site that I've done that doesn't respond to my cookie saves. All others use the same code and work perfectly. I've even emulated the post of their forms and faked the header in case it was checking [those.

The site I'm trying to mimic an add to cart process for is http://zapper.co.uk/

Yes, user agent is usually needed when scrapping websites :) — m3nda
– m3nda, Commented Nov 4, 2013 at 18:24

m3nda · Accepted Answer · 2013-11-01 14:21:02Z

1

Read a possible solution directly from the php.net site about curl_setopt. It's a workaround to get Cookie content from the header output. Seems to be a cool alternative.

Also, you can get surprising results modifiying some of your rules at curl_setop. Sometimes we use more options than needed.

I also recommend you to echo the whole $ch content (It will print page like the browser does). Sometimes you get a detailed error not present at headers seeing the live result content.

answered Nov 1, 2013 at 14:21

m3nda

2,0574 gold badges34 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Curl not saving all Cookies

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related