Using Urlllib2.urlopen fails for binary data?

Question

I'm using python to programatically download a zip file from a web server. Using a web browser, it's fine. I've written this (partial) script;

response = urllib2.urlopen(url, data, 10)
the_page = response.read()
f = open(filename, 'w')
f.write(the_page)
f.close()

The request succeeds and I get data. The problem is that the file I'm downloading -- a zip file -- doesn't work; the file appears to be corrupt. It seems to be the right sort of length, and looked at in text editor seems to look like a zip file content. Here are the headers from the download;

Content-Length: 9891 Content-Disposition: Content-Disposition:attachment; filename="TrunkBackup_20101230.zip" Date: Wed, 30 Dec 2009 12:22:08 GMT Accept-Ranges: bytes

When I check the length of the response, it is correct at 9891. I suspect what's happening is that when I call response.read() the result is a string with carriage returned 'helpfully' normalized (say, \r to \n). when I write the file, the binary data is slightly wrong, and the zip file is corrupt.

My problem is (A) I'm not sure if I'm right, and (B) if I am right, how to I save the binary data itself?

Does it work when you open file in binary mode (open(filename, 'wb'))? — Denis Otkidach
– Denis Otkidach, Commented Dec 30, 2009 at 12:29

dusan · Accepted Answer · 2009-12-30 12:29:03Z

9

Try opening the file in binary mode:

 f = open(filename, 'wb')

answered Dec 30, 2009 at 12:29

dusan

9,3033 gold badges39 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

3lectrologos · Accepted Answer · 2009-12-30 12:29:38Z

1

You could use the urlretrieve function for downloading raw binary files.

answered Dec 30, 2009 at 12:29

3lectrologos

9,7204 gold badges41 silver badges46 bronze badges

2 Comments

Steve Cooper Over a year ago

I had problems with urlretrieve -- my app just stopped. Dunno why.

3lectrologos Over a year ago

That's curious... I think urlretrieve(url, targetpath) works fine for files (and is probably pretty faster than the "url->string->file" way). Maybe you could show me your code.

Keith Hughitt · Accepted Answer · 2012-04-16 16:14:25Z

0

If anyone is running into the same error even with the write-mode set to "wb" make sure you call either "f.flush()" or "f.close()" before attempting to work with the file, otherwise it may not be completely written.

answered Apr 16, 2012 at 16:14

Keith Hughitt

4,9705 gold badges52 silver badges57 bronze badges

Collectives™ on Stack Overflow

Using Urlllib2.urlopen fails for binary data?

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related