Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > httplib raises ValueError reading chunked content

Reply
Thread Tools

httplib raises ValueError reading chunked content

 
 
philip20060308@gmail.com
Guest
Posts: n/a
 
      03-08-2006
Hi all,
Has anyone ever seen Python 2.4.1's httplib choke when reading chunked
content? I'm using it via urrlib2, and I ran into a particular server
that returns something that httplib doesn't expect. Specifically, in
the code below where the error occurs, line == ''.

Python 2.4.1 (#2, Oct 12 2005, 01:36:32)
[GCC 3.4.4 [FreeBSD] 20050518] on freebsd6
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> req = urllib2.Request("http://www.mistyshaven.com/")
>>> f = urllib2.urlopen(req)
>>> content = f.read()

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/local/lib/python2.4/socket.py", line 285, in read
data = self._sock.recv(recv_size)
File "/usr/local/lib/python2.4/httplib.py", line 456, in read
return self._read_chunked(amt)
File "/usr/local/lib/python2.4/httplib.py", line 495, in
_read_chunked
chunk_left = int(line, 16)
ValueError: invalid literal for int():
>>>


I'm running Python 2.4.1 under FreeBSD 6.0. Interestingly, I can't
recreate the problem using Python 2.3 under OS X.

I've done a little digging for clues. First, the response headers
include:
X-Powered-By: ASP.NET
X-AspNet-Version: 1.1.4322

I reckon that if that popular server was sending out broken chunked
content, it'd be a well-known problem but that doesn't seem to be the
case. So I assume (big assumption) that it is sending correct
responses. Another clue is that the content fits all in one chunk.
Under my 2.3 installation (where I can fetch the content successfully),
len(content) == 0x303. The first chunk size reported by the server is
0x311, so I guess that adds up when one adds a fudge factor for \r\n
and so forth.

My guess is that httplib is somehow reading the blank line that
signifies the end of chunked content as part of the content. I don't
know enough about debugging HTTP conversations to go any further. Can
anyone at least confirm the problem elsewhere?

Thanks
Philip

PS - The email address with which this was posted is live; you can also
email Philip Semanchuk: my first name @ my last name .com

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: socket.makefile raises ValueError when mode = 'rt' Terry Reedy Python 0 01-09-2013 11:36 PM
Re: socket.makefile raises ValueError when mode = 'rt' Antoon Pardon Python 0 01-09-2013 02:14 PM
Re: socket.makefile raises ValueError when mode = 'rt' Dave Angel Python 0 01-09-2013 01:54 PM
socket.makefile raises ValueError when mode = 'rt' Antoon Pardon Python 0 01-09-2013 01:22 PM
Re: httplib raises ValueError reading chunked content Philip Semanchuk Python 0 03-09-2006 02:00 AM



Advertisments