Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > httplib/socket problems reading 404 Not Found response

Reply
Thread Tools

httplib/socket problems reading 404 Not Found response

 
 
Patrick Altman
Guest
Posts: n/a
 
      03-13-2007
I am attempting to use a HEAD request against Amazon S3 to check
whether a file exists or not and if it does parse the md5 hash from
the ETag in the response to verify the contents of the file so as to
save on bandwidth of uploading files when it is not necessary.

If the file exist, the HEAD works as expected and I get valid headers
back that I can parse and pull the ETag out of the dictionary using
getheader('ETag')[1:-1] (using the slice to trim off the double-quotes
in the string.

The problem lies when I attempt to send a HEAD request when no file
exists. As expected, a 404 Not Found response is sent back from
Amazon however, my test scripts seem to hang. I run python with
trace.py and it hangs here:

--- modulename: httplib, funcname: _read_chunked
httplib.py(536): assert self.chunked != _UNKNOWN
httplib.py(537): chunk_left = self.chunk_left
httplib.py(53: value = ''
httplib.py(542): while True:
httplib.py(543): if chunk_left is None:
httplib.py(544): line = self.fp.readline()
--- modulename: socket, funcname: readline
socket.py(321): data = self._rbuf
socket.py(322): if size < 0:
socket.py(324): if self._rbufsize <= 1:
socket.py(326): assert data == ""
socket.py(327): buffers = []
socket.py(32: recv = self._sock.recv
socket.py(329): while data != "\n":
socket.py(330): data = recv(1)

It eventually completes with an exception here:

File "C:\Python25\lib\httplib.py", line 509, in read
return self._read_chunked(amt)
File "C:\Python25\lib\httplib.py", line 548, in _read_chunked
chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: ''

For reference, ethereal captured the following request and response:

HEAD <REMOVED> HTTP/1.1
Host: s3.amazonaws.com
Accept-Encoding: identity
Date: Tue, 13 Mar 2007 02:54:12 GMT
Authorization: AWS <REMOVED>

HTTP/1.1 404 Not Found
x-amz-request-id: E20B4C0D0C48B2EF
x-amz-id-2: <REMOVED>
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 13 Mar 2007 02:54:16 GMT
Server: AmazonS3

Am I doing something wrong? Is this a known issue? I am an
experienced developer, but pretty new to Python and dynamic languages
in general.

Thanks,
Patrick

 
Reply With Quote
 
 
 
 
Gabriel Genellina
Guest
Posts: n/a
 
      03-13-2007
En Tue, 13 Mar 2007 00:07:55 -0300, Patrick Altman <(E-Mail Removed)>
escribió:

> I am attempting to use a HEAD request against Amazon S3 to check
> whether a file exists or not and if it does parse the md5 hash from
> the ETag in the response to verify the contents of the file so as to
> save on bandwidth of uploading files when it is not necessary.
> The problem lies when I attempt to send a HEAD request when no file
> exists. As expected, a 404 Not Found response is sent back from
> Amazon however, my test scripts seem to hang. I run python with
> trace.py and it hangs here:


Yes, it's a known problem. See this message with a self-response:
http://mail.python.org/pipermail/pyt...ch/375087.html

--
Gabriel Genellina

 
Reply With Quote
 
 
 
 
Patrick Altman
Guest
Posts: n/a
 
      03-13-2007
> Yes, it's a known problem. See this message with a self-response:http://mail.python.org/pipermail/pyt...ch/375087.html

Are there plans to include this fix in the standard Python libraries
or must I make the modifications myself (I'm running Python 2.5)?


 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      03-13-2007
En Tue, 13 Mar 2007 10:38:24 -0300, Patrick Altman <(E-Mail Removed)>
escribió:

>> Yes, it's a known problem. See this message with a
>> self-response:http://mail.python.org/pipermail/pyt...ch/375087.html


> Are there plans to include this fix in the standard Python libraries
> or must I make the modifications myself (I'm running Python 2.5)?


Submit a bug report, if not already done.
http://sourceforge.net/tracker/?group_id=5470

--
Gabriel Genellina

 
Reply With Quote
 
Patrick Altman
Guest
Posts: n/a
 
      03-14-2007
On Mar 13, 3:16 pm, "Gabriel Genellina" <(E-Mail Removed)>
wrote:
> En Tue, 13 Mar 2007 10:38:24 -0300, Patrick Altman <(E-Mail Removed)>
> escribió:
>
> >> Yes, it's a known problem. See this message with a
> >> self-response:http://mail.python.org/pipermail/pyt...ch/375087.html

> > Are there plans to include this fix in the standard Python libraries
> > or must I make the modifications myself (I'm running Python 2.5)?

>
> Submit a bug report, if not already done.http://sourceforge.net/tracker/?group_id=5470
>
> --
> Gabriel Genellina


Bug already exists at:
https://sourceforge.net/tracker/inde...70&atid=105470

In the meantime, I implemented a work around for my specific case in
the Amazon S3 library in that I implemented a head() method but am
actually just requesting a GET operation with a very small byte
range. This is essentially yielding all the same header data that I
need (md5 hash in the ETag if the file exists, 404 Not Found if it
doesn't).

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sending htp response 404 not found for a page when anonymous usersgo to it and showing the valid page when a logged in user sees it. Andy B. ASP .Net 0 12-23-2008 03:22 PM
OpenURI::HTTPError: 404 Not Found reading http://gems.rubyforge.org/? hadley wickham Ruby 1 11-24-2007 06:55 PM
Intercept IIS 404 errors? Issue with Viewstate and 404 Jonathan Folland ASP .Net 2 03-17-2005 02:32 AM
Trapping the 404 (resource not found) error Craig Wilson ASP .Net 2 10-08-2004 08:45 PM
IIS reports SmartNav.htm as a 404 File Not Found, Why So? anon ASP .Net 7 04-09-2004 12:06 PM



Advertisments