Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Timeout in urllib2

Thread Tools

Timeout in urllib2

Dan M
Posts: n/a
I'm writing a system monitor script that needs to load web pages. I'm
using urllib2.urlopen to get the pages, and I'm attempting to set the
timeout value using socket.defaulttimeout.

Specifically, I'm calling socket.defaultttimeout(10), then calling
urllib2.urlopen to fetch a web page that never gets delivered. My code
waits about 30 seconds before terminating.

I am about to add threading to my app so that delays on a few servers
won't be a critical issue, but I'd still like to understand why the call
to socket.defaulttimeout doesn't affect the timeout on my urlopen calls.

My code follows.

import socket, time

import urllib2

def doHttpTest():
url = "http://url.that.never.returns"
t_start = time.time()
if httptest(url):
print "Error on site ", url
t_end = time.time()
t_diff = t_end - t_start

def httptest(url):
timeout = 10
req = urllib2.Request(url)
except urllib2.HTTPError, e:
if e.code == 401:
return 1
elif e.code == 404:
return 1
elif e.code == 503:
return 1
return 1
except urllib2.URLError, e:
return 1
return 0

if __name__ == '__main__':
except KeyboardInterrupt:
print "Exiting..."

Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
urllib2 urlopen timeout option warpy Python 0 01-03-2009 06:29 PM
Problem with: urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) Josef Cihal Python 0 09-05-2005 11:26 AM
urllib2 and timeout question Gerard C Blais Python 2 05-03-2005 12:07 PM
Timeout::timeout and Socket timeout Mark Probert Ruby 1 10-06-2004 09:30 AM
urllib2 timeout?? Doug Gray Python 1 04-29-2004 09:06 PM