Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Very slow opening of client connections to localhost with httplib

Reply
Thread Tools

Very slow opening of client connections to localhost with httplib

 
 
Martin Fuzzey
Guest
Posts: n/a
 
      12-12-2003
I am using xmlrpclib (based on httplib) in Python 2.3 on Mandrake
Linux.

When my client attempts to connect to a server using a
"http://localhostort" style URL there is a long delay before the
connection is established (it finally works).

This certainly smells a name resolution problem but ping localhost,
telnet localhost etc all work fine.

Tracking down a bit it appears that the problem comes from the
getaddrinfo() call in code in HttpConnection.connect() :
def connect(self):
"""Connect to the host and port specified in __init__."""
msg = "getaddrinfo returns an empty list"
for res in socket.getaddrinfo(self.host, self.port, 0,
socket.SOCK_STREAM):

To test this I did :

import socket
socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)

and sure enough it blocked for several seconds before finally
returning 127.0.0.1

However
socket.gethostbyname("localhost")
returns immediately

If I remplace the 0 in the getaddrinfo call above by socket.AF_INET
all is fine as well.

So I did a tcpdump to see what it was sending to the nameserver and
got

18:36:01.790045 X.Y.Z.41.32961 > X.Y.Z.12.53: 42057+ AAAA?
localhost.besancon.parkeon.com. (4 (DF)
18:36:01.790526 X.Y.Z.12.53 > X.Y.Z.41.32961: 42057* 0/1/0 (95)
18:36:01.790652 X.Y.Z.41.32961 > X.Y.Z.12.53: 42058+ AAAA? localhost.
(27) (DF)
18:36:06.792898 X.Y.Z.41.32962 > X.Y.Z.215.53: 42058+ AAAA?
localhost. (27) (DF)
18:36:06.793134 X.Y.Z.215.53 > X.Y.Z.41.32962: 42058 ServFail 0/0/0
(27)
18:36:06.793301 X.Y.Z.41.32962 > X.Y.Z.12.53: 42058+ AAAA? localhost.
(27) (DF)
18:36:11.802998 X.Y.Z.41.32963 > X.Y.Z.215.53: 42058+ AAAA?
localhost. (27) (DF)
18:36:11.803222 X.Y.Z.215.53 > X.Y.Z.41.32963: 42058 ServFail 0/0/0
(27)
18:36:13.356344 X.Y.Z.12.53 > X.Y.Z.41.32959: 42056 ServFail 0/0/0
(27)

(my machine is X.Y.Z.41 and the NS is X.Y.Z.12)

When AF_INET or gethostbyname() is used there is no network
communication.

This appears to be a problem with IPV6 DNS queries?? (the AAAA
records). We don't use IPV6 but the 0 parameter (AF_UNSPEC) used in
httplib seems to cause them to be sent.

Anyone else had this problem or got any ideas?

The problem does NOT occur under windows.

Thanks,

Martin
 
Reply With Quote
 
 
 
 
Peter Hansen
Guest
Posts: n/a
 
      12-12-2003
Martin Fuzzey wrote:
>
> To test this I did :
>
> import socket
> socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)
>
> and sure enough it blocked for several seconds before finally
> returning 127.0.0.1
>
> However
> socket.gethostbyname("localhost")
> returns immediately


Any chance your /etc/host.conf, /etc/resolv.conf or /etc/hosts files
are buggering this up somehow? (It doesn't explain gethostbyname()
returning immediately, unless perhaps that function internally handles
localhost "magically".) Just a thought.

In my experience, delays like this generally result from host.conf
having "order bind,hosts" instead of the other way around, plus perhaps
an inappropriate "options timeoutxx" value in resolv.conf.

I haven't ever seen an issue with localhost, however, but I've never
been on a machine that didn't have the right entry in /etc/hosts.

-Peter
 
Reply With Quote
 
 
 
 
Jarkko Torppa
Guest
Posts: n/a
 
      12-13-2003
In article <(E-Mail Removed) >, Martin Fuzzey wrote:
> I am using xmlrpclib (based on httplib) in Python 2.3 on Mandrake
> Linux.
>
> When my client attempts to connect to a server using a
> "http://localhostort" style URL there is a long delay before the
> connection is established (it finally works).

....
> import socket
> socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)
>
> and sure enough it blocked for several seconds before finally
> returning 127.0.0.1
>
> If I remplace the 0 in the getaddrinfo call above by socket.AF_INET
> all is fine as well.


Try adding
::1 localhost

to your /etc/hosts

Or delete IPv6 from your kernel.

--
Jarkko Torppa, Elisa Internet
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with slow httplib connections on Windows (and maybe otherplatforms) Christoph Zwerschke Python 5 02-02-2009 06:43 AM
httplib VERY slow reacocard Python 1 04-06-2008 04:54 AM
httplib and HTTPS Connections runningwild Python 5 10-19-2006 12:08 AM
Quick Book file access very very very slow Thomas Reed Computer Support 7 04-09-2004 08:09 PM
HttpModule -- how to intercept urls like http://localhost/abc/def or http://localhost/abc/def/ where abc, def are non virtual dir Jiong Feng ASP .Net 0 11-19-2003 05:29 AM



Advertisments