Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > HTTP/1.1 persistent connections

Reply
Thread Tools

HTTP/1.1 persistent connections

 
 
Alex Hunsley
Guest
Posts: n/a
 
      02-13-2004

(Sorry, can't post any complete example code for this at the minute
which is really annoying, I know.)


AFAIK, when java is using HttpURLConnections with HTTP/1.1 (as it is in
my situation), repeated connections to the same host will re-use an
existing established HTTP connection. This appears to be the case for
normal HTTP URLS - I'm doing something like this:

URL url = new URL("http://guff.here.etc/");
HttpURLConnection urlConn = url.openConnection();

urlConn.setDoInput(true);
urlConn.setDoOutput(true);
osw = new OutputStreamWriter(httpURLConn.getOutputStream());
bw = new BufferedWriter(osw);

... etc..



while I'm do this repeatedly in java, I can see by running 'netstat -a'
(windows 2000 machine) that there is indeed only one connection being used:

$ netstat -a | grep 199
TCP localhost:2282 remote.host.name:80 ESTABLISHED

$

...despite the fact that I'm doing many GETs from this URL. this is
desirable, only having one connection, as it uses less resources
(connector threads) in tomcat.


Now, if I have an https (SSL) link: (connecting to tomcat running an SSL
connector on port 19929)

URL url = new URL("https://guff.here.etc:19929/");
HttpURLConnection urlConn = url.openConnection();

// call setDefaultSSLSocketFactory(...) here...
// set the HostVerifier here...

urlConn.setDoInput(true);
urlConn.setDoOutput(true);
osw = new OutputStreamWriter(httpURLConn.getOutputStream());
bw = new BufferedWriter(osw);
....etc.

.... it works, but each GET is spawning a new connection, and a new
connector thread on tomcat.
e.g. netstat gives me the following:

$ netstat -a | grep 199
TCP localhost:2049 remote.host.name:19929 ESTABLISHED
TCP localhost:2051 remote.host.name:19929 ESTABLISHED
TCP localhost:knetd remote.host.name:19929 ESTABLISHED
TCP localhost:2055 remote.host.name:19929 ESTABLISHED
TCP localhost:2057 remote.host.name:19929 ESTABLISHED
TCP localhost:2059 remote.host.name:19929 ESTABLISHED
TCP localhost:2061 remote.host.name:19929 ESTABLISHED
...[snipped many more lines like this!]....

Because the link isn't persistent like HTTP, tomcat soon runs out of
threads as it is using a thread for each connection, and I start getting
refused requests as a result of this.

I searched around in the usual places, and found someone with the same
problem, and it turned out the cause in his case was that he wasn't
calling close() on his OutputStreamWriter that was being used to feed
his query into his GET. I am calling close() here, however, so that's
not the problem...

Just on the offchance that someone is familiar with this problem!

thanks,
alex


 
Reply With Quote
 
 
 
 
Ryan Stewart
Guest
Posts: n/a
 
      02-13-2004
"Alex Hunsley" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
>
> (Sorry, can't post any complete example code for this at the minute
> which is really annoying, I know.)
>
>
> AFAIK, when java is using HttpURLConnections with HTTP/1.1 (as it is in
> my situation), repeated connections to the same host will re-use an
> existing established HTTP connection. This appears to be the case for
> normal HTTP URLS - I'm doing something like this:
>
> URL url = new URL("http://guff.here.etc/");
> HttpURLConnection urlConn = url.openConnection();
>
> urlConn.setDoInput(true);
> urlConn.setDoOutput(true);
> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
> bw = new BufferedWriter(osw);
>
> .. etc..
>
>
>
> while I'm do this repeatedly in java, I can see by running 'netstat -a'
> (windows 2000 machine) that there is indeed only one connection being

used:
>
> $ netstat -a | grep 199
> TCP localhost:2282 remote.host.name:80 ESTABLISHED
>
> $
>
> ..despite the fact that I'm doing many GETs from this URL. this is
> desirable, only having one connection, as it uses less resources
> (connector threads) in tomcat.
>
>
> Now, if I have an https (SSL) link: (connecting to tomcat running an SSL
> connector on port 19929)
>
> URL url = new URL("https://guff.here.etc:19929/");
> HttpURLConnection urlConn = url.openConnection();
>
> // call setDefaultSSLSocketFactory(...) here...
> // set the HostVerifier here...
>
> urlConn.setDoInput(true);
> urlConn.setDoOutput(true);
> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
> bw = new BufferedWriter(osw);
> ...etc.
>
> ... it works, but each GET is spawning a new connection, and a new
> connector thread on tomcat.
> e.g. netstat gives me the following:
>
> $ netstat -a | grep 199
> TCP localhost:2049 remote.host.name:19929 ESTABLISHED
> TCP localhost:2051 remote.host.name:19929 ESTABLISHED
> TCP localhost:knetd remote.host.name:19929 ESTABLISHED
> TCP localhost:2055 remote.host.name:19929 ESTABLISHED
> TCP localhost:2057 remote.host.name:19929 ESTABLISHED
> TCP localhost:2059 remote.host.name:19929 ESTABLISHED
> TCP localhost:2061 remote.host.name:19929 ESTABLISHED
> ..[snipped many more lines like this!]....
>
> Because the link isn't persistent like HTTP, tomcat soon runs out of
> threads as it is using a thread for each connection, and I start getting
> refused requests as a result of this.
>
> I searched around in the usual places, and found someone with the same
> problem, and it turned out the cause in his case was that he wasn't
> calling close() on his OutputStreamWriter that was being used to feed
> his query into his GET. I am calling close() here, however, so that's
> not the problem...
>
> Just on the offchance that someone is familiar with this problem!
>
> thanks,
> alex


In my limited experience with SSL, the port hopping is normal. Isn't it part
of the protocol, supposed to enhance security by changing connections
frequently?


 
Reply With Quote
 
 
 
 
Alex Hunsley
Guest
Posts: n/a
 
      02-13-2004
Ryan Stewart wrote:

> "Alex Hunsley" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>
>>(Sorry, can't post any complete example code for this at the minute
>>which is really annoying, I know.)
>>
>>
>>AFAIK, when java is using HttpURLConnections with HTTP/1.1 (as it is in
>>my situation), repeated connections to the same host will re-use an
>>existing established HTTP connection. This appears to be the case for
>>normal HTTP URLS - I'm doing something like this:
>>
>> URL url = new URL("http://guff.here.etc/");
>> HttpURLConnection urlConn = url.openConnection();
>>
>> urlConn.setDoInput(true);
>> urlConn.setDoOutput(true);
>> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
>> bw = new BufferedWriter(osw);
>>
>>.. etc..
>>
>>
>>
>>while I'm do this repeatedly in java, I can see by running 'netstat -a'
>>(windows 2000 machine) that there is indeed only one connection being

>
> used:
>
>>$ netstat -a | grep 199
>> TCP localhost:2282 remote.host.name:80 ESTABLISHED
>>
>>$
>>
>>..despite the fact that I'm doing many GETs from this URL. this is
>>desirable, only having one connection, as it uses less resources
>>(connector threads) in tomcat.
>>
>>
>>Now, if I have an https (SSL) link: (connecting to tomcat running an SSL
>>connector on port 19929)
>>
>> URL url = new URL("https://guff.here.etc:19929/");
>> HttpURLConnection urlConn = url.openConnection();
>>
>> // call setDefaultSSLSocketFactory(...) here...
>>// set the HostVerifier here...
>>
>> urlConn.setDoInput(true);
>> urlConn.setDoOutput(true);
>> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
>> bw = new BufferedWriter(osw);
>>...etc.
>>
>>... it works, but each GET is spawning a new connection, and a new
>>connector thread on tomcat.
>>e.g. netstat gives me the following:
>>
>>$ netstat -a | grep 199
>> TCP localhost:2049 remote.host.name:19929 ESTABLISHED
>> TCP localhost:2051 remote.host.name:19929 ESTABLISHED
>> TCP localhost:knetd remote.host.name:19929 ESTABLISHED
>> TCP localhost:2055 remote.host.name:19929 ESTABLISHED
>> TCP localhost:2057 remote.host.name:19929 ESTABLISHED
>> TCP localhost:2059 remote.host.name:19929 ESTABLISHED
>> TCP localhost:2061 remote.host.name:19929 ESTABLISHED
>>..[snipped many more lines like this!]....
>>
>>Because the link isn't persistent like HTTP, tomcat soon runs out of
>>threads as it is using a thread for each connection, and I start getting
>>refused requests as a result of this.
>>
>>I searched around in the usual places, and found someone with the same
>>problem, and it turned out the cause in his case was that he wasn't
>>calling close() on his OutputStreamWriter that was being used to feed
>>his query into his GET. I am calling close() here, however, so that's
>>not the problem...
>>
>>Just on the offchance that someone is familiar with this problem!
>>
>>thanks,
>>alex

>
>
> In my limited experience with SSL, the port hopping is normal. Isn't it part
> of the protocol, supposed to enhance security by changing connections
> frequently?


I don't think so... and it's majorly taking up server resources for
tomcat as well, as each different connection uses up a different thread.
This all suggests that I'm not shutting down my connection properly at
my end....

Well, I took out the guts of the code to make a self-contained runnable
example, it's put up here:

http://www.lardus.myby.co.uk/SSLthrasher.html

(it's preformatted text with no line wrap, so watch out!)

The bummer is that when I run *this*, netstat -a shows only one
connection being made, which is what I want! <g> bums.. now have to
work out why this code does things correctly, whereas the code it's
yanked out from (and based on!) doesn't...

This code may be of help to others wanting to do SSL queries as well.

thanks,
alex



 
Reply With Quote
 
Chris Smith
Guest
Posts: n/a
 
      02-13-2004
Alex Hunsley wrote:
> AFAIK, when java is using HttpURLConnections with HTTP/1.1 (as it is in
> my situation), repeated connections to the same host will re-use an
> existing established HTTP connection. This appears to be the case for
> normal HTTP URLS [...]


> Now, if I have an https (SSL) link: (connecting to tomcat running an SSL
> connector on port 19929) [...]


> ... it works, but each GET is spawning a new connection, and a new
> connector thread on tomcat.


Yep. IME, Java's implementation of HttpsURLConnection doesn't do keep-
alive connections. You could switch to a different HTTP access library
(for example, Jakarta Commons HttpClient) or try to provide a better
implementation of HttpsURLConnection through the pluggable URLConnection
architecture...

Or you could realize that there's something fishy going on in your
client. If you want to establish a connection and then close it, then
establish another connection, and so forth, then go ahead and do that.
If you're accumulating connections over time, it looks like you aren't
properly closing prior connections.

> I searched around in the usual places, and found someone with the same
> problem, and it turned out the cause in his case was that he wasn't
> calling close() on his OutputStreamWriter that was being used to feed
> his query into his GET. I am calling close() here, however, so that's
> not the problem...


Well, if that's really the case, then you do need to post a reproducible
test case.

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
Reply With Quote
 
Alex Hunsley
Guest
Posts: n/a
 
      02-13-2004
Alex Hunsley wrote:
> Ryan Stewart wrote:
>
>> "Alex Hunsley" <(E-Mail Removed)> wrote in message
>> news:(E-Mail Removed)...
>>
>>> (Sorry, can't post any complete example code for this at the minute
>>> which is really annoying, I know.)
>>>
>>>
>>> AFAIK, when java is using HttpURLConnections with HTTP/1.1 (as it is in
>>> my situation), repeated connections to the same host will re-use an
>>> existing established HTTP connection. This appears to be the case for
>>> normal HTTP URLS - I'm doing something like this:
>>>
>>> URL url = new URL("http://guff.here.etc/");
>>> HttpURLConnection urlConn = url.openConnection();
>>>
>>> urlConn.setDoInput(true);
>>> urlConn.setDoOutput(true);
>>> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
>>> bw = new BufferedWriter(osw);
>>>
>>> .. etc..
>>>
>>>
>>>
>>> while I'm do this repeatedly in java, I can see by running 'netstat -a'
>>> (windows 2000 machine) that there is indeed only one connection being

>>
>>
>> used:
>>
>>> $ netstat -a | grep 199
>>> TCP localhost:2282 remote.host.name:80 ESTABLISHED
>>>
>>> $
>>>
>>> ..despite the fact that I'm doing many GETs from this URL. this is
>>> desirable, only having one connection, as it uses less resources
>>> (connector threads) in tomcat.
>>>
>>>
>>> Now, if I have an https (SSL) link: (connecting to tomcat running an SSL
>>> connector on port 19929)
>>>
>>> URL url = new URL("https://guff.here.etc:19929/");
>>> HttpURLConnection urlConn = url.openConnection();
>>>
>>> // call setDefaultSSLSocketFactory(...) here...
>>> // set the HostVerifier here...
>>>
>>> urlConn.setDoInput(true);
>>> urlConn.setDoOutput(true);
>>> osw = new OutputStreamWriter(httpURLConn.getOutputStream());
>>> bw = new BufferedWriter(osw);
>>> ...etc.
>>>
>>> ... it works, but each GET is spawning a new connection, and a new
>>> connector thread on tomcat.
>>> e.g. netstat gives me the following:
>>>
>>> $ netstat -a | grep 199
>>> TCP localhost:2049 remote.host.name:19929 ESTABLISHED
>>> TCP localhost:2051 remote.host.name:19929 ESTABLISHED
>>> TCP localhost:knetd remote.host.name:19929 ESTABLISHED
>>> TCP localhost:2055 remote.host.name:19929 ESTABLISHED
>>> TCP localhost:2057 remote.host.name:19929 ESTABLISHED
>>> TCP localhost:2059 remote.host.name:19929 ESTABLISHED
>>> TCP localhost:2061 remote.host.name:19929 ESTABLISHED
>>> ..[snipped many more lines like this!]....
>>>
>>> Because the link isn't persistent like HTTP, tomcat soon runs out of
>>> threads as it is using a thread for each connection, and I start getting
>>> refused requests as a result of this.
>>>
>>> I searched around in the usual places, and found someone with the same
>>> problem, and it turned out the cause in his case was that he wasn't
>>> calling close() on his OutputStreamWriter that was being used to feed
>>> his query into his GET. I am calling close() here, however, so that's
>>> not the problem...
>>>
>>> Just on the offchance that someone is familiar with this problem!
>>>
>>> thanks,
>>> alex

>>
>>
>>
>> In my limited experience with SSL, the port hopping is normal. Isn't
>> it part
>> of the protocol, supposed to enhance security by changing connections
>> frequently?

>
>
> I don't think so... and it's majorly taking up server resources for
> tomcat as well, as each different connection uses up a different thread.


Aha, I've just noticed that my test program *does* actually open a new
SSL connection on a different (increasing) port... my problem , when it
occurs, is that old connections just aren't closing.
The detective work continues....

alex



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Persistent field and Persistent properties - difference gk Java 7 10-12-2010 09:43 PM
Lingering Idle Oracle Connections? Persistent connections? jobs ASP .Net 2 11-16-2007 03:37 AM
HTTP Persistent connections on Apache web-server webdev50 Java 0 09-26-2005 05:50 AM
ASP.net Debugger and Persistent Connections =?Utf-8?B?bWFuamk=?= ASP .Net 0 08-04-2005 06:05 AM
Persistent connections possible in ASP.Net? Jim Hubbard ASP .Net 2 01-21-2005 09:10 PM



Advertisments