Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > URLConnection tricks

Reply
Thread Tools

URLConnection tricks

 
 
Roedy Green
Guest
Posts: n/a
 
      05-04-2004
URLConnection urlc = url.openConnection();
urlc.setAllowUserInteraction( false );
urlc.setDoInput( true );
urlc.setDoOutput( false );
urlc.setUseCaches( true );
urlc.connect();
String mime = urlc.getContentType();


How would I modify that code to avoid getting the contents of the
file, get just the headers?

How would I turn off Keep-Alive?

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
 
 
 
Thomas Fritsch
Guest
Posts: n/a
 
      05-04-2004
Roedy Green schrieb:

> URLConnection urlc = url.openConnection();
> urlc.setAllowUserInteraction( false );
> urlc.setDoInput( true );
> urlc.setDoOutput( false );
> urlc.setUseCaches( true );
> urlc.connect();
> String mime = urlc.getContentType();
>
>
>How would I modify that code to avoid getting the contents of the
>file, get just the headers?
>

don't know...

>
>How would I turn off Keep-Alive?
>

I think
urlc.setRequestProperty("Connection", "Close");
should do the trick. See http://www.faqs.org/rfcs/rfc2068.html (chapter
"14.10 Connection").
But I didn't actually test it.

--
Thomas<dot>Fritsch<squiggle>ops<dot>de

 
Reply With Quote
 
 
 
 
Gordon Beaton
Guest
Posts: n/a
 
      05-04-2004
On Tue, 04 May 2004 05:30:03 GMT, Roedy Green wrote:
> URLConnection urlc = url.openConnection();
> urlc.setAllowUserInteraction( false );
> urlc.setDoInput( true );
> urlc.setDoOutput( false );
> urlc.setUseCaches( true );
> urlc.connect();
> String mime = urlc.getContentType();
>
>
> How would I modify that code to avoid getting the contents of the
> file, get just the headers?


Use HEAD instead of GET. They are identical, except that the server
response must not include a message body. That probably rules out the
use of URLConnection though.

> How would I turn off Keep-Alive?


Include the request header "Connection: close".

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e
 
Reply With Quote
 
Chris Uppal
Guest
Posts: n/a
 
      05-04-2004
Roedy Green wrote:
> URLConnection urlc = url.openConnection();
> urlc.setAllowUserInteraction( false );
> urlc.setDoInput( true );
> urlc.setDoOutput( false );
> urlc.setUseCaches( true );
> urlc.connect();
> String mime = urlc.getContentType();
>
>
> How would I modify that code to avoid getting the contents of the
> file, get just the headers?


IIRC: cast the URLConnection to an HttpURLConnection and use
setRequestMethod("HEAD") before you connect().

BTW, a surprisingly large number of servers ignore the "HEAD" and treat it as a
"GET" -- that shouldn't affect your logic, but it can be confusing if you are
debugging with a network sniffer.

-- chris


 
Reply With Quote
 
Thomas Fritsch
Guest
Posts: n/a
 
      05-04-2004
Roedy Green schrieb:

> URLConnection urlc = url.openConnection();
> urlc.setAllowUserInteraction( false );
> urlc.setDoInput( true );
> urlc.setDoOutput( false );
> urlc.setUseCaches( true );
> urlc.connect();
> String mime = urlc.getContentType();
>
>
>How would I modify that code to avoid getting the contents of the
>file, get just the headers?
>

See
http://java.sun.com/j2se/1.4.2/docs/...va.lang.String)
==> Probably
((HttpURLConnection) urlc).setRequestMethod("HEAD");
will do. But obviously this depends on my unproved assumption, that
url.openConnection()
returns an instance of java.net.HttpURLConnection.

>
>How would I turn off Keep-Alive?
>
>
>
>

--
Thomas<dot>Fritsch<squiggle>ops<dot>de

 
Reply With Quote
 
Dave Monroe
Guest
Posts: n/a
 
      05-04-2004
Roedy Green <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>. ..
> URLConnection urlc = url.openConnection();
> urlc.setAllowUserInteraction( false );
> urlc.setDoInput( true );
> urlc.setDoOutput( false );
> urlc.setUseCaches( true );
> urlc.connect();
> String mime = urlc.getContentType();
>
>
> How would I modify that code to avoid getting the contents of the
> file, get just the headers?
>
> How would I turn off Keep-Alive?



Have a look at my implementation of wget. The HttpUrlConnection class
inherits from URLConnection. The header info is visible there.

I'm pretty sure Keep-Alive is a tcp function.

import java.net.*;
import java.io.*;
import java.util.*;

public class wget {
public static void main(String [] args) {
if(args.length == 0) {
System.err.println("usage: java wget url");
System.exit(1);
}

try {
URL u = new URL(args[0]);
HttpURLConnection h =
(HttpURLConnection)u.openConnection();

InputStream is = h.getInputStream();

BufferedReader in = new BufferedReader(
new InputStreamReader(is));

String line;

while ((line = in.readLine()) != null) {
System.out.println(line);
}

}
catch(Exception e) {
System.err.println(e);
System.exit(1);
}
}
}
 
Reply With Quote
 
John C. Bollinger
Guest
Posts: n/a
 
      05-04-2004
Dave Monroe wrote:

> Have a look at my implementation of wget. The HttpUrlConnection class
> inherits from URLConnection. The header info is visible there.


Okay.

> I'm pretty sure Keep-Alive is a tcp function.


It is, but I suspect Roedy actually meant to ask about the "Persistent
Connection" feature of HTTP 1.1. The HttpURLConnection API docs do say
"Calling the disconnect() method may close the underlying socket if a
persistent connection is otherwise idle at that time," which is at least
one avenue of approach. If checking many URLs on the same server,
however, then it may be counterproductive to do that.

> public class wget {
> public static void main(String [] args) {


[...]

> URL u = new URL(args[0]);
> HttpURLConnection h =
> (HttpURLConnection)u.openConnection();


And that should work reliably, as long as the URL is an HTTP URL.

> InputStream is = h.getInputStream();
>
> BufferedReader in = new BufferedReader(
> new InputStreamReader(is));


That, however, is pretty broken. It depends on the system's default
charset being suitable. It will work for ASCII, UTF-8, or any of the
ISO-8859 family of charsets, as they all agree on the first 128 code
points, but it will break on many others.

> String line;
>
> while ((line = in.readLine()) != null) {
> System.out.println(line);
> }


And that's pretty naive. It does not account for any content coding
that might have been specified in the header, nor (in the case of HTML
content) for a charset specified via a meta tag. It also doesn't do
anything sensible with content that is inherently binary.

[...]

That also doesn't answer the actual question, which was about obtaining
the content type without retrieving the entire content. Assuming that
the point is to reduce impact on the server, using the "HEAD" HTTP
method to make the request is the only way that has a reasonable chance
of solving the problem. If the concern is more for the client side then
the client could always close the connection without reading the content
-- for large entities that might prevent the content from being fully
retrieved when the server treats "HEAD" as "GET".


John Bollinger
http://www.velocityreviews.com/forums/(E-Mail Removed)

 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      05-04-2004
On 4 May 2004 05:23:10 -0700, (E-Mail Removed) (Dave Monroe)
wrote or quoted :

>
>I'm pretty sure Keep-Alive is a tcp function.


Connect: Keep-Alive is also a field in the GET header to say, keep the
socket open after sending the response for more traffic. Most of the
time you don't need it and you needlessly keep the server dangling.


--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      05-04-2004
On Tue, 04 May 2004 05:30:03 GMT, Roedy Green
<(E-Mail Removed)> wrote or quoted :

>How would I modify that code to avoid getting the contents of the
>file, get just the headers?
>
>How would I turn off Keep-Alive?


thanks everyone. MimeCheck 1.1 now uses this information and FileI/O
4.5 teaches it to the newbies.

see http://mindprod.com/jgloss/mime.html
http://mindprod.com/fileio.html

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Andrew Thompson
Guest
Posts: n/a
 
      05-04-2004
On Tue, 04 May 2004 17:19:57 GMT, Roedy Green wrote:

> see http://mindprod.com/jgloss/mime.html


Neat! Would you consider adding some JS(!)
to allow configuring it by URL?

If your applet accepted a 'url' parameter,
you could write the parameter into the applet
tag using JS, or default to the 'no param'
applet for those UA's with no JS.

Yes, yes. I am familiar with your aversion
to mixing JS/Java. But you can gain extra
functionality while losing nothing in this
case.

The ability to specify the entire check
as an URL would be most handy.
[ I can recall situations in the c.i.w.a.h.
and c.i.w.a.s groups where I could show
someone what is going wrong using the
added functionality. ]

--
Andrew Thompson
http://www.PhySci.org/ Open-source software suite
http://www.PhySci.org/codes/ Web & IT Help
http://www.1point1C.org/ Science & Technology
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
OutputStream from a URLConnection produces an OutOfMemory OutputStream from a URLConnection produces an OutOfMemory WinstonSmith_101@hotmail.com Java 2 10-25-2006 04:45 PM
write a string from URLConnection to a servlet dave Java 0 12-03-2003 07:47 PM
Problems encoding in utf8 with urlconnection response Bronwyn Howard Java 0 10-01-2003 06:10 AM
URLConnection to WebService gets FileNotFoundException Brock Jones Java 0 09-04-2003 12:10 AM
URLConnection and proxy Kristoffel Java 0 07-27-2003 08:27 PM



Advertisments