Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > web page retrieval

Reply
Thread Tools

web page retrieval

 
 
anonymous@coolgroups.com
Guest
Posts: n/a
 
      05-27-2004
What is the simplest way to retrieve a web page in Java?
Ideally, I want to pass in a URL and get back a String.

 
Reply With Quote
 
 
 
 
Chris Smith
Guest
Posts: n/a
 
      05-27-2004
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> What is the simplest way to retrieve a web page in Java?
> Ideally, I want to pass in a URL and get back a String.


Since a URL isn't guaranteed to return text data and you don't know the
encoding head of time, it's not that simple. You can certainly create a
URL and get an InputStream, but to convert to characters, you'll need to
use the Content-Type response header to deduce the encoding.
Unfortunately, that's rather complex so I'll wait for you to say you
need it, and then maybe I or someone else will have time to write it for
you. The basics are:

URL url = new URL("http://something");
InputStream stream = url.openStream();

try
{
// read the data from the stream
}
finally
{
stream.close();
}

or, if you want to convert to text with a given character encoding:

URL url = new URL("http://something");
InputStream stream = url.openStream();

try
{
InputStreamReader reader = new InputStreamReader(
stream, "encoding-name");

// read the data from the reader
}
finally
{
stream.close();
}

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
Reply With Quote
 
 
 
 
Andrew Thompson
Guest
Posts: n/a
 
      05-27-2004
On Thu, 27 May 2004 18:03:23 GMT, (E-Mail Removed) wrote:

> What is the simplest way to retrieve a web page in Java?
> Ideally, I want to pass in a URL and get back a String.


Isn't it about time you Bob, that you
found the JavaDocs and tutorials, and
headed over to c.l.j.help?

--
Andrew Thompson
http://www.PhySci.org/ Open-source software suite
http://www.PhySci.org/codes/ Web & IT Help
http://www.1point1C.org/ Science & Technology
 
Reply With Quote
 
kaeli
Guest
Posts: n/a
 
      05-27-2004
In article <(E-Mail Removed) >,
(E-Mail Removed) enlightened us with...
> What is the simplest way to retrieve a web page in Java?
> Ideally, I want to pass in a URL and get back a String.
>
>


A snippet from code I have that does this very thing...


URL url=null;
String line=null;
InputStream in=null;
BufferedReader input=null;
URLConnection connection=null;
String pageContent="";


url = new URL("http://www.google.com");
connection = url.openConnection();
connection.setDoInput(true);
in = connection.getInputStream();
input = new BufferedReader(new InputStreamReader(in));

while ((line = input.readLine()) != null)
{
// put the whole thing into a string
pageContent += line;
}


--
--
~kaeli~
The man who fell into an upholstery machine is fully
recovered.
http://www.ipwebdesign.net/wildAtHeart
http://www.ipwebdesign.net/kaelisSpace

 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      05-28-2004
On Thu, 27 May 2004 18:03:23 GMT, (E-Mail Removed) wrote or
quoted :

>What is the simplest way to retrieve a web page in Java?
>Ideally, I want to pass in a URL and get back a String.


see http://mindprod.com/fileio.html

tell it you want to learn to do a CGI get. This what browsers do to
get webpages.

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
BlackFoliage
Guest
Posts: n/a
 
      05-28-2004
(E-Mail Removed) wrote:
> What is the simplest way to retrieve a web page in Java?
> Ideally, I want to pass in a URL and get back a String.
>


The Jakarta commons HttpClient works great.

http://jakarta.apache.org/commons/httpclient/

-Patrick
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
ASP.net ques: Webpage data retrieval and parsing news.microsoft.com ASP .Net 5 02-01-2005 05:15 AM
ASP.net Ques: Image Retrieval news.microsoft.com ASP .Net 2 01-17-2005 02:02 AM
Re: HTML Form value retrieval Rick Spiewak ASP .Net 1 05-07-2004 04:36 PM
Form information retrieval =?Utf-8?B?U3dheQ==?= ASP .Net 1 04-18-2004 09:54 PM
PDF "Title" and "Comments" field retrieval George Sew ASP .Net 2 02-12-2004 06:49 PM



Advertisments