Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > recursively pull web site?

Reply
Thread Tools

recursively pull web site?

 
 
Thomas Weidenfeller
Guest
Posts: n/a
 
      06-24-2004
Andy Fish wrote:
> Whether it's an appropriate mechanism for mirroring software is another
> question


rsync or if nothing else, rdist. No need to re-invent the wheel.

/Thomas
 
Reply With Quote
 
 
 
 
Roedy Green
Guest
Posts: n/a
 
      06-24-2004
On Thu, 24 Jun 2004 08:54:38 +0200, Thomas Weidenfeller
<(E-Mail Removed)> wrote or quoted :

>
>rsync or if nothing else, rdist. No need to re-invent the wheel.


Rsync has two problems. It requires the Rsync software to run on a
webserver, and it uses its own protocol you have to arrange to tunnel
through firewalls. In some bureaucratic situations these can be
showstoppers.

I argued for Rsync with a researchers from the pharmaceutical
companies. They insisted I write something without these two
drawbacks. The result is called The Replicatior. See
http://mindprod.com/zips/java/replicator.html

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
 
 
 
Thomas Weidenfeller
Guest
Posts: n/a
 
      06-28-2004
Roedy Green wrote:
> Rsync has two problems. It requires the Rsync software to run on a
> webserver,


No, a remote shell account (ssh would be best, rsh would also do), is
all you need on the remote system. You could set up a remote sync
server, but you don't have to.

> and it uses its own protocol


No, it uses the shell's protocol, unless you use a sync server.

> you have to arrange to tunnel
> through firewalls.


Well, if you have to remote administer a webserver, ssh is usually
permitted.

> I argued for Rsync with a researchers from the pharmaceutical
> companies.


If you told them the same as you told us here, well, yes, I can
understand why they argued.

/Thomas
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      06-29-2004
On Mon, 28 Jun 2004 10:12:00 +0200, Thomas Weidenfeller
<(E-Mail Removed)> wrote or quoted :

>No, a remote shell account (ssh would be best, rsh would also do), is
>all you need on the remote system. You could set up a remote sync
>server, but you don't have to.


Even that can be hard to arrange. Most ISPs won't let you run any
software at all. All you get is a vanilla HTTP server running none of
your code. Life in a big company can be almost as restrictive.

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      06-29-2004
On Mon, 28 Jun 2004 10:12:00 +0200, Thomas Weidenfeller
<(E-Mail Removed)> wrote or quoted :

>If you told them the same as you told us here, well, yes, I can
>understand why they argued.


They could not run any code at all on their webservers. Further, all
clients were deeply behind a variety of corporate firewalls with no
political clout to get anything modified.

All it would take would be one client to be unable to communicate to
spoil the project.

I argued for running on a third party server, but that too was not
permitted. Part of the problem was the high confidentiality.



--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      06-29-2004
On Mon, 28 Jun 2004 10:12:00 +0200, Thomas Weidenfeller
<(E-Mail Removed)> wrote or quoted :

>If you told them the same as you told us here, well, yes, I can
>understand why they argued.


You might want to check the rsync Java glossary entry for accuracy.

See http://mindprod.com/jgloss/rsync.html


--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Concurrent threads to pull web pages? Gilles Ganault Python 5 10-02-2009 03:09 PM
weak pull up and pull down krithiga81@yahoo.com VHDL 2 06-28-2006 02:18 PM
Sorting bookmarks recursively? taras.di@gmail.com Firefox 0 07-31-2005 06:02 AM
Best way to recursively populate a tree based on SQL Server 2000 backend Mark ASP .Net 9 05-28-2005 08:54 PM
pull image file from SQL server then displayed in web browser Jason Java 1 04-20-2004 05:40 AM



Advertisments