«

Hi,

I want to write a Java application that can download a file (url)
fast.

To do this, I want to open multiple connections on the same file(url)
and each connection will download a chunk of the file.

Any clue how I can do this ?

thanks,
JD

On 13 Jul 2004 21:08:00 -0700, (Jean-Daniel
Gamache) wrote or quoted :

>To do this, I want to open multiple connections on the same file(url)
>and each connection will download a chunk of the file.
>
>Any clue how I can do this ?
see http://mindprod.com/fileio.html

Tell it you want to do a CGI get.

Look up the HTTP headers to control the offset and length of the chunk
you want, and spin off three threads after you figure out how big the
whole thing is with a HEAD.


--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.

"Jean-Daniel Gamache" <> wrote in message
news: m...
> Hi,
>
> I want to write a Java application that can download a file (url)
> fast.
>
> To do this, I want to open multiple connections on the same
file(url)
> and each connection will download a chunk of the file.
>
> Any clue how I can do this ?
>
> thanks,
> JD

Note: I would expect your technique to be SLOWER,
not faster.

The speed of the download will almost certainly be
determined by the speed of the physical link between
the two computers. Creating two separate TCP/IP
connections over the same link (e.g., modem and
telephone line, modem and cable, etc.) will increase
the overhead, i.e., the number of bits dedicated to
connection creation and control, but will not increase
the total number of bits per second that can flow over
the link. So a smaller number of data bits from the file
you want will be downloaded per second than if you
had one connection with more of the bits dedicated
to data instead of control.

If you have two physical links, e.g., two phone lines
that you can use at once, that's another story.

However, even in that case, you'd also need to get
the server to cooperate with you. To the best of
my knowledge, neither the http nor the ftp protocols
will download two halves of a file in this way. So
you'd need a program on the server to split the
file in half into two separate files - then go after
each half on each connection. If it's a server that's
not under your control, I don't think you'll be able
to do that.

Alan

Alan Meyer wrote:
>
> Note: I would expect your technique to be SLOWER,
> not faster.
>

[snip]

>
> However, even in that case, you'd also need to get
> the server to cooperate with you. To the best of
> my knowledge, neither the http nor the ftp protocols
> will download two halves of a file in this way. So
> you'd need a program on the server to split the
> file in half into two separate files - then go after
> each half on each connection. If it's a server that's
> not under your control, I don't think you'll be able
> to do that.

Well, that technique has made a significant speed increase in the past,
due to a couple of factors you're missing.

First, many servers cap their upload speed *per connection*. If you
open multiple connections, you can get much greater speed, up to the
available bandwidth between your computer and the server.

Second, the server does cooperate with the user on this, if it supports
"resume" style functionality. Many web and FTP servers do support
resuming a download, and therefore support this technique.

And third... Many programs made when this style of download manager was
popular included functionality to search for the same file on multiple
servers, and download different parts of it from different servers.
When this worked correctly (not that often, which is why it's pretty
much abandoned for this type of downloading now), it also provided a
giant speed boost.