Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Need to download 20000 pdf files

Reply
Thread Tools

Need to download 20000 pdf files

 
 
Hemant
Guest
Posts: n/a
 
      03-02-2005
I am working on a project that requires me to have access to more than
20000 pdf files. Any suggestions on how to go about searching over the
internet and be able to download the files?

 
Reply With Quote
 
 
 
 
Tad McClellan
Guest
Posts: n/a
 
      03-02-2005
Hemant <> wrote:

> be able to download the files?



use LWP::Simple;
my $pdf = get 'http://some.domain/directory/file.pdf';


--
Tad McClellan SGML consulting
Perl programming
Fort Worth, Texas
 
Reply With Quote
 
 
 
 
Peter Wyzl
Guest
Posts: n/a
 
      03-02-2005
"Hemant" <> wrote in message
news: oups.com...
:I am working on a project that requires me to have access to more than
: 20000 pdf files. Any suggestions on how to go about searching over the
: internet and be able to download the files?


Here is something I ran to download a bunch of .swf files into the swf
directory on my c: drive
chdir 'c:/swf';
for ('001' .. '340'){
next if (-e "c:/swf/a_0${_}.swf");
system "lwp-download http://www.swfsite.com/a_0${_}.swf";
}
exit;



lwp-download comes with Activeperl

The files I need were all named a_0***.swf where *** represents the numbers
from 001 to 340

Modify it to suit your needs (assuming you have Windows and lwp-download. I
don't know if Activestate's perl for Unix includes lwp-download.



P

--
print "Just another Perl Hacker";



 
Reply With Quote
 
Gregory Toomey
Guest
Posts: n/a
 
      03-03-2005
Hemant wrote:

> I am working on a project that requires me to have access to more than
> 20000 pdf files. Any suggestions on how to go about searching over the
> internet and be able to download the files?


Assuming one directory tree:
http://www.gnu.org/software/wget/wget.html

gtoomey
 
Reply With Quote
 
Shane (aka froggy)
Guest
Posts: n/a
 
      03-03-2005
On Wed, 02 Mar 2005 12:22:56 -0800, Hemant wrote:

> I am working on a project that requires me to have access to more than
> 20000 pdf files. Any suggestions on how to go about searching over the
> internet and be able to download the files?


er.. from all the other posts it seems an easy task
but I dont get it.. are you asking how to just download the pdf
documents.. or search them out over the net.. or search out random pdf
documents for downloading?
--

Hardware, n.: The parts of a computer system that can be kicked

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
SproutCore--over 20000 lines of new code! David Mark Javascript 49 01-14-2010 01:10 PM
Postscript to PDF with pdf-tools, pdf-writer, or other Sean Nakasone Ruby 1 04-14-2008 09:13 PM
PDF::Writer, create pdf and insert in other pdf file. Ricardo Pog Ruby 1 03-26-2008 08:24 PM
how to get 20000 html pages content quickly from one server? JuHui Python 1 03-17-2006 03:09 AM
20000 List Items Peter ASP .Net 3 01-29-2006 07:22 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57