Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Creating lots of folders on ftp server

Reply
Thread Tools

Creating lots of folders on ftp server

 
 
Raimon Fs
Guest
Posts: n/a
 
      10-06-2009
Hello,


I have to check some folders on a ftp server, if they don't exist,
create it.


The main folder is:

project

the project has some folders, like 1,2,3,4,5,...

and each folder have 12 more folders, one for each month, and this
folders have many folders as days the current month have.


My code works, but I think is pretty slow and always, I'm getting an
EOFError: end of file reached after creating 50 folders, always.

How can I trace to find the error ?

thanks,

r.


date=Date.new(2008,1,1)
res.each{ |tuple|

row_current+=1

print "\n\n Row "+row_current.to_s+"/"+rows_to_fetch.to_s+" => "

print tuple['id'], ' ', tuple['name']

project=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
date.month+"/"+'%02d' % date.day
project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d'
% date.month
project_year=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s
project_id=ftp_acct+"/"+tuple['id']


begin # project_id
ftp.chdir(project_id)
rescue Net::FTPPermError, NameError => boom
ftp.mkdir("/"+project_id)

print "Folder #{project_id} created"
end


begin # project_year
ftp.chdir(project_year)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_year)
print "Folder #{project_year} created"
end


for i in 1..12

project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
i

begin # project_month
ftp.chdir(project_month)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_month)
print "Folder #{project_month} created"
end


if i == 12
d=Date.new(date.year+1,1,1)-1
else
d=Date.new(date.year.to_i,i+1,1)-1
end



puts "\n #{d.to_s}"


for days in 1..d.day

project_day=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
i+"/"+'%02d' % days

begin # project_day
ftp.chdir(project_day)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_day)
print "Folder #{project_day} created"
end

print " \nFolder #{project_day} exists "


end # for days



end # for i in 1..12


} # res.each{ |tuple|
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Chris Mowforth
Guest
Posts: n/a
 
      10-06-2009
Paste the error you get. You should see line numbers indicating the
method call(s) that bailed on you.

Incidentally I think you could do this more simply using some recursion
and a couple more ruby idioms.

m0wf
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Raimon Fs
Guest
Posts: n/a
 
      10-06-2009
Chris Mowforth wrote:
> Paste the error you get. You should see line numbers indicating the
> method call(s) that bailed on you.
>
> Incidentally I think you could do this more simply using some recursion
> and a couple more ruby idioms.
>
> m0wf


The error I get is from the ftp library: reach end of line

As it starts creating folders, is getting slower and slower and finally,
at 50, end of reach line

and yes, I think I can do it more 'ruby like', but I'm coming from a
really bored language ...



thanks,

r.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Hassan Schroeder
Guest
Posts: n/a
 
      10-06-2009
On Mon, Oct 5, 2009 at 11:48 PM, Raimon Fs <(E-Mail Removed)> wrote:

> I have to check some folders on a ftp server, if they don't exist,
> create it.


This may not fit your use case, but it might be easier and faster to
create the directory structure locally and rsync it to the server.

Just a thought.
--
Hassan Schroeder ------------------------ http://www.velocityreviews.com/forums/(E-Mail Removed)
twitter: @hassan

 
Reply With Quote
 
Raimon Fs
Guest
Posts: n/a
 
      10-08-2009
Chris Mowforth wrote:
> Paste the error you get. You should see line numbers indicating the
> method call(s) that bailed on you.



EOFError: end of file reached

method readline in ftp.rb at line 211
method getline in ftp.rb at line 211
method getmultiline in ftp.rb at line 221
method getresp in ftp.rb at line 235
method voidresp in ftp.rb at line 251
method voidcmd in ftp.rb at line 274
method synchronize in monitor.rb at line 242
method voidcmd in ftp.rb at line 272
method chdir in ftp.rb at line 674
at top level in createFoldersOnFTP.rb at line 161
method each in createFoldersOnFTP.rb at line 156
at top level in createFoldersOnFTP.rb at line 156
method each in createFoldersOnFTP.rb at line 133
at top level in createFoldersOnFTP.rb at line 133
method each in createFoldersOnFTP.rb at line 96
at top level in createFoldersOnFTP.rb at line 96


maybe could be a memory leak, as it goes slower, slower, slower, and
after 50 folders created, the error appears...


>
> Incidentally I think you could do this more simply using some recursion
> and a couple more ruby idioms.


can you give me some hints on this, please ?


thanks,


regards,

r.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Raimon Fs
Guest
Posts: n/a
 
      10-08-2009
I have more information ....

Just create an array and add elements with the folders names you want to
create.


ftp_folder_create.each{ |folder|

print "\n Creating folder ",folder

ftp.mkdir(folder)

end


}

All works great, if they folders doesn't exist and you can create them.


Add error control and check before creating the folder, if it really
exists, and you'll get the error after creating 50 folders.


ftp_folder_create.each{ |folder|

print "\n Creating folder ",folder

begin

ftp.chdir(folder)

rescue Net::FTPPermError, NameError => boom # it doesn't exist => create
it

ftp.mkdir(folder)

end


}


Anyone can confirm this ?

thanks,


r.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Chris Mowforth
Guest
Posts: n/a
 
      10-09-2009
Sounds reasonable.

If you HAVE to use ftp, personally I'd start creating a nested array
representing the folder structure, then start trying to get Net::FTP to
iterate through it.

Alternatively if you could store the absolute paths as array elements, I
think it'd be much more efficient than calling #chdir all the time. Of
course this conflates the folder structure with the ruby array.

And try puts rather than print, might save you some typing.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Looking for lots of words in lots of files brad Python 9 06-19-2008 07:59 AM
Downloading lots and lots and lots of files coolneo Perl Misc 9 01-30-2007 02:34 PM
'Undifined commands' on FTP Server, when using FTP 'put' and 'quit' in Client Asaf Sinai Perl Misc 1 07-04-2006 01:02 PM
$ftp->rget() does not work for MS Windows FTP Server semovrs@concord.edu Perl Misc 3 12-18-2004 08:03 AM
Net::FTP problems getting files from Windows FTP server, but not Linux FTP Server. D. Buck Perl Misc 2 06-29-2004 02:05 PM



Advertisments