Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: files,folders,paths

Reply
Thread Tools

Re: files,folders,paths

 
 
Dave Angel
Guest
Posts: n/a
 
      02-14-2011
(You forgot to include the list in your response, and you neglected to
properly quote the message you were replying to. That makes it quite
hard to separate your remarks from mine)

On 02/13/2011 07:19 PM, jon hayes wrote:
> c:\users\name\backup\ #win7
>> c:\docs and settings\name\app data\backup\ #winxp
>> /home/name/backup #linux and mac

>
> In what way do those imply a need for chdir() ? What's wrong with os.path.join() ?
> the only commonality is the \backup
> the chdir() is there to build the path name of the destination.
> so c:\users\name\backup\folder1\folder2\file3
> in my code, doing the substution.


Well, since chdir() is illegal inside the os.walk() context and
unnecesary, I suggest you debug your original technique. Build the
names with os.path.join(), as I've suggested twice before, and others
have mentioned as well.

>
> these 3 lines
> os.chdir(path)
> cwd = os.getcwd()
> leftover = cwd.replace(source, '')
> in this case leftover = \folder1\folder2
> which gets added to the path for the destination in
> shutil.copytree(os.getcwd(), destination+leftover)
>
> i looked at the relpath, it did not do exacly what i wanted. in version 11 of the code this works out well.


I can't see how os.path.relpath() would be useful here.



>
> "And I quote from facbac-012a.py:
> os.chdir("v:")"
> i don't understand. is this a question?


It is in context. If you read my message, I had just quoted you saying
you did not change drive letters.

>
> "I suggest you actually try writing a simple loop using os.walk to just
> display the names of all the files in and under a specified directory.
> Then maybe you'll understand what path, dirs, and files actually are."
> i started with that. look at the attached file, date-linuxversion.py.
> the relative paths are not, quite, right.


Well file you enclosed shows that you don't understand os.walk(). So
learn it before trying to use it in a more complex setting. Why would
you use the following sequence?

filestring = str(files)
filestring = filestring.strip("[")
filestring = filestring.strip("]")
filestring = filestring.strip("'")

This is no way to process a list of filenames.


> the thing with the copytree is, the line above is sopposed to check whether that folder exists. if it does not exist at all, then why not copy the whole thing?
> same sort of thing with files. if not exist then do copy. that way nothing should get clobbered.
> this is the observed behavior in version 11.
>


Why not copy the whole tree? Because once the code works, you'll be
copying it many times.

DaveA
 
Reply With Quote
 
 
 
 
ecu_jon
Guest
Posts: n/a
 
      02-14-2011
On Feb 13, 8:48*pm, Dave Angel <(E-Mail Removed)> wrote:
> (You forgot to include the list in your response, and you neglected to
> properly quote the message you were replying to. *That makes it quite
> hard to separate your remarks from mine)
>
> On 02/13/2011 07:19 PM, jon hayes wrote:
>
> > c:\users\name\backup\ *#win7
> >> c:\docs and settings\name\app data\backup\ *#winxp
> >> /home/name/backup *#linux and mac

>
> > In what way do those imply a need for chdir() ? *What's wrong with os..path.join() ?
> > the only commonality is the \backup
> > the chdir() is there to build the path name of the destination.
> > so c:\users\name\backup\folder1\folder2\file3
> > in my code, doing the substution.

>
> Well, since chdir() is illegal inside the os.walk() context and
> unnecesary, I suggest you debug your original technique. *Build the
> names with os.path.join(), as I've suggested twice before, and others
> have mentioned as well.
>
>
>
> > these 3 lines
> > os.chdir(path)
> > cwd = os.getcwd()
> > leftover = cwd.replace(source, '')
> > in this case leftover = \folder1\folder2
> > which gets added to the path for the destination in
> > shutil.copytree(os.getcwd(), destination+leftover)

>
> > i looked at the relpath, it did not do exacly what i wanted. in version 11 of the code this works out well.

>
> I can't see how os.path.relpath() would be useful here.
>
>
>
> > "And I quote from facbac-012a.py:
> > * * * * *os.chdir("v:")"
> > i don't understand. is this a question?

>
> It is in context. *If you read my message, I had just quoted you saying
> you did not change drive letters.
>
>
>
> > "I suggest you actually try writing a simple loop using os.walk to just
> > display the names of all the files in and under a specified directory.
> > Then maybe you'll understand what *path, dirs, and files actually are.."
> > i started with that. look at the attached file, date-linuxversion.py.
> > the relative paths are not, quite, right.

>
> Well file you enclosed shows that you don't understand os.walk(). *So
> learn it before trying to use it in a more complex setting. *Why would
> you use the following sequence?
>
> * * *filestring = str(files)
> * * *filestring = filestring.strip("[")
> * * *filestring = filestring.strip("]")
> * * *filestring = filestring.strip("'")
>
> This is no way to process a list of filenames.
>
> > the thing with the copytree is, the line above is sopposed to check whether that folder exists. if it does not exist at all, then why not copy the whole thing?
> > same sort of thing with files. if not exist then do copy. that way nothing should get clobbered.
> > this is the observed behavior in version 11.

>
> Why not copy the whole tree? *Because once the code works, you'll be
> copying it many times.
>
> DaveA


if i do not change the directory how do i get the relative folder
path. for example,
starting with c:\users\name\backup\folder1\folder2\file
as the oswalk drills down through the folders, when it is in the
'folder2' folder, if you do something like
print dirs
it would output just 'folder2' not the whole path or any of the other
parts.
so how do i know where to write this to on the destination relative
to
\\servername\username\week[1-4]\ or possibly that folder mounted as a
drive in windows as
v:\week[1-4]\ ?
how do i build the output folder name.
bear in mind it works great in version 011.
i spent weeks, and tried maybe 7 different other ways before going
down this path, so to speak.
i thought about using a list. or an array. but you wouldnt know when
oswalk starts bak from top level to go int another folder. let me
think if i can say it another way. ...
i think what you guys want me to do the osjoin on is this line
destination = r"\\\\Mothera\\"+username+"\\"+weekstr or
shutil.copytree(os.getcwd(), destination+leftover)
please clarify.
 
Reply With Quote
 
 
 
 
ecu_jon
Guest
Posts: n/a
 
      02-14-2011
this is a reply to both Dave Angel and Ben Finney. this version of
testing i think incorperates what you guys are saying.
<a href="http://thanksforallthefish.endofinternet.net/
testing1.py">testin1.py</a>
except maybe the os.join.path in the last function.
here is the traceback
Traceback (most recent call last):
File "I:\college\spring11\capstone-project\testing.py", line 88, in
<module>
backupall()
File "I:\college\spring11\capstone-project\testing.py", line 84, in
backupall
shutil.copy2(filesource, filedest1)
File "C:\Python27\lib\shutil.py", line 127, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 82, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
\college\\spring11\\capstone-project\\changelog.txt'

it looks like it needs os.walk and the other for line to dig through
the folders.
this being the whole path
I:\\college\\spring11\\capstone-project\\changelog.txt of source.
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
 
Reply With Quote
 
Dave Angel
Guest
Posts: n/a
 
      02-14-2011
On 01/-10/-28163 02:59 PM, ecu_jon wrote:
> this is a reply to both Dave Angel and Ben Finney. this version of
> testing i think incorperates what you guys are saying.
> <a href="http://thanksforallthefish.endofinternet.net/
> testing1.py">testin1.py</a>
> except maybe the os.join.path in the last function.
> here is the traceback
> Traceback (most recent call last):
> File "I:\college\spring11\capstone-project\testing.py", line 88, in
> <module>
> backupall()
> File "I:\college\spring11\capstone-project\testing.py", line 84, in
> backupall
> shutil.copy2(filesource, filedest1)
> File "C:\Python27\lib\shutil.py", line 127, in copy2
> copyfile(src, dst)
> File "C:\Python27\lib\shutil.py", line 82, in copyfile
> with open(dst, 'wb') as fdst:
> IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
> \college\\spring11\\capstone-project\\changelog.txt'
>
> it looks like it needs os.walk and the other for line to dig through
> the folders.
> this being the whole path
> I:\\college\\spring11\\capstone-project\\changelog.txt of source.
>


Simplify the code.


def weekChoice()
return 1 + (datetime.now().day -1) // 7 #weeks go from 1 to 5

def backupall():
source = source1()
destination = destination2()
print "copy tree ", source, destination
if os.path.exists(destination):
if os.path.isdir(destination):
shutil.rmtree(destination)
else:
os.unlink(destination)
shutil.copytree(source, destination)
return

All that nonsense with curdir was messing up your thinking.


I'm not going to try to debug your destination2() function, but I would
point out that when you use os.path.join(), please let it do the work
for you. It takes any number of arguments, and those arguments should
not have extra slashes or backslashes in them. The leading slashes are
okay for the first node, if you want a unc.

It would have been much easier if you had made source and destination
arguments to the backupall() function. It then might have become clear
that it's simply copytree(), with the extra requirement of deleting
whatever used to be there.

Now if you aren't allowed to delete what was already there, then you
can't use copytree, and need to start over with os.walk().

DaveA
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-14-2011
On Feb 14, 4:43*am, Dave Angel <(E-Mail Removed)> wrote:
> On 01/-10/-28163 02:59 PM, ecu_jon wrote:
>
>
>
> > this is a reply to both Dave Angel and Ben Finney. this version of
> > testing i think incorperates what you guys are saying.
> > <a href="http://thanksforallthefish.endofinternet.net/
> > testing1.py">testin1.py</a>
> > except maybe the os.join.path in the last function.
> > here is the traceback
> > Traceback (most recent call last):
> > * *File "I:\college\spring11\capstone-project\testing.py", line 88, in
> > <module>
> > * * *backupall()
> > * *File "I:\college\spring11\capstone-project\testing.py", line 84, in
> > backupall
> > * * *shutil.copy2(filesource, filedest1)
> > * *File "C:\Python27\lib\shutil.py", line 127, in copy2
> > * * *copyfile(src, dst)
> > * *File "C:\Python27\lib\shutil.py", line 82, in copyfile
> > * * *with open(dst, 'wb') as fdst:
> > IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
> > \college\\spring11\\capstone-project\\changelog.txt'

>
> > it looks like it needs os.walk and the other for line to dig through
> > the folders.
> > this being the whole path
> > I:\\college\\spring11\\capstone-project\\changelog.txt of source.

>
> Simplify the code.
>
> def weekChoice()
> * * *return 1 + (datetime.now().day -1) // 7 * #weeks go from 1 to 5
>
> def backupall():
> * * *source = source1()
> * * *destination = destination2()
> * * *print "copy tree ", source, destination
> * * *if os.path.exists(destination):
> * * * * *if os.path.isdir(destination):
> * * * * * * *shutil.rmtree(destination)
> * * * * else:
> * * * * * * *os.unlink(destination)
> * * *shutil.copytree(source, destination)
> * * *return
>
> All that nonsense with curdir was messing up your thinking.
>
> I'm not going to try to debug your destination2() function, but I would
> point out that when you use os.path.join(), please let it do the work
> for you. *It takes any number of arguments, and those arguments should
> not have extra slashes or backslashes in them. *The leading slashes are
> okay for the first node, if you want a unc.
>
> It would have been much easier if you had made source and destination
> arguments to the backupall() function. *It then might have become clear
> that it's simply copytree(), with the extra requirement of deleting
> whatever used to be there.
>
> Now if you aren't allowed to delete what was already there, then you
> can't use copytree, and need to start over with os.walk().
>
> DaveA


and dest2 was me trying different ways to write to the sever, unc or
mounted drive. that's for later
i missed this "as missing parent directories."
shutil.copytree(src, dst[, symlinks=False[, ignore=None]])

Recursively copy an entire directory tree rooted at src. The
destination directory, named by dst, must not already exist; it will
be created as well as missing parent directories.

so for testing i did the copy once. deleted the files in top folder,
and deleted a folder deeper in.
gave error when tried to copytree the deeper in folder.

is there something like mkdir?
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-14-2011
On Feb 14, 4:43*am, Dave Angel <(E-Mail Removed)> wrote:
> On 01/-10/-28163 02:59 PM, ecu_jon wrote:
>
>
>
> > this is a reply to both Dave Angel and Ben Finney. this version of
> > testing i think incorperates what you guys are saying.
> > <a href="http://thanksforallthefish.endofinternet.net/
> > testing1.py">testin1.py</a>
> > except maybe the os.join.path in the last function.
> > here is the traceback
> > Traceback (most recent call last):
> > * *File "I:\college\spring11\capstone-project\testing.py", line 88, in
> > <module>
> > * * *backupall()
> > * *File "I:\college\spring11\capstone-project\testing.py", line 84, in
> > backupall
> > * * *shutil.copy2(filesource, filedest1)
> > * *File "C:\Python27\lib\shutil.py", line 127, in copy2
> > * * *copyfile(src, dst)
> > * *File "C:\Python27\lib\shutil.py", line 82, in copyfile
> > * * *with open(dst, 'wb') as fdst:
> > IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
> > \college\\spring11\\capstone-project\\changelog.txt'

>
> > it looks like it needs os.walk and the other for line to dig through
> > the folders.
> > this being the whole path
> > I:\\college\\spring11\\capstone-project\\changelog.txt of source.

>
> Simplify the code.
>
> def weekChoice()
> * * *return 1 + (datetime.now().day -1) // 7 * #weeks go from 1 to 5
>
> def backupall():
> * * *source = source1()
> * * *destination = destination2()
> * * *print "copy tree ", source, destination
> * * *if os.path.exists(destination):
> * * * * *if os.path.isdir(destination):
> * * * * * * *shutil.rmtree(destination)
> * * * * else:
> * * * * * * *os.unlink(destination)
> * * *shutil.copytree(source, destination)
> * * *return
>
> All that nonsense with curdir was messing up your thinking.
>
> I'm not going to try to debug your destination2() function, but I would
> point out that when you use os.path.join(), please let it do the work
> for you. *It takes any number of arguments, and those arguments should
> not have extra slashes or backslashes in them. *The leading slashes are
> okay for the first node, if you want a unc.
>
> It would have been much easier if you had made source and destination
> arguments to the backupall() function. *It then might have become clear
> that it's simply copytree(), with the extra requirement of deleting
> whatever used to be there.
>
> Now if you aren't allowed to delete what was already there, then you
> can't use copytree, and need to start over with os.walk().
>
> DaveA


i thought i had it. i think im going to sleep on it.
here is what im working with right now. jsut trying to make folders
stuff work.
def backupall():
week = weekChoice()
weekstr = "week"+str(week)
source = source1() #was backupdir
destination = destination1() #was remotedir
#destinationweek = os.path.join(destination, weekstr)
for (path, dirs, files) in os.walk(source):
for fname in dirs:
os.chdir(path)
leftover = os.getcwd().replace(source, '')
#print "leftover is:",leftover
currentdir = destination+leftover
if not os.path.isdir(currentdir):
print "leftover is:",leftover
print "currentdir1:",currentdir
print "i should copy teh
folderz",os.path.join(currentdir,fname)
os.mkdir(os.path.join(currentdir,fname),0755)

it is not making the top level folder, then tries to create deeper
folders and fails.
part of the traceback.
os.mkdir(os.path.join(currentdir,fname),0755)
WindowsError: [Error 3] The system cannot find the path specified: 'C:\
\Users\\jon\\backup2\\week2\\configs\\squid_files'
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-15-2011
well i think i will close this out. redid recursive foldering now for
3rd or 4th time. below script works from a windows to another folder
(destination1) or a unc path(destination2). i tried it on my linux
box, and the error deals with writing to a remote smb share. i think
if it was mounted somewhere, it would work fine with something like
dest1, but changing the path(or mounting the remote drive to backup2
for the lazy ).

#!/usr/bin/python
import wx,os,string,fnmatch,shutil,tempfile,time,getpass
from datetime import *
from os.path import join, getsize


def weekChoice():
#week = 0
dateNum = datetime.now().day
if dateNum <=7:
week = 1
elif (dateNum >= and (dateNum <= 14):
week = 2
elif (dateNum >= 15) and (dateNum <= 21):
week = 3
elif dateNum > 22:
week = 4
else:
print "error!!!!"
return week

def getusername():
username = getpass.getuser()
return username

def homeDir():
homedir = os.path.expanduser('~')
try:
from win32com.shell import shellcon, shell
homedir = shell.SHGetFolderPath(0, shellcon.CSIDL_APPDATA, 0,
0)
except ImportError:
homedir = os.path.expanduser("~")
return homedir

def source1():
week = weekChoice()
homedir = homeDir()
#changed backupdir to source
source = os.path.join(homedir, "backup")
return source

def destination1():
week = weekChoice()
homedir = homeDir()
remotedir = os.path.join(homedir, "backup2")
weekstr = "week"+str(week)
destination = os.path.join(remotedir, weekstr)
return destination

def destination2():
username = getusername()
week = weekChoice()
weekstr = "week"+str(week)
destination = os.path.join("//Mothera",username,weekstr) #need to
mount the smb share somewhere
return destination

def backupall():
week = weekChoice()
weekstr = "week"+str(week)
source = source1()
destination = destination2() # change this around to suit needs
for (path, dirs, files) in os.walk(source):
for fname in dirs:
leftover = path.replace(source, '')
#print "leftover = ",leftover
currentdir = destination+leftover
#print "currentdir = ",currentdir
try:
#print "dest folder to
make",os.path.join(currentdir,fname)
os.mkdir(os.path.join(currentdir,fname),0755)
except:
print "error folder"
for fname in files:
leftover = path.replace(source, '')
#print "leftover = ",leftover
currentdir = destination+leftover
#print "currentdir = ",currentdir
try:
#print "dest file to
make",os.path.join(currentdir,fname)

shutil.copy2(os.path.join(path,fname),os.path.join (currentdir,fname))
except:
print "error file"
backupall()
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments