Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > files,folders,paths

Reply
Thread Tools

files,folders,paths

 
 
ecu_jon
Guest
Posts: n/a
 
      02-13-2011
i have a samba server at home (acting as my test environment) with one
of the 'shares' mounted as v: on my windows box. inside of that are 4
folders week[1-4]. i have created c:\users\name\backup , and put a few
files/folders in there to test with. please ignore the wxpython parts
of the script, the last 1/3, that part is functional. initial test was
looking good. (in the code, ver011 source1, destination1) coping files/
folders and not clobbering files. going to version 12a, it seems like
it is trying to copy the files under the folder, before making the
folder. or doing the copy2 part before the copytree part. i dont know
why it works on ver011, and not in ver12a, the only difference being
the destination.
<a href="http://thanksforallthefish.endofinternet.net/
facbac-011.py">facbac 011</a>
<a href="http://thanksforallthefish.endofinternet.net/
facbac-012a.py">facbac 012a</a>
Traceback (most recent call last):
File "I:\college\spring11\capstone-project\facbac-012a.py", line
161, in button1Click
backupall()
File "I:\college\spring11\capstone-project\facbac-012a.py", line 81,
in backupall
shutil.copy2(filesource, filedest1)
File "C:\Python27\lib\shutil.py", line 127, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 82, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
\apache2.conf'

here, apache2.conf = c:\users\name\backup\config\apache2.conf
trying to copy to v:\week2\config\apache2.conf
but there is no v:\week2\config
as it has not made the config folder yet
i think the problem might be the \\ in the destination name.
 
Reply With Quote
 
 
 
 
aspineux
Guest
Posts: n/a
 
      02-13-2011
Without trying to understand your code, I see one difference

if not os.path.isfile(destination+leftover+fname):

become

if not os.path.isfile(destination+leftover):


Regards


On Feb 13, 11:06*am, ecu_jon <(E-Mail Removed)> wrote:
> i have a samba server at home (acting as my test environment) with one
> of the 'shares' mounted as v: on my windows box. inside of that are 4
> folders week[1-4]. i have created c:\users\name\backup , and put a few
> files/folders in there to test with. please ignore the wxpython parts
> of the script, the last 1/3, that part is functional. initial test was
> looking good. (in the code, ver011 source1, destination1) coping files/
> folders and not clobbering files. going to version 12a, it seems like
> it is trying to copy the files under the folder, before making the
> folder. or doing the copy2 part before the copytree part. i dont know
> why it works on ver011, and not in ver12a, the only difference being
> the destination.
> <a href="http://thanksforallthefish.endofinternet.net/
> facbac-011.py">facbac 011</a>
> <a href="http://thanksforallthefish.endofinternet.net/
> facbac-012a.py">facbac 012a</a>
> Traceback (most recent call last):
> * File "I:\college\spring11\capstone-project\facbac-012a.py", line
> 161, in button1Click
> * * backupall()
> * File "I:\college\spring11\capstone-project\facbac-012a.py", line 81,
> in backupall
> * * shutil.copy2(filesource, filedest1)
> * File "C:\Python27\lib\shutil.py", line 127, in copy2
> * * copyfile(src, dst)
> * File "C:\Python27\lib\shutil.py", line 82, in copyfile
> * * with open(dst, 'wb') as fdst:
> IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
> \apache2.conf'
>
> here, apache2.conf = c:\users\name\backup\config\apache2.conf
> trying to copy to v:\week2\config\apache2.conf
> but there is no v:\week2\config
> as it has not made the config folder yet
> i think the problem might be the \\ in the destination name.


 
Reply With Quote
 
 
 
 
ecu_jon
Guest
Posts: n/a
 
      02-13-2011
i just tried changing that in ver12a, still get
IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
\apache2.conf'
good catch tho as that is needed. thanks.
weekchoice is used to pick week 1-4. really jsut returns 1-4. i plan
to do full weekly backups with another script.
getusername i was using when i was trying to build the path as part of
a unc path. isdir/isfile did not play well with that.
homedir returns the users home directory. works in xp,7, and linux.
source and destination should be self explanatory.
the backupall() gets called from clicking the button. for now its the
meat and potatoes im working on. it is trying to recursively copy
files/folders from source to dest. the for ... os.walk line digs
through the backup folder. i use os.chgdir , os.getcwd and leftover to
build the path for file/folder names on dest. basically start at c:
\users\name\backup\somefolder and end with just somefolderadded to
dest.
if os.path.isdir(fname): checks if it is a folder. copytree is a
recursive folder copy.
elif os.path.isfile(fname): if it is a file. copy2 copies files and
metadata.
class MyForm(wx.Frame): and down is wxpython, that works. ( tho i will
take suggestions but i know this is not the right forum to as
questions for that part)
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      02-13-2011
On Sun, 13 Feb 2011 02:06:15 -0800, ecu_jon wrote:

> i have a samba server at home (acting as my test environment) with one
> of the 'shares' mounted as v: on my windows box. inside of that are 4
> folders week[1-4]. i have created c:\users\name\backup , and put a few
> files/folders in there to test with. please ignore the wxpython parts of
> the script, the last 1/3, that part is functional.


Please don't dump code on us and tell us to ignore parts of it. We're
offering our time and experience without charging you, the least you can
do is take the time to eliminate the irrelevant parts of the code
yourself, and show us the smallest piece of code which demonstrates the
problem.

Doing so will not only demonstrate respect for others, but it might even
help you solve the problem yourself.


> initial test was
> looking good. (in the code, ver011 source1, destination1) coping files/
> folders and not clobbering files. going to version 12a, it seems like it
> is trying to copy the files under the folder, before making the folder.
> or doing the copy2 part before the copytree part. i dont know why it
> works on ver011, and not in ver12a, the only difference being the
> destination.


That's not exactly true, is it? I count at least three other differences:

* facbac-012a.py imports getpass and defines a redundant
getusername function (redundant because all it does is
wrap getpass.getuser);

* it also defines a function destination2 which is different
from destination1, and uses that;

* there's a difference in the call to os.path.isfile;

I don't know if these are relevant.

You say the difference between the script that works and the script that
doesn't is the destination. And then you show an error that appears to be
because the destination doesn't exist:


> Traceback (most recent call last):
> File "I:\college\spring11\capstone-project\facbac-012a.py", line
> 161, in button1Click
> backupall()
> File "I:\college\spring11\capstone-project\facbac-012a.py", line 81,
> in backupall
> shutil.copy2(filesource, filedest1)
> File "C:\Python27\lib\shutil.py", line 127, in copy2
> copyfile(src, dst)
> File "C:\Python27\lib\shutil.py", line 82, in copyfile
> with open(dst, 'wb') as fdst:
> IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
> \apache2.conf'



Seems like an open and shut case to me.



> here, apache2.conf = c:\users\name\backup\config\apache2.conf trying to
> copy to v:\week2\config\apache2.conf but there is no v:\week2\config
> as it has not made the config folder yet i think the problem might be
> the \\ in the destination name.



There is no double-backslash in the destination name. Backslashes are
special in Python, and when displaying strings, they are escaped:


>>> s = "x\yz"
>>> len(s)

4
>>> s

'x\\yz'
>>> print s

x\yz


Your suggestion, that the problem is that the config directory doesn't
exist, sounds reasonable to me. Only you can test that, as we don't have
access to your file system.




--
Steven
 
Reply With Quote
 
Dave Angel
Guest
Posts: n/a
 
      02-13-2011
On 01/-10/-28163 02:59 PM, ecu_jon wrote:
> i just tried changing that in ver12a, still get
> IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
> \apache2.conf'
> good catch tho as that is needed. thanks.
> weekchoice is used to pick week 1-4. really jsut returns 1-4. i plan
> to do full weekly backups with another script.
> getusername i was using when i was trying to build the path as part of
> a unc path. isdir/isfile did not play well with that.
> homedir returns the users home directory. works in xp,7, and linux.
> source and destination should be self explanatory.
> the backupall() gets called from clicking the button. for now its the
> meat and potatoes im working on. it is trying to recursively copy
> files/folders from source to dest. the for ... os.walk line digs
> through the backup folder. i use os.chgdir , os.getcwd and leftover to
> build the path for file/folder names on dest. basically start at c:
> \users\name\backup\somefolder and end with just somefolderadded to
> dest.
> if os.path.isdir(fname): checks if it is a folder. copytree is a
> recursive folder copy.
> elif os.path.isfile(fname): if it is a file. copy2 copies files and
> metadata.
> class MyForm(wx.Frame): and down is wxpython, that works. ( tho i will
> take suggestions but i know this is not the right forum to as
> questions for that part)
>


I haven't tried to run your script, since much of it is Windows
dependent, and I'm on Linux. So observations are just by inspection.

I suspect your problem is that you're trying to change drives as well as
directories. In Windows, there's a current directory on each drive,
that Windows remembers. So changing directories on a different drive
doesn't mean what you might think it does. I'd recommend doing no
chdir()'s at all, and using only absolute paths.

But I see other complications or maybe risks in the code, concentrating
on function backupall(). You're using os.walk(), which walks through a
whole tree of files. But then when you encounter a directory, you're
doing a copytree(). So you'll be copying the same files many-many times.

Similarly, you have logic for copying a single file, but you invoke it
on a name from dirlist, which is a list of directories, so the isfile()
probably never fires.

os.walk() assumes the current directory is unchanged during its run.
See: http://docs.python.org/library/os.html#module-os
"walk() never changes the current directory, and assumes that its
caller doesn’t either."

You're not copying the files at the top level at all; you don't process
the list called 'files', while you could probably safely copy *only* the
files.

Your name manipulations in this same function look risky, using replace
when you presumably know that a string is either at the begin or end.

In function homeDir(), you never return the value calculated in the try
block. If you don't get an exception, you return None.

DaveA


 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-13-2011
On Feb 13, 8:17*am, Dave Angel <(E-Mail Removed)> wrote:
> On 01/-10/-28163 02:59 PM, ecu_jon wrote:
>
>
>
> > i just tried changing that in ver12a, still get
> > IOError: [Errno 2] No such file or directory: 'V:\\week2\\configs\
> > \apache2.conf'
> > good catch tho as that is needed. thanks.
> > weekchoice is used to pick week 1-4. really jsut returns 1-4. i plan
> > to do full weekly backups with another script.
> > getusername i was using when i was trying to build the path as part of
> > a unc path. isdir/isfile did not play well with that.
> > homedir returns the users home directory. works in xp,7, and linux.
> > source and destination should be self explanatory.
> > the backupall() gets called from clicking the button. for now its the
> > meat and potatoes im working on. it is trying to recursively copy
> > files/folders from source to dest. the for ... os.walk line digs
> > through the backup folder. i use os.chgdir , os.getcwd and leftover to
> > build the path for file/folder names on dest. basically start at c:
> > \users\name\backup\somefolder and end with just somefolderadded to
> > dest.
> > if os.path.isdir(fname): checks if it is a folder. copytree is a
> > recursive folder *copy.
> > elif os.path.isfile(fname): if it is a file. copy2 copies files and
> > metadata.
> > class MyForm(wx.Frame): and down is wxpython, that works. ( tho i will
> > take suggestions but i know this is not the right forum to as
> > questions for that part)

>
> I haven't tried to run your script, since much of it is Windows
> dependent, and I'm on Linux. *So observations are just by inspection.
>
> I suspect your problem is that you're trying to change drives as well as
> directories. *In Windows, there's a current directory on each drive,
> that Windows remembers. *So changing directories on a different drive
> doesn't mean what you might think it does. *I'd recommend doing no
> chdir()'s at all, and using only absolute paths.
>
> But I see other complications or maybe risks in the code, concentrating
> on function *backupall(). *You're using os.walk(), which walks through a
> whole tree of files. *But then when you encounter a directory, you're
> doing a copytree(). *So you'll be copying the same files many-many times.
>
> Similarly, you have logic for copying a single file, but you invoke it
> on a name from dirlist, which is a list of directories, so the isfile()
> probably never fires.
>
> os.walk() assumes the current directory is unchanged during its run.
> See: *http://docs.python.org/library/os.html#module-os
> * *"walk() never changes the current directory, and assumes that its
> caller doesn’t either."
>
> You're not copying the files at the top level at all; *you don't process
> the list called 'files', while you could probably safely copy *only* the
> files.
>
> Your name manipulations in this same function look risky, using replace
> when you presumably know that a string is either at the begin or end.
>
> In function homeDir(), you never return the value calculated in the try
> block. *If you don't get an exception, you return None.
>
> DaveA


first let me say, you guys are helping me and i thank you for that. i
do not want to sound confrontational. i have don my best to make this
os agnostic. yes i know, right now in 21a, destination2 is windows
specific. that's mostly because os.path.isdir() and unc paths do not
play well with each other.

the os.chdir() came about out of need to build relative path names.
some examples :
c:\users\name\backup\ #win7
c:\docs and settings\name\app data\backup\ #winxp
/home/name/backup #linux and mac

i wanted to start from 'backup' in all these cases. my destination is
going to look like
\\servername\username\week[1-4]

yes os.walk goes thru the whole tree (under backup in my case). the
os.chdir() under the for line in def backupall(): basically juct
chages folders to whever the os.walk() is. it never tried to change to
a different drive letter.the folder copy thing, these 2 lines
if os.path.isdir(fname): #if it is a folder
if not os.path.isdir(destination+leftover): #and id
dont exist on dest drive
work on version 011, determine if its a folder, then if it exists.
only copy if both: it is a folder, and not exist on destination.
this "return homedir" needed to be moved 1 tab left. copy error from
random code to functionalizing parts. good catch thanks.
dirlsit, returns the names of folders and files. i was using a
different method
http://thanksforallthefish.endofinte...inuxversion.py

try version 11 for yourself. in your home directory (ie: /home/name)
make a backup folder, put a few files/folders in it. then make a
backup2 folder, and make week1,week2week3week4 folders. version 11
will copy the top level files under /home/name/backup , and recreate
the directory structure. so "You're not copying the files at the top
level at all; you don't process
the list called 'files', while you could probably safely copy *only*
the
files. " atleats for version 11 is not true. atleast not within the
bounds i have tested it.

"Your name manipulations in this same function look risky, using
replace
when you presumably know that a string is either at the begin or end.
"
i know. i have several things that need to build up. like starting
from a relative position and going to a different position. username
would be part of the unc path if i could get that to work, and
week[1-4] is part of the name.

the thing that gets me is version 11 works. well. but its is reading/
writing to almost the same place (from backup, to backup2). defiantly
open to suggestions.
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-13-2011
"Please don't dump code on us and tell us to ignore parts of it. We're
offering our time and experience without charging you, the least you
can
do is take the time to eliminate the irrelevant parts of the code
yourself, and show us the smallest piece of code which demonstrates
the
problem. " i appreciate your opinion. im fairly new here so i might
not get etiquette right.i only asked you all to ignore it for 2
reasons, it's wxpython to not really targets to the crowd, and it does
not interact with the python parts. other than simply as a means to
call the functions. i could chop off the bottom and replace it with a
functname(), it would be the same for my question.

i know backslashes are special. there a special pain in my #$%.
without a windows environment, you would have to change destination2
significantly to try it out. but like i said above you can try version
11 in a linux environment. i know config does not exist, for some
reason the isfolder check is passing it off saying it exists, when it
does not. then moving on to the isfiles parts. tries to copy a folder
deep files with no folder to write to, and fails.
i jsut don't understand why the folder check is not working right in
version 12a.

and destination2, IS basically the difference between 11 and 12a. it
should be the only difference. the isfile difference someone else
pointed out i have since changed that, stil lsame problem.
and again thanks all for time/trouble.
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      02-13-2011
On Sun, 13 Feb 2011 10:58:23 -0800, ecu_jon wrote:

> i know config does not exist, for some reason the isfolder check is
> passing it off saying it exists, when it does not.



[steve@sylar ~]$ grep isfolder facbac-011.py
[steve@sylar ~]$ grep isfolder facbac-012a.py
[steve@sylar ~]$


There is no such "isfolder" check in your source code.



--
Steven
 
Reply With Quote
 
ecu_jon
Guest
Posts: n/a
 
      02-13-2011
On Feb 13, 5:11*pm, Tim Roberts <(E-Mail Removed)> wrote:
> ecu_jon <(E-Mail Removed)> wrote:
>
> >i know backslashes are special. there a special pain in my #$%.
> >without a windows environment, you would have to change destination2
> >significantly to try it out. but like i said above you can try version
> >11 in a linux environment.

>
> If they are a pain in your #$%, then don't use them. *You can use forward
> slashes everywhere in Windows, except when typing commands at the command
> prompt. *ALL of the Windows file system APIs accept forward slashes.
> --
> Tim Roberts, (E-Mail Removed)
> Providenza & Boekelheide, Inc.


can you give an example of how to build a folder name, maybe as a
string, with parts of it as variables, that have to be strung
together. like x = "//servername/+variable+"/"+variable+"/"
when i did something liek that the end result string was like
//servername/folder/folder\file and i got an error
 
Reply With Quote
 
Dave Angel
Guest
Posts: n/a
 
      02-13-2011
On 01/-10/-28163 02:59 PM, ecu_jon wrote:
>
>> <snip>

>
> first let me say, you guys are helping me and i thank you for that. i
> do not want to sound confrontational. i have don my best to make this
> os agnostic. yes i know, right now in 21a, destination2 is windows
> specific. that's mostly because os.path.isdir() and unc paths do not
> play well with each other.
>
> the os.chdir() came about out of need to build relative path names.
> some examples :
> c:\users\name\backup\ #win7
> c:\docs and settings\name\app data\backup\ #winxp
> /home/name/backup #linux and mac


In what way do those imply a need for chdir() ? What's wrong with
os.path.join() ?

>
> i wanted to start from 'backup' in all these cases. my destination is
> going to look like
> \\servername\username\week[1-4]
>
> yes os.walk goes thru the whole tree (under backup in my case). the
> os.chdir() under the for line in def backupall(): basically juct
> chages folders to whever the os.walk() is. it never tried to change to
> a different drive letter.


And I quote from facbac-012a.py:
os.chdir("v:")


> the folder copy thing, these 2 lines
> if os.path.isdir(fname): #if it is a folder
> if not os.path.isdir(destination+leftover): #and id
> dont exist on dest drive
> work on version 011, determine if its a folder, then if it exists.
> only copy if both: it is a folder, and not exist on destination.
> this "return homedir" needed to be moved 1 tab left. copy error from
> random code to functionalizing parts. good catch thanks.
> dirlsit, returns the names of folders and files. i was using a
> different method
> http://thanksforallthefish.endofinte...inuxversion.py
>
> try version 11 for yourself. in your home directory (ie: /home/name)
> make a backup folder, put a few files/folders in it. then make a
> backup2 folder, and make week1,week2week3week4 folders. version 11
> will copy the top level files under /home/name/backup , and recreate
> the directory structure. so "You're not copying the files at the top
> level at all; you don't process
> the list called 'files', while you could probably safely copy *only*
> the
> files. " atleats for version 11 is not true. atleast not within the
> bounds i have tested it.
>
> "Your name manipulations in this same function look risky, using
> replace
> when you presumably know that a string is either at the begin or end.
> "
> i know. i have several things that need to build up. like starting
> from a relative position and going to a different position. username
> would be part of the unc path if i could get that to work, and
> week[1-4] is part of the name.
>
> the thing that gets me is version 11 works. well. but its is reading/
> writing to almost the same place (from backup, to backup2). defiantly
> open to suggestions.
>


I didn't spot the dirlist before, since it's totally unnecessary. It
just builds the same list you already have as 'files'. I thought you
were looping over the names in dirs. copytree() is also redundant, as
it'll cause the same files to be repeatedly copied. If you plan to use
copytree, why bother using os.walk at all?

I suggest you actually try writing a simple loop using os.walk to just
display the names of all the files in and under a specified directory.
Then maybe you'll understand what path, dirs, and files actually are.

Once you have that, use string manipulation to change the source path
name into a destination path, and print, rather than actually doing the
shutilcopy2.



DaveA



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments