Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > file system iteration

Reply
Thread Tools

file system iteration

 
 
rick
Guest
Posts: n/a
 
      10-09-2006
In Unix, the file system hierarchy is like a tree that has a base or
'root' that exposes objects (files and folders) that can easily be
iterated over.


\ \ | / /
\ \ | / /
\ \|/ /
\ | /
\|/
|
|
Root

So, when I do os.chdir('/') I am at the base of the tree and can now use
something like os.walk() to work with all of the file system objects.

In Windows, the file system is disjointed and there is now real 'root'
At least none that I can see. It looks more like this:

| | | | | | |
|_|_|_|_|_|_|
A B C D E F G

How do you guys handle this when working with scripts that need to touch
all files and folders on a Windows machine? I've been looping through
A-Z like this:

import os.path

paths = []

if os.path.isdir('A:/'):
paths.append('A:/')

if os.path.isdir('B:/'):
paths.append('B:/')

....

That's a kludge, but it works OK. I'm sure WMI may have a function that
returns mounted volumes, but under the circumstances currently, I can
only use the standard Python library. Any ideas on how to do this better?

Thanks





 
Reply With Quote
 
 
 
 
rick
Guest
Posts: n/a
 
      10-09-2006
Gerrit Holl wrote:
> The very least you can try:
>
> import string
> string.ascii_uppercase
>
> for c in string.ascii_uppercase:
> if os.path.isdir('%s:/' % c):
> ...
>
> etc.
> But I suppose there should be a better way.


Oh yes, I do that. I spelled out the example very explicitly for
clarity. I don't actually type in A-Z
 
Reply With Quote
 
 
 
 
Gerrit Holl
Guest
Posts: n/a
 
      10-09-2006
On 2006-10-09 14:45:35 +0200, rick wrote:
> import os.path
>
> paths = []
>
> if os.path.isdir('A:/'):
> paths.append('A:/')
>
> if os.path.isdir('B:/'):
> paths.append('B:/')
>
> ...
>
> That's a kludge, but it works OK. I'm sure WMI may have a function that
> returns mounted volumes, but under the circumstances currently, I can
> only use the standard Python library. Any ideas on how to do this better?


The very least you can try:

import string
string.ascii_uppercase

for c in string.ascii_uppercase:
if os.path.isdir('%s:/' % c):
...

etc.
But I suppose there should be a better way.

Gerrit.
 
Reply With Quote
 
Georg Brandl
Guest
Posts: n/a
 
      10-09-2006
rick wrote:
> In Unix, the file system hierarchy is like a tree that has a base or
> 'root' that exposes objects (files and folders) that can easily be
> iterated over.
>
>
> \ \ | / /
> \ \ | / /
> \ \|/ /
> \ | /
> \|/
> |
> |
> Root
>
> So, when I do os.chdir('/') I am at the base of the tree and can now use
> something like os.walk() to work with all of the file system objects.
>
> In Windows, the file system is disjointed and there is now real 'root'
> At least none that I can see. It looks more like this:
>
> | | | | | | |
> |_|_|_|_|_|_|
> A B C D E F G
>
> How do you guys handle this when working with scripts that need to touch
> all files and folders on a Windows machine? I've been looping through
> A-Z like this:


Which application needs to walk over ALL files? Normally, you just have a
starting path and walk over everything under it.

In Unix, things aren't so clear either. For example, there are symbolic links
that make the tree more complicated. Or different file system mounted on
different mount points, perhaps not even representing real files like the
/proc filesystem. All that needs caution when iterating over "all files".

Georg
 
Reply With Quote
 
rick
Guest
Posts: n/a
 
      10-09-2006
Georg Brandl wrote:

> Which application needs to walk over ALL files? Normally, you just have a
> starting path and walk over everything under it.


Searching for a file by name. Scanning for viruses. Etc. There are lots
of legitimate reason to walk all paths from a central starting point, no???
 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      10-09-2006
"rick" <ath-> wrote:

>> Which application needs to walk over ALL files? Normally, you just have a
>> starting path and walk over everything under it.

>
> Searching for a file by name. Scanning for viruses. Etc. There are lots
> of legitimate reason to walk all paths from a central starting point, no???


what's the difference between a "starting path" and a "starting point" ?

</F>



 
Reply With Quote
 
rick
Guest
Posts: n/a
 
      10-09-2006
Fredrik Lundh wrote:
> what's the difference between a "starting path" and a "starting point" ?


None. What starting path or point would you suggest under Windows? Is
there something obvious that I'm missing? I see no starting point under
windows as my initial question clearly stated.
 
Reply With Quote
 
Jonathan Hartley
Guest
Posts: n/a
 
      10-09-2006
Georg Brandl wrote:
>> Which application needs to walk over ALL files?


How about 'updatedb' for starters, the index-maintainer for the common
*nix command-line utility 'locate'.

I'm pretty sure that os.walk( ) deals with symbolic links (by not
visiting them) and ' /proc' type complexities by not doing anything to
walked directories that '/proc' type entries cannot deal with. I think
(no sarcasm intended) the point of offering a directory-like interface
to '/proc' was so one can perform directory-like operations on it.

--

Jonathan Hartley

+44 7737 062 225

 
Reply With Quote
 
Georg Brandl
Guest
Posts: n/a
 
      10-09-2006
rick wrote:
> Georg Brandl wrote:
>
>> Which application needs to walk over ALL files? Normally, you just have a
>> starting path and walk over everything under it.

>
> Searching for a file by name. Scanning for viruses. Etc. There are lots
> of legitimate reason to walk all paths from a central starting point, no???


Yes. Still, the user may not want to scan all files, or exclude non-locally
mounted filesystem etc.

So you'll always have to give the user control over where to start, and
therefore there's no problem in letting him choose which drives he wants
to search on.

Georg
 
Reply With Quote
 
Georg Brandl
Guest
Posts: n/a
 
      10-09-2006
Jonathan Hartley wrote:
> Georg Brandl wrote:
> >> Which application needs to walk over ALL files?

>
> How about 'updatedb' for starters, the index-maintainer for the common
> *nix command-line utility 'locate'.
>
> I'm pretty sure that os.walk( ) deals with symbolic links (by not
> visiting them) and ' /proc' type complexities by not doing anything to
> walked directories that '/proc' type entries cannot deal with. I think
> (no sarcasm intended) the point of offering a directory-like interface
> to '/proc' was so one can perform directory-like operations on it.


Sure, and I don't say that this is not useful.

But for all applications mentioned in the thread (virus scanning, searching
for a file by name, updating the locate db), including /proc is not very
useful, to say the least.

Georg
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Iteration through File.file? misses entries for whichFile.file?(entry) == true Kyle Barbour Ruby 10 08-02-2010 08:55 PM
Struts - Problem with nested iteration or double iteration Rudi Java 5 10-01-2008 03:30 AM
RE: file system iteration Tim Golden Python 3 10-09-2006 06:44 PM
iteration through a file of structs Dennis Schulz C Programming 2 05-08-2004 08:13 PM
open new file each loop iteration Danny Anderson C++ 0 01-21-2004 08:56 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57