Velocity Reviews > Python 3 minor irritation

# Python 3 minor irritation

David Monaghan
Guest
Posts: n/a

 02-04-2010
I have a small program which reads files from the directory in which it
resides. It's written in Python 3 and when run through IDLE or PythonWin
works fine. If I double-click the file, it works fine in Python 2.6, but in
3 it fails because it looks for the files to load in the Python31 folder,
not the one the script is in.

It's not a big deal, but browsing around I haven't found why the behaviour
has been changed or any comment about it (That might be my poor search
technique, I suppose).

The program fails at:

try:
tutdoc = minidom.parse(".//Myfile.xml")
except IOError:
<snip>

DaveM

Alf P. Steinbach
Guest
Posts: n/a

 02-04-2010
* David Monaghan:
> I have a small program which reads files from the directory in which it
> resides. It's written in Python 3 and when run through IDLE or PythonWin
> works fine. If I double-click the file, it works fine in Python 2.6, but in
> 3 it fails because it looks for the files to load in the Python31 folder,
> not the one the script is in.
>
> It's not a big deal, but browsing around I haven't found why the behaviour
> has been changed or any comment about it (That might be my poor search
> technique, I suppose).
>
> The program fails at:
>
> try:
> tutdoc = minidom.parse(".//Myfile.xml")
> except IOError:
> <snip>

The "//" is wrong, but should not cause the behavior that you describe.

Try to post a complete smallest possible program that exhibits the problem.

Possibly, in creating that example you'll also find what's cause the problem.

Cheers & hth.,

- Alf

Benjamin Kaplan
Guest
Posts: n/a

 02-04-2010
On Wed, Feb 3, 2010 at 9:56 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
> * David Monaghan:
>>
>> I have a small program which reads files from the directory in which it
>> resides. It's written in Python 3 and when run through IDLE or PythonWin
>> works fine. If I double-click the file, it works fine in Python 2.6, but
>> in
>> 3 it fails because it looks for the files to load in the Python31 folder,
>> not the one the script is in.
>>
>> It's not a big deal, but browsing around I haven't found why the behaviour
>> has been changed or any comment about it (That might be my poor search
>> technique, I suppose).
>>
>> The program fails at:
>>
>> * *try:
>> * * * *tutdoc = minidom.parse(".//Myfile.xml")
>> * *except IOError:
>> * * * *<snip>

>
> The "//" is wrong, but should not cause the behavior that you describe.
>
> Try to post a complete smallest possible program that exhibits the problem.
>
> Possibly, in creating that example you'll also find what's cause the
> problem.
>
>
> Cheers & hth.,
>
> - Alf

That is the smallest example the exhibits the problem. It's not an
issue with the Python code, it's an issue with how Windows is running
it. I don't know enough about the way Windows Explorer runs files, but
it seems to be doing the equivalent of
cd C:\Python31
python31.exe C:\full\path\to\script\foo.py

cd C:\full\path\path\to\script
C:\Python31\python.exe foo.py
which is David expected. This throws off the relative filepath.

The easiest way to solve this permanently, by the way, is to not use
relative paths. All it takes is one script to call os.chdir and the
script breaks. You can use __file__ and the os.path module to figure
out exactly where you are in an OS-agnostic way.

import os.path
#get the absolute path to the current script
abs_path = os.path.abspath(__file__)

# get the full path to the directory of the script
directory = os.path.dirname(abs_path)

#get the full path to your file
my_file = os.path.join(directory, "MyFile.xml")

> --
> http://mail.python.org/mailman/listinfo/python-list
>

Alf P. Steinbach
Guest
Posts: n/a

 02-04-2010
* Benjamin Kaplan:
> On Wed, Feb 3, 2010 at 9:56 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
>> * David Monaghan:
>>> I have a small program which reads files from the directory in which it
>>> resides. It's written in Python 3 and when run through IDLE or PythonWin
>>> works fine. If I double-click the file, it works fine in Python 2.6, but
>>> in
>>> 3 it fails because it looks for the files to load in the Python31 folder,
>>> not the one the script is in.
>>>
>>> It's not a big deal, but browsing around I haven't found why the behaviour
>>> has been changed or any comment about it (That might be my poor search
>>> technique, I suppose).
>>>
>>> The program fails at:
>>>
>>> try:
>>> tutdoc = minidom.parse(".//Myfile.xml")
>>> except IOError:
>>> <snip>

>> The "//" is wrong, but should not cause the behavior that you describe.
>>
>> Try to post a complete smallest possible program that exhibits the problem.
>>
>> Possibly, in creating that example you'll also find what's cause the
>> problem.
>>
>>
>> Cheers & hth.,
>>
>> - Alf

>
> That is the smallest example the exhibits the problem.

No, it doesn't seem to exhibit the problem at all.

> It's not an
> issue with the Python code, it's an issue with how Windows is running
> it. I don't know enough about the way Windows Explorer runs files, but
> it seems to be doing the equivalent of
> cd C:\Python31
> python31.exe C:\full\path\to\script\foo.py
>
> cd C:\full\path\path\to\script
> C:\Python31\python.exe foo.py
> which is David expected. This throws off the relative filepath.

No, this is not what happens.

What happens is that when you double-click the script, then __file__ is set to
the absolute path instead of just the filename, at least on my machine (XP);
whether the full path or just the filename is passed on the OS level depends,
however, on the Windows version.

The current directory is always (initially) correct, as the script's directory.

> The easiest way to solve this permanently, by the way, is to not use
> relative paths. All it takes is one script to call os.chdir and the
> script breaks. You can use __file__ and the os.path module to figure
> out exactly where you are in an OS-agnostic way.
>
> import os.path
> #get the absolute path to the current script
> abs_path = os.path.abspath(__file__)

According to the docs: "On most platforms, this is equivalent to
normpath(join(os.getcwd(), path))."

Therefore, if getcwd() is not the script's directory, as you hypothesize above,
then most likely the result of this code is Not The Path You're Looking For.

However, since the current directory is in fact OK, the above is one way to get
a sort of canonical (a where-you-know-the-format) representation of __file__.

> # get the full path to the directory of the script
> directory = os.path.dirname(abs_path)

This is very much likely to yield the same result as calling os.getcwd(); see above.

> #get the full path to your file
> my_file = os.path.join(directory, "MyFile.xml")

Cheers & hth.,

- Alf

Benjamin Kaplan
Guest
Posts: n/a

 02-04-2010
On Wed, Feb 3, 2010 at 11:18 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
> * Benjamin Kaplan:
>>
>> On Wed, Feb 3, 2010 at 9:56 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
>>>
>>> * David Monaghan:
>>>>
>>>> I have a small program which reads files from the directory in which it
>>>> resides. It's written in Python 3 and when run through IDLE or PythonWin
>>>> works fine. If I double-click the file, it works fine in Python 2.6, but
>>>> in
>>>> 3 it fails because it looks for the files to load in the Python31
>>>> folder,
>>>> not the one the script is in.
>>>>
>>>> It's not a big deal, but browsing around I haven't found why the
>>>> behaviour
>>>> has been changed or any comment about it (That might be my poor search
>>>> technique, I suppose).
>>>>
>>>> The program fails at:
>>>>
>>>> * try:
>>>> * * * tutdoc = minidom.parse(".//Myfile.xml")
>>>> * except IOError:
>>>> * * * <snip>
>>>
>>> The "//" is wrong, but should not cause the behavior that you describe.
>>>
>>> Try to post a complete smallest possible program that exhibits the
>>> problem.
>>>
>>> Possibly, in creating that example you'll also find what's cause the
>>> problem.
>>>
>>>
>>> Cheers & hth.,
>>>
>>> - Alf

>>
>> That is the smallest example the exhibits the problem.

>
> No, it doesn't seem to exhibit the problem at all.
>
>
>> It's not an
>> issue with the Python code, it's an issue with how Windows is running
>> it. I don't know enough about the way Windows Explorer runs files, but
>> it seems to be doing the equivalent of
>> cd C:\Python31
>> python31.exe C:\full\path\to\script\foo.py
>>
>> cd C:\full\path\path\to\script
>> C:\Python31\python.exe foo.py
>> which is David expected. This throws off the relative filepath.

>
> No, this is not what happens.
>
> What happens is that when you double-click the script, then __file__ is set
> to the absolute path instead of just the filename, at least on my machine
> (XP); whether the full path or just the filename is passed on the OS level
> depends, however, on the Windows version.
>
> The current directory is always (initially) correct, as the script's
> directory.
>

Read my first paragraph again- it has absolutely nothing to do with
Python. It has to do with how the Windows is running Python 3 on the
OP's computer. The OP's description said that it was looking for the
file .\\myfile.xml in C:\Python31 which means it translated '.', the
current working directory, to be C:\Python31 and not the directory the
script is in.

>
>> The easiest way to solve this permanently, by the way, is to not use
>> relative paths. All it takes is one script to call os.chdir and the
>> script breaks. You can use __file__ and the os.path module to figure
>> out exactly where you are in an OS-agnostic way.
>>
>> import os.path
>> #get the absolute path to the current script
>> abs_path = os.path.abspath(__file__)

>
> According to the docs: "On most platforms, this is equivalent to
> normpath(join(os.getcwd(), path))."
>

os.path.abspath will always work in this case (unless something
changes the current working directory before that statement runs)
because __file__ is given either as an absolute path or as relative to
the current working directory.

----- /Users/bkaplan/test/test.py
print(__file__)
import os.path
print(os.path.abspath(__file__))
------------------

$cd /users/bkaplan$ python3 test/test.py
python3 test/test.py
test/test.py
/Users/bkaplan/test/test.py

$cd /users/bkaplan/test$ python3 test.py
test.py
/Users/bkaplan/test/test.py

If abspath is given an absolute path, it won't touch it

Output from double clicking on the file:
/Users/bkaplan/test/test.py
/Users/bkaplan/test/test.py

> Therefore, if getcwd() is not the script's directory, as you hypothesize
> above, then most likely the result of this code is Not The Path You're
> Looking For.
>

Except that if the cwd is not the directory the script is running in,
__file__ should still point to it either through an absolute path
(highly likely since it's run through Windows Explorer) or some
(however convoluted) relative path.

> However, since the current directory is in fact OK, the above is one way to
> get a sort of canonical (a where-you-know-the-format) representation of
> __file__.
>
>
>> *# get the full path to the directory of the script
>> directory = os.path.dirname(abs_path)

>
> This is very much likely to yield the same result as calling os.getcwd();
> see above.
>

Not if there is more than one folder between __file__ and cwd. For
instance, if the script is in a package several directories down from
the main script.

>
>> #get the full path to your file
>> my_file = os.path.join(directory, "MyFile.xml")

>
>
> Cheers & hth.,
>
> - Alf
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Alf P. Steinbach
Guest
Posts: n/a

 02-04-2010
* Benjamin Kaplan:
> On Wed, Feb 3, 2010 at 11:18 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
>> * Benjamin Kaplan:
>>> On Wed, Feb 3, 2010 at 9:56 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
>>>> * David Monaghan:
>>>>> I have a small program which reads files from the directory in which it
>>>>> resides. It's written in Python 3 and when run through IDLE or PythonWin
>>>>> works fine. If I double-click the file, it works fine in Python 2.6, but
>>>>> in
>>>>> 3 it fails because it looks for the files to load in the Python31
>>>>> folder,
>>>>> not the one the script is in.
>>>>>
>>>>> It's not a big deal, but browsing around I haven't found why the
>>>>> behaviour
>>>>> has been changed or any comment about it (That might be my poor search
>>>>> technique, I suppose).
>>>>>
>>>>> The program fails at:
>>>>>
>>>>> try:
>>>>> tutdoc = minidom.parse(".//Myfile.xml")
>>>>> except IOError:
>>>>> <snip>
>>>> The "//" is wrong, but should not cause the behavior that you describe.
>>>>
>>>> Try to post a complete smallest possible program that exhibits the
>>>> problem.
>>>>
>>>> Possibly, in creating that example you'll also find what's cause the
>>>> problem.
>>>>
>>>>
>>>> Cheers & hth.,
>>>>
>>>> - Alf
>>> That is the smallest example the exhibits the problem.

>> No, it doesn't seem to exhibit the problem at all.
>>
>>
>>> It's not an
>>> issue with the Python code, it's an issue with how Windows is running
>>> it. I don't know enough about the way Windows Explorer runs files, but
>>> it seems to be doing the equivalent of
>>> cd C:\Python31
>>> python31.exe C:\full\path\to\script\foo.py
>>>
>>> cd C:\full\path\path\to\script
>>> C:\Python31\python.exe foo.py
>>> which is David expected. This throws off the relative filepath.

>> No, this is not what happens.
>>
>> What happens is that when you double-click the script, then __file__ is set
>> to the absolute path instead of just the filename, at least on my machine
>> (XP); whether the full path or just the filename is passed on the OS level
>> depends, however, on the Windows version.
>>
>> The current directory is always (initially) correct, as the script's
>> directory.
>>

>
> Read my first paragraph again- it has absolutely nothing to do with
> Python. It has to do with how the Windows is running Python 3 on the
> OP's computer.

I'm not sure what you're arguing here.

But anyway, first, I haven't stated that "it" is (only) an issue with Python: on
the contrary, I explained how the program is run by Windows Explorer, where in
Windows XP the full path is passed by Windows Explorer, and how that results in
(can result in) a full path in __file__ when a script is run by double-clicking.

So the explanation of an incorrect result involves not only Windows Explorer,
the Windows version, and Python, but also the OP's code is involved.

Hence the request for a minimal, complete example -- which is nearly always a
good idea.

> The OP's description said that it was looking for the
> file .\\myfile.xml in C:\Python31 which means it translated '.', the
> current working directory, to be C:\Python31 and not the directory the
> script is in.

I fail to reproduce that behavior.

By the way, are you the OP (just with another name)?

If not, have you reproduced the described behavior? Then kindly post the code.

>>> The easiest way to solve this permanently, by the way, is to not use
>>> relative paths. All it takes is one script to call os.chdir and the
>>> script breaks. You can use __file__ and the os.path module to figure
>>> out exactly where you are in an OS-agnostic way.
>>>
>>> import os.path
>>> #get the absolute path to the current script
>>> abs_path = os.path.abspath(__file__)

>> According to the docs: "On most platforms, this is equivalent to
>> normpath(join(os.getcwd(), path))."
>>

>
> os.path.abspath will always work in this case (unless something
> changes the current working directory before that statement runs)

Which change is what you surmised as a cause of the original problem.

Hello.

> because __file__ is given either as an absolute path or as relative to
> the current working directory.

No, that's incorrect.

<code py="3.1">
import os

print( "Current directory: [" + os.getcwd() + "]" )
print( "__file__ = " + __file__ )
os.chdir( "blah" )
print( "Current directory: [" + os.getcwd() + "]" )
print( "__file__ = " + __file__ )
</code>

<example>
C:\Documents and Settings\Alf\test> python curdir.py
Current directory: [C:\Documents and Settings\Alf\test]
__file__ = curdir.py
Current directory: [C:\Documents and Settings\Alf\test\blah]
__file__ = curdir.py <-- *Not* relative to cwd.

C:\Documents and Settings\Alf\test> _
</example>

[snip]
> Except that if the cwd is not the directory the script is running in,
> __file__ should still point to it either through an absolute path
> (highly likely since it's run through Windows Explorer) or some
> (however convoluted) relative path.

Don't know about "should", but if you're talking reality, no, that's incorrect;
see above.

[snip]

Cheers & hth.,

- Alf

Alf P. Steinbach
Guest
Posts: n/a

 02-04-2010
* Alf P. Steinbach:
> * Benjamin Kaplan:
>> On Wed, Feb 3, 2010 at 11:18 PM, Alf P. Steinbach <(E-Mail Removed)> wrote:
>>> * Benjamin Kaplan:
>>>> The easiest way to solve this permanently, by the way, is to not use
>>>> relative paths. All it takes is one script to call os.chdir and the
>>>> script breaks. You can use __file__ and the os.path module to figure
>>>> out exactly where you are in an OS-agnostic way.
>>>>
>>>> import os.path
>>>> #get the absolute path to the current script
>>>> abs_path = os.path.abspath(__file__)
>>> According to the docs: "On most platforms, this is equivalent to
>>> normpath(join(os.getcwd(), path))."
>>>

>>
>> os.path.abspath will always work in this case (unless something
>> changes the current working directory before that statement runs)

>
> Which change is what you surmised as a cause of the original problem.
>
> Hello.
>
>
>> because __file__ is given either as an absolute path or as relative to
>> the current working directory.

>
> No, that's incorrect.

Oh sorry, now I see what you mean. I read it too literally. You mean that at
script startup __file__ is a valid relative or absolute path to the script.

But anyways, Windows Explorer doesn't change the current directory to that of
the associated program, at least not in Windows XP.

Where there *is* a difference with double-clicking the script is in the path
that ends up as __file__.

[snip my silly code counter-example + even more silly comment]

Cheers,

- Alf

Gabriel Genellina
Guest
Posts: n/a

 02-04-2010
En Thu, 04 Feb 2010 02:46:45 -0300, Alf P. Steinbach <(E-Mail Removed)>
escribió:

> Oh sorry, now I see what you mean. I read it too literally. You mean
> that at script startup __file__ is a valid relative or absolute path to
> the script.
>
> But anyways, Windows Explorer doesn't change the current directory to
> that of the associated program, at least not in Windows XP.

But the associated program might change the current directory - that's not
the case with the default associations created by the Python installer,
but one should verify this.
To the OP: please create this small test script

import os
print("curdir=", os.getcwd())
print("__file__=", __file__)
input()

and post what you get when you double-click on it.

Also, from the command line, execute:

D:\temp>reg query HKCR\.py

! REG.EXE VERSION 3.0

HKEY_CLASSES_ROOT\.py
<Sin nombre> REG_SZ Python.File
Content Type REG_SZ text/plain

HKEY_CLASSES_ROOT\.py\PersistentHandler

D:\temp>reg query HKCR\Python.File\shell\open\command

! REG.EXE VERSION 3.0

HKEY_CLASSES_ROOT\Python.File\shell\open\command
<Sin nombre> REG_SZ "D:\Apps\Python26\python.exe" "%1" %*

The first command says that files ending with .py are of type
"Python.File", the second one says which command is executed to open a
"Python.File" file.

--
Gabriel Genellina

Gib Bogle
Guest
Posts: n/a

 02-04-2010
Gabriel Genellina wrote:
> But the associated program might change the current directory - that's
> not the case with the default associations created by the Python
> installer, but one should verify this.
> To the OP: please create this small test script
>
> import os
> print("curdir=", os.getcwd())
> print("__file__=", __file__)
> input()
>
> and post what you get when you double-click on it.
>
> Also, from the command line, execute:
>
> D:\temp>reg query HKCR\.py
>
> ! REG.EXE VERSION 3.0
>
> HKEY_CLASSES_ROOT\.py
> <Sin nombre> REG_SZ Python.File
> Content Type REG_SZ text/plain
>
> HKEY_CLASSES_ROOT\.py\PersistentHandler

I'm interested in this, because I'm using Windows XP, and when I execute this
command I see the first part but not the second (with PersistentHandler). Is
this related to the fact that when I double-click on a .py file the command
window disappears after the execution is completed?

Gabriel Genellina
Guest
Posts: n/a

 02-05-2010
En Thu, 04 Feb 2010 17:50:29 -0300, Gib Bogle
<(E-Mail Removed)> escribió:
> Gabriel Genellina wrote:

>> Also, from the command line, execute:
>> D:\temp>reg query HKCR\.py
>> ! REG.EXE VERSION 3.0
>> HKEY_CLASSES_ROOT\.py
>> <Sin nombre> REG_SZ Python.File
>> Content Type REG_SZ text/plain
>> HKEY_CLASSES_ROOT\.py\PersistentHandler

>
> I'm interested in this, because I'm using Windows XP, and when I execute
> this command I see the first part but not the second (with
> PersistentHandler).

Sorry, I should have removed that line. This is just my setup; a normal
Python install doesn't create that registry entry. It allows Desktop
Search (or Windows Search, or whatever it is called nowadays; the F3 key)
to search inside .py files (default behavior is to just ignore their
contents).
See http://support.microsoft.com/kb/309173

> Is this related to the fact that when I double-click on a .py file the
> command window disappears after the execution is completed?

(I bet the "Persistent" word confused you.) No, as you can see, it's
completely unrelated. AFAIK, there is no way (on XP and later at least) to
keep a console window open after the program exited. Three choices:

- Open a cmd window and execute the script there. You may drag&drop the
file over the window to avoid typing the full path (I think this last part
does not work on Vista nor Win7)

- Add a raw_input() [2.x] or input() [3.x] line at the end of the script

- Rename it with a '.cmd' extension and add this line at the very top:

@(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF

(see this post by Duncan Booth last month:

--
Gabriel Genellina

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is OffTrackbacks are On Pingbacks are On Refbacks are Off Forum Rules

 Similar Threads Thread Thread Starter Forum Replies Last Post CBIC MCSE 6 09-29-2006 06:55 PM larry Computer Support 1 09-09-2005 07:14 PM edgy Computer Support 4 10-29-2004 11:07 AM foldface@yahoo.co.uk ASP .Net 0 06-11-2004 02:26 PM karlpov DVD Video 11 12-28-2003 11:47 PM