Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > tempfile.NamedTemporaryFile wont work

Reply
Thread Tools

tempfile.NamedTemporaryFile wont work

 
 
Imbaud Pierre
Guest
Posts: n/a
 
      11-19-2006
On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected.
(I found a permanent workaround, so I dont ask for help)
I expected to write to a file, and access it thru a shell command.
This code, in a loop:
tf = tempfile.NamedTemporaryFile()
tfName = tf.name
#tf.seek(0) # rewind the file
tf.write(chunk); tf.flush()
print >> sys.stderr, '%s: %s' % (tfName, ['no',
'yes'][os.path.exists(tfName)])
subprocess.Popen(['strings', tfName])

Symptom: the file does not always exist, after the call to
NamedTemporaryFile(). Or at least its not seen by the strings command,
or by os.path.exists.

I guess the bug is pretty much os dependent, or even filesystem
dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the
python interface. Or did I miss something?
Shame, I didnt even try to check for a python bug tracker.
 
Reply With Quote
 
 
 
 
Bjoern Schliessmann
Guest
Posts: n/a
 
      11-19-2006
Imbaud Pierre wrote:

> tf = tempfile.NamedTemporaryFile()
> tfName = tf.name
> [...]
> print >> sys.stderr, '%s: %s' % (tfName, ['no',
> 'yes'][os.path.exists(tfName)])
> subprocess.Popen(['strings', tfName])


Just out of curiosity: Why did you assign tf.name to tfname?

Hypothetically, if tf.name changed, tfname wouldn't follow since
strings are immutable.

Regards,


Björn

--
BOFH excuse #149:

Dew on the telephone lines.

 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      11-19-2006
On Sun, 19 Nov 2006 13:11:13 +0100, Imbaud Pierre wrote:

> On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected.

[snip]

> Symptom: the file does not always exist, after the call to
> NamedTemporaryFile(). Or at least its not seen by the strings command,
> or by os.path.exists.
>
> I guess the bug is pretty much os dependent, or even filesystem
> dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the
> python interface. Or did I miss something?
> Shame, I didnt even try to check for a python bug tracker.


I can verify this problem occurs on Fedora Core 5 too:

import os
import sys
import tempfile
import subprocess
def test(n):
chunk = ': +++ abcd +++'
for i in xrange(n):
tf = tempfile.NamedTemporaryFile()
tfName = tf.name
tf.seek(0)
tf.write(str(i) + chunk)
tf.flush()
if not os.path.exists(tfName):
print 'pre-check: %s not there' % tfName
subprocess.Popen(['strings', tfName])
if not os.path.exists(tfName):
print 'post-check: %s not there' % tfName


And here is a typical run, with the boring bits removed for ease of
reading:

>>> test(30)

0: +++ abcd +++
1: +++ abcd +++
[ more of the same ]
14: +++ abcd +++
strings: '/tmp/tmpOALbx9': No such file
16: +++ abcd +++
17: +++ abcd +++
18: +++ abcd +++
[ more of the same ]
27: +++ abcd +++
strings: /tmp/tmpdc52Nz: No such file or directory
29: +++ abcd +++


Curiouser and curiouser... not only does os.path.exist always report the
temp file as existing (at least in my tests), even when strings can't find
it, but strings returns different error messages.

Is it possible this is a bug in strings?


--
Steven.

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      11-19-2006
On Sun, 19 Nov 2006 13:18:39 +0100, Bjoern Schliessmann wrote:

> Imbaud Pierre wrote:
>
>> tf = tempfile.NamedTemporaryFile()
>> tfName = tf.name
>> [...]
>> print >> sys.stderr, '%s: %s' % (tfName, ['no',
>> 'yes'][os.path.exists(tfName)])
>> subprocess.Popen(['strings', tfName])

>
> Just out of curiosity: Why did you assign tf.name to tfname?
>
> Hypothetically, if tf.name changed, tfname wouldn't follow since
> strings are immutable.


Well, yes, but if tf.name changed, that won't change the file name on disk
either:

>>> tf = tempfile.NamedTemporaryFile()
>>> tf.name

'/tmp/tmpYVV1Ij'
>>> os.path.exists(tf.name)

True
>>> oldname = tf.name
>>> tf.name = "/tmp/something"
>>> os.path.exists(tf.name)

False
>>> os.path.exists(oldname)

True


I'm guessing that binding tf.name to tfName is a micro-optimization. In a
very tight loop, name lookups can take considerable time, and one
optimization can be to reduce the number of lookups:

method = something.method
while 1:
something.method # needs at least two lookups
method # needs a single lookup


--
Steve.

 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      11-19-2006
Steven D'Aprano wrote:

> On Sun, 19 Nov 2006 13:11:13 +0100, Imbaud Pierre wrote:
>
>> On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected.

> [snip]
>
>> Symptom: the file does not always exist, after the call to
>> NamedTemporaryFile(). Or at least its not seen by the strings command,
>> or by os.path.exists.
>>
>> I guess the bug is pretty much os dependent, or even filesystem
>> dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the
>> python interface. Or did I miss something?
>> Shame, I didnt even try to check for a python bug tracker.

>
> I can verify this problem occurs on Fedora Core 5 too:
>
> import os
> import sys
> import tempfile
> import subprocess
> def test(n):
> chunk = ': +++ abcd +++'
> for i in xrange(n):
> tf = tempfile.NamedTemporaryFile()
> tfName = tf.name
> tf.seek(0)
> tf.write(str(i) + chunk)
> tf.flush()
> if not os.path.exists(tfName):
> print 'pre-check: %s not there' % tfName
> subprocess.Popen(['strings', tfName])
> if not os.path.exists(tfName):
> print 'post-check: %s not there' % tfName
>
>
> And here is a typical run, with the boring bits removed for ease of
> reading:
>
>>>> test(30)

> 0: +++ abcd +++
> 1: +++ abcd +++
> [ more of the same ]
> 14: +++ abcd +++
> strings: '/tmp/tmpOALbx9': No such file
> 16: +++ abcd +++
> 17: +++ abcd +++
> 18: +++ abcd +++
> [ more of the same ]
> 27: +++ abcd +++
> strings: /tmp/tmpdc52Nz: No such file or directory
> 29: +++ abcd +++
>
>
> Curiouser and curiouser... not only does os.path.exist always report the
> temp file as existing (at least in my tests), even when strings can't find
> it, but strings returns different error messages.
>
> Is it possible this is a bug in strings?


What /you/ are seeing is not a bug, I think. Popen() is asynchronous,
therefore you may enter the second iteration -- which implicitly closes the
temporary file -- before strings actually tries to access it. Use call()
and everything should be fine.

Peter

 
Reply With Quote
 
Imbaud Pierre
Guest
Posts: n/a
 
      11-19-2006
Steven D'Aprano a écrit :
> On Sun, 19 Nov 2006 13:18:39 +0100, Bjoern Schliessmann wrote:
>
>
>>Imbaud Pierre wrote:
>>
>>
>>> tf = tempfile.NamedTemporaryFile()
>>> tfName = tf.name
>>>[...]
>>> print >> sys.stderr, '%s: %s' % (tfName, ['no',
>>>'yes'][os.path.exists(tfName)])
>>> subprocess.Popen(['strings', tfName])

>>
>>Just out of curiosity: Why did you assign tf.name to tfname?
>>
>>Hypothetically, if tf.name changed, tfname wouldn't follow since
>>strings are immutable.

>
>
> Well, yes, but if tf.name changed, that won't change the file name on disk
> either:
>
>
>>>>tf = tempfile.NamedTemporaryFile()
>>>>tf.name

>
> '/tmp/tmpYVV1Ij'
>
>>>>os.path.exists(tf.name)

>
> True
>
>>>>oldname = tf.name
>>>>tf.name = "/tmp/something"
>>>>os.path.exists(tf.name)

>
> False
>
>>>>os.path.exists(oldname)

>
> True
>
>
> I'm guessing that binding tf.name to tfName is a micro-optimization.

indeed. And I dont see why tf.name would change.
In a
> very tight loop, name lookups can take considerable time, and one
> optimization can be to reduce the number of lookups:
>
> method = something.method
> while 1:
> something.method # needs at least two lookups
> method # needs a single lookup
>
>


 
Reply With Quote
 
Imbaud Pierre
Guest
Posts: n/a
 
      11-19-2006
Peter Otten a écrit :
> Steven D'Aprano wrote:
>
>
>>On Sun, 19 Nov 2006 13:11:13 +0100, Imbaud Pierre wrote:
>>
>>
>>>On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected.

>>
>>[snip]
>>
>>
>>>Symptom: the file does not always exist, after the call to
>>>NamedTemporaryFile(). Or at least its not seen by the strings command,
>>>or by os.path.exists.
>>>
>>>I guess the bug is pretty much os dependent, or even filesystem
>>>dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the
>>>python interface. Or did I miss something?
>>>Shame, I didnt even try to check for a python bug tracker.

>>
>>I can verify this problem occurs on Fedora Core 5 too:
>>
>>import os
>>import sys
>>import tempfile
>>import subprocess
>>def test(n):
>>chunk = ': +++ abcd +++'
>>for i in xrange(n):
>>tf = tempfile.NamedTemporaryFile()
>>tfName = tf.name
>>tf.seek(0)
>>tf.write(str(i) + chunk)
>> tf.flush()
>>if not os.path.exists(tfName):
>>print 'pre-check: %s not there' % tfName
>>subprocess.Popen(['strings', tfName])
>>if not os.path.exists(tfName):
>>print 'post-check: %s not there' % tfName
>>
>>
>>And here is a typical run, with the boring bits removed for ease of
>>reading:
>>
>>
>>>>>test(30)

>>
>>0: +++ abcd +++
>>1: +++ abcd +++
>> [ more of the same ]
>>14: +++ abcd +++
>>strings: '/tmp/tmpOALbx9': No such file
>>16: +++ abcd +++
>>17: +++ abcd +++
>>18: +++ abcd +++
>> [ more of the same ]
>>27: +++ abcd +++
>>strings: /tmp/tmpdc52Nz: No such file or directory
>>29: +++ abcd +++
>>
>>
>>Curiouser and curiouser... not only does os.path.exist always report the
>>temp file as existing (at least in my tests), even when strings can't find
>>it, but strings returns different error messages.
>>
>>Is it possible this is a bug in strings?

>
>
> What /you/ are seeing is not a bug, I think. Popen() is asynchronous,
> therefore you may enter the second iteration -- which implicitly closes the
> temporary file -- before strings actually tries to access it. Use call()
> and everything should be fine.

Thanks A LOT, works fine, I feel kind of silly; your diagnostic is
pretty obvious, afterward... I felt uneasy not closing the Popen, but
it worked, so why bother? Its so easy to make ugly code!

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
192.168.1.1 wont work Nigel Day Wireless Networking 4 11-22-2009 03:12 PM
wont start,wont stop hunty Computer Support 4 06-04-2006 05:26 PM
Adhoc wifi bridge on server wont work =?Utf-8?B?U2lDa0NvVw==?= Wireless Networking 5 10-26-2005 09:31 PM
firefox quit and wont work Vernon Firefox 10 10-11-2005 04:12 PM
Java wont work on FB 0.7...? Axl Firefox 7 12-20-2003 06:55 AM



Advertisments