Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > interaction of mode 'r+', file.write(), and file.tell(): a bug orundefined behavior?

Reply
Thread Tools

interaction of mode 'r+', file.write(), and file.tell(): a bug orundefined behavior?

 
 
Lie Ryan
Guest
Posts: n/a
 
      01-28-2010
In the code:

"""
f = open('input.txt', 'r+')
for line in f:
s = line.replace('python', 'PYTHON')
# f.tell()
f.write(s)
"""

When f.tell() is commented, 'input.txt' does not change; but when
uncommented, the f.write() succeeded writing into the 'input.txt'
(surprisingly, but not entirely unexpected, at the end of the file).


$ #####################################
$
$ cp orig.txt input.txt
$ cat input.txt
abcde
abc python abc
python abc python
$ python
Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27)
[GCC 4.3.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('input.txt', 'r+')
>>> for line in f:

.... s = line.replace('python', 'PYTHON')
.... f.write(s)
....
>>>

$ cat input.txt
abcde
abc python abc
python abc python
$
$ #####################################
$
$ cp orig.txt input.txt
$ cat input.txt
abcde
abc python abc
python abc python
$ python
Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27)
[GCC 4.3.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('input.txt', 'r+')
>>> for line in f:

.... s = line.replace('python', 'PYTHON')
.... f.tell()
.... f.write(s)
....
39
45
60
>>>

$ cat input.txt
abcde
abc python abc
python abc python
abcde
abc PYTHON abc
$
$ #####################################



Do you think this should be a bug or undefined behavior governed by the
underlying OS and C library? Shouldn't file.tell() be purely
informational, and not have side effect?




The machine is Gentoo (amd64, gcc-4.3.4, glibc-2.10.1-r1), Linux
(2.6.31-gentoo-r6), and Python 2.6.4
 
Reply With Quote
 
 
 
 
Anthony Tolle
Guest
Posts: n/a
 
      01-28-2010
On Jan 28, 7:12*am, Lie Ryan <(E-Mail Removed)> wrote:
> In the code:
>
> """
> f = open('input.txt', 'r+')
> for line in f:
> * * s = line.replace('python', 'PYTHON')
> * * # f.tell()
> * * f.write(s)
> """
> [snip]


My guess is that there are a few possible problems:

1) In this case, writing to file opened with 'r+' without an explicit
f.seek is probably not a good idea. The file iterator (for line in f)
uses a readahead buffer, which means you can't guarantee what the
current file position will be.

2) It may be necessary to do an explicit f.flush or f.close when
writing to an 'r+' file. In your case, the close should automatically
happen when the f object falls out of scope, which tells me that were
still looking at some other problem, like not using f.seek

3) It is possible that f.tell implicitly flushes buffers used by the
file object. That would explain why uncommenting the f.tell causes
the writes to show up.


What are you trying to accomplish? Overwrite the original file, or
append to it? If you want to overwrite the file, it may be better to
generate a new file, delete the old one, then rename the new one. If
you want to append, then it would be better to open the file with
append mode ('a')
 
Reply With Quote
 
 
 
 
Alf P. Steinbach
Guest
Posts: n/a
 
      01-28-2010
* Anthony Tolle:
> On Jan 28, 7:12 am, Lie Ryan <(E-Mail Removed)> wrote:
>> In the code:
>>
>> """
>> f = open('input.txt', 'r+')
>> for line in f:
>> s = line.replace('python', 'PYTHON')
>> # f.tell()
>> f.write(s)
>> """
>> [snip]

>
> My guess is that there are a few possible problems:
>
> 1) In this case, writing to file opened with 'r+' without an explicit
> f.seek is probably not a good idea. The file iterator (for line in f)
> uses a readahead buffer, which means you can't guarantee what the
> current file position will be.
>
> 2) It may be necessary to do an explicit f.flush or f.close when
> writing to an 'r+' file. In your case, the close should automatically
> happen when the f object falls out of scope, which tells me that were
> still looking at some other problem, like not using f.seek
>
> 3) It is possible that f.tell implicitly flushes buffers used by the
> file object. That would explain why uncommenting the f.tell causes
> the writes to show up.


As far as I understand it the behavior stems from CPython file operations being
implemented fairly directly as forwarding to C library FILE* operations, and the
C standard prescribes Undefined Behavior to the case above.

I think the Python language/library specification should specify the effect
(perhaps just as UB, but anyway, specified).

For as it is, it may/will be different with different Python implementations,
meaning that code that works OK with one implementation may fail with another
implementation.



> What are you trying to accomplish? Overwrite the original file, or
> append to it? If you want to overwrite the file, it may be better to
> generate a new file, delete the old one, then rename the new one. If
> you want to append, then it would be better to open the file with
> append mode ('a')


Cheers,

- Alf
 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      02-01-2010
In article <4b617f4a$(E-Mail Removed)>,
Lie Ryan <(E-Mail Removed)> wrote:
>
>f = open('input.txt', 'r+')
>for line in f:
> s = line.replace('python', 'PYTHON')
> # f.tell()
> f.write(s)
>
>When f.tell() is commented, 'input.txt' does not change; but when
>uncommented, the f.write() succeeded writing into the 'input.txt'
>(surprisingly, but not entirely unexpected, at the end of the file).


Another possible issue is that using a file iterator is generally not
compatible with direct file operations.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

import antigravity
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: mmm-mode, python-mode and doctest-mode? Neal Becker Python 0 08-09-2007 12:10 PM
mmm-mode, python-mode and doctest-mode? Edward Loper Python 0 08-09-2007 05:40 AM
re: mmm-mode, python-mode and doctest-mode? John J Lee Python 0 08-07-2007 07:49 PM
re: mmm-mode, python-mode and doctest-mode? Edward Loper Python 0 08-07-2007 08:58 AM
mmm-mode, python-mode and doctest-mode? John J Lee Python 3 12-01-2005 08:35 PM



Advertisments