Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > when should I explicitely close a file?

Reply
Thread Tools

when should I explicitely close a file?

 
 
gelonida
Guest
Posts: n/a
 
      04-13-2010
Hi,


I've been told, that following code snippet is not good.


open("myfile","w").write(astring) , because I'm neither explicitely
closing nor using the new 'with' syntax.

What exactly is the impact of not closing the file explicitely
(implicitley with a 'with' block)?


Even with my example
I'd expected to get an exception raised if not all data could have
been written.

I'd also expected, that all write data is flushed as soon as the
filehandle is out of scope (meaning in the next line of my source
code).


Thanks for explaining me exactly what kind of evil I could encounter
with not explicitely closing.

 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      04-13-2010
On Tue, Apr 13, 2010 at 3:01 PM, gelonida <(E-Mail Removed)> wrote:
> Hi,
>
> I've been told, that following code snippet is not good.
>
> open("myfile","w").write(astring) , because I'm neither explicitely
> closing nor using the new 'with' syntax.
>
> What exactly is the impact of not closing the file explicitely
> (implicitley with a 'with' block)?
>
> Even with my example
> I'd expected to get an exception raised if not all data could have
> been written.
>
> I'd also expected, that all write data is flushed as soon as the
> filehandle is out of scope (meaning in the next line of my source
> code).


That extremely-quick responsiveness of the garbage-collection
machinery is only guaranteed by CPython, not the language
specification itself, and indeed some of the other implementations
*explicitly don't* make that guarantee (and hence the data may not get
flushed in a timely manner on those implementations). And portability
of code is encouraged, hence the admonishment you encountered.

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-14-2010
On Tue, 13 Apr 2010 15:01:25 -0700, gelonida wrote:

> Hi,
>
>
> I've been told, that following code snippet is not good.
>
>
> open("myfile","w").write(astring) , because I'm neither explicitely
> closing nor using the new 'with' syntax.
>
> What exactly is the impact of not closing the file explicitely
> (implicitley with a 'with' block)?



Your data may not be actually written to disk until the file closes. If
you have a reference loop, and Python crashes badly enough, the garbage
collector may never run and the file will never be closed, hence you will
get data loss.

If you are running something other than CPython (e.g. IronPython or
Jython) then the file might not be closed until your program exits. If
you have a long-running program that opens many, many files, it is
possible for you to run out of system file handles and be unable to open
any more.

Best practice is to explicitly close the file when you are done with it,
but for short scripts, I generally don't bother. Laziness is a virtue

But for library code and larger applications, I always explicitly close
the file, because I want to control exactly when the file is closed
rather than leave it up to the interpreter. I don't know if my code might
one day end up in a long-running Jython application so I try to code
defensively and avoid even the possibility of a problem.


> Even with my example
> I'd expected to get an exception raised if not all data could have been
> written.


Generally if you get an exception while trying to *close* a file, you're
pretty much stuffed. What are you going to do? How do you recover?

My feeling is that you're probably safe with something as simple as

file("myfile", "w").write("my data\n")

but if you do something like

some_data_structure.filehandle = file("myfile", "w")
some_data_structure.filehandle.write("my data\n")
# ... lots more code here

and some_data_structure keeps the file open until the interpreter shuts
down, there *might* be rare circumstances where you won't be notified of
an exception, depending on the exact circumstances of timing of when the
file gets closed. In the worst case, the file might not be closed until
the interpreter is shutting down *and* has already dismantled the
exception infrastructure, and so you can't get an exception. I don't know
enough about the Python runtime (particularly about how it works during
shutdown) to know how real this danger is, but if it is a danger, I bet
it involves __del__ methods.



> I'd also expected, that all write data is flushed as soon as the
> filehandle is out of scope (meaning in the next line of my source code).


This is only guaranteed with CPython, not other implementations.

My feeling is that explicit closing is pedantic and careful, implicit
closing is lazy and easy. You make your choice and take your chance




--
Steven
 
Reply With Quote
 
Giampaolo Rodola'
Guest
Posts: n/a
 
      04-14-2010
What about open('foo', 'w').close().
Does it have the same problems?


--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil
 
Reply With Quote
 
Chris Rebert
Guest
Posts: n/a
 
      04-14-2010
On Tue, Apr 13, 2010 at 5:45 PM, Giampaolo Rodola' <(E-Mail Removed)> wrote:
> What about open('foo', 'w').close().
> Does it have the same problems?


Well, no, but that's only because it's a pointless no-op that doesn't
really do anything besides possibly throwing an exception (e.g. if the
script didn't have write access to the current directory).

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
Dave Angel
Guest
Posts: n/a
 
      04-14-2010
gelonida wrote:
> Hi,
>
>
> I've been told, that following code snippet is not good.
>
>
> open("myfile","w").write(astring) , because I'm neither explicitely
> closing nor using the new 'with' syntax.
>
> What exactly is the impact of not closing the file explicitely
> (implicitley with a 'with' block)?
>
>
> Even with my example
> I'd expected to get an exception raised if not all data could have
> been written.
>
> I'd also expected, that all write data is flushed as soon as the
> filehandle is out of scope (meaning in the next line of my source
> code).
>
>
> Thanks for explaining me exactly what kind of evil I could encounter
> with not explicitely closing.
>
>
>

Evil? No. Just undefined behavior.

The language does NOT guarantee that a close or even a flush will occur
when an object "goes out of scope." This is the same in Python as it is
in Java. There's also no exception for data not being flushed.

In one particular implementation of Python, called CPython, there are
some things that tend to help. So if you're sure you're always going to
be using this particular implementation, and understand what the
restrictions are, then go ahead and be sloppy. Similarly, on some OS
systems, files are flushed when a process ends. So if you know your
application is only going to run on those environments, you might not
bother closing files at the end of execution.

It all depends on how restrictive your execution environment is going to be.

DaveA

 
Reply With Quote
 
Ryan Kelly
Guest
Posts: n/a
 
      04-14-2010
On Tue, 2010-04-13 at 18:19 -0700, Chris Rebert wrote:
> On Tue, Apr 13, 2010 at 5:45 PM, Giampaolo Rodola' <(E-Mail Removed)> wrote:
> > What about open('foo', 'w').close().
> > Does it have the same problems?

>
> Well, no, but that's only because it's a pointless no-op that doesn't
> really do anything besides possibly throwing an exception (e.g. if the
> script didn't have write access to the current directory).


Actually, it will create the file if it doesn't exist, and truncate it
to zero length if it does.


Ryan


--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
http://www.velocityreviews.com/forums/(E-Mail Removed) | http://www.rfk.id.au/ramblings/gpg/ for details


 
Reply With Quote
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      04-17-2010
In message
<(E-Mail Removed)>, gelonida
wrote:

> I've been told, that following code snippet is not good.
>
> open("myfile","w").write(astring) ...


I do that for reads, but never for writes.

For writes, you want to give a chance for write errors to raise an exception
and alert the user, instead of failing silently, to avoid inadvertent data
loss. Hence the explicit close.
 
Reply With Quote
 
Lie Ryan
Guest
Posts: n/a
 
      04-17-2010
On 04/17/10 21:23, Lawrence D'Oliveiro wrote:
> In message
> <(E-Mail Removed)>, gelonida
> wrote:
>
>> I've been told, that following code snippet is not good.
>>
>> open("myfile","w").write(astring) ...

>
> I do that for reads, but never for writes.
>
> For writes, you want to give a chance for write errors to raise an exception
> and alert the user, instead of failing silently, to avoid inadvertent data
> loss. Hence the explicit close.


In short, in case of doubt, just be explicit.

Since in python nothing is guaranteed about implicit file close, you
must always explicitly close it.
 
Reply With Quote
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      04-22-2010
In message <4bc9aadb$(E-Mail Removed)>, Lie Ryan wrote:

> Since in python nothing is guaranteed about implicit file close ...


It is guaranteed that objects with a reference count of zero will be
disposed. In my experiments, this happens immediately.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to close a TCP socket? (TCPSocket#close doesn't close it) Iñaki Baz Castillo Ruby 7 01-12-2010 01:32 PM
Explicitely specifying the size of a std::vector mathieu C++ 4 08-12-2006 09:47 PM
Must explicitely the ID of control? Why? Amelyan ASP .Net 0 05-23-2005 11:50 PM
Calling constructor explicitely Victor Bazarov C++ 9 02-24-2005 11:28 AM
Explicitely calling Thread.run Phaedrus Java 3 07-06-2003 07:28 PM



Advertisments