Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Read and re-write file with one open?

Reply
Thread Tools

Read and re-write file with one open?

 
 
Adam Bender
Guest
Posts: n/a
 
      04-30-2009
[Note: parts of this message were removed to make it a legal post.]

I would like to write a Ruby script that opens a text file, performs a gsub
on each line, and then overwrites the file with the updated contents. Right
now I open the file twice: once to read and once to write. The reason for
this is that if I try to perform both operations on the same IO object by
calling io.rewind and writing from the beginning, and the substituted word
is shorter than what it is replacing, some portion of the end of the
original file remains. Is there an idiom for "clearing" the contents of the
file before writing?

Thanks,

Adam

 
Reply With Quote
 
 
 
 
Siddick Ebramsha
Guest
Posts: n/a
 
      04-30-2009
Instead of using the rewind method, you can use the reopen method to
open a same file in Write mode.
Example :-
file = File.open( "filename", "r" )
file.reopen( "filename", "w" )
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Gregory Brown
Guest
Posts: n/a
 
      04-30-2009
On Thu, Apr 30, 2009 at 12:27 AM, Adam Bender <(E-Mail Removed)> wrote:
> I would like to write a Ruby script that opens a text file, performs a gs=

ub
> on each line, and then overwrites the file with the updated contents. =A0=

Right
> now I open the file twice: once to read and once to write. =A0The reason =

for
> this is that if I try to perform both operations on the same IO object by
> calling io.rewind and writing from the beginning, and the substituted wor=

d
> is shorter than what it is replacing, some portion of the end of the
> original file remains. =A0Is there an idiom for "clearing" the contents o=

f the
> file before writing?


Yes, but typically this is done by creating a new file and then
renaming it to replace the old one.

Here's an example from my upcoming book "Ruby Best Practices"[0]. It
naively strips comments from source files.
You can modify it to fit your needs.

----------

require "tempfile"
require "fileutils"

temp =3D Tempfile.new("without_comments")
File.foreach(ARGV[0]) do |line|
temp << line unless line =3D~ /^\s*#/
end
temp.close

FileUtils.cp(ARGV[0],"#{ARGV[0]}.bak") # comment out if you don't want back=
ups.
FileUtils.mv(temp.path,ARGV[0])

----------

-greg

PS: sorry for the shameless book plug, but it really might be helpful
for questions like these.


[0] http://rubybestpractices.com

 
Reply With Quote
 
7stud --
Guest
Posts: n/a
 
      04-30-2009
Adam Bender wrote:
> I would like to write a Ruby script that opens a text file, performs a
> gsub
> on each line, and then overwrites the file with the updated contents.
> Right
> now I open the file twice: once to read and once to write. The reason
> for
> this is that if I try to perform both operations on the same IO object
> by
> calling io.rewind and writing from the beginning, and the substituted
> word
> is shorter than what it is replacing, some portion of the end of the
> original file remains. Is there an idiom for "clearing" the contents of
> the
> file before writing?
>


Yes, opening the file in write mode! However, you are going to expose
yourself to this catastrophe. Suppose you read the contents of the
file into a variable, then open the file for writing, which then erases
the file, but immediately thereafter your program crashes or the power
goes out in your city. What are you left with? You will be left with
an empty file, and the variable that contained the contents of the file
will have evaporated into the ether. In other words, you will lose all
your data!

So the idiom for rewriting a file is:

1) Open the file for *reading*.
2) Open another file for writing with a name like origName-edited.txt
3) Read the original file line by line(saves memory, but is slower)
4) Write each altered line to the file origName-edited.txt
5) Delete the original file.
6) Change the name of the new file (origName-edited.txt) to origName.txt









--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Adam Bender
Guest
Posts: n/a
 
      04-30-2009
[Note: parts of this message were removed to make it a legal post.]

On Thu, Apr 30, 2009 at 1:26 AM, 7stud -- <(E-Mail Removed)> wrote:

> Yes, opening the file in write mode! However, you are going to expose
> yourself to this catastrophe. Suppose you read the contents of the
> file into a variable, then open the file for writing, which then erases
> the file, but immediately thereafter your program crashes or the power
> goes out in your city.
>
> So the idiom for rewriting a file is:
>
> 1) Open the file for *reading*.
> 2) Open another file for writing with a name like origName-edited.txt
> 3) Read the original file line by line(saves memory, but is slower)
> 4) Write each altered line to the file origName-edited.txt
> 5) Delete the original file.
> 6) Change the name of the new file (origName-edited.txt) to origName.txt



I see the potential for catastrophe with the way I suggested, however, there
is potential for catastrophe here if there is already an
"origName-edited.txt" file (I know, slim chance, but you never know). You
could get around this by generating new file names until you found one that
didn't exist, or writing to /tmp, of course. I think I'll switch to Greg
Brown's suggestion. Does Tempfile guarantee that it won't overwrite an
existing file?

Adam

 
Reply With Quote
 
7stud --
Guest
Posts: n/a
 
      04-30-2009
Adam Bender wrote:
> On Thu, Apr 30, 2009 at 1:26 AM, 7stud -- <(E-Mail Removed)>
> wrote:
>
>> 3) Read the original file line by line(saves memory, but is slower)
>> 4) Write each altered line to the file origName-edited.txt
>> 5) Delete the original file.
>> 6) Change the name of the new file (origName-edited.txt) to origName.txt

>
>
> I see the potential for catastrophe with the way I suggested, however,
> there
> is potential for catastrophe here if there is already an
> "origName-edited.txt" file (I know, slim chance, but you never know).
>


Of course, if that was a possibility then you would take extra measures
like create a new file name with rand, and then check it with
File.exists?, which is probably what Tempfile does.

> Does Tempfile guarantee that it won't overwrite an

existing file?

What the standard library docs aren't clear enough for you:

----
tempfile - manipulates temporary files
----

???!! lol. pathetic. But once in a great while you can actually find
some information on a standard library module using google:

http://www.rubytips.org/2008/01/11/u...y-tempfilenew/

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      04-30-2009
2009/4/30 Gregory Brown <(E-Mail Removed)>:
> On Thu, Apr 30, 2009 at 12:27 AM, Adam Bender <(E-Mail Removed)> wrote:
>> I would like to write a Ruby script that opens a text file, performs a g=

sub
>> on each line, and then overwrites the file with the updated contents. =

=A0Right
>> now I open the file twice: once to read and once to write. =A0The reason=

for
>> this is that if I try to perform both operations on the same IO object b=

y
>> calling io.rewind and writing from the beginning, and the substituted wo=

rd
>> is shorter than what it is replacing, some portion of the end of the
>> original file remains. =A0Is there an idiom for "clearing" the contents =

of the
>> file before writing?

>
> Yes, but typically this is done by creating a new file and then
> renaming it to replace the old one.
>
> Here's an example from my upcoming book "Ruby Best Practices"[0]. =A0It
> naively strips comments from source files.


A variant exploiting Ruby's command line parameters:

11:05:08 Temp$ ruby -e '10.times {|i| puts i}' >| x
11:05:22 Temp$ cat x
0
1
2
3
4
5
6
7
8
9
11:05:23 Temp$ ./x.rb x
11:05:29 Temp$ cat x
<<<0>>>
<<<1>>>
<<<2>>>
<<<3>>>
<<<4>>>
<<<5>>>
<<<6>>>
<<<7>>>
<<<8>>>
<<<9>>>
11:05:34 Temp$ cat x.bak
0
1
2
3
4
5
6
7
8
9
11:05:37 Temp$ cat x.rb
#!/opt/bin/ruby19 -pi.bak

$_.sub! /^/, '<<<'
$_.sub! /$/, '>>>'
11:05:40 Temp$

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 
Reply With Quote
 
Gregory Brown
Guest
Posts: n/a
 
      04-30-2009
On Thu, Apr 30, 2009 at 3:11 AM, Adam Bender <(E-Mail Removed)> wrote:

> I see the potential for catastrophe with the way I suggested, however, th=

ere
> is potential for catastrophe here if there is already an
> "origName-edited.txt" file (I know, slim chance, but you never know). =A0=

You
> could get around this by generating new file names until you found one th=

at
> didn't exist, or writing to /tmp, of course. =A0I think I'll switch to Gr=

eg
> Brown's suggestion. =A0Does Tempfile guarantee that it won't overwrite an
> existing file?


Yes, Tempfile avoids file collisions.

-greg

 
Reply With Quote
 
James Dinkel
Guest
Posts: n/a
 
      04-30-2009
Gregory Brown wrote:
>
> Here's an example from my upcoming book "Ruby Best Practices"[0]. It
> naively strips comments from source files.
> You can modify it to fit your needs.
>
> ----------
>
> require "tempfile"
> require "fileutils"
>
> temp = Tempfile.new("without_comments")
> File.foreach(ARGV[0]) do |line|
> temp << line unless line =~ /^\s*#/
> end
> temp.close
>
> FileUtils.cp(ARGV[0],"#{ARGV[0]}.bak") # comment out if you don't want
> backups.
> FileUtils.mv(temp.path,ARGV[0])
>
> ----------
>
> -greg


What about if another program opens the file after it has been read by
the Ruby program but before the Ruby program has copied the temp file?
If the second program makes a change and saves it, those changes will be
lost when the Ruby program copies the temp file over it. Or if the
second program still has it open and the Ruby program finishes, then
when the second program saves it's open file, it will overwrite the Ruby
program's changes.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Gregory Brown
Guest
Posts: n/a
 
      04-30-2009
On Thu, Apr 30, 2009 at 10:41 AM, James Dinkel <(E-Mail Removed)> wrote:

> What about if another program opens the file after it has been read by
> the Ruby program but before the Ruby program has copied the temp file?
> If the second program makes a change and saves it, those changes will be
> lost when the Ruby program copies the temp file over it. =A0Or if the
> second program still has it open and the Ruby program finishes, then
> when the second program saves it's open file, it will overwrite the Ruby
> program's changes.


Is this related to the OP's concerns? In this case, you'd need file
locking (see the Ruby API).
But I didn't see any mention of these sorts of issues in Adam's original po=
st.

If you need this feature, read the API docs for File#flock

-greg

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shortest way to read all lines (one by one) from a text file? Robin Wenger Java 11 02-14-2011 12:08 PM
File.read(fname) vs. File.read(fname,File.size(fname)) Alex Dowad Ruby 4 05-01-2010 08:20 AM
how to read parse data from two file to generate one one file Asif Iqbal Ruby 0 08-06-2009 04:47 PM
file.read() doesn't read the whole file Sreejith K Python 24 03-24-2009 12:20 PM
Need to concatenate all files in a dir together into one file and read the first 225 characters from each file into another file. Tony Perl Misc 5 04-19-2004 03:28 PM



Advertisments