Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Reading and writing a binary file

Reply
Thread Tools

Reading and writing a binary file

 
 
Chris Guenther
Guest
Posts: n/a
 
      03-21-2005
Hi,

I am having trouble reading values from an existing (or maybe not yet
exisiting) binary file:
My target is to open a binary file for reading and writing (if it does not
exisit it should be created).
If the file was not yes existing I'd like to fill it with binary data to a
certain predefined size.
Later on I'd like to position my file position, and read and write whatever
I like.

The following piece does not work as intended:

require 'pp'
UNDER_UNIX = CONFIG['target_os'].to_s.index('linux')!=nil
filename='foo.bin'
size=1024
dont_write=FileTest.exists?(filename) && File.stat(filename).size==size
begin
fd=File.open(filename,"ab+")
fd.binmode unless UNDER_UNIX # assuming !UNIX=WINDOWS
rescue => e
pp e
fd=File.open(filename,"wb+")
puts "opened #{filename} with 'wib+'"
end

fd.sync=true
fd.seek(0)
unless dont_write
bin_data = [0].pack('c')*(size)
fd.write(bin_data)
# OK now I really have a file of size 1024 with a binay 0 contents
end

###.... then sometime later using the same file descriptor (fd) :
requestes_stream_position = 900
fd.pos=(requestes_stream_position)
p "tell=#{fd.tell}"
# Now please read 30 binary 0s into a binary string
bin_value=fd.read(30)

#### I ASSUMED THAT bin_value now is a "binary string of size 30", but
unfortunately it is nil !!!



Thanks in adcance,
C.





 
Reply With Quote
 
 
 
 
Robert Klemme
Guest
Posts: n/a
 
      03-21-2005

"Chris Guenther" <(E-Mail Removed)> schrieb im Newsbeitrag
news:d1mhu3$bi6$(E-Mail Removed)...
> Hi,
>
> I am having trouble reading values from an existing (or maybe not yet
> exisiting) binary file:
> My target is to open a binary file for reading and writing (if it does

not
> exisit it should be created).
> If the file was not yes existing I'd like to fill it with binary data to

a
> certain predefined size.
> Later on I'd like to position my file position, and read and write

whatever
> I like.
>
> The following piece does not work as intended:
>
> require 'pp'
> UNDER_UNIX = CONFIG['target_os'].to_s.index('linux')!=nil
> filename='foo.bin'
> size=1024
> dont_write=FileTest.exists?(filename) &&

File.stat(filename).size==size
> begin
> fd=File.open(filename,"ab+")
> fd.binmode unless UNDER_UNIX # assuming !UNIX=WINDOWS
> rescue => e
> pp e
> fd=File.open(filename,"wb+")
> puts "opened #{filename} with 'wib+'"
> end
>
> fd.sync=true
> fd.seek(0)
> unless dont_write
> bin_data = [0].pack('c')*(size)
> fd.write(bin_data)
> # OK now I really have a file of size 1024 with a binay 0 contents
> end
>
> ###.... then sometime later using the same file descriptor (fd) :
> requestes_stream_position = 900
> fd.pos=(requestes_stream_position)
> p "tell=#{fd.tell}"
> # Now please read 30 binary 0s into a binary string
> bin_value=fd.read(30)
>
> #### I ASSUMED THAT bin_value now is a "binary string of size 30",

but
> unfortunately it is nil !!!
>
>
>
> Thanks in adcance,
> C.


I don't know what you're doing wrong. Your script works fine for me.
Btw, you don't close the fd.

Here's how I'd do it:

filename='foo.bin'
size=1024

File.open(filename, "ab+") do |fd|
fd.seek(0, IO::SEEK_END)

if fd.tell < size
fd.seek 0
fd.write( "\000" * size );
elsif fd.tell > size
# if you need the file to be exact this size
fd.truncate size
end

requestes_stream_position = 900
fd.seek requestes_stream_position

bin_value = fd.read(30)
p bin_value
end

Notes:

- don't change binmode depending on platform, always use "b" for binary
files.

- use File.open with block for proper close / cleanup

Kind regards

robert



 
Reply With Quote
 
 
 
 
Chris GŁnther
Guest
Posts: n/a
 
      03-22-2005
Thanks Robert,

you are absolutely right with your remarks using a block. The code I
posted was just a small copied fraction of the real code, in which I
cannor use a block because the same fd will be stored in a hash for
later usage by another method.
Anyway, meanwhile I found the cause of this problem (again you are
right, it was not in the code I posted) , but now I have another related
problem that tricks my mind. I am trying to patch binary data in a file.
Unfortunately the data I am patching is not the data I get when reading
it back from the patched binary file.
Any ideas??



Thanks again in advance,
Chris





#!/usr/bin/env ruby

FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')

File.open(FNAME,"ab+") {|fd|
fd.pos=0
bin_data = [0].pack('c')*FSIZE
fd.write(bin_data)
}

File.truncate(FNAME,FSIZE)

File.open(FNAME,"ab+") {|fd|
fd.pos=0
data=fd.read(FSIZE)
data[OFFSET,VALUE.length]=VALUE
fd.pos=0
fd.write(data)
p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
}

File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}
 
Reply With Quote
 
Chris GŁnther
Guest
Posts: n/a
 
      03-22-2005
Chris Guenther wrote:
> Hi,
>
> I am having trouble reading values from an existing (or maybe not yet
> exisiting) binary file:
> My target is to open a binary file for reading and writing (if it does not
> exisit it should be created).
> If the file was not yes existing I'd like to fill it with binary data to a
> certain predefined size.
> Later on I'd like to position my file position, and read and write whatever
> I like.
>
> The following piece does not work as intended:
>
> require 'pp'
> UNDER_UNIX = CONFIG['target_os'].to_s.index('linux')!=nil
> filename='foo.bin'
> size=1024
> dont_write=FileTest.exists?(filename) && File.stat(filename).size==size
> begin
> fd=File.open(filename,"ab+")
> fd.binmode unless UNDER_UNIX # assuming !UNIX=WINDOWS
> rescue => e
> pp e
> fd=File.open(filename,"wb+")
> puts "opened #{filename} with 'wib+'"
> end
>
> fd.sync=true
> fd.seek(0)
> unless dont_write
> bin_data = [0].pack('c')*(size)
> fd.write(bin_data)
> # OK now I really have a file of size 1024 with a binay 0 contents
> end
>
> ###.... then sometime later using the same file descriptor (fd) :
> requestes_stream_position = 900
> fd.pos=(requestes_stream_position)
> p "tell=#{fd.tell}"
> # Now please read 30 binary 0s into a binary string
> bin_value=fd.read(30)
>
> #### I ASSUMED THAT bin_value now is a "binary string of size 30", but
> unfortunately it is nil !!!
>
>
>
> Thanks in adcance,
> C.
>
>
>
>
>

Thanks Robert,

you are absolutely right with your remarks using a block. The code I
posted was just a small copied fraction of the real code, in which I
cannor use a block because the same fd will be stored in a hash for
later usage by another method.
Anyway, meanwhile I found the cause of this problem (again you are
right, it was not in the code I posted) , but now I have another related
problem that tricks my mind. I am trying to patch binary data in a file.
Unfortunately the data I am patching is not the data I get when reading
it back from the patched binary file.
Any ideas??



Thanks again in advance,
Chris





#!/usr/bin/env ruby

FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')

File.open(FNAME,"ab+") {|fd|
fd.pos=0
bin_data = [0].pack('c')*FSIZE
fd.write(bin_data)
}

File.truncate(FNAME,FSIZE)

File.open(FNAME,"ab+") {|fd|
fd.pos=0
data=fd.read(FSIZE)
data[OFFSET,VALUE.length]=VALUE
fd.pos=0
fd.write(data)
p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
}

File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}
 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      03-22-2005

"Chris GŁnther" <(E-Mail Removed)> schrieb im Newsbeitrag
news:(E-Mail Removed)...
> Thanks Robert,
>
> you are absolutely right with your remarks using a block. The code I
> posted was just a small copied fraction of the real code, in which I
> cannor use a block because the same fd will be stored in a hash for
> later usage by another method.
> Anyway, meanwhile I found the cause of this problem (again you are
> right, it was not in the code I posted) , but now I have another related
> problem that tricks my mind. I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading
> it back from the patched binary file.
> Any ideas??
>
>
>
> Thanks again in advance,
> Chris
>
>
>
>
>
> #!/usr/bin/env ruby
>
> FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
> [?c,?a,?f,?e].pack('cccc')
>
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> bin_data = [0].pack('c')*FSIZE
> fd.write(bin_data)
> }
>
> File.truncate(FNAME,FSIZE)
>
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> data=fd.read(FSIZE)
> data[OFFSET,VALUE.length]=VALUE
> fd.pos=0
> fd.write(data)
> p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
> }


You truncate too often:
> File.truncate(FNAME,FSIZE)


> File.open(FNAME,"ab+") {|fd|
> fd.pos=OFFSET
> patched=fd.read(VALUE.length)
> p "read modified piece of data = '#{patched}'"
> puts "Oooops !!!" unless VALUE==patched
> }


Careful when you're copying and pasting.

Regards

robert

 
Reply With Quote
 
Chris GŁnther
Guest
Posts: n/a
 
      03-22-2005
Robert Klemme wrote:
> "Chris GŁnther" <(E-Mail Removed)> schrieb im Newsbeitrag
> news:(E-Mail Removed)...
>
>>Thanks Robert,
>>
>>you are absolutely right with your remarks using a block. The code I
>>posted was just a small copied fraction of the real code, in which I
>>cannor use a block because the same fd will be stored in a hash for
>>later usage by another method.
>>Anyway, meanwhile I found the cause of this problem (again you are
>>right, it was not in the code I posted) , but now I have another related
>>problem that tricks my mind. I am trying to patch binary data in a file.
>>Unfortunately the data I am patching is not the data I get when reading
>>it back from the patched binary file.
>>Any ideas??
>>
>>
>>
>>Thanks again in advance,
>>Chris
>>
>>
>>
>>
>>
>>#!/usr/bin/env ruby
>>
>>FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
>>[?c,?a,?f,?e].pack('cccc')
>>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=0
>> bin_data = [0].pack('c')*FSIZE
>> fd.write(bin_data)
>>}
>>
>>File.truncate(FNAME,FSIZE)
>>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=0
>> data=fd.read(FSIZE)
>> data[OFFSET,VALUE.length]=VALUE
>> fd.pos=0
>> fd.write(data)
>> p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
>>}

>
>
> You truncate too often:
>
>>File.truncate(FNAME,FSIZE)

>
>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=OFFSET
>> patched=fd.read(VALUE.length)
>> p "read modified piece of data = '#{patched}'"
>> puts "Oooops !!!" unless VALUE==patched
>>}

>
>
> Careful when you're copying and pasting.
>
> Regards
>
> robert
>


I dont see your point. I have to truncate after every write to ensure
the file size does not change. Where is the connection to the problem
that the read value does not match the written value ?

regards,
C.


 
Reply With Quote
 
Chris GŁnther
Guest
Posts: n/a
 
      03-22-2005
Robert Klemme wrote:
> "Chris GŁnther" <(E-Mail Removed)> schrieb im Newsbeitrag
> news:(E-Mail Removed)...
>
>>Thanks Robert,
>>
>>you are absolutely right with your remarks using a block. The code I
>>posted was just a small copied fraction of the real code, in which I
>>cannor use a block because the same fd will be stored in a hash for
>>later usage by another method.
>>Anyway, meanwhile I found the cause of this problem (again you are
>>right, it was not in the code I posted) , but now I have another related
>>problem that tricks my mind. I am trying to patch binary data in a file.
>>Unfortunately the data I am patching is not the data I get when reading
>>it back from the patched binary file.
>>Any ideas??
>>
>>
>>
>>Thanks again in advance,
>>Chris
>>
>>
>>
>>
>>
>>#!/usr/bin/env ruby
>>
>>FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
>>[?c,?a,?f,?e].pack('cccc')
>>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=0
>> bin_data = [0].pack('c')*FSIZE
>> fd.write(bin_data)
>>}
>>
>>File.truncate(FNAME,FSIZE)
>>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=0
>> data=fd.read(FSIZE)
>> data[OFFSET,VALUE.length]=VALUE
>> fd.pos=0
>> fd.write(data)
>> p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
>>}

>
>
> You truncate too often:
>
>>File.truncate(FNAME,FSIZE)

>
>
>>File.open(FNAME,"ab+") {|fd|
>> fd.pos=OFFSET
>> patched=fd.read(VALUE.length)
>> p "read modified piece of data = '#{patched}'"
>> puts "Oooops !!!" unless VALUE==patched
>>}

>
>
> Careful when you're copying and pasting.
>
> Regards
>
> robert
>


I dont see your point. I have to truncate after every write to ensure
the file size does not change. Where is the connection to the problem
that the read value does not match the written value ?

regards,
C.
 
Reply With Quote
 
Pit Capitain
Guest
Posts: n/a
 
      03-22-2005
Chris GŁnther schrieb:
> ... I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading
> it back from the patched binary file.
> Any ideas??
> ...
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> ...


Hi Chris,

it seems that the mode "ab+" is your problem. According to [1], "a+"
means open the file for reading and writing, starting at the end of the
file. This also means (at least on windows), that the end of file is
position 0. So what you are doing above is writing the patched data
after the end of the original file.

For patching a binary file I'd change your code to:

FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')

# "wb+" for initial data: truncate existing content
File.open(FNAME,"wb+") {|fd|
bin_data = [0].pack('c')*FSIZE
fd.write(bin_data)
}

# "rb+" for the patch: open at the beginning
File.open(FNAME,"rb+") {|fd|
fd.pos=OFFSET
fd.write(VALUE)
p "written modified piece of data = '#{VALUE}'"
}

# "rb" for reading the data
File.open(FNAME,"rb") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}

Regards,
Pit

[1] http://www.ruby-doc.org/core/classes/IO.html


 
Reply With Quote
 
Carlos
Guest
Posts: n/a
 
      03-22-2005
[Chris GŁnther <(E-Mail Removed)>, 2005-03-22 12.29 CET]
> Thanks Robert,
>
> you are absolutely right with your remarks using a block. The code I
> posted was just a small copied fraction of the real code, in which I
> cannor use a block because the same fd will be stored in a hash for
> later usage by another method.
> Anyway, meanwhile I found the cause of this problem (again you are
> right, it was not in the code I posted) , but now I have another related
> problem that tricks my mind. I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading
> it back from the patched binary file.
> Any ideas??


All your problems come from the fact that your are opening the file for
appending ("ab+") instead of for reading ("r+b"). With "ab+", you can set fd.pos=0,
if you want, but it will always write at the end of file.



 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      03-23-2005

"Chris GŁnther" <(E-Mail Removed)> schrieb im Newsbeitrag
news:(E-Mail Removed)...

<snip/>

> I dont see your point. I have to truncate after every write to ensure
> the file size does not change. Where is the connection to the problem
> that the read value does not match the written value ?


I don't see the point either. :-} Sorry for the confusion, I overlooked
that you used the other form of truncate with a length.

Regards

robert

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
writing binary file (ios::binary) Ron Eggler C++ 9 04-28-2008 08:20 AM
Reading and Writing to Binary Files Daniel Moree C++ 9 11-24-2004 10:20 PM
need help reading/writing binary Steve Bennett Perl 1 08-25-2004 06:08 PM
Reading/Writing pure binary files Daniel Gowans VHDL 2 06-12-2004 01:25 AM
URGENT: Reading from socket and writing into binary... HOW? Louis Java 6 10-15-2003 07:30 PM



Advertisments