Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > stripping the first byte from a binary file

Reply
Thread Tools

stripping the first byte from a binary file

 
 
rvr
Guest
Posts: n/a
 
      07-10-2007
Would someone mind showing me how to strip the first byte from a
binary file? For some reason I can't figure this out from the binary
file editing examples I've read. Thanks.

~rvr

 
Reply With Quote
 
 
 
 
Jeremy Sanders
Guest
Posts: n/a
 
      07-10-2007
rvr wrote:

> Would someone mind showing me how to strip the first byte from a
> binary file? For some reason I can't figure this out from the binary
> file editing examples I've read. Thanks.


Do you mean something like this?

f = open('test.dat', 'rb')
f.read(1) # read 1st byte and ignore it
rest = f.read() # read rest

or

data = f.read()
data = data[1:] # skip 1st byte

?

--
Jeremy Sanders
http://www.jeremysanders.net/
 
Reply With Quote
 
 
 
 
rvr
Guest
Posts: n/a
 
      07-11-2007
On Jul 10, 6:37 pm, Jeremy Sanders <jeremy
(E-Mail Removed)> wrote:
> rvr wrote:
> > Would someone mind showing me how to strip the first byte from a
> > binary file? For some reason I can't figure this out from the binary
> > file editing examples I've read. Thanks.

>
> Do you mean something like this?
>
> f = open('test.dat', 'rb')
> f.read(1) # read 1st byte and ignore it
> rest = f.read() # read rest
>
> or
>
> data = f.read()
> data = data[1:] # skip 1st byte


Is there a way to edit the file in place? The best I seem to be able
to do is to use your second solution to read the file into the string,
then re-open the file for writing and put the whole thing back (minus
the first byte). Thanks.

~rvr

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      07-11-2007
On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:

> Is there a way to edit the file in place? The best I seem to be able to
> do is to use your second solution to read the file into the string, then
> re-open the file for writing and put the whole thing back (minus the
> first byte). Thanks.


I don't believe that any of the popular operating systems in common use
(Windows, Linux, Mac, *BSD) have any such functionality.

For safety, you are best off copying the file (minus the first byte) to a
temporary file, then renaming the copy over the original. That way if
your process dies midway through copying the file, you don't lose data.

Renaming the file is atomic under Linux and (probably) Mac, so it is as
safe as possible. Even under Windows, which isn't atomic, it has a
smaller margin for disaster than over-writing the file in place.


--
Steven.
 
Reply With Quote
 
rvr
Guest
Posts: n/a
 
      07-11-2007
On Jul 11, 1:28 pm, Steven D'Aprano
<(E-Mail Removed)> wrote:
> On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
> > Is there a way to edit the file in place? The best I seem to be able to
> > do is to use your second solution to read the file into the string, then
> > re-open the file for writing and put the whole thing back (minus the
> > first byte). Thanks.

>
> I don't believe that any of the popular operating systems in common use
> (Windows, Linux, Mac, *BSD) have any such functionality.
>
> For safety, you are best off copying the file (minus the first byte) to a
> temporary file, then renaming the copy over the original. That way if
> your process dies midway through copying the file, you don't lose data.
>
> Renaming the file is atomic under Linux and (probably) Mac, so it is as
> safe as possible. Even under Windows, which isn't atomic, it has a
> smaller margin for disaster than over-writing the file in place.


Thanks for your response. While searching for solution, I found this:

http://mail.python.org/pipermail/pyt...er/116519.html

Quoting from it:

"""
Replace 2 bytes in place beginning at offset 100 (101st byte):

f = open('text_input', 'r+b')
f.seek(100)
f.write(chr(123) + chr(0x80))
f.seek(0,2)
f.close()
"""

Can I use the seek() and write() methods in a similar way to remove
the first byte? For whatever reason I can't seem to make it work
myself. Thanks again.

~rvr

 
Reply With Quote
 
Stefan Behnel
Guest
Posts: n/a
 
      07-11-2007
rvr wrote:
> On Jul 11, 1:28 pm, Steven D'Aprano
> <(E-Mail Removed)> wrote:
>> On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
>>> Is there a way to edit the file in place? The best I seem to be able to
>>> do is to use your second solution to read the file into the string, then
>>> re-open the file for writing and put the whole thing back (minus the
>>> first byte). Thanks.

>> I don't believe that any of the popular operating systems in common use
>> (Windows, Linux, Mac, *BSD) have any such functionality.
>>
>> For safety, you are best off copying the file (minus the first byte) to a
>> temporary file, then renaming the copy over the original. That way if
>> your process dies midway through copying the file, you don't lose data.
>>
>> Renaming the file is atomic under Linux and (probably) Mac, so it is as
>> safe as possible. Even under Windows, which isn't atomic, it has a
>> smaller margin for disaster than over-writing the file in place.

>
> Thanks for your response. While searching for solution, I found this:
>
> http://mail.python.org/pipermail/pyt...er/116519.html
>
> Quoting from it:
>
> """
> Replace 2 bytes in place beginning at offset 100 (101st byte):
>
> f = open('text_input', 'r+b')
> f.seek(100)
> f.write(chr(123) + chr(0x80))
> f.seek(0,2)
> f.close()
> """
>
> Can I use the seek() and write() methods in a similar way to remove
> the first byte? For whatever reason I can't seem to make it work
> myself. Thanks again.


Funny. I just happened to read ESR's "how to ask questions the smart way" and
your posts match quite a few of the examples.

No, you can't. Steven's solution is what I'd go for.

Stefan
 
Reply With Quote
 
Alex Popescu
Guest
Posts: n/a
 
      07-11-2007
On Jul 11, 1:25 pm, Stefan Behnel <(E-Mail Removed)> wrote:
> rvr wrote:
> > On Jul 11, 1:28 pm, Steven D'Aprano
> > <(E-Mail Removed)> wrote:
> >> On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
> >>> Is there a way to edit the file in place? The best I seem to be able to
> >>> do is to use your second solution to read the file into the string, then
> >>> re-open the file for writing and put the whole thing back (minus the
> >>> first byte). Thanks.
> >> I don't believe that any of the popular operating systems in common use
> >> (Windows, Linux, Mac, *BSD) have any such functionality.

>
> >> For safety, you are best off copying the file (minus the first byte) to a
> >> temporary file, then renaming the copy over the original. That way if
> >> your process dies midway through copying the file, you don't lose data.

>
> >> Renaming the file is atomic under Linux and (probably) Mac, so it is as
> >> safe as possible. Even under Windows, which isn't atomic, it has a
> >> smaller margin for disaster than over-writing the file in place.

>
> > Thanks for your response. While searching for solution, I found this:

>
> > http://mail.python.org/pipermail/pyt...er/116519.html

>
> > Quoting from it:

>
> > """
> > Replace 2 bytes in place beginning at offset 100 (101st byte):

>
> > f = open('text_input', 'r+b')
> > f.seek(100)
> > f.write(chr(123) + chr(0x80))
> > f.seek(0,2)
> > f.close()
> > """

>
> > Can I use the seek() and write() methods in a similar way to remove
> > the first byte? For whatever reason I can't seem to make it work
> > myself. Thanks again.

>
> Funny. I just happened to read ESR's "how to ask questions the smart way" and
> your posts match quite a few of the examples.
>
> No, you can't. Steven's solution is what I'd go for.
>
> Stefan


Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.

TIA,

../alex
--
..w( the_mindstorm )p.



 
Reply With Quote
 
Diez B. Roggisch
Guest
Posts: n/a
 
      07-11-2007
>
> Forgive my newbie ignorance, but I am wondering why the other method
> would not work? I mean it may not be very safe,
> but I guess it may perform a lot better, than having to read the whole
> file just to cut out the first byte.


Because seeking is not moving? Shifting data bytewise isn't something that
is supported by the underlying OS filesystems, and thus not supported. But
replacing bytes with others is. Which seek is for.

Diez
 
Reply With Quote
 
Stefan Behnel
Guest
Posts: n/a
 
      07-11-2007
Alex Popescu wrote:
> Forgive my newbie ignorance, but I am wondering why the other method
> would not work? I mean it may not be very safe,
> but I guess it may perform a lot better, than having to read the whole
> file just to cut out the first byte.


Why would you expect that? It *might* perform better if there was a system
call for removing bytes from inside a file, as that could reduce the
intermediate space requirements to the size of a hard disk sector rather than
the remaining size of the file (note that the time consumption would not be
reduced significantly, if the remaining file has to be copied around to fill
the sectors). But since that is a rather rare use case that most people would
prefer being handled in a safe rather than space-optimal way, I don't see the
need for such a function.

Stefan
 
Reply With Quote
 
Alex Popescu
Guest
Posts: n/a
 
      07-11-2007
On Jul 11, 4:15 pm, "Diez B. Roggisch" <(E-Mail Removed)> wrote:
> > Forgive my newbie ignorance, but I am wondering why the other method
> > would not work? I mean it may not be very safe,
> > but I guess it may perform a lot better, than having to read the whole
> > file just to cut out the first byte.

>
> Because seeking is not moving? Shifting data bytewise isn't something that
> is supported by the underlying OS filesystems, and thus not supported. But
> replacing bytes with others is. Which seek is for.
>
> Diez


As far as I know seek is just about positioning and nothing else.
So, in fact the problem boils down to os support for deleting a bytes.

bests,

../alex
--
..w( the_mindstorm )p.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
read text file byte by byte daved170 Python 30 12-16-2009 09:54 PM
convert form byte[4] to Int32 while retaining the binary value of the byte array jeff@foundrymusic.com C++ 20 09-07-2009 08:54 PM
Stripping specific bytes of a file without opening the entire file... balgach@gmail.com C++ 7 02-06-2005 04:11 PM
Search for byte pattern in a binary file. Ryan Tan via JavaKB.com Java 20 11-19-2004 08:41 AM
byte B= 0x0F;// Hex. representaion --> How byte B=0001111 // Binary rep.? C++ 2 07-26-2004 08:08 PM



Advertisments