Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > piping out binaries properly

Reply
Thread Tools

piping out binaries properly

 
 
Andy Leszczynski
Guest
Posts: n/a
 
      10-12-2005
I have got following program:

import sys
import binascii
from string import *
sys.stdout.write(binascii.unhexlify("41410A4141"))


when I run under Unix I got:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0a 41 41

and under Windows/Cygwin following:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0d 0a 41 41
0000006


The question is how can I pipe out binary content properly and platform
independently?


A.
 
Reply With Quote
 
 
 
 
Mike Meyer
Guest
Posts: n/a
 
      10-12-2005
Andy Leszczynski <leszczynscyATnospam.yahoo.com.nospam> writes:
> I have got following program:
>
> import sys
> import binascii
> from string import *
> sys.stdout.write(binascii.unhexlify("41410A4141"))
>
>
> when I run under Unix I got:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0a 41 41
>
> and under Windows/Cygwin following:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0d 0a 41 41
> 0000006
>
> The question is how can I pipe out binary content properly and platform
> independently?


It's not normal to write binary content to stdout - you normally write
it to a file. Open the file with open(name, 'wb') to write binaries.

There doesn't appear to be any way to retroactively change the mode on
a file. Which is probably a good thing.

<mike
--
Mike Meyer <(E-Mail Removed)> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
 
Reply With Quote
 
 
 
 
Andy Leszczynski
Guest
Posts: n/a
 
      10-12-2005
Mike Meyer wrote:
> It's not normal to write binary content to stdout - you normally write


Well, I grew up in the Unix world and it is normal over there.

I am still curious which layer adds that 0xd. Is it python, cygwin,
windows ...

Thx for reply, Andy
 
Reply With Quote
 
marduk
Guest
Posts: n/a
 
      10-12-2005
On Wed, 2005-10-12 at 00:16 -0400, Mike Meyer wrote:
[...]
> It's not normal to write binary content to stdout - you normally write
> it to a file. Open the file with open(name, 'wb') to write binaries.
>


It is interesting that as a "Unix consultant" you should make that
claim. Especially since

>>> sys.stdout

<open file '<stdout>', mode 'w' at 0x2aaaaaac9198>
^^^^

Indeed there would be a lot of broken Unix systems out there if that
were not the case.

As for the OP, you may want to check how stdout is opened on your
system. In Windows there are two "write" modes for files, 'w', and 'wb'
where apparently 'w' is text mode and 'wb' is binary. You may want to
check what mode your stdout is in. I don't have a Windows box handy
right now to verify.

> There doesn't appear to be any way to retroactively change the mode on
> a file. Which is probably a good thing.
>
> <mike--
> Mike Meyer <(E-Mail Removed)>

http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more

information.


 
Reply With Quote
 
Mike Meyer
Guest
Posts: n/a
 
      10-12-2005
Andy Leszczynski <leszczynscyATnospam.yahoo.com.nospam> writes:

> Mike Meyer wrote:
>> It's not normal to write binary content to stdout - you normally write

>
> Well, I grew up in the Unix world and it is normal over there.


I watched the Unix world grow up, and it ain't normal to me. I don't
think I've ever written a program that wrote binary data to standard
out, not in nearly 30 years of unix programming. I've written lots of
things whose standard out was designed specifically to be read by
another program, but never as binary data.

> I am still curious which layer adds that 0xd. Is it python, cygwin,
> windows ...


It's Python. You tell Python whether or not it needs to do
platform-dependent newline mangling when you open the file.

<mike
--
Mike Meyer <(E-Mail Removed)> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
 
Reply With Quote
 
Erik Max Francis
Guest
Posts: n/a
 
      10-12-2005
Mike Meyer wrote:

> I watched the Unix world grow up, and it ain't normal to me.


Since there's no distinction between a file opened in binary mode and in
text mode on Unix, there is no difference.

> I don't
> think I've ever written a program that wrote binary data to standard
> out, not in nearly 30 years of unix programming. I've written lots of
> things whose standard out was designed specifically to be read by
> another program, but never as binary data.


Plenty of applications use that functionality and depend on it. See
cjpeg, djpeg, the pbmplus library, and so forth.

--
Erik Max Francis && http://www.velocityreviews.com/forums/(E-Mail Removed) && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Every human being is a problem in search of a solution.
-- Ashley Montagu
 
Reply With Quote
 
Mike Meyer
Guest
Posts: n/a
 
      10-12-2005
marduk <(E-Mail Removed)> writes:

> On Wed, 2005-10-12 at 00:16 -0400, Mike Meyer wrote:
> [...]
>> It's not normal to write binary content to stdout - you normally write
>> it to a file. Open the file with open(name, 'wb') to write binaries.
>>

>
> It is interesting that as a "Unix consultant" you should make that
> claim. Especially since
>
>>>> sys.stdout

> <open file '<stdout>', mode 'w' at 0x2aaaaaac9198>
> ^^^^
>
> Indeed there would be a lot of broken Unix systems out there if that
> were not the case.


Ain't ambiguity wonderful? That we use the same word to donote a
collection of bytes on a disk and a software entity that one can write
bytes to allows for all kind of mistaken interpretations.

<mike
--
Mike Meyer <(E-Mail Removed)> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      10-12-2005
Erik Max Francis <(E-Mail Removed)> writes:
> > I've written lots of things whose standard out was designed
> > specifically to be read by another program, but never as binary data.

>
> Plenty of applications use that functionality and depend on it. See
> cjpeg, djpeg, the pbmplus library, and so forth.


Also "tar cf files... | compress > tarball.Z" used to be a standard
idiom, now replaced by "tar cfz ..." which does about the same thing under
the covers.
 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      10-12-2005
Andy Leszczynski wrote:

> when I run under Unix I got:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0a 41 41
>
> and under Windows/Cygwin following:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0d 0a 41 41
> 0000006
>
> The question is how can I pipe out binary content properly and platform
> independently?


$ python -h
usage: python [option] ... [-c cmd | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
....
-u : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
....

</F>



 
Reply With Quote
 
Paul Watson
Guest
Posts: n/a
 
      10-12-2005
"Andy Leszczynski" <leszczynscyATnospam.yahoo.com.nospam> wrote in message
news(E-Mail Removed)...
>I have got following program:
>
> import sys
> import binascii
> from string import *
> sys.stdout.write(binascii.unhexlify("41410A4141"))
>
>
> when I run under Unix I got:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0a 41 41
>
> and under Windows/Cygwin following:
>
> $ python u.py > u.bin
> $ od -t x1 u.bin
> 0000000 41 41 0d 0a 41 41
> 0000006
>
>
> The question is how can I pipe out binary content properly and platform
> independently?
>
>
> A.


try:
import msvcrt, os
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
except:
pass


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Writing Binaries to and Reading Binaries from Disk KevinSimonson Java 17 09-28-2009 05:40 PM
piping input when shelling out Caio Chassot Ruby 12 09-21-2006 12:17 AM
piping data Jeff Kish Java 1 10-15-2004 12:29 PM
Piping ping into perl-prog bernd wegener Perl 3 09-22-2004 06:32 PM
More American Graffiti: Properly Framed, Properly Scored? Scot Gardner DVD Video 0 09-02-2003 02:28 AM



Advertisments