«

Does Python have a function that is analogous to C's write() or
fwrite()-

that is , I want to write a file (of arbitrary data) that is 100K, or
1MB (or more) bytes long..

Both write() and fwrite() in C allow the user to specify the size of
the data to be written.

Python's write only allows a string to be passed in.

Sure, I could create a string that is n Megabytes long, and pass it to
write, but it seems as though there should be a better way ???
String manipulation is typically considered slow.

st="X" * 1048576
fh=open("junk","wb")
fh.write(st)
fh.close()



thanks

Tony C enlightened us with:
> Sure, I could create a string that is n Megabytes long, and pass it to
> write, but it seems as though there should be a better way ???
> String manipulation is typically considered slow.

Why not create a string that is 1 KB long and write that n*1024 times?

Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?

"Tony C" <> wrote in message
news: om...
> Does Python have a function that is analogous to C's write() or
> fwrite()-
>
> that is , I want to write a file (of arbitrary data) that is 100K, or
> 1MB (or more) bytes long..
>
> Both write() and fwrite() in C allow the user to specify the size of
> the data to be written.
>
> Python's write only allows a string to be passed in.
>
> Sure, I could create a string that is n Megabytes long, and pass it to
> write, but it seems as though there should be a better way ???
> String manipulation is typically considered slow.

The C coded string methods are not slow.
In Py2.3, 'x' * n calls str.__mul__ which is implemented
using the C library's memset() function.

It is faster still to use itertools.repeat:

st = itertools.repeat('X', 1048576)

Of course, your particular use case is I/O bound
(meaning that string construction isn't the
cause of your performance issues).

> st="X" * 1048576
> fh=open("junk","wb")
> fh.write(st)
> fh.close()


Raymond Hettinger

On 26 Aug 2003 15:04:12 -0700, (Tony C) wrote:

>Does Python have a function that is analogous to C's write() or
>fwrite()-
>
>that is , I want to write a file (of arbitrary data) that is 100K, or
>1MB (or more) bytes long..
>
>Both write() and fwrite() in C allow the user to specify the size of
>the data to be written.
>
>Python's write only allows a string to be passed in.
>
>Sure, I could create a string that is n Megabytes long, and pass it to
>write, but it seems as though there should be a better way ???
>String manipulation is typically considered slow.
>
>st="X" * 1048576
>fh=open("junk","wb")
>fh.write(st)
>fh.close()

Pythons file objects are automatically treated as streams. So you could do
something like this:

st = 'X'
somebignumber = 1048576
fh = open('junk','wb')
for n in xrange(somebignumber):
fh.write(st)
fh.close()

Daniel Klein

"Tony C" <> wrote in message
news: om...
> Does Python have a function that is analogous to C's write() or
> fwrite()-
>
> that is , I want to write a file (of arbitrary data) that is 100K, or
> 1MB (or more) bytes long..
>
> Both write() and fwrite() in C allow the user to specify the size of
> the data to be written.
>
> Python's write only allows a string to be passed in.
>
> Sure, I could create a string that is n Megabytes long, and pass it to
> write, but it seems as though there should be a better way ???
> String manipulation is typically considered slow.
>
> st="X" * 1048576
> fh=open("junk","wb")
> fh.write(st)
> fh.close()
>
>
>
> thanks

Another alternative is to seek to required position and write (at least) one
byte...

reqSize = 1048576
fh = open('junk', 'wb')
fh.seek(reqSize - 1)
fh.write('\0')
fh.close()

Mike.

"Michael Porter" <> wrote in message
news:3f4c8ebc$0$256$.. .
>
> "Tony C" <> wrote in message
> news: om...
> > Does Python have a function that is analogous to C's write() or
> > fwrite()-
> >
> > that is , I want to write a file (of arbitrary data) that is 100K,
or
> > 1MB (or more) bytes long..
> >
> > Both write() and fwrite() in C allow the user to specify the size of
> > the data to be written.

> Another alternative is to seek to required position and write (at
least) one
> byte...
>
> reqSize = 1048576
> fh = open('junk', 'wb')
> fh.seek(reqSize - 1)
> fh.write('\0')
> fh.close()
>
> Mike.

I interpreted his 'arbitrary data' to mean the same thing & came up with
the same solution (which should be the fastest way to do it in C as
well!). Anyway, doing some rough timing also seems to show that
creating the string was only taking about 10% of the time that writing
it is, and that the seek solution is about 10x faster that creating the
string & 100x faster that writing the string! It shows once again that
it pays to find out where the bottleneck is before trying to optimize
the wrong area!

--
Greg

"Dialtone" <dialtone#NOSPAM#.> schrieb im Newsbeitrag
news:...
> (Tony C) writes:
>
[...]

> > st="X" * 1048576
> > fh=open("junk","wb")
> > fh.write(st)
> > fh.close()
>
> For little files (less than 2 or 3 Mb I think) your code is the
> fastest I can think of. But growing There is a new version which is a
> lot faster

All this is valid in a very limited scope only.
- Consider limited RAM (lets say 128 MB) - you will be extremely slowed down
by thrashing.
- Consider on-the-fly compression (Windows NTFS): The 1000 MB testdate will
be reduced to 1 Byte!

Kindly
Michael P