Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > How can I Read/Write multiple sequential Binary/Text data files

Reply
Thread Tools

How can I Read/Write multiple sequential Binary/Text data files

 
 
Albert Tu
Guest
Posts: n/a
 
      03-10-2005
Dear there,

We have an x-ray CT system. The acquisition computer acquires x-ray
projections and outputs multiple data files in binary format (2-byte
unsigned integer) such as projection0.raw, projection1.raw,
projection2.raw ... up to projection500.raw. Each file is
2*1024*768-byte big.

I would like to read those files and convert to ascii files in %5.0f/n
format as projection0.data ... projection500.data so that our
visualization software can undersatnd the projection images. I was
trying to do this conversion using Python. However, I had troubles
declaring the file names using the do-loop index. Anyone had previous
experience?

Thanks,
Albert

 
Reply With Quote
 
 
 
 
Christos TZOTZIOY Georgiou
Guest
Posts: n/a
 
      03-10-2005
On 10 Mar 2005 09:41:05 -0800, rumours say that "Albert Tu"
<(E-Mail Removed)> might have written:

>Dear there,
>
>We have an x-ray CT system. The acquisition computer acquires x-ray
>projections and outputs multiple data files in binary format (2-byte
>unsigned integer) such as projection0.raw, projection1.raw,
>projection2.raw ... up to projection500.raw. Each file is
>2*1024*768-byte big.
>
>I would like to read those files and convert to ascii files in %5.0f/n
>format as projection0.data ... projection500.data so that our
>visualization software can undersatnd the projection images. I was
>trying to do this conversion using Python. However, I had troubles
>declaring the file names using the do-loop index. Anyone had previous
>experience?


Regular expressions could help, but if you *know* that these are the filenames,
you can (untested code):

PREFIX= "projection"
SUFFIX_I= ".raw"
SUFFIX_O= ".data"

import glob, struct

for filename in glob.glob("%s*%s" % (PREFIX, SUFFIX_I)):
number= filename[len(PREFIX):-len(SUFFIX_I)]
fpi= open(filename, "rb")
fpo= open("%s%s%s" % (PREFIX, number, SUFFIX_O), "w")
while 1:
datum= fpi.read(2)
if not datum: break
fpo.write("%5d\n" % struct.unpack("H", datum)) # check endianness!!!
fpi.close()
fpo.close()
--
TZOTZIOY, I speak England very best.
"Be strict when sending and tolerant when receiving." (from RFC195
I really should keep that in mind when talking with people, actually...
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      03-10-2005
On Thu, 10 Mar 2005 20:06:29 +0200, Christos "TZOTZIOY" Georgiou
<(E-Mail Removed)> wrote:

>On 10 Mar 2005 09:41:05 -0800, rumours say that "Albert Tu"
><(E-Mail Removed)> might have written:
>
>>Dear there,
>>
>>We have an x-ray CT system. The acquisition computer acquires x-ray
>>projections and outputs multiple data files in binary format (2-byte
>>unsigned integer) such as projection0.raw, projection1.raw,
>>projection2.raw ... up to projection500.raw. Each file is
>>2*1024*768-byte big.
>>
>>I would like to read those files and convert to ascii files in %5.0f/n
>>format as projection0.data ... projection500.data so that our
>>visualization software can undersatnd the projection images. I was
>>trying to do this conversion using Python. However, I had troubles
>>declaring the file names using the do-loop index. Anyone had previous
>>experience?

>
>Regular expressions could help, but if you *know* that these are the filenames,
>you can (untested code):
>
>PREFIX= "projection"
>SUFFIX_I= ".raw"
>SUFFIX_O= ".data"
>
> import glob, struct


import array

DIFFERENT_ENDIAN = True/False

>
>for filename in glob.glob("%s*%s" % (PREFIX, SUFFIX_I)):
> number= filename[len(PREFIX):-len(SUFFIX_I)]
> fpi= open(filename, "rb")
> fpo= open("%s%s%s" % (PREFIX, number, SUFFIX_O), "w")
> while 1:
> datum= fpi.read(2)
> if not datum: break
> fpo.write("%5d\n" % struct.unpack("H", datum)) # check endianness!!!


If the OP knows that each input file is small enough (1.5Mb each as
stated), then the agony of file.read(2) can be avoided by reading the
whole file in one hit. The agony of struct.unpack() on each datum can
be avoided by using the array module. E.g. replace the whole 'while'
loop by this:

ary = array.array('H', fpi.read())
if DIFFERENT_ENDIAN:
ary.byteswap()
for datum in ary:
fpo.write("%5d\n" % datum)

Even if the input files were too large to fit in memory, they could
still be processed fast enough by reading a big chunk at a time.


> fpi.close()
> fpo.close()


 
Reply With Quote
 
Bengt Richter
Guest
Posts: n/a
 
      03-10-2005
On 10 Mar 2005 09:41:05 -0800, "Albert Tu" <(E-Mail Removed)> wrote:

>Dear there,
>
>We have an x-ray CT system. The acquisition computer acquires x-ray
>projections and outputs multiple data files in binary format (2-byte
>unsigned integer) such as projection0.raw, projection1.raw,
>projection2.raw ... up to projection500.raw. Each file is
>2*1024*768-byte big.
>
>I would like to read those files and convert to ascii files in %5.0f/n
>format as projection0.data ... projection500.data so that our
>visualization software can undersatnd the projection images. I was

Is there no chance of fixing the visualization software instead? The format seems
easy and efficient, and it seems a shame to make a redundant bloated copy
of the same info. What next? XML tags surrounding your ascii floats?

What platform are you on? What is the visualization software's method of
accessing data? Only files as you describe? Are you visualizing interactively,
or setting up batch processing?

>trying to do this conversion using Python. However, I had troubles
>declaring the file names using the do-loop index. Anyone had previous
>experience?
>

Are you the same person who posted re this format some time ago?

Regards,
Bengt Richter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
Organizing Sequential Data (TimeStamps) Overthinking? xkenneth Python 6 10-19-2007 10:40 PM
Text files read multiple files into single file, and then recreate the multiple files googlinggoogler@hotmail.com Python 4 02-13-2005 05:44 PM
Selecting sequential files in Explorer pickle Computer Support 4 09-04-2004 09:05 PM
Creating AVI or MOV files from sequential TGAs? Hobbit Computer Support 0 01-05-2004 04:51 PM



Advertisments