Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > How to parse huge file with timestamps

Reply
Thread Tools

How to parse huge file with timestamps

 
 
jobs239@hotmail.com
Guest
Posts: n/a
 
      11-30-2005
I need to parse a text file where each line is timestamped.
Each line has the form "timestamp, value1, value2"
I have to get max value of value1 between two timestamps and the the
avg value of value2 between another two timestamps. What would be an
efficient way to do this in terms of memory and processing time?
What comes to mind now is to read each line and store values in a
vector of objects. Then traverse that vector later to find the starting
timestamp and ending timestamp and process data in between. But it
doesn't sound like the most efficient way given that the files are huge.

 
Reply With Quote
 
 
 
 
Roedy Green
Guest
Posts: n/a
 
      11-30-2005
On 30 Nov 2005 14:04:10 -0800, http://www.velocityreviews.com/forums/(E-Mail Removed) wrote, quoted or
indirectly quoted someone who said :

> I have to get max value of value1 between two timestamps and the the
>avg value of value2 between another two timestamps. What would be an
>efficient way to do this in terms of memory and processing time?


you process the file sequentially, convert your timestamps to longs,
subtract and keep the longest value you find in variable called
longest.

if (diff > longest ) longest = diff;

--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
replacing timestamps in file [newbie] nukeymusic Python 2 12-14-2011 09:07 PM
Trying to parse a HUGE(1gb) xml file spaceman-spiff Python 41 01-14-2011 02:22 AM
Re: Trying to parse a HUGE(1gb) xml file spaceman-spiff Python 3 12-22-2010 10:28 PM
Memory error due to the huge/huge input file size tejsupra@gmail.com Python 3 11-20-2008 07:21 PM
Re: How to parse timestamps containing milliseconds skip@pobox.com Python 0 06-28-2006 07:05 PM



Advertisments