Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Reading last N lines from large file

Reply
Thread Tools

Reading last N lines from large file

 
 
Chris Nestrud
Guest
Posts: n/a
 
      11-25-2004
I need to read a number of lines from the end of a large file. For
example, the file may have 460484 lines and I need the last 500.

The "tail -500 file" command from the shell works, but I'd like a solution
that doesn't involve calling an external program.

Reading the file and keeping an array with the last 500 lines seen
works, but takes a very long time on large files.

I've looked at the seek function, but other than guessing at how far
from EOF to seek backward, I can't figure out how to make sure that I'm
left with enough data to account for the needed number of lines.

Any suggestions or other approaches would be much appreciated.
--
Chris Nestrud
Email: http://www.velocityreviews.com/forums/(E-Mail Removed)
http://www.panix.com/~ccn/
 
Reply With Quote
 
 
 
 
Peter Corlett
Guest
Posts: n/a
 
      11-25-2004
Chris Nestrud <(E-Mail Removed)> wrote:
[...]
> I've looked at the seek function, but other than guessing at how far
> from EOF to seek backward, I can't figure out how to make sure that
> I'm left with enough data to account for the needed number of lines.


I suspect tail merely guesses a seek position, then reads to find out
if it was right, retrying until it gets enough data.


--
PGP key ID E85DC776 - finger (E-Mail Removed) for full key
 
Reply With Quote
 
 
 
 
Tore Aursand
Guest
Posts: n/a
 
      11-25-2004
On Thu, 25 Nov 2004 14:51:19 +0000, Chris Nestrud wrote:
> I need to read a number of lines from the end of a large file. For
> example, the file may have 460484 lines and I need the last 500.


Take a look at the 'File::ReadBackwards' module;

<http://search.cpan.org/~uri/File-ReadBackwards-1.02/>


--
Tore Aursand <(E-Mail Removed)>
"A teacher is never a giver of truth - he is a guide, a pointer to the
truth that each student must find for himself. A good teacher is
merely a catalyst." (Bruce Lee)
 
Reply With Quote
 
Chris Nestrud
Guest
Posts: n/a
 
      11-25-2004
On Thu, 25 Nov 2004 17:01:49 +0100, Tore Aursand <(E-Mail Removed)> wrote:
> On Thu, 25 Nov 2004 14:51:19 +0000, Chris Nestrud wrote:
>> I need to read a number of lines from the end of a large file. For
>> example, the file may have 460484 lines and I need the last 500.

>
> Take a look at the 'File::ReadBackwards' module;
>
> <http://search.cpan.org/~uri/File-ReadBackwards-1.02/>


This looks like exactly what I need. Thanks.
--
Chris Nestrud
Email: (E-Mail Removed)
http://www.panix.com/~ccn/
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Reading LAST line from text file without iterating through the file? Robin Wenger Java 191 03-26-2011 06:19 PM
To delete few lines and add few lines at the end of a text file using c program Murali C++ 2 03-09-2006 04:45 PM
Reading file lines into large List iksrazal@terra.com.br Java 10 09-14-2005 06:36 PM
How to display the last n lines from a text file nic977 C Programming 14 09-13-2003 02:03 PM
Re: how to read 10 lines from a 200 lines file and write to a new file?? Joe Wright C Programming 0 07-27-2003 08:50 PM



Advertisments