Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > How to extract some lines from a large report and then reverse every character?

Reply
Thread Tools

How to extract some lines from a large report and then reverse every character?

 
 
hzzc1012@gmail.com
Guest
Posts: n/a
 
      01-12-2006
Hello, everyone:
I'm using quite big data file as large as 1 G bytes, and it contain
many lines, the longest of which will be about 1M byte.
And now I need to read the last 100 lines, for example, from the file
and reverse every character in the line adn then print it in another
file ( for example, 12345000 in the original file will be 00054321 in
the new file.)

Now I read the data file into memory, find the end , and then output
the character reversely to the output file, but I do find it very
memory and time consuming

I am a new comer to PERL, so I am not sure if there would be any other
more efficient solution. And I guess there would be.
Thank you in advance!



Jacobus

 
Reply With Quote
 
 
 
 
Gunnar Hjalmarsson
Guest
Posts: n/a
 
      01-12-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> I'm using quite big data file as large as 1 G bytes, and it contain
> many lines, the longest of which will be about 1M byte.
> And now I need to read the last 100 lines, for example, from the file
> and reverse every character in the line adn then print it in another
> file ( for example, 12345000 in the original file will be 00054321 in
> the new file.)
>
> Now I read the data file into memory, find the end , and then output
> the character reversely to the output file, but I do find it very
> memory and time consuming
>
> I am a new comer to PERL, so I am not sure if there would be any other
> more efficient solution. And I guess there would be.


Check out the File::ReadBackwards module.

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
 
Reply With Quote
 
 
 
 
Gunnar Hjalmarsson
Guest
Posts: n/a
 
      01-12-2006
Gunnar Hjalmarsson wrote:
> (E-Mail Removed) wrote:
>> And now I need to read the last 100 lines, for example, from the file
>> and reverse every character in the line

>
> Check out the File::ReadBackwards module.


In addition to that:

perldoc -q reverse
perldoc -f reverse

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
 
Reply With Quote
 
Paul Lalli
Guest
Posts: n/a
 
      01-12-2006
(E-Mail Removed) wrote:
> I'm using quite big data file as large as 1 G bytes, and it contain
> many lines, the longest of which will be about 1M byte.
> And now I need to read the last 100 lines, for example, from the file


perldoc Tie::File
perldoc -q "line in a file"

> and reverse every character in the line


perldoc -f reverse

> adn then print it in another
> file ( for example, 12345000 in the original file will be 00054321 in
> the new file.)


perldoc -f open
perldoc -f print

Paul Lalli

 
Reply With Quote
 
Xicheng
Guest
Posts: n/a
 
      01-12-2006
I guess the key part of the OP's problem is how to track the last 100
lines which may contain upto 100MB spaces. File::ReadBackwards module
looks good, which read the huge file backward and line by line. then I
can reverse the contents of each line and "print" it into a new file,
the new file now has 100 lines and in completely reversed order(lines,
words, and characters).

So what if I dont want to reverse lines, I just want to reverse the
conents of each line. I have to do File::ReadBackwards again(no reverse
contents though)??... This looks weird...

Considering the huge size each line may contain, is there any better
solutions besides the File::ReadBackwards modules??

Many thanks
Xicheng

Gunnar Hjalmarsson wrote:
> Gunnar Hjalmarsson wrote:
> > (E-Mail Removed) wrote:
> >> And now I need to read the last 100 lines, for example, from the file
> >> and reverse every character in the line

> >
> > Check out the File::ReadBackwards module.

>
> In addition to that:
>
> perldoc -q reverse
> perldoc -f reverse
>
> --
> Gunnar Hjalmarsson
> Email: http://www.gunnar.cc/cgi-bin/contact.pl


 
Reply With Quote
 
J. Gleixner
Guest
Posts: n/a
 
      01-12-2006
Xicheng wrote:
> I guess the key part of the OP's problem is how to track the last 100
> lines which may contain upto 100MB spaces. File::ReadBackwards module
> looks good, which read the huge file backward and line by line. then I
> can reverse the contents of each line and "print" it into a new file,
> the new file now has 100 lines and in completely reversed order(lines,
> words, and characters).
>
> So what if I dont want to reverse lines, I just want to reverse the
> conents of each line. I have to do File::ReadBackwards again(no reverse
> contents though)??... This looks weird...
>
> Considering the huge size each line may contain, is there any better
> solutions besides the File::ReadBackwards modules??


Please, stop top-posting....


Non-perl answer, probably the fastest option:

rev < tail -100 file > file.reverse

For a more perlish answer, something similar to: (untested)

my $file = 'blah';
open( my $new_fh, '>', $file.new ) or die "Can't open $file.new: $!";
open( my $tail, "/usr/bin/tail -100 $file |" ) or die "can't open tail: $!";
while ( <$tail> ) { print $new_fh reverse; }
close $tail;
close $new_fh;
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      01-12-2006
"Xicheng" <(E-Mail Removed)> wrote:
> I guess the key part of the OP's problem is how to track the last 100
> lines which may contain upto 100MB spaces. File::ReadBackwards module
> looks good, which read the huge file backward and line by line.


For values of huge being less than 2 Gig (on a 32 bit machine). Otherwise,
you have to hack the module by taking out the "use integer;" line. I've
done this, and it seems to work for truly huge files, but I can't gaurantee
I didn't introduce some subtle bug in the process.

> then I
> can reverse the contents of each line and "print" it into a new file,
> the new file now has 100 lines and in completely reversed order(lines,
> words, and characters).
>
> So what if I dont want to reverse lines, I just want to reverse the
> conents of each line. I have to do File::ReadBackwards again(no reverse
> contents though)??... This looks weird...


Uh, wouldn't it be the other way around? You read normally (not with
File::ReadBackwards), and reverse the contents of each line?

> Considering the huge size each line may contain,


He specified 1 MB. Are you calling that huge, or are you thinking of
something bigger?

> is there any better
> solutions besides the File::ReadBackwards modules??


File::ReadBackwards goes to some slight effort to reverse line order
without reversing the contents of the lines. If you want to reverse both,
then you could surely come up with something using seek and reverse that
would be more efficient at doing that exact thing. But considering that
you would have to go through the effort of actually coding it, maybe you
wouldn't consider that option "better".

Better is so context dependent, it makes little sense to discuss it in the
abstract.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
robic0
Guest
Posts: n/a
 
      01-16-2006
On 12 Jan 2006 07:15:28 -0800, (E-Mail Removed) wrote:

>Hello, everyone:
>I'm using quite big data file as large as 1 G bytes, and it contain
>many lines, the longest of which will be about 1M byte.
>And now I need to read the last 100 lines, for example, from the file
>and reverse every character in the line adn then print it in another
>file ( for example, 12345000 in the original file will be 00054321 in
>the new file.)
>
>Now I read the data file into memory, find the end , and then output
>the character reversely to the output file, but I do find it very
>memory and time consuming
>
>I am a new comer to PERL, so I am not sure if there would be any other
>more efficient solution. And I guess there would be.
>Thank you in advance!
>
>
>
> Jacobus

huh?

olleH, enoyreve:
I'm gnisu etiuq gib atad elif sa egral sa 1 G setyb, dna ti niatnoc
ynam senil, eht tsegnol fo hcihw lliw eb tuoba M1 etyb.
dnA won I deen ot daer eht tsal 001 senil, rof elpmaxe, morf eht elif
dna esrever yreve retcarahc ni eht enil nda neht tnirp ti ni rehtona
elif ( rof elpmaxe, 00054321 ni eht lanigiro elif lliw eb 12345000 ni
eht wen elif.)

woN I daer eht atad elif otni yromem, dnif eht dne , dna neht tuptuo
eht retcarahc ylesrever ot eht tuptuo elif, tub I od dnif ti yrev
yromem dna emit gnimusnoc

I ma a wen remoc ot LREP, os I ma ton erus fi ereht dluow eb yna rehto
erom tneiciffe noitulos. dnA I sseug ereht dluow eb.
knahT uoy ni ecnavda!



subocaJ

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
Explorer.exe error report every time access every type of video me =?Utf-8?B?SXJ3YW5zeWFo?= ASP .Net 4 10-30-2007 07:49 AM
Help. SessionID is x then y then x then y BodiKlamph@gmail.com ASP General 0 09-03-2005 03:02 PM
Reading total lines of file then displaying in reverse? Tristan Knowles Ruby 8 07-19-2005 08:48 AM
Stacks Queues Reverse Reverse Polish dogbite C++ 4 10-10-2003 05:06 AM



Advertisments