Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > How to change stdin separator (perl vs ruby)

Reply
Thread Tools

How to change stdin separator (perl vs ruby)

 
 
Sak ..
Guest
Posts: n/a
 
      02-10-2010
I have to process very very big text files and not line per line, I have
to access to a group of lines seperated for a "%". Then I tryed to
access line per line and detect the separator for each line saving the
lines in a buffer, obviously it works but it's very slow (text files are
bigger than 1GB).

Perl have an awesome feature that is to change the separator for
iterating files (by default in all languages is "\n"), this simple and
extremely efficent code do what I need:

$/ = "%";
while ($article = <STDIN>) {
#Iteration
}


I can write a equivalent code in ruby but not (by far) as efficient as
perl. Someone knows how can I change the separator for iterate the stdin
in ruby?

Thanks!
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Stefan Lang
Guest
Posts: n/a
 
      02-10-2010
2010/2/10 Sak .. <(E-Mail Removed)>:
> I have to process very very big text files and not line per line, I have
> to access to a group of lines seperated for a "%". Then I tryed to
> access line per line and detect the separator for each line saving the
> lines in a buffer, obviously it works but it's very slow (text files are
> bigger than 1GB).
>
> Perl have an awesome feature that is to change the separator for
> iterating files (by default in all languages is "\n"), this simple and
> extremely efficent code do what I need:
>
> $/ =3D "%";
> while ($article =3D <STDIN>) {
> =A0#Iteration
> }
>
>
> I can write a equivalent code in ruby but not (by far) as efficient as
> perl. Someone knows how can I change the separator for iterate the stdin
> in ruby?


STDIN.each_line("%") { |line|
# iteration
}

Stefan

 
Reply With Quote
 
 
 
 
Xavier Noria
Guest
Posts: n/a
 
      02-10-2010
On Wed, Feb 10, 2010 at 4:23 PM, Sak .. <(E-Mail Removed)> wrote:

> I can write a equivalent code in ruby but not (by far) as efficient as
> perl. Someone knows how can I change the separator for iterate the stdin
> in ruby?


Ruby has also $/ among its global variables. The methods #gets and
#readline take it into account. Those methods and #each also accept a
separator as an optional argument.

In Ruby 1.9 they additionally accept an integer, that's the same as
setting $/ to a reference to an integer in Perl.

 
Reply With Quote
 
Daniel Bye
Guest
Posts: n/a
 
      02-10-2010
On Thu, Feb 11, 2010 at 12:23:55AM +0900, Sak .. wrote:
> I have to process very very big text files and not line per line, I have
> to access to a group of lines seperated for a "%". Then I tryed to
> access line per line and detect the separator for each line saving the
> lines in a buffer, obviously it works but it's very slow (text files are
> bigger than 1GB).
>
> Perl have an awesome feature that is to change the separator for
> iterating files (by default in all languages is "\n"), this simple and
> extremely efficent code do what I need:
>
> $/ = "%";
> while ($article = <STDIN>) {
> #Iteration
> }
>


If your delimiter only occurs as a delimiter, and not in the data as well:

File.open(path_to_file, 'r') do |f|
f.each('%') do |record|
# Do stuff.
end
end

>
> I can write a equivalent code in ruby but not (by far) as efficient as
> perl. Someone knows how can I change the separator for iterate the stdin
> in ruby?
>
> Thanks!
> --
> Posted via http://www.ruby-forum.com/.
>
>
> !DSPAM:4b72d13c186061354869328!
>
>


--
Daniel Bye
_
ASCII ribbon campaign ( )
- against HTML, vCards and X
- proprietary attachments in e-mail / \

 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      02-10-2010
2010/2/10 Daniel Bye <(E-Mail Removed)>:
> On Thu, Feb 11, 2010 at 12:23:55AM +0900, Sak .. wrote:
>> I have to process very very big text files and not line per line, I have
>> to access to a group of lines seperated for a "%". Then I tryed to
>> access line per line and detect the separator for each line saving the
>> lines in a buffer, obviously it works but it's very slow (text files are
>> bigger than 1GB).
>>
>> Perl have an awesome feature that is to change the separator for
>> iterating files (by default in all languages is "\n"), this simple and
>> extremely efficent code do what I need:
>>
>> $/ =3D "%";
>> while ($article =3D <STDIN>) {
>> =A0 #Iteration
>> }
>>

>
> If your delimiter only occurs as a delimiter, and not in the data as well=

:

Note that this statement is true for \n as well!

> File.open(path_to_file, 'r') do |f|
> =A0f.each('%') do |record|
> =A0 =A0# Do stuff.
> =A0end
> end


You can have it simpler:

File.foreach path, '%' do |record|
# so something with record
end

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 
Reply With Quote
 
Sak ..
Guest
Posts: n/a
 
      02-10-2010
Stefan Lang wrote:
> 2010/2/10 Sak .. <(E-Mail Removed)>:
>> $/ = "%";
>> while ($article = <STDIN>) {
>> �#Iteration
>> }
>>
>>
>> I can write a equivalent code in ruby but not (by far) as efficient as
>> perl. Someone knows how can I change the separator for iterate the stdin
>> in ruby?

>
> STDIN.each_line("%") { |line|
> # iteration
> }
>
> Stefan


Thanks Stefan for the answer, that's exactly what I need, thanks Xavier
explanation is perfect. Thanks Daniel, good observation.


--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
peek at stdin, flush stdin Johnathan Doe C Programming 5 05-17-2013 04:30 PM
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
How to pass stdin of a C++ program to the stdin of a process createdwith ShellExecute() Ben C Programming 2 08-29-2009 09:47 PM
STDIN, OUT, ERR and $stdin, out, err - Differences? Terry Cooper Ruby 7 06-09-2009 05:48 AM
Reading stdin once confuses second stdin read Charlie Zender C Programming 6 06-21-2004 01:39 PM



Advertisments