Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Ruby (http://www.velocityreviews.com/forums/f66-ruby.html)
-   -   get first and last line from txt file - how? (http://www.velocityreviews.com/forums/t854865-get-first-and-last-line-from-txt-file-how.html)

Mmcolli00 Mom 12-20-2008 02:16 PM

get first and last line from txt file - how?
 
I have txt file with date/time stamps only. I want to grab the first
date/time and the last date/time. For instance, I will be needing
08/09/08 3:00 and 08/24/08 3:00 from the below queued.txt. Do you know
how I can pull these out? Thanks in advance.

queued.txt
8/09/08 3:00
8/10/08 5:00
8/23/08 22:00
8/24/08 3:00

firstDate = ""
lastDate = ""

File.open('queued.txt', 'r') do |f1|
while line = f1.gets
if f1.lineno == 1 then #<-this would only give me 8/09/08 3:00
@@fistDate = f1
end
end
--
Posted via http://www.ruby-forum.com/.


Tim Hunter 12-20-2008 02:30 PM

Re: get first and last line from txt file - how?
 
Mmcolli00 Mom wrote:
> I have txt file with date/time stamps only. I want to grab the first
> date/time and the last date/time. For instance, I will be needing
> 08/09/08 3:00 and 08/24/08 3:00 from the below queued.txt. Do you know
> how I can pull these out? Thanks in advance.
>
> queued.txt
> 8/09/08 3:00
> 8/10/08 5:00
> 8/23/08 22:00
> 8/24/08 3:00
>
> firstDate = ""
> lastDate = ""
>
> File.open('queued.txt', 'r') do |f1|
> while line = f1.gets
> if f1.lineno == 1 then #<-this would only give me 8/09/08 3:00
> @@fistDate = f1
> end
> end


lines = IO.readlines("queued.txt")
first = lines.first
last = lines.last

puts first
puts last


--
RMagick: http://rmagick.rubyforge.org/


Yaser Sulaiman 12-20-2008 03:35 PM

Re: get first and last line from txt file - how?
 
[Note: parts of this message were removed to make it a legal post.]

I'm just wondering..
Let's say that we only need to read the last line. Can we do that without
reading the other lines?

Regards,
Yaser Sulaiman


Ch Ba 12-20-2008 03:54 PM

Re: get first and last line from txt file - how?
 
Yaser Sulaiman wrote:
> I'm just wondering..
> Let's say that we only need to read the last line. Can we do that
> without
> reading the other lines?
>
> Regards,
> Yaser Sulaiman



It would work the same? Or do you mean without loading up the entire
file?

lines = IO.readlines("foo.bar")

puts lines.last
--
Posted via http://www.ruby-forum.com/.


Yaser Sulaiman 12-20-2008 04:02 PM

Re: get first and last line from txt file - how?
 
[Note: parts of this message were removed to make it a legal post.]

On Sat, Dec 20, 2008 at 6:54 PM, Ch Ba <navouri@gmail.com> wrote:
>
> It would work the same? Or do you mean without loading up the entire
> file?


Yep, that is exactly what I mean.


Tim Hunter 12-20-2008 04:14 PM

Re: get first and last line from txt file - how?
 
Yaser Sulaiman wrote:
> On Sat, Dec 20, 2008 at 6:54 PM, Ch Ba <navouri@gmail.com> wrote:
>> It would work the same? Or do you mean without loading up the entire
>> file?

>
> Yep, that is exactly what I mean.
>


If you know where the last line starts (that is, the byte offset of the
first character in the last line) then you could use IO#seek to seek to
that offset and then read.

How do you know where the last line starts? When you write the file,
call IO#tell to get the current byte offset before you write the last line.

--
RMagick: http://rmagick.rubyforge.org/


Thomas Preymesser 12-20-2008 04:46 PM

Re: get first and last line from txt file - how?
 
2008/12/20 Yaser Sulaiman <yaserbuntu@gmail.com>:
> I'm just wondering..
> Let's say that we only need to read the last line. Can we do that without
> reading the other lines?


Yes. Position your file pointer to the last byte in a file, read and
collect backwards each byte until you find a newline character (or the
first byte of the file). This is the last line.

-Thomas




--
Thomas Preymesser
thopre@gmail.com
http://thopre.googlepages.com/
http://thopre.wordpress.com/


Robert Klemme 12-21-2008 02:51 PM

Re: get first and last line from txt file - how?
 
On 20.12.2008 17:46, Thomas Preymesser wrote:
> 2008/12/20 Yaser Sulaiman <yaserbuntu@gmail.com>:
>> I'm just wondering..
>> Let's say that we only need to read the last line. Can we do that without
>> reading the other lines?

>
> Yes. Position your file pointer to the last byte in a file, read and
> collect backwards each byte until you find a newline character (or the
> first byte of the file). This is the last line.


You have to admit that this approach is rather inefficient. Here's a
more efficient variant - especially for large files:

$ cat r.rb
#!/bin/env ruby

OFFSET = 512 # > 2 * assumed avg line length

file = ARGV.shift or abort "ERROR: need a file name"

File.open file do |io|
first = io.gets
break unless first
puts first

limit = io.stat.size
offset = OFFSET
lines = []

while lines.size < 2 && offset <= limit
io.seek -offset, IO::SEEK_END
lines = io.readlines
offset += OFFSET
end # while lines.size < 2

puts lines.last unless lines.empty?
end

Cheers

robert

Thomas Preymesser 12-21-2008 06:16 PM

Re: get first and last line from txt file - how?
 
2008/12/21 Robert Klemme <shortcutter@googlemail.com>:
> On 20.12.2008 17:46, Thomas Preymesser wrote:
>>
>> 2008/12/20 Yaser Sulaiman <yaserbuntu@gmail.com>:
>>>
>>> I'm just wondering..
>>> Let's say that we only need to read the last line. Can we do that without
>>> reading the other lines?

>>
>> Yes. Position your file pointer to the last byte in a file, read and
>> collect backwards each byte until you find a newline character (or the
>> first byte of the file). This is the last line.

>
> You have to admit that this approach is rather inefficient.


Really?

I did a comparision of your code and my idea:

$ time ruby r.rb input
111111111111111111111111111111
999999999999999999999999999999

real 0m0.053s
user 0m0.000s
sys 0m0.004s

$ time ruby t.rb input
999999999999999999999999999999

real 0m0.043s
user 0m0.004s
sys 0m0.000s

the first result is your code, the second is mine.

I did the tests with a test file with almost 8,000,000 lines.

My q&d code:

f=File.open("input")
pos = 2
f.seek(-pos, File::SEEK_END)
c = f.getc
result = ''
while c.chr != "\n"
result.insert(0,c.chr)
pos += 1
f.seek(-pos, File::SEEK_END)
c = f.getc
end
f.close

puts result

-Thomas

--
Thomas Preymesser
thopre@gmail.com
http://thopre.googlepages.com/
http://thopre.wordpress.com/


Simon Krahnke 12-21-2008 08:59 PM

Re: get first and last line from txt file - how?
 
* Yaser Sulaiman <yaserbuntu@gmail.com> (2008-12-20) schrieb:

> I'm just wondering..
> Let's say that we only need to read the last line. Can we do that without
> reading the other lines?


Yes, of course. It's exactly the same problem as reading the first line.
The only difference is that there is a standard function for the first
line: gets.

For the last line you need to implement it yourself.

If I had mmap in Ruby, I'd just map the file into memory and do
mapped_file[/^.*\z/].

mfg, simon .... l


All times are GMT. The time now is 07:39 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.