Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > FasterCSV, Gruff and Large Data Import

Reply
Thread Tools

FasterCSV, Gruff and Large Data Import

 
 
Dan Webb [dbw]
Guest
Posts: n/a
 
      03-10-2009
Hi all,

=20

I've got a CSV file which contains 18k items that I need to import to
graph. I'm using FasterCSV and Gruff to import and graph the data.=20

=20

Obviously I'm going about this wrong as it's overloading the Array
class. Is there another way around this?

http://pastie.org/412741

=20

=20

Kind Regards,

Dan

=20

=20


 
Reply With Quote
 
 
 
 
James Gray
Guest
Posts: n/a
 
      03-10-2009
On Mar 10, 2009, at 9:03 AM, Dan Webb [dbw] wrote:

> I've got a CSV file which contains 18k items that I need to import to
> graph. I'm using FasterCSV and Gruff to import and graph the data.
>
>
>
> Obviously I'm going about this wrong as it's overloading the Array
> class. Is there another way around this?
>
> http://pastie.org/412741


Well, I think you probably need to trim the data down a bit. Could
you maybe build up some averages over time periods and feed just those
averages to Gruff?

James Edward Gray II

 
Reply With Quote
 
 
 
 
Rob Biedenharn
Guest
Posts: n/a
 
      03-10-2009
On Mar 10, 2009, at 10:36 AM, James Gray wrote:

> On Mar 10, 2009, at 9:03 AM, Dan Webb [dbw] wrote:
>
>> I've got a CSV file which contains 18k items that I need to import to
>> graph. I'm using FasterCSV and Gruff to import and graph the data.
>>
>>
>>
>> Obviously I'm going about this wrong as it's overloading the Array
>> class. Is there another way around this?
>>
>> http://pastie.org/412741

>
> Well, I think you probably need to trim the data down a bit. Could =20=


> you maybe build up some averages over time periods and feed just =20
> those averages to Gruff?
>
> James Edward Gray II


What kind of graph? How big is the graph? Think about what kind of =20
resolution is meaningful =96 the default is only 800 pixels wide and you =
=20
can't graph sub-pixel values

-Rob

Rob Biedenharn http://agileconsultingllc.com
http://www.velocityreviews.com/forums/(E-Mail Removed)




 
Reply With Quote
 
Dan Webb [dbw]
Guest
Posts: n/a
 
      03-10-2009
Not really, well I could but I'm trying to compare it with doing it in a
similar program such as GNU Plot, plus there are sometimes
irregularities in the data which I'd like to show if possible.=20

On a second look the error I'm getting is a gruff related one.=20

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
`larger_than
_max?': undefined method `>' for ["1\t8877"]:Array (NoMethodError)
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:453:in
`data'
from
c:/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:15
70:in `each_with_index'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`each'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`each_with_index'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`data'
from main.rb:20

line 20's where the array gets pushed into the graph.=20


Dan


-----Original Message-----
From: James Gray [(E-Mail Removed)]=20
Sent: 10 March 2009 14:37
To: ruby-talk ML
Subject: Re: FasterCSV, Gruff and Large Data Import

On Mar 10, 2009, at 9:03 AM, Dan Webb [dbw] wrote:

> I've got a CSV file which contains 18k items that I need to import to
> graph. I'm using FasterCSV and Gruff to import and graph the data.
>
>
>
> Obviously I'm going about this wrong as it's overloading the Array
> class. Is there another way around this?
>
> http://pastie.org/412741


Well, I think you probably need to trim the data down a bit. Could =20
you maybe build up some averages over time periods and feed just those =20
averages to Gruff?

James Edward Gray II


 
Reply With Quote
 
James Coglan
Guest
Posts: n/a
 
      03-10-2009
[Note: parts of this message were removed to make it a legal post.]

> On a second look the error I'm getting is a gruff related one.
>
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
> `larger_than
> _max?': undefined method `>' for ["1\t8877"]:Array (NoMethodError)
> from
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:453:in
> `data'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:15
> 70:in `each_with_index'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
> `each'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
> `each_with_index'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
> `data'
> from main.rb:20
>
> line 20's where the array gets pushed into the graph.




Can you show us your code? Looks like your array of data may itself contain
arrays -- have you tried flattening the array before giving it to Gruff?

 
Reply With Quote
 
James Gray
Guest
Posts: n/a
 
      03-10-2009
On Mar 10, 2009, at 9:57 AM, Dan Webb [dbw] wrote:

> On a second look the error I'm getting is a gruff related one.
>
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
> `larger_than
> _max?': undefined method `>' for ["1\t8877"]:Array (NoMethodError)
> from


Gruff expects an Array of values to graph. It looks like you might be
feeding it an Array of Arrays and even those values look fishy.

I recommend simplifying the problem. First, try just reading the
first five lines of your data file and graphing just that.

When you have that working, you can expand the data and see how things
go.

James Edward Gray II


 
Reply With Quote
 
Dan Webb [dbw]
Guest
Posts: n/a
 
      03-11-2009
Hi again,

As suggested I've changed tact. It now looks like gruff doesn't like the
array created by FasterCSV, but I don't know why!

If I create an array manually then feed it into gruff, i.e.=20

a =3D [1,2,3,4,5]
g.data("X", a)

it works absolutely fine. However if I've put the same data into a csv
file, then read it in using either:=20

a =3D fastercsv.read("r.csv")

or=20

FasterCSV.foreach("r.csv") do |row|=20
a << row
end

If I puts all three they all print out as
1
2
3
4
5

And both errors given by the array when I use fasterCSV are:

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
`larger_than
_max?': undefined method `>' for ["1", "2", "3", "4", "5"]:Array
(NoMethodError)

from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:453:in
`data'
from
c:/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:15
70:in `each_with_index'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`each'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`each_with_index'
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`data'
from main.rb:27

am I missing something spectacular?
Thanks,
Dan

-----Original Message-----
From: James Gray [(E-Mail Removed)]=20
Sent: 10 March 2009 15:06
To: ruby-talk ML
Subject: Re: FasterCSV, Gruff and Large Data Import

On Mar 10, 2009, at 9:57 AM, Dan Webb [dbw] wrote:

> On a second look the error I'm getting is a gruff related one.
>
> c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
> `larger_than
> _max?': undefined method `>' for ["1\t8877"]:Array (NoMethodError)
> from


Gruff expects an Array of values to graph. It looks like you might be =20
feeding it an Array of Arrays and even those values look fishy.

I recommend simplifying the problem. First, try just reading the =20
first five lines of your data file and graphing just that.

When you have that working, you can expand the data and see how things =20
go.

James Edward Gray II



 
Reply With Quote
 
James Coglan
Guest
Posts: n/a
 
      03-11-2009
[Note: parts of this message were removed to make it a legal post.]

> a = [1,2,3,4,5]
> g.data("X", a)
>
> it works absolutely fine. However if I've put the same data into a csv
> file, then read it in using either:
>
> a = fastercsv.read("r.csv")
>
> or
>
> FasterCSV.foreach("r.csv") do |row|
> a << row
> end




Looks like you're feeding the whole CSV table into Gruff all at once. Try
something like this:

idx = 1
FasterCSV.foreach("r.csv") do |row|
g.data("Series #{idx}", row)
idx += 1
end

--
James Coglan
http://github.com/jcoglan

 
Reply With Quote
 
Dan Webb [dbw]
Guest
Posts: n/a
 
      03-11-2009
However if I feed the fastercsv data into an array then into gruff it
_should_ work, as feeding a whole array that I've written out by hand
into gruff works. If you see my logic here, it should work, no?

Dan


-----Original Message-----
From: James Coglan [(E-Mail Removed)]=20
Sent: 11 March 2009 09:32
To: ruby-talk ML
Subject: Re: FasterCSV, Gruff and Large Data Import

> a =3D [1,2,3,4,5]
> g.data("X", a)
>
> it works absolutely fine. However if I've put the same data into a csv
> file, then read it in using either:
>
> a =3D fastercsv.read("r.csv")
>
> or
>
> FasterCSV.foreach("r.csv") do |row|
> a << row
> end




Looks like you're feeding the whole CSV table into Gruff all at once.
Try
something like this:

idx =3D 1
FasterCSV.foreach("r.csv") do |row|
g.data("Series #{idx}", row)
idx +=3D 1
end

--
James Coglan
http://github.com/jcoglan

 
Reply With Quote
 
James Coglan
Guest
Posts: n/a
 
      03-11-2009
[Note: parts of this message were removed to make it a legal post.]

2009/3/11 Dan Webb [dbw] <(E-Mail Removed)>

> However if I feed the fastercsv data into an array then into gruff it
> _should_ work, as feeding a whole array that I've written out by hand
> into gruff works. If you see my logic here, it should work, no?



No, because reading a CSV produces a table, i.e. an array of arrays,
something like:

FasterCSV.read("r.csv") #=> [[1,2,3,4], [5,6,7,8], ...]

Gruff expects a flat array of data for each data series, so you need to loop
over each row in the table and feed them to Gruff one by one.

--
James Coglan
http://github.com/jcoglan

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Gruff & Uneven Data: Interpolation or Line Connection Nick Gohn Ruby 0 12-12-2009 04:48 PM
Gruff and labeling Trick Nick Ruby 1 11-25-2008 09:58 PM
Gruff and SVG Damaris Fuentes Ruby 0 02-09-2008 11:28 PM
mrplot and Gruff japanese Horacio Sanson Ruby 0 01-21-2008 02:47 AM
NameError - Scruffy and Gruff - Ams Lone Ruby 3 08-12-2007 11:48 AM



Advertisments