Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Bug in 1.9.0 csv parser

Reply
Thread Tools

Bug in 1.9.0 csv parser

 
 
Nicko Kaltner
Guest
Posts: n/a
 
      01-10-2008
Hi,
I've found a bug in the 1.9.0 csvparser. I've got a script and data
that effectively breaks it, but it is 567kb. Is that too large for this
list?

The ruby instance takes 100% of cpu while processing this file, and I
had to stop it after 5 minutes..

the code is

#!/usr/local/bin/ruby1.9.0

require 'csv'

filename = 'broken.csv'

CSV.foreach(filename) do |row|
STDERR.puts row.length
row.each do |entry|
puts entry
end
puts "\n####################################\n"
end


I would try and debug it further, but the debugger seems broken in 1.9.0.


Regards,
Nicko
 
Reply With Quote
 
 
 
 
James Gray
Guest
Posts: n/a
 
      01-10-2008
On Jan 10, 2008, at 2:15 AM, Nicko Kaltner wrote:

> I've found a bug in the 1.9.0 csvparser. I've got a script and data =20=


> that effectively breaks it, but it is 567kb. Is that too large for =20=


> this list?


Probably, but you are welcome to email it to me privately. I maintain =20=

the CSV library in Ruby 1.9.

> The ruby instance takes 100% of cpu while processing this file, and =20=


> I had to stop it after 5 minutes..


I'm 95% percent sure this is an issue of your CSV file being invalid. =20=

Because of the way the format works, a simple file like:

"=8510 Gigs of data without another quote=85

can only be detected invalid by reading the entire thing. I've =20
considered putting limits in the library for controlling how far it =20
would read ahead, but these would break some valid data. Then the =20
problem becomes, do I make the limits default to on? That's the only =20=

way they would have helped here, but that would break some otherwise =20
valid code. It's a tough problem to solve correctly.

James Edward Gray II=

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Best way to parse a csv...... a csv that has CRLF in the fields sso Java 20 04-26-2009 11:04 AM
read and write csv file using csv module jliu66 Python 0 10-19-2007 03:12 PM
How to move data from a CSV file to a JTable, and from a JTable to a CSV file ? Tintin92 Java 1 02-14-2007 06:51 PM
Re: csv writerow creates double spaced excel csv files Skip Montanaro Python 0 02-13-2004 08:50 PM
csv writerow creates double spaced excel csv files Michal Mikolajczyk Python 0 02-13-2004 08:38 PM



Advertisments