Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > reading data files with headers and columns

Reply
Thread Tools

reading data files with headers and columns

 
 
Yaj Bhattacharya
Guest
Posts: n/a
 
      03-13-2009
Hello,

As a noob to Ruby, I need to read a data file with the first few lines
as header, then the rest of the file as columns of different variables
as arrays (in this case, 5 columns).
After reading in the variables, I want to calculate some operations
with each unique data field address (e.g. column3, row4), then write
out the file with headers and data in columns.

Could someone help with a few lines of basic code?

Thanks in advance
Yaj
 
Reply With Quote
 
 
 
 
Luc Traonmilin
Guest
Posts: n/a
 
      03-13-2009
Yaj Bhattacharya a écrit :
> Hello,
>
> As a noob to Ruby, I need to read a data file with the first few lines
> as header, then the rest of the file as columns of different variables
> as arrays (in this case, 5 columns).
> After reading in the variables, I want to calculate some operations
> with each unique data field address (e.g. column3, row4), then write
> out the file with headers and data in columns.
>
> Could someone help with a few lines of basic code?
>
> Thanks in advance
> Yaj
>
>
>

You could start with this basic piece of code (replace <tags> with your
code):
# here's the columns as arrays
@column1 = []
<rest_of_columns>
# open file in read mode (r)
file = File.new("<your_filename", "r")
# read each line one after another and process it
file.each_line do |line|
# process as header if necessary
# or
# extract your variables in the line into an array
array = line.split("<your_column_separator>")
# assign variable to column
@column1 << array[0]
# ... process other columns
end

Additionnally you can google "ruby array" and "ruby file", the API docs
are well documented.

Luc

 
Reply With Quote
 
 
 
 
Yaj Bhattacharya
Guest
Posts: n/a
 
      03-13-2009
Thanks very much Luc, here are a few follow up questions (complete
noob)

# here's the columns as arrays
@column1 = [] # should this be @column1=[variable1] or just the
empty square brackets?
<rest_of_columns>
# open file in read mode (r)
file = File.new("<your_filename", "r")
# read each line one after another and process it
file.each_line do |line| # where do I specify how the header is to
be skipped, how many lines to
#be skipped or otherwise processed?
# process as header if necessary
# or
# extract your variables in the line into an array
array = line.split("<your_column_separator>") # I am guessing that
a space column separator
#would be " ", does it work with arbit number of multiple spaces
between the columns i.e. if the
#column width is variable but between two
# columns there are one or several spaces?
# assign variable to column
@column1 << array[0]
# if @column1=[variable1] comment in the second line of the above code
was not correct, then
#where do the variables get their names?
# ... process other columns
end


On Mar 13, 1:23*pm, Luc Traonmilin <luc.traonmi...@gmail.com> wrote:
> Yaj Bhattacharya a écrit :
>
>
>
> > Hello,

>
> > As a noob to Ruby, I need to read a data file with the first few lines
> > as header, then the rest of the file as columns of different variables
> > as arrays (in this case, 5 columns).
> > After reading in the variables, I want to calculate some operations
> > with each unique data field address (e.g. column3, row4), then write
> > out the file with headers and data in columns.

>
> > Could someone help with a few lines of basic code?

>
> > Thanks in advance
> > Yaj

>
> You could start with this basic piece of code (replace <tags> with your
> code):
> * # here's the columns as arrays
> * @column1 = []
> * <rest_of_columns>
> * # open file in read mode (r)
> * file = File.new("<your_filename", "r")
> * # read each line one after another and process it
> * file.each_line do |line|
> * * # process as header if necessary
> * * # or
> * * # extract your variables in the line into an array
> * * array = line.split("<your_column_separator>")
> * * # assign variable to column
> * * @column1 << array[0]
> * * # ... process other columns
> * end
>
> Additionnally you can google "ruby array" and "ruby file", the API docs
> are well documented.
>
> Luc- Hide quoted text -
>
> - Show quoted text -


 
Reply With Quote
 
Luc Traonmilin
Guest
Posts: n/a
 
      03-13-2009
Well [] only initializes the array as empty, if you want to initialize it
with variables it is [var1, var2...]. Alternatively, you can use a Hash:
@column1 =3D {"var1" =3D> value, ...}
and then access the values with @column1["var1"] if that is what you want t=
o
do.

I don't know what your header looks like so it is difficult to say how to
skip it. Maybe it is possible for you to match the line against the pattern
of your columns if you just want to skip the header. Or if you have a fixed
number of lines, you can use a counter.



On Fri, Mar 13, 2009 at 7:02 PM, Yaj Bhattacharya <>wrote=
:

> Thanks very much Luc, here are a few follow up questions (complete
> noob)
>
> # here's the columns as arrays
> @column1 =3D [] # should this be @column1=3D[variable1] or just the
> empty square brackets?
> <rest_of_columns>
> # open file in read mode (r)
> file =3D File.new("<your_filename", "r")
> # read each line one after another and process it
> file.each_line do |line| # where do I specify how the header is to
> be skipped, how many lines to
> #be skipped or otherwise processed?
> # process as header if necessary
> # or
> # extract your variables in the line into an array
> array =3D line.split("<your_column_separator>") # I am guessing that
> a space column separator
> #would be " ", does it work with arbit number of multiple spaces
> between the columns i.e. if the
> #column width is variable but between two
> # columns there are one or several spaces?
> # assign variable to column
> @column1 << array[0]
> # if @column1=3D[variable1] comment in the second line of the above code
> was not correct, then
> #where do the variables get their names?
> # ... process other columns
> end
>
>
> On Mar 13, 1:23 pm, Luc Traonmilin <luc.traonmi...@gmail.com> wrote:
> > Yaj Bhattacharya a =E9crit :
> >
> >
> >
> > > Hello,

> >
> > > As a noob to Ruby, I need to read a data file with the first few line=

s
> > > as header, then the rest of the file as columns of different variable=

s
> > > as arrays (in this case, 5 columns).
> > > After reading in the variables, I want to calculate some operations
> > > with each unique data field address (e.g. column3, row4), then write
> > > out the file with headers and data in columns.

> >
> > > Could someone help with a few lines of basic code?

> >
> > > Thanks in advance
> > > Yaj

> >
> > You could start with this basic piece of code (replace <tags> with your
> > code):
> > # here's the columns as arrays
> > @column1 =3D []
> > <rest_of_columns>
> > # open file in read mode (r)
> > file =3D File.new("<your_filename", "r")
> > # read each line one after another and process it
> > file.each_line do |line|
> > # process as header if necessary
> > # or
> > # extract your variables in the line into an array
> > array =3D line.split("<your_column_separator>")
> > # assign variable to column
> > @column1 << array[0]
> > # ... process other columns
> > end
> >
> > Additionnally you can google "ruby array" and "ruby file", the API docs
> > are well documented.
> >
> > Luc- Hide quoted text -
> >
> > - Show quoted text -

>
>
>



--=20
Luc Traonmilin

 
Reply With Quote
 
Rob Biedenharn
Guest
Posts: n/a
 
      03-13-2009
If you know how many header lines, I'll assume 3, just read them first =20=

before you start the loop:


headers =3D []
rows =3D []
File.open("your_filename", 'r') do |file|
3.times { headers << file.gets }

file.each do |line|
rows << line.split(' ') # or ',' or /,\s*/ depending
# on how "columns" are formed
end
end # the block given to File.open will automatically close the file

# for the value in 3rd column, 4th row (counting from ZERO)

rows[3][2]

# do your calculations, then open the same or a new file for writing:

File.open("your_output", 'w') do |file|
file.puts headers
rows.each do |row|
file.puts row.join(' ') # or however you separate columns
end
end

-Rob

Rob Biedenharn http://agileconsultingllc.com



On Mar 13, 2009, at 2:30 PM, Luc Traonmilin wrote:

> Well [] only initializes the array as empty, if you want to =20
> initialize it
> with variables it is [var1, var2...]. Alternatively, you can use a =20
> Hash:
> @column1 =3D {"var1" =3D> value, ...}
> and then access the values with @column1["var1"] if that is what you =20=


> want to
> do.
>
> I don't know what your header looks like so it is difficult to say =20
> how to
> skip it. Maybe it is possible for you to match the line against the =20=


> pattern
> of your columns if you just want to skip the header. Or if you have =20=


> a fixed
> number of lines, you can use a counter.
>
>
>
> On Fri, Mar 13, 2009 at 7:02 PM, Yaj Bhattacharya =20
> <>wrote:
>
>> Thanks very much Luc, here are a few follow up questions (complete
>> noob)
>>
>> # here's the columns as arrays
>> @column1 =3D [] # should this be @column1=3D[variable1] or just the
>> empty square brackets?
>> <rest_of_columns>
>> # open file in read mode (r)
>> file =3D File.new("<your_filename", "r")
>> # read each line one after another and process it
>> file.each_line do |line| # where do I specify how the header is to
>> be skipped, how many lines to
>> #be skipped or otherwise processed?
>> # process as header if necessary
>> # or
>> # extract your variables in the line into an array
>> array =3D line.split("<your_column_separator>") # I am guessing =

that
>> a space column separator
>> #would be " ", does it work with arbit number of multiple spaces
>> between the columns i.e. if the
>> #column width is variable but between two
>> # columns there are one or several spaces?
>> # assign variable to column
>> @column1 << array[0]
>> # if @column1=3D[variable1] comment in the second line of the above =20=


>> code
>> was not correct, then
>> #where do the variables get their names?
>> # ... process other columns
>> end
>>
>>
>> On Mar 13, 1:23 pm, Luc Traonmilin <luc.traonmi...@gmail.com> wrote:
>>> Yaj Bhattacharya a =E9crit :
>>>
>>>
>>>
>>>> Hello,
>>>
>>>> As a noob to Ruby, I need to read a data file with the first few =20=


>>>> lines
>>>> as header, then the rest of the file as columns of different =20
>>>> variables
>>>> as arrays (in this case, 5 columns).
>>>> After reading in the variables, I want to calculate some operations
>>>> with each unique data field address (e.g. column3, row4), then =20
>>>> write
>>>> out the file with headers and data in columns.
>>>
>>>> Could someone help with a few lines of basic code?
>>>
>>>> Thanks in advance
>>>> Yaj
>>>
>>> You could start with this basic piece of code (replace <tags> with =20=


>>> your
>>> code):
>>> # here's the columns as arrays
>>> @column1 =3D []
>>> <rest_of_columns>
>>> # open file in read mode (r)
>>> file =3D File.new("<your_filename", "r")
>>> # read each line one after another and process it
>>> file.each_line do |line|
>>> # process as header if necessary
>>> # or
>>> # extract your variables in the line into an array
>>> array =3D line.split("<your_column_separator>")
>>> # assign variable to column
>>> @column1 << array[0]
>>> # ... process other columns
>>> end
>>>
>>> Additionnally you can google "ruby array" and "ruby file", the API =20=


>>> docs
>>> are well documented.
>>>
>>> Luc- Hide quoted text -
>>>
>>> - Show quoted text -

>>

> --=20
> Luc Traonmilin





 
Reply With Quote
 
Yaj Bhattacharya
Guest
Posts: n/a
 
      03-17-2009
Thanks Rob!
Thanks Luc!
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with wsgiref.headers.Headers Phil Python 4 01-17-2010 04:47 PM
Reading CSV or Excel files in java and mapping values to database columns delcas2004@gmail.com Java 7 06-28-2007 02:34 PM
Server cannot clear headers after HTTP headers have been sent Ian ASP .Net Security 2 03-20-2007 09:00 AM
convert rows to columns and columns to rows helpful sql ASP .Net 0 05-19-2005 06:03 PM
Reading 'received' headers: Email Headers Parsing dont bother Python 0 03-03-2004 08:18 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57