Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > avoding the accumulation of array when using loop.

Reply
Thread Tools

avoding the accumulation of array when using loop.

 
 
Isaac Won
Guest
Posts: n/a
 
      01-02-2013
Hi all,

Thanks to Hans, I have had a good progress on my problem.

Followings are Hans's Idea:

import numpy as np

b = []
c = 4
f = open("text.file", "r")

while c < 10:
c = c + 1


f.seek(0,0)

for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])

y = np.array(b, float)
print c, y


It's a bit inefficient to read the same file several times.
You might consider reading it just once. For example:


import numpy as np

b = []



f = open("text.file", "r")

data = [ line.strip().split() for line in f ]
f.close()

for c in xrange(5, 11):
for row in data:
b.append(row[c])


y = np.array(b, float)
print c, y
-------------------------------------------------------------------------------

It is a great idea, but I found some problems. I want each individual arrayof y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize forloop for each time to produce array. This effort has not been very successful.
Do you guys have any idea? I will really appreciate ant help and idea.

Thanks,

Isaac
 
Reply With Quote
 
 
 
 
Dave Angel
Guest
Posts: n/a
 
      01-02-2013
On 01/02/2013 05:21 PM, Isaac Won wrote:
> Hi all,
>
> Thanks to Hans, I have had a good progress on my problem.
>
> Followings are Hans's Idea:
>
> import numpy as np
>
> b = []
> c = 4
> f = open("text.file", "r")
>
> while c < 10:
> c = c + 1
>
>
> f.seek(0,0)
>
> for columns in ( raw.strip().split() for raw in f ):
> b.append(columns[c])
>
> y = np.array(b, float)
> print c, y
>
>
> It's a bit inefficient to read the same file several times.


Don't bet on it. The OS and the libraries and Python each do some
buffering, so it might be nearly as fast to just reread if it's a small
file. And if it's a huge one, the list would be even bigger. So the
only sizes where the second approach is likely better is the mid-size file.

> You might consider reading it just once. For example:
>
>
> import numpy as np
>
> b = []
>
>
>
> f = open("text.file", "r")
>
> data = [ line.strip().split() for line in f ]
> f.close()
>
> for c in xrange(5, 11):
> for row in data:
> b.append(row[c])
>
>
> y = np.array(b, float)
> print c, y
> -------------------------------------------------------------------------------
>
> It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize for loop for each time to produce array. This effort has not been very successful.
> Do you guys have any idea? I will really appreciate ant help and idea.


Your description is very confusing. But i don't see why you just don't
just set b=[] inside the outer loop, rather than doing it at the begin
of the program.

for c in xrange(5, 11):
b = []
for row in data:
b.append(row[c])



--

DaveA

 
Reply With Quote
 
 
 
 
Isaac Won
Guest
Posts: n/a
 
      01-03-2013
On Wednesday, January 2, 2013 5:54:18 PM UTC-6, Dave Angel wrote:
> On 01/02/2013 05:21 PM, Isaac Won wrote:
>
> > Hi all,

>
> >

>
> > Thanks to Hans, I have had a good progress on my problem.

>
> >

>
> > Followings are Hans's Idea:

>
> >

>
> > import numpy as np

>
> >

>
> > b = []

>
> > c = 4

>
> > f = open("text.file", "r")

>
> >

>
> > while c < 10:

>
> > c = c + 1

>
> >

>
> >

>
> > f.seek(0,0)

>
> >

>
> > for columns in ( raw.strip().split() for raw in f ):

>
> > b.append(columns[c])

>
> >

>
> > y = np.array(b, float)

>
> > print c, y

>
> >

>
> >

>
> > It's a bit inefficient to read the same file several times.

>
>
>
> Don't bet on it. The OS and the libraries and Python each do some
>
> buffering, so it might be nearly as fast to just reread if it's a small
>
> file. And if it's a huge one, the list would be even bigger. So the
>
> only sizes where the second approach is likely better is the mid-size file.
>
>
>
> > You might consider reading it just once. For example:

>
> >

>
> >

>
> > import numpy as np

>
> >

>
> > b = []

>
> >

>
> >

>
> >

>
> > f = open("text.file", "r")

>
> >

>
> > data = [ line.strip().split() for line in f ]

>
> > f.close()

>
> >

>
> > for c in xrange(5, 11):

>
> > for row in data:

>
> > b.append(row[c])

>
> >

>
> >

>
> > y = np.array(b, float)

>
> > print c, y

>
> > -------------------------------------------------------------------------------

>
> >

>
> > It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initializefor loop for each time to produce array. This effort has not been very successful.

>
> > Do you guys have any idea? I will really appreciate ant help and idea.

>
>
>
> Your description is very confusing. But i don't see why you just don't
>
> just set b=[] inside the outer loop, rather than doing it at the begin
>
> of the program.
>
>
>
> for c in xrange(5, 11):
>
> b = []
>
> for row in data:
>
> b.append(row[c])
>
>
>
>
>
>
>
> --
>
>
>
> DaveA


Hi Dave,

I really appreciate your advice. It was really helpful.

Isaac
 
Reply With Quote
 
Isaac Won
Guest
Posts: n/a
 
      01-03-2013
On Wednesday, January 2, 2013 5:54:18 PM UTC-6, Dave Angel wrote:
> On 01/02/2013 05:21 PM, Isaac Won wrote:
>
> > Hi all,

>
> >

>
> > Thanks to Hans, I have had a good progress on my problem.

>
> >

>
> > Followings are Hans's Idea:

>
> >

>
> > import numpy as np

>
> >

>
> > b = []

>
> > c = 4

>
> > f = open("text.file", "r")

>
> >

>
> > while c < 10:

>
> > c = c + 1

>
> >

>
> >

>
> > f.seek(0,0)

>
> >

>
> > for columns in ( raw.strip().split() for raw in f ):

>
> > b.append(columns[c])

>
> >

>
> > y = np.array(b, float)

>
> > print c, y

>
> >

>
> >

>
> > It's a bit inefficient to read the same file several times.

>
>
>
> Don't bet on it. The OS and the libraries and Python each do some
>
> buffering, so it might be nearly as fast to just reread if it's a small
>
> file. And if it's a huge one, the list would be even bigger. So the
>
> only sizes where the second approach is likely better is the mid-size file.
>
>
>
> > You might consider reading it just once. For example:

>
> >

>
> >

>
> > import numpy as np

>
> >

>
> > b = []

>
> >

>
> >

>
> >

>
> > f = open("text.file", "r")

>
> >

>
> > data = [ line.strip().split() for line in f ]

>
> > f.close()

>
> >

>
> > for c in xrange(5, 11):

>
> > for row in data:

>
> > b.append(row[c])

>
> >

>
> >

>
> > y = np.array(b, float)

>
> > print c, y

>
> > -------------------------------------------------------------------------------

>
> >

>
> > It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initializefor loop for each time to produce array. This effort has not been very successful.

>
> > Do you guys have any idea? I will really appreciate ant help and idea.

>
>
>
> Your description is very confusing. But i don't see why you just don't
>
> just set b=[] inside the outer loop, rather than doing it at the begin
>
> of the program.
>
>
>
> for c in xrange(5, 11):
>
> b = []
>
> for row in data:
>
> b.append(row[c])
>
>
>
>
>
>
>
> --
>
>
>
> DaveA


Hi Dave,

I really appreciate your advice. It was really helpful.

Isaac
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
const and array of array (of array ...) Mara Guida C Programming 3 09-03-2009 07:54 AM
Length of Array of Array of Array Tom Perl Misc 3 12-20-2004 05:23 PM
Avoding Cross-Domain Security Error before accessing windowattributes? Erwin S. Andreasen Javascript 2 06-22-2004 10:38 AM
Solutions to photo accumulation on trips? Anthony Buckland Digital Photography 30 12-08-2003 12:15 PM
Memory accumulation in web service Ragnar ASP .Net Web Services 0 11-05-2003 12:58 PM



Advertisments