Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > A question about yield

Reply
Thread Tools

A question about yield

 
 
chad
Guest
Posts: n/a
 
      11-07-2010
I have an input file named 'freq' which contains the following data

123 0

133 3
146 1
200 0
233 10
400 2


Now I've attempted to write a script that would take a number from the
standard input and then
have the program return the number in the input file that is closest
to that input file.

#!/usr/local/bin/python

import sys

def construct_set(data):
for line in data:
lines = line.splitlines()
for curline in lines:
if curline.strip():
key = curline.split(' ')
value = int(key[0])
yield value

def approximate(first, second):
midpoint = (first + second) / 2
return midpoint

def format(input):
prev = 0
value = int(input)

with open("/home/cdalten/oakland/freq") as f:
for next in construct_set(f):
if value > prev:
current = prev
prev = next

middle = approximate(current, prev)
if middle < prev and value > middle:
return prev
elif value > current and current < middle:
return current

if __name__ == "__main__":
if len(sys.argv) != 2:
print >> sys.stderr, "You need to enter a number\n"
sys.exit(1)

nearest = format(sys.argv[1])
print "The closest value to", sys.argv[1], "is", nearest


When I run it, I get the following...

[cdalten@localhost oakland]$ ./android4.py 123
The closest value to 123 is 123
[cdalten@localhost oakland]$ ./android4.py 130
The closest value to 130 is 133
[cdalten@localhost oakland]$ ./android4.py 140
The closest value to 140 is 146
[cdalten@localhost oakland]$ ./android4.py 146
The closest value to 146 is 146
[cdalten@localhost oakland]$ ./android4.py 190
The closest value to 190 is 200
[cdalten@localhost oakland]$ ./android4.py 200
The closest value to 200 is 200
[cdalten@localhost oakland]$ ./android4.py 205
The closest value to 205 is 200
[cdalten@localhost oakland]$ ./android4.py 210
The closest value to 210 is 200
[cdalten@localhost oakland]$ ./android4.py 300
The closest value to 300 is 233
[cdalten@localhost oakland]$ ./android4.py 500
The closest value to 500 is 400
[cdalten@localhost oakland]$ ./android4.py 1000000
The closest value to 1000000 is 400
[cdalten@localhost oakland]$

The question is about the construct_set() function.

def construct_set(data):
for line in data:
lines = line.splitlines()
for curline in lines:
if curline.strip():
key = curline.split(' ')
value = int(key[0])
yield value

I have it yield on 'value' instead of 'curline'. Will the program
still read the input file named freq line by line even though I don't
have it yielding on 'curline'? Or since I have it yield on 'value',
will it read the entire input file into memory at once?

Chad
 
Reply With Quote
 
 
 
 
chad
Guest
Posts: n/a
 
      11-07-2010
On Nov 7, 9:34*am, chad <(E-Mail Removed)> wrote:
> I have an input file named 'freq' which contains the following data
>
> 123 0
>
> 133 3
> 146 1
> 200 0
> 233 10
> 400 2
>
> Now I've attempted to write a script that would take a number from the
> standard input and then
> have the program return the number in the input file that is closest
> to that input file.


*and then have the program return the number in the input file that is
closest to the number the user inputs (or enters).*
 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      11-07-2010
On Sun, Nov 7, 2010 at 9:34 AM, chad <(E-Mail Removed)> wrote:
<snip>
> #!/usr/local/bin/python
>
> import sys
>
> def construct_set(data):
> * *for line in data:
> * * * *lines = line.splitlines()
> * * * *for curline in lines:
> * * * * * *if curline.strip():
> * * * * * * * *key = curline.split(' ')
> * * * * * * * *value = int(key[0])
> * * * * * * * *yield value
>
> def approximate(first, second):
> * *midpoint = (first + second) / 2
> * *return midpoint
>
> def format(input):
> * *prev = 0
> * *value = int(input)
>
> * *with open("/home/cdalten/oakland/freq") as f:
> * * * *for next in construct_set(f):
> * * * * * *if value > prev:
> * * * * * * * *current = prev
> * * * * * * * *prev = next
>
> * * * *middle = approximate(current, prev)
> * * * *if middle < prev and value > middle:
> * * * * * *return prev
> * * * *elif value > current and current < middle:
> * * * * * *return current

<snip>
> The question is about the construct_set() function.

<snip>
> I have it yield on 'value' instead of 'curline'. Will the program
> still read the input file named freq line by line even though I don't
> have it yielding on 'curline'? Or since I have it yield on 'value',
> will it read the entire input file into memory at once?


The former. The yield has no effect at all on how the file is read.
The "for line in data:" iteration over the file object is what makes
Python read from the file line-by-line. Incidentally, the use of
splitlines() is pointless; you're already getting single lines from
the file object by iterating over it, so splitlines() will always
return a single-element list.

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
chad
Guest
Posts: n/a
 
      11-07-2010
On Nov 7, 9:47*am, Chris Rebert <(E-Mail Removed)> wrote:
> On Sun, Nov 7, 2010 at 9:34 AM, chad <(E-Mail Removed)> wrote:
>
> <snip>
>
>
>
> > #!/usr/local/bin/python

>
> > import sys

>
> > def construct_set(data):
> > * *for line in data:
> > * * * *lines = line.splitlines()
> > * * * *for curline in lines:
> > * * * * * *if curline.strip():
> > * * * * * * * *key = curline.split(' ')
> > * * * * * * * *value = int(key[0])
> > * * * * * * * *yield value

>
> > def approximate(first, second):
> > * *midpoint = (first + second) / 2
> > * *return midpoint

>
> > def format(input):
> > * *prev = 0
> > * *value = int(input)

>
> > * *with open("/home/cdalten/oakland/freq") as f:
> > * * * *for next in construct_set(f):
> > * * * * * *if value > prev:
> > * * * * * * * *current = prev
> > * * * * * * * *prev = next

>
> > * * * *middle = approximate(current, prev)
> > * * * *if middle < prev and value > middle:
> > * * * * * *return prev
> > * * * *elif value > current and current < middle:
> > * * * * * *return current

> <snip>
> > The question is about the construct_set() function.

> <snip>
> > I have it yield on 'value' instead of 'curline'. Will the program
> > still read the input file named freq line by line even though I don't
> > have it yielding on 'curline'? Or since I have it yield on 'value',
> > will it read the entire input file into memory at once?

>
> The former. The yield has no effect at all on how the file is read.
> The "for line in data:" iteration over the file object is what makes
> Python read from the file line-by-line. Incidentally, the use of
> splitlines() is pointless; you're already getting single lines from
> the file object by iterating over it, so splitlines() will always
> return a single-element list.
>


But what happens if the input file is say 250MB? Will all 250MB be
loaded into memory at once? Just curious, because I thought maybe
using something like 'yield curline' would prevent this scenario.
 
Reply With Quote
 
Chris Rebert
Guest
Posts: n/a
 
      11-07-2010
On Sun, Nov 7, 2010 at 9:56 AM, chad <(E-Mail Removed)> wrote:
> On Nov 7, 9:47*am, Chris Rebert <(E-Mail Removed)> wrote:
>> On Sun, Nov 7, 2010 at 9:34 AM, chad <(E-Mail Removed)> wrote:
>> <snip>
>> > #!/usr/local/bin/python

>>
>> > import sys

>>
>> > def construct_set(data):
>> > * *for line in data:
>> > * * * *lines = line.splitlines()
>> > * * * *for curline in lines:
>> > * * * * * *if curline.strip():
>> > * * * * * * * *key = curline..split(' ')
>> > * * * * * * * *value = int(key[0])
>> > * * * * * * * *yield value

>>
>> > def approximate(first, second):
>> > * *midpoint = (first + second) / 2
>> > * *return midpoint

>>
>> > def format(input):
>> > * *prev = 0
>> > * *value = int(input)

>>
>> > * *with open("/home/cdalten/oakland/freq") as f:
>> > * * * *for next in construct_set(f):
>> > * * * * * *if value > prev:
>> > * * * * * * * *current = prev
>> > * * * * * * * *prev = next

>>
>> > * * * *middle = approximate(current, prev)
>> > * * * *if middle < prev and value > middle:
>> > * * * * * *return prev
>> > * * * *elif value > current and current < middle:
>> > * * * * * *return current

>> <snip>
>> > The question is about the construct_set() function.

>> <snip>
>> > I have it yield on 'value' instead of 'curline'. Will the program
>> > still read the input file named freq line by line even though I don't
>> > have it yielding on 'curline'? Or since I have it yield on 'value',
>> > will it read the entire input file into memory at once?

>>
>> The former. The yield has no effect at all on how the file is read.
>> The "for line in data:" iteration over the file object is what makes
>> Python read from the file line-by-line. Incidentally, the use of
>> splitlines() is pointless; you're already getting single lines from
>> the file object by iterating over it, so splitlines() will always
>> return a single-element list.

>
> But what happens if the input file is say 250MB? Will all 250MB be
> loaded into memory at once?


No. As I said, the file will be read from 1 line at a time, on an
as-needed basis; which is to say, "line-by-line".

> Just curious, because I thought maybe
> using something like 'yield curline' would prevent this scenario.


Using "for line in data:" is what prevents that scenario.
The "yield" is only relevant to how the file is read insofar as the
the alternative to yield-ing would be to return a list, which would
necessitate going through the entire file in continuous go and then
returning a very large list; but even then, the file's content would
still be read from line-by-line, not all at once as one humongous
string.

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
Simon Brunning
Guest
Posts: n/a
 
      11-08-2010
On 7 November 2010 18:14, Chris Rebert <(E-Mail Removed)> wrote:
> On Sun, Nov 7, 2010 at 9:56 AM, chad <(E-Mail Removed)> wrote:
>> But what happens if the input file is say 250MB? Will all 250MB be
>> loaded into memory at once?

>
> No. As I said, the file will be read from 1 line at a time, on an
> as-needed basis; which is to say, "line-by-line".


IIRC, it's somewhere in between. Python will read the file in blocks.
If only *looks* like it's reading the file line by line.

--
Cheers,
Simon B.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[ANN] To Yield or Not to Yield: An Inferable Question Michael Edgar Ruby 13 04-21-2011 05:55 PM
yield expression programmized-formal interpretation. (interpretationof yield expression.) castironpi@gmail.com Python 1 04-22-2008 01:32 AM
Yield & associations (was Proc / def /yield semantics) Markus Ruby 1 09-27-2004 07:04 PM
RE: Question about bytecode and yield Robert Brewer Python 1 07-20-2004 01:49 PM
Question about bytecode and yield Daniel Brodie Python 1 07-18-2004 05:51 AM



Advertisments