«

hello,

i'm trying to understand how i could build following consecutive sets
from a root one using generator :

l = [1,2,3,4]

would like to produce :

[1,2], [2,3], [3,4], [1,2,3], [2,3,4]

but unfortunately can not, i guess i can do it by using sub generator
and maybe enumerate, please if you help could you explain a bit the
trick ? looks like this sub generator thing mess me up.

(i think it should may be also have [1,], [2,], [3,], [1,2,3,4] , and
then be filtered)

bests

On 21 Jan 2005 05:58:03 -0800
(Joh) wrote:

> i'm trying to understand how i could build following consecutive sets
> from a root one using generator :
>
> l = [1,2,3,4]
>
> would like to produce :
>
> [1,2], [2,3], [3,4], [1,2,3], [2,3,4]

>>> def consecutive_sets(l):
.... for i in xrange(2, len(l)):
.... for j in xrange(0, len(l)-i+1):
.... yield l[j:j+i]
....
>>> list(consecutive_sets([1,2,3,4]))
[[1, 2], [2, 3], [3, 4], [1, 2, 3], [2, 3, 4]]

--
Denis S. Otkidach
http://www.python.ru/ [ru]

Joh wrote:

>hello,
>
>i'm trying to understand how i could build following consecutive sets
>from a root one using generator :
>
>l = [1,2,3,4]
>
>would like to produce :
>
>[1,2], [2,3], [3,4], [1,2,3], [2,3,4]
>
>
Do you mean:

[1,2], [2,3], [3,4], [1,2,3], [2,3,4], [1,3,4]


(E.g. all elements in the power set except the empty set, the sets with
one element and the sets with all elements.)
Otherwise, please describe your desired sets in verbal - I cannot see
the point.

On Fri, 2005-01-21 at 17:14 +0300, Denis S. Otkidach wrote:
> On 21 Jan 2005 05:58:03 -0800
> (Joh) wrote:
>
> > i'm trying to understand how i could build following consecutive sets
> > from a root one using generator :
> >
> > l = [1,2,3,4]
> >
> > would like to produce :
> >
> > [1,2], [2,3], [3,4], [1,2,3], [2,3,4]
>
> >>> def consecutive_sets(l):
> ... for i in xrange(2, len(l)):
> ... for j in xrange(0, len(l)-i+1):
> ... yield l[j:j+i]

Since you have a much faster brain than I (though I ended up with
exactly the same thing barring variable names) and beat me to posting
the answer, I'll post the inevitable awful generator expression version
instead:

consecutive_sets = ( x[offsetffset+subset_size]
for subset_size in xrange(2, len(x))
for offset in xrange(0, len(x) + 1 - subset_size) )

--
Craig Ringer

Hi,

I recently read David Mertz (IBM DeveloperWorks) about generators and got
excited about using lazy constructs in my Python programming.

But besides the fact that generators are either produced with the new "yield"
reserved word or by defining the __new__ method in a class definition, I
don't know much about them.

In particular, I don't know what Python constructs does generate a generator.
I know this is now the case for reading lines in a file or with the new
"iterator" package. But what else ? Does Craig Ringer answer mean that list
comprehensions are lazy ? Where can I find a comprehensive list of all the
lazy constructions built in Python ? (I think that to easily distinguish lazy
from strict constructs is an absolute programmer need -- otherwise you always
end up wondering when is it that code is actually executed like in Haskell).

Thank you

Francis Girard
FRANCE

Le vendredi 21 Janvier 2005 15:38, Craig Ringer a écritÂ*:
> On Fri, 2005-01-21 at 17:14 +0300, Denis S. Otkidach wrote:
> > On 21 Jan 2005 05:58:03 -0800
> >
> > (Joh) wrote:
> > > i'm trying to understand how i could build following consecutive sets
> > > from a root one using generator :
> > >
> > > l = [1,2,3,4]
> > >
> > > would like to produce :
> > >
> > > [1,2], [2,3], [3,4], [1,2,3], [2,3,4]
> > >
> > >>> def consecutive_sets(l):
> >
> > ... for i in xrange(2, len(l)):
> > ... for j in xrange(0, len(l)-i+1):
> > ... yield l[j:j+i]
>
> Since you have a much faster brain than I (though I ended up with
> exactly the same thing barring variable names) and beat me to posting
> the answer, I'll post the inevitable awful generator expression version
> instead:
>
> consecutive_sets = ( x[offsetffset+subset_size]
> for subset_size in xrange(2, len(x))
> for offset in xrange(0, len(x) + 1 - subset_size) )
>
> --
> Craig Ringer

On Fri, 2005-01-21 at 22:38 +0800, Craig Ringer wrote:

> consecutive_sets = ( x[offsetffset+subset_size]
> for subset_size in xrange(2, len(x))
> for offset in xrange(0, len(x) + 1 - subset_size) )

Where 'x' is list to operate on, as I should've initially noted. Sorry
for the reply-to-self.

I did say "awful" for a reason

--
Craig Ringer

Joh wrote:

> i'm trying to understand how i could build following consecutive sets
> from a root one using generator :
>
> l = [1,2,3,4]
>
> would like to produce :
>
> [1,2], [2,3], [3,4], [1,2,3], [2,3,4]
>
> but unfortunately can not, i guess i can do it by using sub generator
> and maybe enumerate, please if you help could you explain a bit the
> trick ? looks like this sub generator thing mess me up.

Here is an (untested) variant that accepts any iterable while trying to
remain memory-efficient. This makes it necessary to shuffle the order of
the output a bit.

from itertools import tee, islice

def gen(iterable, start, end):
it = iter(iterable)
while True:
it, a = tee(it)
a = tuple(islice(a, end-1))
for sz in xrange(start, len(a)+1):
yield a[:sz]
it.next()


if __name__ == "__main__":
print list(gen(range(1, 5), 2, 4))

# prints:
# [(1, 2), (1, 2, 3), (2, 3), (2, 3, 4), (3, 4)]

Peter

On Fri, 2005-01-21 at 16:05 +0100, Francis Girard wrote:
> I recently read David Mertz (IBM DeveloperWorks) about generators and
> got excited about using lazy constructs in my Python programming.

Speaking of totally great articles, and indirectly to lazyness (though
not lazyily evaluated constructs), I was really impressed by this
fantastic article on metaclasses:

http://gnosis.cx/publish/programming/metaclass_1.html
http://gnosis.cx/publish/programming/metaclass_2.html

which shows that they're really just not that hard. That saved me an
IMMENSE amount of utterly tedious coding just recently.

> But besides the fact that generators are either produced with the new
> "yield" reserved word or by defining the __new__ method in a class
> definition, I don't know much about them.

They can also be created with a generator expression under Python 2.4. A
generator expression works much like a list comprehension, but returns a
generator instead of a list, and is evaluated lazily. (It also doesn't
pollute the outside namespace with its working variables).

>>> print [ x for x in range(1,10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> print ( x for x in xrange(1,10) )
<generator object at 0x401e40ac>
>>> print list(( x for x in xrange(1,10) ))
[1, 2, 3, 4, 5, 6, 7, 8, 9]

Not the use of xrange above for efficiency in the generator expressions.
These examples are trivial and pointless, but hopefully get the point
across.

> In particular, I don't know what Python constructs does generate a
> generator.

As far as I know, functions that use yield, and generator expressions. I
was unaware of the ability to create them using a class with a __new__
method, and need to check that out - I can imagine situations in which
it might be rather handy.

I'm not sure how many Python built-in functions and library modules
return generators for things.

> I know this is now the case for reading lines in a file or with the
> new "iterator" package. But what else ? Does Craig Ringer answer mean
> that list comprehensions are lazy ?

Nope, but generator expressions are, and they're pretty similar.

> Where can I find a comprehensive list of all the lazy constructions
> built in Python ? (I think that to easily distinguish lazy from strict
> constructs is an absolute programmer need -- otherwise you always end
> up wondering when is it that code is actually executed like in
> Haskell).

I'm afraid I can't help you with that. I tend to take the view that side
effects in lazily executed code are a bad plan, and use lazy execution
for things where there is no reason to care when the code is executed.

--
Craig Ringer

Le vendredi 21 Janvier 2005 16:06, Craig Ringer a écritÂ*:
> On Fri, 2005-01-21 at 22:38 +0800, Craig Ringer wrote:
> > consecutive_sets = ( x[offsetffset+subset_size]
> > for subset_size in xrange(2, len(x))
> > for offset in xrange(0, len(x) + 1 - subset_size) )
>
> Where 'x' is list to operate on, as I should've initially noted. Sorry
> for the reply-to-self.
>
> I did say "awful" for a reason
>
> --
> Craig Ringer

First, I think that you mean :

consecutive_sets = [ x[offsetffset+subset_size]
for subset_size in xrange(2, len(x))
for offset in xrange(0, len(x) + 1 - subset_size)]

(with square brackets).

Second,

this is not lazy anymore (like Denis S. Otkidach previous answer was) because
the __whole__ list get constructed __before__ any other piece of code have a
chance to execute. The type of consecutive_sets is simply a list, not a
generator.

I'm just trying to understand and obviously I'm missing the point.

Thank you

Francis Girard
FRANCE

Really, thank you Craig Ringer for your great answer.


> I'm afraid I can't help you with that. I tend to take the view that side
> effects in lazily executed code are a bad plan, and use lazy execution
> for things where there is no reason to care when the code is executed.


I completly agree with this. But this is much more true in theory than in
practice. In practice you might end up with big big memory usage with lazy
constructs as the "system" has to store intermediate results (the execution
context in the case of Python). These kinds of phenomena happen all the time
in a language like Haskell -- at least for a beginner like me -- if you don't
pay attention to it ; and this makes the language a lot more difficult to
master. Thus you have to keep an eye on performance even though, in FP, you
shoould just have to "declare" your intentions and let the system manage the
execution path.


> http://gnosis.cx/publish/programming/metaclass_1.html
> http://gnosis.cx/publish/programming/metaclass_2.html

Thank you, I'll read that.

Francis Girard
FRANCE

Le vendredi 21 Janvier 2005 16:42, Craig Ringer a écritÂ*:
> On Fri, 2005-01-21 at 16:05 +0100, Francis Girard wrote:
> > I recently read David Mertz (IBM DeveloperWorks) about generators and
> > got excited about using lazy constructs in my Python programming.
>
> Speaking of totally great articles, and indirectly to lazyness (though
> not lazyily evaluated constructs), I was really impressed by this
> fantastic article on metaclasses:
>
> http://gnosis.cx/publish/programming/metaclass_1.html
> http://gnosis.cx/publish/programming/metaclass_2.html
>
> which shows that they're really just not that hard. That saved me an
> IMMENSE amount of utterly tedious coding just recently.
>
> > But besides the fact that generators are either produced with the new
> > "yield" reserved word or by defining the __new__ method in a class
> > definition, I don't know much about them.
>
> They can also be created with a generator expression under Python 2.4. A
> generator expression works much like a list comprehension, but returns a
> generator instead of a list, and is evaluated lazily. (It also doesn't
> pollute the outside namespace with its working variables).
>
> >>> print [ x for x in range(1,10)]
>
> [1, 2, 3, 4, 5, 6, 7, 8, 9]
>
> >>> print ( x for x in xrange(1,10) )
>
> <generator object at 0x401e40ac>
>
> >>> print list(( x for x in xrange(1,10) ))
>
> [1, 2, 3, 4, 5, 6, 7, 8, 9]
>
> Not the use of xrange above for efficiency in the generator expressions.
> These examples are trivial and pointless, but hopefully get the point
> across.
>
> > In particular, I don't know what Python constructs does generate a
> > generator.
>
> As far as I know, functions that use yield, and generator expressions. I
> was unaware of the ability to create them using a class with a __new__
> method, and need to check that out - I can imagine situations in which
> it might be rather handy.
>
> I'm not sure how many Python built-in functions and library modules
> return generators for things.
>
> > I know this is now the case for reading lines in a file or with the
> > new "iterator" package. But what else ? Does Craig Ringer answer mean
> > that list comprehensions are lazy ?
>
> Nope, but generator expressions are, and they're pretty similar.
>
> > Where can I find a comprehensive list of all the lazy constructions
> > built in Python ? (I think that to easily distinguish lazy from strict
> > constructs is an absolute programmer need -- otherwise you always end
> > up wondering when is it that code is actually executed like in
> > Haskell).
>
> I'm afraid I can't help you with that. I tend to take the view that side
> effects in lazily executed code are a bad plan, and use lazy execution
> for things where there is no reason to care when the code is executed.
>
> --
> Craig Ringer