Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Flattening lists

Reply
Thread Tools

Flattening lists

 
 
Michele Simionato
Guest
Posts: n/a
 
      02-05-2009
On Feb 5, 7:24*pm, (E-Mail Removed) (Aahz) wrote:
> In article <(E-Mail Removed)..com>,
> Michele Simionato *<(E-Mail Removed)> wrote:
>
>
>
> >Looks fine to me. In some situations you may also use hasattr(el,
> >'__iter__') instead of isinstance(el, list) (it depends if you want to
> >flatten generic iterables or only lists).

>
> Of course, once you do that, you need to special-case strings...


Strings are iterable but have no __iter__ method, which is fine in
this context, since I would say 99.9% of times one wants to treat them
as atomic objects, so no need to special case.
 
Reply With Quote
 
 
 
 
Rhamphoryncus
Guest
Posts: n/a
 
      02-06-2009
On Feb 5, 1:16*pm, Michele Simionato <(E-Mail Removed)>
wrote:
> On Feb 5, 7:24*pm, (E-Mail Removed) (Aahz) wrote:
>
> > In article <(E-Mail Removed)>,
> > Michele Simionato *<(E-Mail Removed)> wrote:

>
> > >Looks fine to me. In some situations you may also use hasattr(el,
> > >'__iter__') instead of isinstance(el, list) (it depends if you want to
> > >flatten generic iterables or only lists).

>
> > Of course, once you do that, you need to special-case strings...

>
> Strings are iterable but have no __iter__ method, which is fine in
> this context, since I would say 99.9% of times one wants to treat them
> as atomic objects, so no need to special case.


Don't worry, that little oddity was fixed for you:

Python 3.0+ (unknown, Dec 8 2008, 14:26:15)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> str.__iter__

<slot wrapper '__iter__' of 'str' objects>
>>> bytes.__iter__

<slot wrapper '__iter__' of 'bytes' objects>
>>> bytearray.__iter__

<slot wrapper '__iter__' of 'bytearray' objects>


I'm in the "why do you need more than 1 depth?" camp. Dispatching
based on your own type should be given an extra look. Dispatching
based passed in types should be given three extra looks.

I didn't realize itertools.chain(*iterable) worked. I guess that
needs to be pushed as the canonical form.
 
Reply With Quote
 
 
 
 
Mensanator
Guest
Posts: n/a
 
      02-06-2009
On Feb 6, 3:23*pm, Rhamphoryncus <(E-Mail Removed)> wrote:
> On Feb 5, 1:16*pm, Michele Simionato <(E-Mail Removed)>
> wrote:
>
> > On Feb 5, 7:24*pm, (E-Mail Removed) (Aahz) wrote:

>
> > > In article <(E-Mail Removed)>,
> > > Michele Simionato *<(E-Mail Removed)> wrote:

>
> > > >Looks fine to me. In some situations you may also use hasattr(el,
> > > >'__iter__') instead of isinstance(el, list) (it depends if you want to
> > > >flatten generic iterables or only lists).

>
> > > Of course, once you do that, you need to special-case strings...

>
> > Strings are iterable but have no __iter__ method, which is fine in
> > this context, since I would say 99.9% of times one wants to treat them
> > as atomic objects, so no need to special case.

>
> Don't worry, that little oddity was fixed for you:
>
> Python 3.0+ (unknown, Dec *8 2008, 14:26:15)
> [GCC 4.3.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.>>> str.__iter__
>
> <slot wrapper '__iter__' of 'str' objects>>>> bytes.__iter__
>
> <slot wrapper '__iter__' of 'bytes' objects>>>> bytearray.__iter__
>
> <slot wrapper '__iter__' of 'bytearray' objects>
>
> I'm in the "why do you need more than 1 depth?" camp. *Dispatching
> based on your own type should be given an extra look. *Dispatching
> based passed in types should be given three extra looks.
>
> I didn't realize itertools.chain(*iterable) worked. *I guess that
> needs to be pushed as the canonical form.


What about this (from the Recipes section of the itertools manual)?

def flatten(listOfLists):
return list(chain.from_iterable(listOfLists))
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      02-07-2009
On Feb 6, 10:23*pm, Rhamphoryncus <(E-Mail Removed)> wrote:
> On Feb 5, 1:16*pm, Michele Simionato <(E-Mail Removed)>
> wrote:
>
> > On Feb 5, 7:24*pm, (E-Mail Removed) (Aahz) wrote:

>
> > > In article <(E-Mail Removed)>,
> > > Michele Simionato *<(E-Mail Removed)> wrote:

>
> > > >Looks fine to me. In some situations you may also use hasattr(el,
> > > >'__iter__') instead of isinstance(el, list) (it depends if you want to
> > > >flatten generic iterables or only lists).

>
> > > Of course, once you do that, you need to special-case strings...

>
> > Strings are iterable but have no __iter__ method, which is fine in
> > this context, since I would say 99.9% of times one wants to treat them
> > as atomic objects, so no need to special case.

>
> Don't worry, that little oddity was fixed for you:


Acc! I have a few places in my code with checks of the
kind ``hasattr(x, '__iter__')`` and I guess those spots
will be tricky when converting to Python 3. I guess
2to3 cannot help either
 
Reply With Quote
 
rdmurray@bitdance.com
Guest
Posts: n/a
 
      02-07-2009
Quoth Mensanator <(E-Mail Removed)>:
> On Feb 6, 3:23=A0pm, Rhamphoryncus <(E-Mail Removed)> wrote:
> > On Feb 5, 1:16=A0pm, Michele Simionato <(E-Mail Removed)>
> > wrote:
> >
> > > On Feb 5, 7:24=A0pm, (E-Mail Removed) (Aahz) wrote:
> > > > In article <(E-Mail Removed)>,
> > > > Michele Simionato =A0<(E-Mail Removed)> wrote:
> > > > >Looks fine to me. In some situations you may also use hasattr(el,
> > > > >'__iter__') instead of isinstance(el, list) (it depends if you want to
> > > > >flatten generic iterables or only lists).
> > > > Of course, once you do that, you need to special-case strings...

> >
> > > Strings are iterable but have no __iter__ method, which is fine in
> > > this context, since I would say 99.9% of times one wants to treat them
> > > as atomic objects, so no need to special case.

> >
> > Don't worry, that little oddity was fixed for you:
> >
> > Python 3.0+ (unknown, Dec =A08 2008, 14:26:15)
> > [GCC 4.3.2] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> str.__iter__

> > <slot wrapper '__iter__' of 'str' objects
> > >>> bytes.__iter__

> > <slot wrapper '__iter__' of 'bytes' objects
> > >>> bytearray.__iter__

> > <slot wrapper '__iter__' of 'bytearray' objects>
> >
> > I'm in the "why do you need more than 1 depth?" camp. Dispatching
> > based on your own type should be given an extra look. Dispatching
> > based passed in types should be given three extra looks.
> >
> > I didn't realize itertools.chain(*iterable) worked. I guess that
> > needs to be pushed as the canonical form.

>
> What about this (from the Recipes section of the itertools manual)?
>
> def flatten(listOfLists):
> return list(chain.from_iterable(listOfLists))


Python 2.6.1 (r261:67515, Jan 7 2009, 17:09:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from itertools import chain
>>> list(chain.from_iterable([1, 2, [3, 4]]))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> list(chain(*[1, 2, [3, 4]]))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))

['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]

--RDM

 
Reply With Quote
 
Rhamphoryncus
Guest
Posts: n/a
 
      02-07-2009
On Feb 6, 10:21*pm, (E-Mail Removed) wrote:
> Quoth Mensanator <(E-Mail Removed)>:
> > def flatten(listOfLists):
> > * * return list(chain.from_iterable(listOfLists))

>
> * * Python 2.6.1 (r261:67515, Jan *7 2009, 17:09:13)
> * * [GCC 4.3.2] on linux2
> * * Type "help", "copyright", "credits" or "license" for more information.
> * * >>> from itertools import chain
> * * >>> list(chain.from_iterable([1, 2, [3, 4]]))
> * * Traceback (most recent call last):
> * * * File "<stdin>", line 1, in <module>
> * * TypeError: 'int' object is not iterable
> * * >>> list(chain(*[1, 2, [3, 4]]))
> * * Traceback (most recent call last):
> * * * File "<stdin>", line 1, in <module>
> * * TypeError: 'int' object is not iterable
> * * >>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))
> * * ['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]


What usecase do you have for such inconsistently structured data?

If I'm building a tree I use my own type for the nodes, keeping them
purely internal, so I can always use isinstance without worrying about
getting something inconvenient passed in.
 
Reply With Quote
 
rdmurray@bitdance.com
Guest
Posts: n/a
 
      02-07-2009
Rhamphoryncus <(E-Mail Removed)> wrote:
> On Feb 6, 10:21=A0pm, (E-Mail Removed) wrote:
> > Quoth Mensanator <(E-Mail Removed)>:
> > > def flatten(listOfLists):
> > > =A0 =A0 return list(chain.from_iterable(listOfLists))

> >
> > =A0 =A0 Python 2.6.1 (r261:67515, Jan =A07 2009, 17:09:13)
> > =A0 =A0 [GCC 4.3.2] on linux2
> > =A0 =A0 Type "help", "copyright", "credits" or "license" for more informa=

> tion.
> > =A0 =A0 >>> from itertools import chain
> > =A0 =A0 >>> list(chain.from_iterable([1, 2, [3, 4]]))
> > =A0 =A0 Traceback (most recent call last):
> > =A0 =A0 =A0 File "<stdin>", line 1, in <module>
> > =A0 =A0 TypeError: 'int' object is not iterable
> > =A0 =A0 >>> list(chain(*[1, 2, [3, 4]]))
> > =A0 =A0 Traceback (most recent call last):
> > =A0 =A0 =A0 File "<stdin>", line 1, in <module>
> > =A0 =A0 TypeError: 'int' object is not iterable
> > =A0 =A0 >>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))
> > =A0 =A0 ['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]

>
> What usecase do you have for such inconsistently structured data?
>
> If I'm building a tree I use my own type for the nodes, keeping them
> purely internal, so I can always use isinstance without worrying about
> getting something inconvenient passed in.


I don't have any use cases myself, I'm just pointing out that this
doesn't answer the concerns of the OP, who presumably does.

--RDM

 
Reply With Quote
 
Guest
Posts: n/a
 
      02-07-2009
On Sat, 7 Feb 2009 01:06:06 -0800 (PST)
Rhamphoryncus <(E-Mail Removed)> wrote:

> On Feb 6, 10:21*pm, (E-Mail Removed) wrote:
> > Quoth Mensanator <(E-Mail Removed)>:
> > > def flatten(listOfLists):
> > > * * return list(chain.from_iterable(listOfLists))

> >
> > * * Python 2.6.1 (r261:67515, Jan *7 2009, 17:09:13)
> > * * [GCC 4.3.2] on linux2
> > * * Type "help", "copyright", "credits" or "license" for more
> > information. >>> from itertools import chain
> > * * >>> list(chain.from_iterable([1, 2, [3, 4]]))
> > * * Traceback (most recent call last):
> > * * * File "<stdin>", line 1, in <module>
> > * * TypeError: 'int' object is not iterable
> > * * >>> list(chain(*[1, 2, [3, 4]]))
> > * * Traceback (most recent call last):
> > * * * File "<stdin>", line 1, in <module>
> > * * TypeError: 'int' object is not iterable
> > * * >>> list(chain.from_iterable(['abcd', 'efg', [3, 4]]))
> > * * ['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4]

>
> What usecase do you have for such inconsistently structured data?


I have a similar use case in pyspread, which is a Python spreadsheet
that employs numpy object arrays. Since the Python objects in the numpy
arrays are derived from user input, they can be anything, including
nested lists as well as strings, etc.

Since I consider my work-around that treats strings as a special case a
rather ugly hack, I would welcome a robust, generic approach to the
OP's problem.

Martin

 
Reply With Quote
 
Rhamphoryncus
Guest
Posts: n/a
 
      02-07-2009
On Feb 7, 1:39*pm, <(E-Mail Removed)> wrote:
> On Sat, 7 Feb 2009 01:06:06 -0800 (PST)
> Rhamphoryncus <(E-Mail Removed)> wrote:
>
> > What usecase do you have for such inconsistently structured data?

>
> I have a similar use case in pyspread, which is a Python spreadsheet
> that employs numpy object arrays. Since the Python objects in the numpy
> arrays are derived from user input, they can be anything, including
> nested lists as well as strings, etc.
>
> Since I consider my work-around that treats strings as a special case a
> rather ugly hack, I would welcome a robust, generic approach to the
> OP's problem.


Can you explain this in a little more detail?
 
Reply With Quote
 
Guest
Posts: n/a
 
      02-07-2009
On Sat, 7 Feb 2009 12:50:22 -0800 (PST)
Rhamphoryncus <(E-Mail Removed)> wrote:

> On Feb 7, 1:39*pm, <(E-Mail Removed)> wrote:
> > On Sat, 7 Feb 2009 01:06:06 -0800 (PST)
> > Rhamphoryncus <(E-Mail Removed)> wrote:
> >
> > > What usecase do you have for such inconsistently structured data?

> >
> > I have a similar use case in pyspread, which is a Python spreadsheet
> > that employs numpy object arrays. Since the Python objects in the
> > numpy arrays are derived from user input, they can be anything,
> > including nested lists as well as strings, etc.
> >
> > Since I consider my work-around that treats strings as a special
> > case a rather ugly hack, I would welcome a robust, generic approach
> > to the OP's problem.

>
> Can you explain this in a little more detail?


In the application, there is one main numpy array of type "O".
Each element of the array corresponds to one cell in a grid. The user
may enter a Python expression into the grid cell. The input is
evaled and the result is stored in the numpy array (the actual process
is a bit more complicated). Therefore, the object inside a numpy array
element may be an inconsistent, nested, iterable type.

The user now may access the result grid via __getitem__. When doing
this, a numpy array that is as flat as possible while comprising the
maximum possible data depth is returned, i.e.:

1. Non-string and non-unicode iterables of similar length for each of
the cells form extra dimensions.

2. In order to remove different container types, the result is
flattened, cast into a numpy.array and re-shaped.

3. Dimensions of length 1 are eliminated.

Therefore, the user can conveniently use numpy ufuncs on the results.

I am referring to the flatten operation in step 2

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
flattening and rebuilding a simple list of lists Esmail Python 2 11-30-2009 02:06 PM
flattening/rolling up/aggregating a large sorted text file adtvff@yahoo.com Python 3 03-22-2007 04:10 PM
a flattening operator? gangesmaster Python 2 04-22-2006 01:41 PM
Flattening out an XML document David Gersic XML 0 05-24-2005 07:14 AM
Descendent's Relative position neede when flattening XSL delgados129 XML 2 04-25-2005 03:41 PM



Advertisments