Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > issubclass(dict, Mapping)

Reply
Thread Tools

issubclass(dict, Mapping)

 
 
kj
Guest
Posts: n/a
 
      12-22-2010

In a message (<4cf97c94$0$30003$c3e8da3$(E-Mail Removed). com>)
on a different thread, Steven D'Aprano tells me:

>I suspect you're trying to make this more complicated than it actually
>is. You keep finding little corner cases that expose implementation
>details (such as the heap-types issue above) and leaping to the erroneous
>conclusion that because you didn't understand this tiny little corner of
>Python's class model, you didn't understand any of it. Python's object
>model is relatively simple, but it does occasionally expose a few messy
>corners.


I disagree with your assessment. What you call "little corner
cases" I call "fundamental", as in "you can't really call yourself
competent with Python if you're ignorant about them".

To use a term I first saw in an article by Joel Spolsky
(http://is.gd/je42O), Python's object model is a rather "leaky
abstraction". This refers to the situation in which a user is not
shielded from the "implementation details". When an abstraction
leaks, implementation details are no longer negligible, they cease
to be "little corner cases".

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)
>>> from collections import Mapping
>>> issubclass(dict, Mapping)

True
>>> dict.__bases__

(<type 'object'>,)
>>> [issubclass(b, Mapping) for b in dict.__bases__]

[False]


So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

I suspect this is another abstraction leak ("dict is *supposed* to
be a Python class like all others, but in fact it's not *really*.
You see, once upon a time...").

I conclude that, for me to understand Python's (rather leaky) object
model abstraction, I have to understand its underlying implementation.
Unfortunately, as far as I know, there's no other choice but to
study the source code, since there's no other more readable
description of this implementation.

Maybe there are fewer "abstraction leaks" in 3.0...

~kj
 
Reply With Quote
 
 
 
 
Adam Tauno Williams
Guest
Posts: n/a
 
      12-22-2010
On Wed, 2010-12-22 at 14:20 +0000, kj wrote:
> In a message (<4cf97c94$0$30003$c3e8da3$(E-Mail Removed). com>)
> on a different thread, Steven D'Aprano tells me:
> >I suspect you're trying to make this more complicated than it actually
> >is. You keep finding little corner cases that expose implementation
> >details (such as the heap-types issue above) and leaping to the erroneous
> >conclusion that because you didn't understand this tiny little corner of
> >Python's class model, you didn't understand any of it. Python's object
> >model is relatively simple, but it does occasionally expose a few messy
> >corners.

> I disagree with your assessment. What you call "little corner
> cases" I call "fundamental", as in "you can't really call yourself
> competent with Python if you're ignorant about them".
> To use a term I first saw in an article by Joel Spolsky
> (http://is.gd/je42O), Python's object model is a rather "leaky
> abstraction". This refers to the situation in which a user is not
> shielded from the "implementation details". When an abstraction
> leaks, implementation details are no longer negligible, they cease
> to be "little corner cases".
> Here's another example, fresh from today's crop of wonders:
> (v. 2.7.0)
> >>> from collections import Mapping
> >>> issubclass(dict, Mapping)

> True
> >>> dict.__bases__

> (<type 'object'>,)
> >>> [issubclass(b, Mapping) for b in dict.__bases__]

> [False]
> So dict is a subclass of Mapping, even though none of the bases of
> dict is either Mapping or a subclass of Mapping. Great.
> I suspect this is another abstraction leak ("dict is *supposed* to
> be a Python class like all others, but in fact it's not *really*.
> You see, once upon a time...").
> I conclude that, for me to understand Python's (rather leaky) object
> model abstraction, I have to understand its underlying implementation.
> Unfortunately, as far as I know, there's no other choice but to
> study the source code, since there's no other more readable
> description of this implementation.
> Maybe there are fewer "abstraction leaks" in 3.0...


Boy howdy are you going to incite the ire of the Pythonistas!

IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
really a "model" at all [write as many 800 page books as you want: if it
walks like a zombie duck, smells like a zombie duck - it is still a
zombie duck]. Performing introspection in Python is awful and a
veritable land-mine of "implementation details". The short and honest
answer is: avoid doing it whenever possible, try to figure out how to
accomplish the task some other way.


 
Reply With Quote
 
 
 
 
Antoine Pitrou
Guest
Posts: n/a
 
      12-22-2010
On Wed, 22 Dec 2010 14:20:51 +0000 (UTC)
kj <(E-Mail Removed)> wrote:
>
> So dict is a subclass of Mapping, even though none of the bases of
> dict is either Mapping or a subclass of Mapping. Great.
>
> I suspect this is another abstraction leak ("dict is *supposed* to
> be a Python class like all others, but in fact it's not *really*.


It is. You just haven't read about Python's ABCs (abstract base
classes):

http://docs.python.org/library/abc.html#abc.ABCMeta

« You can also register unrelated concrete classes (even built-in
classes) and unrelated ABCs as “virtual subclasses” – these and their
descendants will be considered subclasses of the registering ABC by the
built-in issubclass() function, but the registering ABC won’t show up
in their MRO (Method Resolution Order) nor will method implementations
defined by the registering ABC be callable (not even via super()). »

With a very simple example in the register() doc:

http://docs.python.org/library/abc.h...CMeta.register

Regards

Antoine.


 
Reply With Quote
 
Antoine Pitrou
Guest
Posts: n/a
 
      12-22-2010
On Wed, 22 Dec 2010 09:35:48 -0500
Adam Tauno Williams <(E-Mail Removed)> wrote:
>
> IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
> really a "model" at all [write as many 800 page books as you want: if it
> walks like a zombie duck, smells like a zombie duck - it is still a
> zombie duck]. Performing introspection in Python is awful and a
> veritable land-mine of "implementation details".


Introspection is fine as long as you stick to officially promoted tools
such as isinstance(), issubclass(), dir() or the inspect module.
If you start looking inside the pants of the object model, you can have
surprises

Regards

Antoine.


 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      12-22-2010
On 12/22/2010 9:20 AM, kj wrote:
[...]
> I suspect this is another abstraction leak ("dict is *supposed* to
> be a Python class like all others, but in fact it's not *really*.
> You see, once upon a time...").
>

So your suspicions are to be placed above the knowledge of those who
really do understand Python's object model? That seems like a recipe for
cargo cult programming ...

> I conclude that, for me to understand Python's (rather leaky) object
> model abstraction, I have to understand its underlying implementation.
> Unfortunately, as far as I know, there's no other choice but to
> study the source code, since there's no other more readable
> description of this implementation.
>

You don't have to understand "the implementation" (there are at least
five different implementations, which one will you choose as your standard?)

> Maybe there are fewer "abstraction leaks" in 3.0...
>

Python deliberately exposes introspection interfaces, which you may use
if you wish. As with all introspectable languages (including Java) if
you push the envelope you are likely to hit corner cases. As Steven
d'Aprano has already said, these *are* corner cases and not the whole of
the language.

Don't worry about having a complete knowledge of the language before you
start to use it. That can induce paralysis ...

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
See Python Video! http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

 
Reply With Quote
 
Ethan Furman
Guest
Posts: n/a
 
      12-22-2010
kj wrote:
> In a message (<4cf97c94$0$30003$c3e8da3$(E-Mail Removed). com>)
> on a different thread, Steven D'Aprano tells me:
>
>> I suspect you're trying to make this more complicated than it actually
>> is. You keep finding little corner cases that expose implementation
>> details (such as the heap-types issue above) and leaping to the erroneous
>> conclusion that because you didn't understand this tiny little corner of
>> Python's class model, you didn't understand any of it. Python's object
>> model is relatively simple, but it does occasionally expose a few messy
>> corners.

>
> I disagree with your assessment. What you call "little corner
> cases" I call "fundamental", as in "you can't really call yourself
> competent with Python if you're ignorant about them".


So where on the sliding scale do you place 'competent'? It sounds to me
like you are looking at 'master'.


> Here's another example, fresh from today's crop of wonders:
>
> (v. 2.7.0)
>>>> from collections import Mapping
>>>> issubclass(dict, Mapping)

> True
>>>> dict.__bases__

> (<type 'object'>,)
>>>> [issubclass(b, Mapping) for b in dict.__bases__]

> [False]


Firstly, as I'm sure you know, if you don't import Mapping from
collections the issubclass test fails with a NameError.

Secondly, why do you care? Did you get bitten by something? Some
error, or worse, silently got wrong results? (Sincere question.)


> So dict is a subclass of Mapping, even though none of the bases of
> dict is either Mapping or a subclass of Mapping. Great.
>
> I suspect this is another abstraction leak


My take on abstraction leaks is when the underlying actuality shows
through in a non-ignorable way -- so I ask again, how is this
discrepancy making it so you can't ignore it?

~Ethan~
 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      12-22-2010
On 12/22/2010 9:20 AM, kj wrote:

>>>> from collections import Mapping


Documented as an *ABSTRACT* base class. ABCs were added in 3.0 and
backparted to 2.7. One can be quite competant in Python completely
ignoring ABCs.

>>>> issubclass(dict, Mapping)

> True


Yes, dict is a concrete Mapping class. I suppose we could have instead
added a new builtin function 'isconcretetizationof' but is seemed easier
to reuse issubclass for the test. Most people have no problem with that.

>>>> dict.__bases__

> (<type 'object'>,)


The one and only *CONCRETE* base class. In 3.x, all classes are
subclasses of object, which simplifies the class model a bit.

>>>> [issubclass(b, Mapping) for b in dict.__bases__]

> [False]


> So dict is a subclass of Mapping, even though none of the bases of
> dict is either Mapping or a subclass of Mapping. Great.


Right. dict is direct concrete Mapping implementation and not subclassed
from one.

The main reason for introducing ABCs was to make it easier to test
whether an object passed to a function is an instance of a possibly
unknown or yet-to-be-written class in an abstract category, which has a
certain api or interface. The actual usage of an ABC would be more like
this:

from collections import Mapping
def f(m):
if not isinstance(m, Mapping):
raise ValueError('input is not a mapping')
else: return True

f(dict())
# True
f([])
# produces
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
f([])
File "<pyshell#6>", line 3, in f
raise ValueError('input is not a mapping')
ValueError: input is not a mapping

--
Terry Jan Reedy

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      12-22-2010
On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:

> Here's another example, fresh from today's crop of wonders:
>
> (v. 2.7.0)
>>>> from collections import Mapping
>>>> issubclass(dict, Mapping)

> True
>>>> dict.__bases__

> (<type 'object'>,)
>>>> [issubclass(b, Mapping) for b in dict.__bases__]

> [False]
>
>
> So dict is a subclass of Mapping, even though none of the bases of dict
> is either Mapping or a subclass of Mapping. Great.


Yes. So what?

(1) What *actual* problem does this cause you?

(2) Do you have an example of code that breaks because of this?

(3) Do you understand that since the introduction of ABC (abstract base
classes) in Python 2.6 (I think), isinstance and issubclass checks are
performed cooperatively? The instance or class are asked if they wish to
be known as an instance/subclass of the second argument. Classes can
register themselves as subclasses of (say) Mapping without sharing any
actual code with Mapping.

This is a good thing, and the problem isn't that the abstraction leaks,
as you believe, but the opposite: you're *ignoring* the abstraction and
looking for concrete details that may or may not exist.

I fear that you have fundamentally misunderstood the concept of "leaky
abstraction". It does not mean, as you seem to think, that some concrete
implementation detail differs between two classes (or functions). It
means that some difference in behaviour is exposed, that difference being
irrelevant to the abstraction but nevertheless important in some other
sense. A contrived example:

class MyList(list):
def __len__(self):
import time
time.sleep(3600000)
return list.__len__(self)

MyList can be used anywhere a regular list can be used. Functionally the
two are identical. The abstraction is that MyList is the same as list.
But the leak is that len(MyList()) is *incredibly* slow.


Coming back to Mapping:

Abstraction: issubclass(dict, Mapping)

One possible concrete implementation detail of how issubclass is
implemented:
any(base is Mapping for base in dict.__bases__)


The statement "dict is a subclass of Mapping" is about an abstract
relationship. It's not necessarily a statement about __bases__.

To give an analogy, if you insist on doing DNA testing to determine
whether a boy is a son of a man, you're going to be confused and
distressed every time you find fathers whose sons are genetically
unrelated to them.


--
Steven
 
Reply With Quote
 
kj
Guest
Posts: n/a
 
      12-23-2010
In <4d127d5e$0$29997$c3e8da3$(E-Mail Removed) om> Steven D'Aprano <(E-Mail Removed)> writes:

>On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:


>> Here's another example, fresh from today's crop of wonders:
>>
>> (v. 2.7.0)
>>>>> from collections import Mapping
>>>>> issubclass(dict, Mapping)

>> True
>>>>> dict.__bases__

>> (<type 'object'>,)
>>>>> [issubclass(b, Mapping) for b in dict.__bases__]

>> [False]
>>
>> So dict is a subclass of Mapping, even though none of the bases of dict
>> is either Mapping or a subclass of Mapping. Great.



>Yes. So what?


That's being deliberately obtuse. The situation described goes
smack against standard OOP semantics, which would be fine if all
this stuff was documented clearly and reasonably, i.e. in one
(preferably "official") place rather than scattered over a bazillion
separate documents, PEP this, module that, GvR musing #42, etc.

Let's just say that I'm looking forward to the end to these surprises.

~kj
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      12-23-2010
On Thu, 23 Dec 2010 01:41:08 +0000, kj wrote:

> In <4d127d5e$0$29997$c3e8da3$(E-Mail Removed) om> Steven D'Aprano
> <(E-Mail Removed)> writes:
>
>>On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:

>
>>> Here's another example, fresh from today's crop of wonders:
>>>
>>> (v. 2.7.0)
>>>>>> from collections import Mapping
>>>>>> issubclass(dict, Mapping)
>>> True
>>>>>> dict.__bases__
>>> (<type 'object'>,)
>>>>>> [issubclass(b, Mapping) for b in dict.__bases__]
>>> [False]
>>>
>>> So dict is a subclass of Mapping, even though none of the bases of
>>> dict is either Mapping or a subclass of Mapping. Great.

>
>
>>Yes. So what?

>
> That's being deliberately obtuse. The situation described goes smack
> against standard OOP semantics,



What are these "standard OOP semantics" you're referring to, and who made
them "standard"? If people can't even decide whether multiple inheritance
should be allowed or not, what makes you think that there is any such
thing as "standard OOP"?

I think you are confusing concrete implementation details with the
interface. The interface for subclass testing in Python is as follows:

A class K is a subclass of class C if, and only if,
issubclass(C, K) returns a true result.


That's it. Everything else is implementation. __bases__ is
implementation. __subclasscheck__ is implementation. If Python adds a
third mechanism for implementing subclass checks in version 3.3 or 3.4,
the interface will just continue to work correctly.



> which would be fine if all this stuff
> was documented clearly and reasonably, i.e. in one (preferably
> "official") place rather than scattered over a bazillion separate
> documents, PEP this, module that, GvR musing #42, etc.


This is documented, in the docs.

http://docs.python.org/reference/dat...ubclass-checks

Nobody says the Python docs are perfect, but most things you ask are in
there. Patches for the docs to improve them are welcome.



--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments