Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Inheritance and name clashes

Reply
Thread Tools

Inheritance and name clashes

 
 
Carl Banks
Guest
Posts: n/a
 
      10-04-2010
On Oct 3, 3:04*pm, Rock <(E-Mail Removed)> wrote:
> No, I was just working with a normal library
> class which was supposed to be derived. So that's what I did, but in
> the process I found myself needing to create an instance variable and
> it dawned on me: "how do I know I'm not clobbering something
> here???" ... I'd have to look at the docs, right? But I still wasn't
> sure ... so, then I thought "let's look at the source", and then I
> found out. But! It took me some time to make sure, and I was puzzled
> as well. I mean, what if I have no source to look at? What if the
> library I'm using doesn't realase the source, or what if I just can't
> get my hands on it for some reason or another?
>
> That was a big disappointment with Python for sure. Somehow PHP makes
> me feel a little safer, in that respect at least.


Name collisions are only one of several pitfalls that can happen when
you subclass a third-party clasa. Another pitfall is uncertainly over
which methods a certain behavior is implemented, which is something
you can't determine from the interface alone, in any language I'm
aware of. If you want to override such a behavior you have to look at
the class's implementation to find out how. Point is, if Python
corrected this "defect", you still wouldn't be "safe" because there
are other dangers, which exist in other langauges, too.

Now, if a class is specifically designed to be subclassed by third-
party users, and it's not using name mangling or some other way to
avoid name collisions, then I would call it defective.


Carl Banks
 
Reply With Quote
 
 
 
 
MRAB
Guest
Posts: n/a
 
      10-04-2010
On 04/10/2010 00:06, Steve Howell wrote:
> On Oct 3, 3:57 pm, Gregory Ewing<(E-Mail Removed)> wrote:
>> Rock wrote:
>>> What if the
>>> library I'm using doesn't realase the source, or what if I just can't
>>> get my hands on it for some reason or another?

>>
>> You can always use dir() on an instance of the class to
>> find out what names it's using.
>>

>
> Indeed but the OP should be aware that dir() only reflects the current
> state of the object.
>

Could something like pyflakes tell you what private attributes there
are, basically looking for self._foo or whatever?
 
Reply With Quote
 
 
 
 
Steve Howell
Guest
Posts: n/a
 
      10-04-2010
On Oct 3, 5:13*pm, Carl Banks <(E-Mail Removed)> wrote:
> On Oct 3, 3:04*pm, Rock <(E-Mail Removed)> wrote:
>
> > No, I was just working with a normal library
> > class which was supposed to be derived. So that's what I did, but in
> > the process I found myself needing to create an instance variable and
> > it dawned on me: "how do I know I'm not clobbering something
> > here???" ... I'd have to look at the docs, right? But I still wasn't
> > sure ... so, then I thought "let's look at the source", and then I
> > found out. But! It took me some time to make sure, and I was puzzled
> > as well. I mean, what if I have no source to look at? What if the
> > library I'm using doesn't realase the source, or what if I just can't
> > get my hands on it for some reason or another?

>
> > That was a big disappointment with Python for sure. Somehow PHP makes
> > me feel a little safer, in that respect at least.

>
> Name collisions are only one of several pitfalls that can happen when
> you subclass a third-party clasa. *Another pitfall is uncertainly over
> which methods a certain behavior is implemented, which is something
> you can't determine from the interface alone, in any language I'm
> aware of. *If you want to override such a behavior you have to look at
> the class's implementation to find out how. *Point is, if Python
> corrected this "defect", you still wouldn't be "safe" because there
> are other dangers, which exist in other langauges, too.
>
> Now, if a class is specifically designed to be subclassed by third-
> party users, and it's not using name mangling or some other way to
> avoid name collisions, then I would call it defective.
>


Yep, and a well-designed library can go a long way toward avoiding
name collisions simply by providing an API that facilitates logical
extensions. If a library knows when it needs to delegate to a caller,
it can have a callback mechanism. If a library knows how subclasses
are likely to change its state, it can provide an API that lowers the
temptation to muck with internals.

 
Reply With Quote
 
Steve Howell
Guest
Posts: n/a
 
      10-04-2010
On Oct 3, 5:17*pm, MRAB <(E-Mail Removed)> wrote:
> On 04/10/2010 00:06, Steve Howell wrote:> On Oct 3, 3:57 pm, Gregory Ewing<(E-Mail Removed)> *wrote:
> >> Rock wrote:
> >>> What if the
> >>> library I'm using doesn't realase the source, or what if I just can't
> >>> get my hands on it for some reason or another?

>
> >> You can always use dir() on an instance of the class to
> >> find out what names it's using.

>
> > Indeed but the OP should be aware that dir() only reflects the current
> > state of the object.

>
> Could something like pyflakes tell you what private attributes there
> are, basically looking for self._foo or whatever?


Using pyflakes has the same problem as dir(). It can always be
thwarted by the dynamic nature of the language.

I think it's more worthwhile to educate Python newbies on how to
choose libraries. Well-written libraries can often have tremendous
reuse without the need for subclassing in the first place, by allowing
for callbacks or rich configuration. Libraries that choose
subclassing as their primary extension mechanism should be selected by
how well they hide their own internal state and by how well they
provide a mechanism to store state in the subclasses. There are lots
of techniques for library writers to employ here--naming conventions,
delegation, readable documentation, APIs for the subclass to call back
to, etc.




 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      10-04-2010
On Sun, 03 Oct 2010 15:04:17 -0700, Rock wrote:

> Thanks for the reply. No, I was just working with a normal library class
> which was supposed to be derived. So that's what I did, but in the
> process I found myself needing to create an instance variable and it
> dawned on me: "how do I know I'm not clobbering something here???"


Because then all your tests will start failing.

What's that you say? You have no tests? Then how do you know your program
does what you think it does?



[...]
> That was a big disappointment with Python for sure. Somehow PHP makes me
> feel a little safer, in that respect at least.


Oh my, that's hilarious! Nice one! PHP feels safer, *grins like a loon*



--
Steven
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      10-04-2010
On Mon, 04 Oct 2010 11:57:18 +1300, Gregory Ewing wrote:

> Rock wrote:
>> What if the
>> library I'm using doesn't realase the source, or what if I just can't
>> get my hands on it for some reason or another?

>
> You can always use dir() on an instance of the class to find out what
> names it's using.


*Some* names that it is using.

dir() is meant to be "interesting" rather than complete, so it may not
display all attribute names, and it won't list dynamic attributes
controlled by __getattr__ and friends. See the Fine Manual:

http://docs.python.org/library/functions.html#dir



--
Steven
 
Reply With Quote
 
Chris Torek
Guest
Posts: n/a
 
      10-04-2010
In article <(E-Mail Removed)>
Rock <(E-Mail Removed)> wrote:
>I've really been wondering about the following lately. The question is
>this: if there are no (real) private or protected members in Python,
>how can you be sure, when inheriting from another class, that you
>won't wind up overriding, and possibly clobbering some important data
>field of the parent class, which might compromise its entire
>functionality?


You are right, but this is a double-edged feature/bug as it allows
you to override stuff deliberately, including things the original
code-writer did not think you should be able to override. You
cannot get this particular benefit without a corresponding cost.

For instance (note, this may have been changed in a later version,
I am using rather old code for this particular project) I ran into
an issue with the SimpleXMLRPCServer code (really BaseHTTPServer),
which I fixed with the pipewrap() code below. I also needed a
number of extra features; see comments. To do this I had to make
use of a number of things that were not officially exported, nor
really designed to be override-able.

(I've snipped out some [I think] irrelevant code below, but left
enough to illustrate all this.)

--------

import socket, SocketServer, SimpleXMLRPCServer, xmlrpclib
import errno

def format_ipaddr(addr, do_reverse_lookup = True):
(host, port) = addr[:2]
fqdn = socket.getfqdn(host) if do_reverse_lookup else ''
return '%s[%s]' % (fqdn, host)

# For exceptions to be passed back to rpc caller (other exceptions
# are caught in the MgrServer and logged), use MgrError (or xmlrpclib.Fault).
class MgrError:
def __init__(self, exc_type = None, exc_val = None):
self.exc_type = exc_type
self.exc_val = exc_val
if exc_type is None or exc_val is None:
i = sys.exc_info()[:2]
if exc_type is None:
self.exc_type = i[0]
if exc_val is None:
self.exc_val = i[1]


# This gives us an opportunity to fix the "broken pipe" error that
# occurs when a client disconnects in mid-RPC.
# XXX I think this should really be done in BaseHTTPServer or similar.
# See also <http://trac.edgewall.org/ticket/1183>.
#
# However, we'd still want to override do_POST so that we can
# sneak the client address to the _dispatch function in self.server.
class pipe_eating_rqh(SimpleXMLRPCServer.SimpleXMLRPCReq uestHandler):
def pipewrap(self, f):
try:
f(self)
except socket.error, (code, msg):
if (code == errno.EPIPE or code == errno.ECONNRESET or
code == 10053): # 10053 for Windows
# self.log_message('Lost connection to client: %s',
# self.address_string())
logger.info('Lost connection to client: %s',
format_ipaddr(self.client_address))
else:
raise

def do_POST(self):
self.server.client_address = self.client_address
return self.pipewrap(
SimpleXMLRPCServer.SimpleXMLRPCRequestHandler.do_P OST)
def report_404(self):
return self.pipewrap(
SimpleXMLRPCServer.SimpleXMLRPCRequestHandler.repo rt_404)

class MgrServer(SocketServer.ThreadingMixIn,
SimpleXMLRPCServer.SimpleXMLRPCServer):
"""
The "Manager Server" adds a few things over a basic XML-RPC server:

- Uses the threading mix-in to run each rpc request in a
separate thread.
- Runs an (optional) periodic handler.
- Logs all requests with the logger.
- Handles "admin" requests specially.
- Logs "unexpected" exceptions, so that we catch server bugs
in the server log.
"""

def __init__(self, addr, periodic = None, *args, **kwargs):
SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(sel f, addr,
requestHandler = pipe_eating_rqh,
*args, **kwargs)

# Note, can't just change self.funcs[] into a dict of
# tuples without overriding system_methodHelp() too,
# so we'll use a separate parallel dictionary.
self.admin_label = {}

self.periodic = periodic
if periodic:
self.socket.settimeout(periodic[0])
# see __nonzero__ below
self.register_function(self.__nonzero__)

def get_request(self):
while True:
try:
result = self.socket.accept()
except socket.timeout:
self.periodic[1](*self.periodic[2])
else:
return result
# not reached

def _dispatch(self, method, params):
# Taken from SimpleXMLRPCServer.py but then stripped down and
# modified.
if method in self.admin_label:
... stuff snipped out here ...
try:
func = self.funcs[method]
except KeyError:
# regular SimpleXMLRPCServer checks for self.instance
# and if so, for its _dispatch here ... we're not using
# that so I omit it
func = None
if func is not None:
logger.debug('%s: %s%s',
format_ipaddr(self.client_address), method, str(params))
try:
return func(*params)
except MgrError, e:
# Given, e.g., MgrError(ValueError('bad value'))),
# send the corresponding exc_type / exc_val back
# via xmlrpclib, which transforms it into a Fault.
raise e.exc_type, e.exc_val
except xmlrpclib.Fault:
# Already a Fault, pass it back unchanged.
raise
except TypeError, e:
# If the parameter count did not match, we will get
# a TypeError with the traceback ending with our own
# call at "func(*params)". We want to pass that back,
# rather than logging it.
#
# If the TypeError happened inside func() or one of
# its sub-functions, the traceback will continue beyond
# here, i.e., its tb_next will not be None.
if sys.exc_info()[2].tb_next is None:
raise
# else fall through to error-logging code
except:
pass # fall through to error-logging code

# Any other exception is assumed to be a bug in the server.
# Log a traceback for server debugging.
# is logger.error exc_info thread-safe? let's assume so
logger.error('internal failure in %s', method, exc_info = True)
# traceback.format_exc().rstrip()
raise xmlrpclib.Fault(2000, 'internal failure in ' + method)
else:
logger.info('%s: bad request: %s%s',
format_ipaddr(self.client_address), method, str(params))
raise Exception('method "%s" is not supported' % method)

# Tests of the form:
# c = new_class_object(params)
# if c: ...
# are turned into calls to the class's __nonzero__ method.
# We don't do "if server:" in our own server code, but if we did
# this would get called, and it's reasonable to just define it as
# True. Probably the existing SimpleXMLRPCServer (or one of its
# base classes) should have done this, but they did not.
#
# For whatever reason, the xml-rpc library routines also pass
# a client's __nonzero__ (on his server proxy connection) to us,
# which reaches our dispatcher above. By registering this in
# our __init__, clients can do "if server:" to see if their
# connection is up. It's a frill, I admit....
def __nonzero__(self):
return True

def register_admin_function(self, f, name = None):
... more stuff snipped out ...

# --END-- threading XML RPC server code
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (4039.22'N, 11150.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html
 
Reply With Quote
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      10-04-2010
On Sun, 3 Oct 2010 15:04:17 -0700 (PDT), Rock <(E-Mail Removed)>
declaimed the following in gmane.comp.python.general:

>
> Thanks for the reply. No, I was just working with a normal library
> class which was supposed to be derived. So that's what I did, but in
> the process I found myself needing to create an instance variable and
> it dawned on me: "how do I know I'm not clobbering something
> here???" ... I'd have to look at the docs, right? But I still wasn't
> sure ... so, then I thought "let's look at the source", and then I
> found out. But! It took me some time to make sure, and I was puzzled
> as well. I mean, what if I have no source to look at? What if the
> library I'm using doesn't realase the source, or what if I just can't
> get my hands on it for some reason or another?
>

Well... then use the double __ prefix so your instance variables
have a class specific name... That IS what the __ are designed to do --
ensure that the name, as used /in/ that class itself is unique, and not
referencing something from some unknown parent class.
--
Wulfraed Dennis Lee Bieber AF6VN
http://www.velocityreviews.com/forums/(E-Mail Removed) HTTP://wlfraed.home.netcom.com/

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      10-04-2010
On Sun, 03 Oct 2010 21:32:25 -0700, Dennis Lee Bieber wrote:

> Well... then use the double __ prefix so your instance variables
> have a class specific name... That IS what the __ are designed to do --
> ensure that the name, as used /in/ that class itself is unique, and not
> referencing something from some unknown parent class.



Unfortunately name mangling doesn't *quite* do that. It is still possible
to have name clashes.

Imagine you inherit from a class Ham:


from luncheon_meats import Ham

class Spam(Ham):
def __init__(self):
self.__x = "yummy"


You think you're safe, because the attribute __x is mangled to _Spam__x.
But little do you know, Ham itself inherits from another class Meat,
which inherits from Food, which inherits from FoodLikeProducts, which
inherits from... Spam. Which also has an __x attribute.

You now have a name clash.



--
Steven
 
Reply With Quote
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      10-04-2010
On 04 Oct 2010 05:08:20 GMT, Steven D'Aprano
<(E-Mail Removed)> declaimed the following in
gmane.comp.python.general:

>
> from luncheon_meats import Ham
>
> class Spam(Ham):
> def __init__(self):
> self.__x = "yummy"
>
>
> You think you're safe, because the attribute __x is mangled to _Spam__x.
> But little do you know, Ham itself inherits from another class Meat,
> which inherits from Food, which inherits from FoodLikeProducts, which
> inherits from... Spam. Which also has an __x attribute.
>
> You now have a name clash.


And you've just defined a circular inheritance tree -- I wouldn't
expect ANY language to handle such... Presuming it doesn't choke on the
parsing. Or a very confusing set of imports where you are
re-using/re-defining a class name itself.

--
Wulfraed Dennis Lee Bieber AF6VN
(E-Mail Removed) HTTP://wlfraed.home.netcom.com/

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Dealing with name clashes in pypi Patrick Sabin Python 0 05-22-2011 06:34 PM
Modular classes and avoiding extension clashes Trans Ruby 0 06-21-2007 08:17 PM
Clashes between type conversion and operator[] Arne Schmitz C++ 6 11-14-2005 01:46 PM
Embedding ruby - name clashes Brian Candler Ruby 0 07-22-2003 05:08 PM
Why aren't signed/unsigned type clashes checked? Clint Olsen C Programming 10 07-11-2003 01:38 PM



Advertisments