Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Class extension confusion :( (http://www.velocityreviews.com/forums/t737613-class-extension-confusion.html)

r0g 11-10-2010 05:45 AM

Class extension confusion :(
 
I have a subclass of BaseHHTPRequestHandler which uses a dictonary
"paths" and a function "api_call" which are defined in the main
namespace of the module. I'd rather I was able to pass these object to
the constructor and store them as data attributes "self.paths" and
"self.api_call" but I'm not sure how to do that properly. My
understanding is that one may extend a constructor by defining it's
__init__ method, calling the parents constructor and then adding ones
own attributes to taste. What I don't understand is where or how I am
supposed to get these extra constructor arguments into the class given
that I don't instantiate it myself, it is seemingly instantiated by
HTTPServer class that I pass it to e.g.

httpd = HTTPServer(server_address, PlainAJAXRequestHandler)

I wondered if I ought to instantiate an instance of
PlainAJAXRequestHandler, set the attributes (either manually or by
extending it's constructor) and pass that to HTTPServer but I figured it
expects a class not an instance as it probably wants to spawn one
instance for each request so that would be a non starter. Might I need
to subclass HTTPServer, find the bit that instantiates the request
handler and override that so it passes it's constructor more parameters?
Right now I'm pretty confused, can somebody please tell me how I might
accomplish this, what I'm failing to grasp or point me to the docs that
explain it - I've spent the last hour or two plowing through docs to no
avail, I guess it's a case of keyword ignorance on my part! Code follows...

Thanks for reading!

Roger.



class PlainAJAXRequestHandler(BaseHTTPRequestHandler):

paths = { "/": pages.main,
"/jqtest/": pages.jqtest
}

def do_GET(self):

# Handle JSON api calls
if self.path[:6] == "/ajax?":
getvars = urlparse.parse_qs( self.path[6:] )
api_key = getvars[ "api" ][0]
json_string = getvars[ "qry" ][0]
json_object = json.loads( json_string )
response = api_call( api_key, json_object )
if response:
self.send_response(200)
self.send_header("Content-type", "application/json")
self.end_headers()
self.wfile.write( response )
else:
self.send_response(404)
self.end_headers()
return

# Handle web pages
try:
page = self.paths[self.path]()
except KeyError:
self.send_response(404)
self.end_headers()
self.wfile.write( "404 - Document not found!" )
return
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write( page )
return

Peter Otten 11-10-2010 09:52 AM

Re: Class extension confusion :(
 
r0g wrote:

> I have a subclass of BaseHHTPRequestHandler which uses a dictonary
> "paths" and a function "api_call" which are defined in the main
> namespace of the module. I'd rather I was able to pass these object to
> the constructor and store them as data attributes "self.paths" and
> "self.api_call" but I'm not sure how to do that properly. My
> understanding is that one may extend a constructor by defining it's
> __init__ method, calling the parents constructor and then adding ones
> own attributes to taste. What I don't understand is where or how I am
> supposed to get these extra constructor arguments into the class given
> that I don't instantiate it myself, it is seemingly instantiated by
> HTTPServer class that I pass it to e.g.
>
> httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
>
> I wondered if I ought to instantiate an instance of
> PlainAJAXRequestHandler, set the attributes (either manually or by
> extending it's constructor) and pass that to HTTPServer but I figured it
> expects a class not an instance as it probably wants to spawn one
> instance for each request so that would be a non starter. Might I need
> to subclass HTTPServer, find the bit that instantiates the request
> handler and override that so it passes it's constructor more parameters?
> Right now I'm pretty confused, can somebody please tell me how I might
> accomplish this, what I'm failing to grasp or point me to the docs that
> explain it - I've spent the last hour or two plowing through docs to no
> avail, I guess it's a case of keyword ignorance on my part! Code
> follows...


Try passing a factory function instead of a class. Untested:

from functools import partial

class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
def __init__(self, api_call, paths, *args, **kw):
BaseHTTPRequestHandler.__init__(self, *args, **kw)
self.api_call = api_call
self.paths = paths

paths = ...
api_call = ...

httpd = HTTPServer(
server_address,
partial(PlainAJAXRequestHandler, paths, api_call))


r0g 11-11-2010 04:22 AM

Re: Class extension confusion :(
 
On 10/11/10 09:52, Peter Otten wrote:
> r0g wrote:
>
>> I have a subclass of BaseHHTPRequestHandler which uses a dictonary
>> "paths" and a function "api_call" which are defined in the main
>> namespace of the module. I'd rather I was able to pass these object to
>> the constructor and store them as data attributes "self.paths" and
>> "self.api_call" but I'm not sure how to do that properly. My
>> understanding is that one may extend a constructor by defining it's
>> __init__ method, calling the parents constructor and then adding ones
>> own attributes to taste. What I don't understand is where or how I am
>> supposed to get these extra constructor arguments into the class given
>> that I don't instantiate it myself, it is seemingly instantiated by
>> HTTPServer class that I pass it to e.g.
>>
>> httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
>>
>> I wondered if I ought to instantiate an instance of
>> PlainAJAXRequestHandler, set the attributes (either manually or by
>> extending it's constructor) and pass that to HTTPServer but I figured it
>> expects a class not an instance as it probably wants to spawn one
>> instance for each request so that would be a non starter. Might I need
>> to subclass HTTPServer, find the bit that instantiates the request
>> handler and override that so it passes it's constructor more parameters?
>> Right now I'm pretty confused, can somebody please tell me how I might
>> accomplish this, what I'm failing to grasp or point me to the docs that
>> explain it - I've spent the last hour or two plowing through docs to no
>> avail, I guess it's a case of keyword ignorance on my part! Code
>> follows...

>
> Try passing a factory function instead of a class. Untested:
>
> from functools import partial
>
> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
> def __init__(self, api_call, paths, *args, **kw):
> BaseHTTPRequestHandler.__init__(self, *args, **kw)
> self.api_call = api_call
> self.paths = paths
>
> paths = ...
> api_call = ...
>
> httpd = HTTPServer(
> server_address,
> partial(PlainAJAXRequestHandler, paths, api_call))
>



Great, that looks ideal Peter, thanks very much :)

One more question quick question if you have time... I actually subclass
the class in question later on to provide an SSL secured connection, is
it possible (or even advisable) to do the following to spare me having
to modify the derived classes init (and any subsequent classes' inits)
too?...

PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths, api_call))

^??? Remap name to new extended class

class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
def setup(self):
self.connection = self.request
self.rfile = socket._fileobject(self.request, "rb", self.rbufsize)
self.wfile = socket._fileobject(self.request, "wb", self.wbufsize)

^??? therefore no init needed?


Cheers,

Roger


Peter Otten 11-11-2010 08:01 AM

Re: Class extension confusion :(
 
r0g wrote:

> On 10/11/10 09:52, Peter Otten wrote:
>> r0g wrote:
>>
>>> I have a subclass of BaseHHTPRequestHandler which uses a dictonary
>>> "paths" and a function "api_call" which are defined in the main
>>> namespace of the module. I'd rather I was able to pass these object to
>>> the constructor and store them as data attributes "self.paths" and
>>> "self.api_call" but I'm not sure how to do that properly. My
>>> understanding is that one may extend a constructor by defining it's
>>> __init__ method, calling the parents constructor and then adding ones
>>> own attributes to taste. What I don't understand is where or how I am
>>> supposed to get these extra constructor arguments into the class given
>>> that I don't instantiate it myself, it is seemingly instantiated by
>>> HTTPServer class that I pass it to e.g.
>>>
>>> httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
>>>
>>> I wondered if I ought to instantiate an instance of
>>> PlainAJAXRequestHandler, set the attributes (either manually or by
>>> extending it's constructor) and pass that to HTTPServer but I figured it
>>> expects a class not an instance as it probably wants to spawn one
>>> instance for each request so that would be a non starter. Might I need
>>> to subclass HTTPServer, find the bit that instantiates the request
>>> handler and override that so it passes it's constructor more parameters?
>>> Right now I'm pretty confused, can somebody please tell me how I might
>>> accomplish this, what I'm failing to grasp or point me to the docs that
>>> explain it - I've spent the last hour or two plowing through docs to no
>>> avail, I guess it's a case of keyword ignorance on my part! Code
>>> follows...

>>
>> Try passing a factory function instead of a class. Untested:
>>
>> from functools import partial
>>
>> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
>> def __init__(self, api_call, paths, *args, **kw):
>> BaseHTTPRequestHandler.__init__(self, *args, **kw)
>> self.api_call = api_call
>> self.paths = paths
>>
>> paths = ...
>> api_call = ...
>>
>> httpd = HTTPServer(
>> server_address,
>> partial(PlainAJAXRequestHandler, paths, api_call))
>>

>
>
> Great, that looks ideal Peter, thanks very much :)
>
> One more question quick question if you have time... I actually subclass
> the class in question later on to provide an SSL secured connection, is
> it possible (or even advisable) to do the following to spare me having
> to modify the derived classes init (and any subsequent classes' inits)
> too?...
>
> PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths,
> api_call))
>
> ^??? Remap name to new extended class
>
> class SecureAJAXRequestHandler(PlainAJAXRequestHandler):


That's not possible; you are trying to subclass a function.
What you can do (again untested):

def make_handler_class(paths, api_call):
class MyHandler(BaseHTTPRequestHandler):
paths = paths
api_call = staticmethod(api_call)
return MyHandler

PlainAJAXRequestHandler = make_handler_class(paths, api_call)

class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
...

Peter

r0g 11-11-2010 08:56 AM

Re: Class extension confusion :(
 
On 10/11/10 09:52, Peter Otten wrote:
> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
> def __init__(self, api_call, paths, *args, **kw):
> BaseHTTPRequestHandler.__init__(self, *args, **kw)
> self.api_call = api_call
> self.paths = paths



Hmm, the plot thickens! I always thought you had to call the parent
constructor first (as above) when extending a constructor (not that I've
had occasion to do that in a long time), but it turns out when I do this
the lines below it never get executed and when I move them above that
line they seem to work fine so it appears I was wrong about that. I've
tried typing many variants of "python class extend constructor" into
google over the last few days but I'm damned if I can find the docs
explaining this. I'm sure I found them several years back when I first
too up python, maybe by google-fu is on the wane!

Anyway, that's not my main question, this is... The extra names that I
pass to functools.partial seem to be bound permanently into the
namespace of my class now i.e. I can reference them as 'api_call' and
'paths' anywhere in the classes' methods as opposed to having to assign
them in the constructor and reference them as 'self.api_call' and
'self.paths'. I'm not 100% how that's working but in practical terms it
suggests to two lines assigning those names to data attributes are
redundant as I can access them anywhere anyway. Indeed, I've commented
them out and my app still seems to work fine so...

Question A) Are there any good reasons why I shouldn't just do that?
(other than B!)

Question B) The only reason I can think of so far is that I don't have a
clear picture of how those names came to end up in that scope, it seems
very convenient but I'm worried it's black magic of some sort! Could
anyone explain or point me to the right docs please?

Please shout if you would like to see the source.

Thanks muchly :)


Roger.



Peter Otten 11-11-2010 09:34 AM

Re: Class extension confusion :(
 
r0g wrote:

> On 10/11/10 09:52, Peter Otten wrote:
>> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
>> def __init__(self, api_call, paths, *args, **kw):
>> BaseHTTPRequestHandler.__init__(self, *args, **kw)
>> self.api_call = api_call
>> self.paths = paths

>
>
> Hmm, the plot thickens! I always thought you had to call the parent
> constructor first (as above) when extending a constructor (not that I've
> had occasion to do that in a long time), but it turns out when I do this
> the lines below it never get executed and when I move them above that
> line they seem to work fine so it appears I was wrong about that. I've


I didn't believe you until I had a look into the source. The meat is in
SocketServer.py:

class BaseRequestHandler:

[snip]

def __init__(self, request, client_address, server):
self.request = request
self.client_address = client_address
self.server = server
try:
self.setup()
self.handle()
self.finish()
finally:
sys.exc_traceback = None # Help garbage collection

def setup(self):
pass

def handle(self):
pass

def finish(self):
pass


As you can see this doesn't use __init__() just to set up the instance, it
makes it the only method that is called by client code.

That's an unusual design decision, to say the least.

> tried typing many variants of "python class extend constructor" into
> google over the last few days but I'm damned if I can find the docs
> explaining this. I'm sure I found them several years back when I first
> too up python, maybe by google-fu is on the wane!
>
> Anyway, that's not my main question, this is... The extra names that I
> pass to functools.partial seem to be bound permanently into the
> namespace of my class now i.e. I can reference them as 'api_call' and
> 'paths' anywhere in the classes' methods as opposed to having to assign
> them in the constructor and reference them as 'self.api_call' and
> 'self.paths'. I'm not 100% how that's working but in practical terms it
> suggests to two lines assigning those names to data attributes are
> redundant as I can access them anywhere anyway. Indeed, I've commented
> them out and my app still seems to work fine so...
>
> Question A) Are there any good reasons why I shouldn't just do that?
> (other than B!)


No. Use either

class B(A):
path = ...

or

class B(A):
def __init__(self, path, *args, **kw):
self.path = path
A.__init__(self, *args, **kw)

not both.

> Question B) The only reason I can think of so far is that I don't have a
> clear picture of how those names came to end up in that scope, it seems
> very convenient but I'm worried it's black magic of some sort! Could
> anyone explain or point me to the right docs please?


Python looks for attributes in the instance first, and then in the class as
a fallback. You only need to put them in the instance if you expect that you
want a different value for every instance.

Peter


r0g 11-11-2010 07:34 PM

Re: Class extension confusion :(
 
On 11/11/10 09:34, Peter Otten wrote:
> r0g wrote:
>
>> Question B) The only reason I can think of so far is that I don't have a
>> clear picture of how those names came to end up in that scope, it seems
>> very convenient but I'm worried it's black magic of some sort! Could
>> anyone explain or point me to the right docs please?

>
> Python looks for attributes in the instance first, and then in the class as
> a fallback. You only need to put them in the instance if you expect that you
> want a different value for every instance.
>
> Peter
>



Thanks Peter, I think I understand now, I'll quickly explain my picture
of things though so you or some other denizens of the mighty
comp.lang.python can correct me if I'm wrong!...

I can access the parameters I pass to __init__ within the classes'
methods without using self. as these methods are run within the
constructor itself and are therefore within it's local scope.

That also explains why I had to call the constructor _after_ creating
the new data attributes to have them be included. My initial belief that
one has to call the parent constructor as the first action in the
extended constructor is not technically valid, in most cases it can be
called at any point but in situations like the above it can mess things
up so I ought to put it at the end of my constructors for it to always
work. Are there any drawbacks to calling it last or is that how it is
supposed to work?

The BaseHTTPRequestHandler seemed curiously constructed to me at first
too but I think I can see why now... as it's a handler it's only ever
meant to be a transient thing and it requires no external parameters
(unless you're me!) as it's always called by HTTPServer which has all
the info necessary so there's be no sense requiring users to create an
instance then call a method to get the servers response, you may as well
just bundle it all into one. Also as it might be called hundreds or
thousands of times in quick succession it would be important to have it
finish and get garbage collected.

I also see that I ought to at least assign these extended params to data
attributes and access them via self. like I would normally do as I can't
guarantee that implementation of BaseHTTPREquestHandler will remain
constant over time. If I understand correctly it may also be possible
(and more efficient) to use setattr() to inject the parameters I want
into the class as class attributes before use, rather than assigning
them to data attributes every time I instantiate an new instance.

Actually looking at the code from BaseRequestHandler it seems I may have
overlooked the proper way of associating a callback function anyway. It
looks like I should just define self.handle() in my subclass so that's
one less parameter to worry about, although I think I may still need to
use the factory function method you showed me to get the "paths"
dictionary in there.

Thanks so much for all your help, I really appreciate it and, assuming
I'm not totally wrong about all of the above, I find it reassuring that
I'm not going mad!

Cheers,

Roger.

r0g 11-11-2010 09:17 PM

Re: Class extension confusion :(
 
On 11/11/10 19:34, r0g wrote:
> On 11/11/10 09:34, Peter Otten wrote:
>> r0g wrote:

> If I understand correctly it may also be possible
> (and more efficient) to use setattr() to inject the parameters I want
> into the class as class attributes before use, rather than assigning
> them to data attributes every time I instantiate an new instance.


Ah, it's even simpler than that, turns out I've been significantly
overcomplicating things, I can just set the class attributes by calling
classname.attributename = whatever once at the start of my script. The
one thing I might need to change as the server runs is the paths
dictionary but dicts are mutable so that's no problem and I don't need
to inject any data into the instances at all. I'm guessing I can
subclass that easily now too - happy days!

Well, far from wasted time it's been a very educational diversion, cheers :)


Roger.


All times are GMT. The time now is 04:34 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.