Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > use of __new__ to permit "dynamic" completion within (any?) IDE ?

Reply
Thread Tools

use of __new__ to permit "dynamic" completion within (any?) IDE ?

 
 
gst
Guest
Posts: n/a
 
      12-07-2010
Hi,

I met a situation where I was passing an object created in/with an
upper level module class to a lower level module class' instance in
one of its __init__ argument and saving a ref of the upper object in
that lower level class' new instance.

But in my IDE I want the completion to also work from within the lower
level module when it's refering to the object passed from the upper
level:


Well, I'm sure I'm not very clear in my terms (and probably a bit long
in the sentence) so here it is in code:


files:
module1.py
subpackage/module2.py


file module1.py:

from subpackage.module2 import class2

class class1(object):

def __new__(cls, _self=None, *args, **kwargs):
if _self: ## we've been passed an instance already
initialyzed
## so directly return it instead of creating a
new object.
return _self
return object.__new__(cls)

def __init__(self, _self=None, *args, **kwargs):
if _self: ## we've been passed an instance already
initialyzed
## so directly returns
## assert(self is _self) ?
return
self.object2 = class2(object1=self, "blip", "blop")
# others init


file module2.py:

class class2(object):

def __init__(self, object1, *args, **kwargs):

from ..module1 import class1

self.object1 = class1(_self=object1) ## instead of:
self.object1 = object1

## others functions and/or init..
## where now I've completion working on self.object1 :
## if I add(or remove) fields/methods in module1 (and save) then
## I have them available(or disappeared) in the completion when
executed from this submodule.
## This ofcourse permits to save me of remembering all of the
class1 attributes/methods when I'm working with self.object1 from
within one of class2 methods.


What do you think of this ?

I guess there can be others ways of doing this.. ?

Thanks,

Regards,

Greg.

note: I'm using Eclipse 3.5.2 with pydev so (I don't really know how
others IDE can handle this case) .

 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      12-08-2010
On Tue, 07 Dec 2010 07:52:06 -0800, gst wrote:

> Hi,
>
> I met a situation where I was passing an object created in/with an upper
> level module class to a lower level module class' instance in one of its
> __init__ argument and saving a ref of the upper object in that lower
> level class' new instance.
>
> But in my IDE I want the completion to also work from within the lower
> level module when it's refering to the object passed from the upper
> level:


"The completion"? What do you mean?


> Well, I'm sure I'm not very clear in my terms (and probably a bit long
> in the sentence) so here it is in code:


I'm afraid I have to agree with your, your description is not clear to me.

Unfortunately, neither is your code, because I don't understand *why* you
do the things you do.


> files:
> module1.py
> subpackage/module2.py
>
>
> file module1.py:
>
> from subpackage.module2 import class2


Is it relevant that class2 comes from another module? Would your concept
work if class1 and class2 were defined in the same file?

By the way, it's conventional (but not compulsory) to name classes with
an initial capital letter, and instances in all lowercase.



> class class1(object):
>
> def __new__(cls, _self=None, *args, **kwargs):
> if _self: ## we've been passed an instance already
> initialyzed
> ## so directly return it instead of creating a
> new object.
> return _self
> return object.__new__(cls)


Depending on your application, this may be an unsafe assumption. If the
caller says class1("spam"), your class will return "spam". This may or
may not be what you want.


> def __init__(self, _self=None, *args, **kwargs):
> if _self: ## we've been passed an instance already
> initialyzed
> ## so directly returns
> ## assert(self is _self) ?
> return
> self.object2 = class2(object1=self, "blip", "blop") # others
> init


Okay, this seems like a fairly straightforward case of adding an
attribute to your instance that includes a reference to itself. No big
deal... there's nothing wrong with this, although you are creating a
reference cycle, which is somewhat of a (mild) code-smell.


> file module2.py:
>
> class class2(object):
>
> def __init__(self, object1, *args, **kwargs):
>
> from ..module1 import class1


Now you have a circular import, and that's a pretty major code smell.
That's dangerous. Try to avoid it. Perhaps the easiest way to avoid it is
to place class1 and class2 in the same module, and stop writing Java


> self.object1 = class1(_self=object1) ## instead of:
> self.object1 = object1
>
> ## others functions and/or init..
> ## where now I've completion working on self.object1 : ## if I
> add(or remove) fields/methods in module1 (and save) then ## I have
> them available(or disappeared) in the completion when
> executed from this submodule.
> ## This ofcourse permits to save me of remembering all of the
> class1 attributes/methods when I'm working with self.object1 from within
> one of class2 methods.


I'm afraid I don't understand what you mean here in these comments. What
do you mean, "save me of (from?) remembering all the class1 attributes/
methods..." -- how does it save you from knowing the methods? Whether you
write this:

instance = class1()
instance.method()

or this:

instance = class1()
another_instance = class2(instance)
another_instance.object1.method()

you still need to know the name "method".

It seems to me that this scheme is unnecessarily complex, convoluted and
confusing, for very little gain. Perhaps you could try explaining what
you hope to accomplish, giving examples?


--
Steven
 
Reply With Quote
 
 
 
 
gst
Guest
Posts: n/a
 
      12-08-2010
On 8 déc, 11:45, Steven D'Aprano <steve
(E-Mail Removed)> wrote:
> On Tue, 07 Dec 2010 07:52:06 -0800, gst wrote:
> > Hi,

>
> > But in my IDE I want the completion to also work from within the lower
> > level module when it's refering to the object passed from the upper
> > level:

>
> "The completion"? What do you mean?


Hi,

I mean the autocompletion feature of my IDE :

when I type "object." then on the "." it shows me possible members/
methods of the 'object'. depending on what my IDE can guess about the
object possible type(s) ; in the worst case it can't guess nothing
about it and that's my point below..


> > Well, I'm sure I'm not very clear in my terms (and probably a bit long
> > in the sentence) so here it is in code:

>
> I'm afraid I have to agree with your, your description is not clear to me..


I hope I'm now

> Unfortunately, neither is your code, because I don't understand *why* you
> do the things you do.


damn. ok let's go..


> > file module1.py:

>
> > from subpackage.module2 import class2

>
> Is it relevant that class2 comes from another module?


yes. hmmm or maybe not completely.. I think it only highly depends on
what the IDE can guess on an object type used inside some methods
depending on the files hierarchy..


> Would your concept work if class1 and class2 were defined in the same file?


I guess yes also.


> > class class1(object):

>
> > * * def __new__(cls, _self=None, *args, **kwargs):
> > * * * * if _self: *## we've been passed an instance already
> > initialyzed
> > * * * * * * * * * * *## so directly return it instead of creating a
> > new object.
> > * * * * * * return _self
> > * * * * return object.__new__(cls)

>
> Depending on your application, this may be an unsafe assumption. If the
> caller says class1("spam"), your class will return "spam". This may or
> may not be what you want.


I'm always passing the good object type when I do this.


> > * * def __init__(self, _self=None, *args, **kwargs):
> > * * * * if _self: *## we've been passed an instance already
> > initialyzed
> > * * * * * * * * * * *## so directly returns
> > * * * * * * ## assert(self is _self) ?
> > * * * * * * return
> > * * * * self.object2 = class2(object1=self, "blip", "blop") # others
> > * * * * init

>
> Okay, this seems like a fairly straightforward case of adding an
> attribute to your instance that includes a reference to itself. No big
> deal... there's nothing wrong with this, although you are creating a
> reference cycle, which is somewhat of a (mild) code-smell.


that is a worry effectively. (hmm when you say it's a case of adding
an attribute that includes a ref to itself it's not really that (or i
badly understand) : it's adding an attribute to a lower level class
instance that contains a ref to an upper level class instance (which
also contains a ref to the lower level class' instance and that's the
cycle (and that's a bit my worry))

> > file module2.py:

>
> > class class2(object):

>
> > * * def __init__(self, object1, *args, **kwargs):

>
> > * * * * from ..module1 import class1

>
> Now you have a circular import, and that's a pretty major code smell.
> That's dangerous. Try to avoid it. Perhaps the easiest way to avoid it is
> to place class1 and class2 in the same module, and stop writing Java


damn if this looks like Java it's really not my will ; I've not made
Java anymore for probably 10 years !

I'll try by putting class1 and class2 in same module (or at same
directory level first) and see if that can do it..

>
> > * * * * self.object1 = class1(_self=object1) * *## instead of:
> > self.object1 = object1

>
> > * * ## others functions and/or init..
> > * * ## where *now I've completion working on self.object1 : ## if I
> > * * add(or remove) fields/methods in module1 (and save) then ## I have
> > * * them available(or disappeared) in the completion when
> > executed from this submodule.
> > * * ## This ofcourse permits to save me of remembering all of the
> > class1 attributes/methods when I'm working with self.object1 from within
> > one of class2 methods.

>
> I'm afraid I don't understand what you mean here in these comments. What
> do you mean, "save me of (from?) remembering all the class1 attributes/
> methods..." -- how does it save you from knowing the methods? Whether you
> write this:
>
> instance = class1()
> instance.method()
>
> or this:
>
> instance = class1()
> another_instance = class2(instance)
> another_instance.object1.method()
>
> you still need to know the name "method".
>
> It seems to me that this scheme is unnecessarily complex, convoluted and
> confusing, for very little gain. Perhaps you could try explaining what
> you hope to accomplish, giving examples?


Effectively I recognise that this seems like a bit silly/odd .. but as
it's working I wanted some advises so

As I told I try to have some kind of "ultimate" autocompletion, always
working for this kind of case (I know exactly that a certain member of
a certain class' instance will always have the same type but my IDE
doesn't seem to guess/know it.. ; I guess the best answer would be :
bad IDE, change IDE but I'm guessing that others IDE can't do a lot
better in this case.

Anyway many thanks for your reply ; I'll do some further tests with
your comments already..

regards,

greg.

nb: so this "hack" is only relevant during dev ; once the project
would be finished the "hack" could be removed (i.e : in class2 init I
would directly do : self.object1 = object1)
 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      12-08-2010
On 12/8/2010 1:01 PM, gst wrote:
> As I told I try to have some kind of "ultimate" autocompletion, always
> working for this kind of case (I know exactly that a certain member of
> a certain class' instance will always have the same type but my IDE
> doesn't seem to guess/know it.. ; I guess the best answer would be :
> bad IDE, change IDE but I'm guessing that others IDE can't do a lot
> better in this case.
>

If you'd told us which IDE you were using we might have offered better
advice, but you seem to want to keep that a secret ("my IDE" tells us
nothing).

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
See Python Video! http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

 
Reply With Quote
 
gst
Guest
Posts: n/a
 
      12-08-2010
On 8 déc, 14:09, Steve Holden <(E-Mail Removed)> wrote:
>
> If you'd told us which IDE you were using we might have offered better
> advice, but you seem to want to keep that a secret ("my IDE" tells us
> nothing).
>


sorry it was totally bottom of my first message :

> note: I'm using Eclipse 3.5.2 with pydev so (I don't really know how

others IDE can handle this case) .

now I realyze that my question has not a lot to do with python
language itself and I should resubmit that probably on pydev forums/..

but as I told I wanted to have some advises on the "good" (or not) use
of this "way of doing" with python itself. You already explained me
it's not the best so far.. and I agree although during (my) dev it
helps (but I do this only with very few classes that have lot of
attributes/methods ofcourse).

regards,

greg.
 
Reply With Quote
 
Jean-Michel Pichavant
Guest
Posts: n/a
 
      12-08-2010
gst wrote:
> greg.
>
> nb: so this "hack" is only relevant during dev ; once the project
> would be finished the "hack" could be removed (i.e : in class2 init I
> would directly do : self.object1 = object1)
>


Expect some bugs then on the 'release' version.
I'm not sure I understood everything you mentioned in your OP, however
writing python code on class creation to make Eclipse completion 'work'
is ... a very bad idea IMO.
Just don't do it.

quoting eclipse page:

"Pydev [...] uses advanced type inference techniques to provide features
such code completion and code analysis"

I don't know exactly what's hidden behind this marketing stuff. Did you
try to document your method with a markup language supported by Eclipse
(if there is any)?

class class2(object):
def __init__(self, object1, *args, **kwargs):
"""blablabla

@param object1: a L{class1} reference
"""
self.object1 = object1

The docstring would be the only way for an IDE to infer a argument type.
Note that I used the epydoc markup language because it's the only one I
know but it's unlikely supported by Eclipse. You better try
reStructuredText.

JM
 
Reply With Quote
 
gst
Guest
Posts: n/a
 
      12-08-2010
On 8 déc, 15:51, Jean-Michel Pichavant <(E-Mail Removed)> wrote:

Hi,


> gst wrote:
> > nb: so this "hack" is only relevant during dev ; once the project
> > would be finished the "hack" could be removed (i.e : in class2 init I
> > would directly do : self.object1 = object1)

>
> Expect some bugs then on the 'release' version.


even with the "assert(self is _self)" in the "if _self:" condition in
the new and/or init methods of the class1 ?


> I'm not sure I understood everything you mentioned in your OP, however
> writing python code on class creation to make Eclipse completion 'work'
> is ... a very bad idea IMO.
> Just don't do it.


yes I'm trying to find better ways for having completion works out of
the box in this case.. (that would be a high help to my opinion).


> quoting eclipse page:
>
> "Pydev [...] uses advanced type inference techniques to provide features
> such code completion and code analysis"
>
> I don't know exactly what's hidden behind this marketing stuff. Did you
> try to document your method with a markup language supported by Eclipse
> (if there is any)?


pydev completion apparently is restricted (as far as i see) to some
very specific cases (basically it works when you import a module at
top of another one and that you instantiate objects from the imported
module within the init methods of the classes of the module which is
importing the other one (but so "circular" references created on some
objects from the first module (the one imported) in the second one
won't have the completion working for them (that's what I see)).


> class class2(object):
> * * def __init__(self, object1, *args, **kwargs):
> * * """blablabla
>
> * * @param object1: a L{class1} reference
> * * """
> * * self.object1 = object1
>
> The docstring would be the only way for an IDE to infer a argument type.
> Note that I used the epydoc markup language because it's the only one I
> know but it's unlikely supported by Eclipse. You better try
> reStructuredText.
>
> JM


well, I'm not very used to docstrings in fact but it's effectively a
way to achieve this ; well certainly better than doing this kind of
"ugly" hack so.
I'll have a try at that.

Thanks for your reply,

regards,

greg.
 
Reply With Quote
 
gst
Guest
Posts: n/a
 
      12-08-2010
On 8 déc, 16:20, gst <(E-Mail Removed)> wrote:
> On 8 déc, 15:51, Jean-Michel Pichavant <(E-Mail Removed)> wrote:
>
> Hi,
>
> > gst wrote:
> > > nb: so this "hack" is only relevant during dev ; once the project
> > > would be finished the "hack" could be removed (i.e : in class2 init I
> > > would directly do : self.object1 = object1)

>
> > Expect some bugs then on the 'release' version.

>
> even with the "assert(self is _self)" in the "if _self:" condition in
> the new and/or *init methods of the class1 ?


note: I don't know if that matters but I'm using __slots__ in this
class1 ..

greg.

 
Reply With Quote
 
gst
Guest
Posts: n/a
 
      12-08-2010
On 8 déc, 16:20, gst <(E-Mail Removed)> wrote:

> even with the "assert(self is _self)" in the "if _self:" condition in
> the new and/or *init methods of the class1 ?


damn : in the init method only of course..
 
Reply With Quote
 
Jean-Michel Pichavant
Guest
Posts: n/a
 
      12-08-2010

>> quoting eclipse page:
>>
>> "Pydev [...] uses advanced type inference techniques to provide features
>> such code completion and code analysis"
>>
>> I don't know exactly what's hidden behind this marketing stuff. Did you
>> try to document your method with a markup language supported by Eclipse
>> (if there is any)?
>>

>
> pydev completion apparently is restricted (as far as i see) to some
> very specific cases (basically it works when you import a module at
> top of another one and that you instantiate objects from the imported
> module within the init methods of the classes of the module which is
> importing the other one (but so "circular" references created on some
> objects from the first module (the one imported) in the second one
> won't have the completion working for them (that's what I see)).
>


To understand what is possible and what is not, you simply need to think
like a completion feature.

case 1:
self.object = MyClass()

self.object is a MyClass instance, easy stuff I need to parse MyClass
and get the list of attributes/method

case 2:

self.object = object1

There may be simply no way to know the type/class of object1, object1
could be even of an inconsistent type, sometimes None, int, MyClass, who
knows ?
Your trick worked because you found a way to write self.object =
MyClass(object1) and getting barely the same effect than self.object =
object1.

I'm still thinking that's it's a bad idea. In any case your IDE can
still be fooled by the dynamic of python (like pylint is)

How to write python code then ? Well I guess most people either knows
all the attributes by heart, have the documentation, or a split screen
to the class definition, possibly folded using advanced text editor
feature. You need to forget about code completion with any dynamic
language. I also sometimes use an ipython shell to instanciate objects
and inspect their attributes.

JM

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
__new__ does not call __init__ as described in descrintro.html (WAS:Can __new__ prevent __init__ from being called?) Steven Bethard Python 2 02-16-2005 06:50 AM
IDE - code completion Michael Schmid VHDL 1 12-15-2004 01:54 PM
Spelling Completion or Code completion in ASP? ziggs ASP General 2 04-21-2004 02:02 PM
Best Python IDE Code Completion! John Python 3 01-17-2004 05:17 PM
Re: Best Python IDE Code Completion! Stephan Deibel Python 0 01-15-2004 04:02 PM



Advertisments