Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Deep Black Magic in Python: please help

Reply
Thread Tools

Deep Black Magic in Python: please help

 
 
Jan Burgy
Guest
Posts: n/a
 
      08-13-2004
Hi everyone,

I am trying to convince my managers that python can replace the outdated and
soon no-longer maintained proprietary system (Tool for Calculator Design) we
use here. In order to achieve this, I need to analyze python code which will
look somethink like this:

def foo(arg_dict):
return arg_dict["one"] + bar(arg_dict)

def bar(other_dict):
return other_dict["two"] + other_dict[True and "three" or "four"]

The result of my analysis should return a list

['one', 'two', 'three']

Alright, that doesn't sound so bad. With a little RTFM-ing and hacking, I got

-- black magic starts here --

import compiler

def _evaluate(partial, name=""):
from compiler.ast import Expression
from compiler.misc import set_filename
from compiler.pycodegen import ExpressionCodeGenerator

tree = Expression(partial)
set_filename(name, tree)
gen = ExpressionCodeGenerator(tree)
return eval(gen.getCode())

class RecursiveVisitor(compiler.visitor.ASTVisitor):
def __init__(self, name):
self.name = "Name(%s)" % repr(name)
self.names = {}
def visitSubscript(self, node, *args):
if repr(node.expr) == self.name:
try:
name = _evaluate(node.subs[0])
except:
name = str(node.subs[0])
self.names[name] = 1
def visitCallFunc(self, node, *args):
try:
from inspect import getsource, getargspec
from compiler import parse, walk
func = _evaluate(node.node)
pos = map(repr, node.args).index(self.name)
src, arg = getsource(func), getargspec(func)[0]
tmp, self.name = self.name, "Name(%s)" % repr(arg[pos])
walk(parse(src), self)
self.name = tmp
except Exception, e:
print str(e)

if __name__ == "__main__":

from inspect import getsource
from compiler import parse, walk

src = getsource(foo)
mod = parse(src)
visitor = RecursiveVisitor("kw")
walk(mod, visitor)
print visitor.names.keys()

-- black magic ends here, ouch --

Once again, I'm in total awe at the possibilities python offers us lowly
users. Unfortunately we're all the same: you give us a finger, we want the
whole arm! I want this method to generalize to method calls as in

class baz:

def foo(self, arg_dict):
return arg_dict["one"] + self.bar(arg_dict)

def bar(self, other_dict):
return other_dict["two"] + other_dict[True and "three" or "four"]

It shouldn't be all that hard. My problem is the lookup of 'self.bar'. In
the AST it looks something like

CallFunc(Getattr(Name('self'), 'bar'), [Name('arg_dict')], None, None)

How do I find the corresponding function? Anybody feels like wrapping
their head on this?

Cheers,

Jan Burgy
 
Reply With Quote
 
 
 
 
Terry Reedy
Guest
Posts: n/a
 
      08-13-2004

"Jan Burgy" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) om...
> return arg_dict["one"] + self.bar(arg_dict)


> It shouldn't be all that hard. My problem is the lookup of 'self.bar'. In
> the AST it looks something like
>
> CallFunc(Getattr(Name('self'), 'bar'), [Name('arg_dict')], None,

None)
>
> How do I find the corresponding function?


The lookup returns a bound method, whose structure is an implementation
detail.

Completely untested suggestion, possibly not correct or possible:

Write an implementation-specific unwrap function. Then either augment the
source code

..... + unwrap(self.bar)(self, arg_dict) # self now an explict arg

or the analysis code to do the unwrapping. For the latter, recognize
Name('self') and replace the result of Getattr(Name('self'), 'bar') with
unwrap(thereof) and, again, augment the arg list.

Terry J. Reedy



 
Reply With Quote
 
 
 
 
Jan Burgy
Guest
Posts: n/a
 
      08-16-2004
"Terry Reedy" <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> "Jan Burgy" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed) om...
> > return arg_dict["one"] + self.bar(arg_dict)

>
> > It shouldn't be all that hard. My problem is the lookup of 'self.bar'. In
> > the AST it looks something like
> >
> > CallFunc(Getattr(Name('self'), 'bar'), [Name('arg_dict')], None,

> None)
> >
> > How do I find the corresponding function?

>
> The lookup returns a bound method, whose structure is an implementation
> detail.
>
> Completely untested suggestion, possibly not correct or possible:
>
> Write an implementation-specific unwrap function. Then either augment the
> source code
>
> .... + unwrap(self.bar)(self, arg_dict) # self now an explict arg
>
> or the analysis code to do the unwrapping. For the latter, recognize
> Name('self') and replace the result of Getattr(Name('self'), 'bar') with
> unwrap(thereof) and, again, augment the arg list.
>
> Terry J. Reedy


Hi Terry,

Thank you very much for your suggestion. I am not sure how it would
help though. What is preventing me now from already replacing the
result of Getattr(Name('self'), 'bar') with what I need? That's
precisely my problem however. Because Python is a dynamically typed
language, I have no information about 'self' at compile-time. Short of
saving context information while I am visiting the parse tree, I don't
know how to tell what class I am in.

Would-Meta-Classes-help-me-in-this-case-ly yours?

Jan
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
magic / counter-magic? (detect loading file?) Giles Bowkett Ruby 9 12-17-2007 05:42 AM
recontextualizing a block (looking for deep magic) Phil Tomson Ruby 16 02-25-2006 07:33 PM
Deep Freeze In Deep Trouble johntangelo@gmail.com Computer Security 3 10-25-2005 11:49 PM



Advertisments