Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > per-function jit compiler

Reply
Thread Tools

per-function jit compiler

 
 
Luis M. GonzŠlez
Guest
Posts: n/a
 
      04-06-2010
This post gave me an idea: http://groups.google.com/group/comp....75080707104b76

What if I write a simple decorator to figure out the types of every
function, and then we use it as a base for a simple method-jit
compiler for python?

example:

def typer(f):
def wrap(*args):
a = f.func_code.co_varnames
b = [type(i) for i in args]
return dict(zip(a,b))
return wrap

@typer
def spam(a, b, c=3, d=4):
pass

>>> spam(8,'hello',9.9, 10)

{'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
'int'>}

So by using this information, we record all the argument types used
the first time each function/method is executed, and then we generate
optimized code for them.
From this point on, a guard should check if all arguments remain the
same and, if so, the optimized code is run.
Otherwise, just fall back to the interpreter.

He! I have no idea how to implement it...
Any guru out there?
Luis
 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      04-06-2010
2010/4/5 Luis M. Gonz√°lez <(E-Mail Removed)>:
> This post gave me an idea: http://groups.google.com/group/comp....75080707104b76
>
> What if I write a simple decorator to figure out the types of every
> function, and then we use it as a base for a simple method-jit
> compiler for python?
>
> example:
>
> def typer(f):
> ¬* ¬* ¬* ¬*def wrap(*args):
> ¬* ¬* ¬* ¬* ¬* ¬*a = f.func_code.co_varnames
> ¬* ¬* ¬* ¬* ¬* ¬*b = [type(i) for i in args]
> ¬* ¬* ¬* ¬* ¬* ¬*return dict(zip(a,b))
> ¬* ¬* ¬* ¬*return wrap
>
> @typer
> def spam(a, b, c=3, d=4):
> ¬* ¬* ¬* ¬*pass
>
>>>> spam(8,'hello',9.9, 10)

> {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
> 'int'>}
>
> So by using this information, we record all the argument types used
> the first time each function/method is executed, and then we generate
> optimized code for them.
> >From this point on, a guard should check if all arguments remain the

> same and, if so, the optimized code is run.
> Otherwise, just fall back to the interpreter.
>
> He! I have no idea how to implement it...


Guido's been lending out his time machine again:
http://psyco.sourceforge.net/introduction.html

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
 
 
 
Luis M. GonzŠlez
Guest
Posts: n/a
 
      04-06-2010
On 6 abr, 03:40, Chris Rebert <(E-Mail Removed)> wrote:
> 2010/4/5 Luis M. GonzŠlez <(E-Mail Removed)>:
>
>
>
>
>
> > This post gave me an idea:http://groups.google.com/group/comp....75080707104b76

>
> > What if I write a simple decorator to figure out the types of every
> > function, and then we use it as a base for a simple method-jit
> > compiler for python?

>
> > example:

>
> > def typer(f):
> > * * * *def wrap(*args):
> > * * * * * *a = f.func_code.co_varnames
> > * * * * * *b = [type(i) for i in args]
> > * * * * * *return dict(zip(a,b))
> > * * * *return wrap

>
> > @typer
> > def spam(a, b, c=3, d=4):
> > * * * *pass

>
> >>>> spam(8,'hello',9.9, 10)

> > {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
> > 'int'>}

>
> > So by using this information, we record all the argument types used
> > the first time each function/method is executed, and then we generate
> > optimized code for them.
> > >From this point on, a guard should check if all arguments remain the

> > same and, if so, the optimized code is run.
> > Otherwise, just fall back to the interpreter.

>
> > He! I have no idea how to implement it...

>
> Guido's been lending out his time machine again:http://psyco.sourceforge.net/introduction.html
>
> Cheers,
> Chris
> --http://blog.rebertia.com


Well, psyco is a complex beast. Actually it's a Just-In-Time
Specializer, not really a compiler at all (Guido's machine told me
so).
It's been superseded by pypy, which is a way more complex beast, and
it's a tracing compiler (and many things more).
What I mean is, why not using python's simple introspection
capabilities (instead of partial evaluation, type inference or other
complex mechanism) to gather type information?
I can imagine that this, coupled with pyrex, cython or shedskin could
give some results perhaps... by selectively compiling some function or
all of them.

But probably someone will throw Guido's time machine again at me. If
it was that easier, someone would have done it already...
I just wonder why this simple approach is not feasible.

Luis
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      04-06-2010
En Tue, 06 Apr 2010 07:23:19 -0300, Luis M. GonzŠlez <(E-Mail Removed)>
escribiů:

> On 6 abr, 03:40, Chris Rebert <(E-Mail Removed)> wrote:
>> 2010/4/5 Luis M. GonzŠlez <(E-Mail Removed)>:
>>
>>
>>
>>
>>
>> > This post gave me an

>> idea:http://groups.google.com/group/comp....75080707104b76
>>
>> > What if I write a simple decorator to figure out the types of every
>> > function, and then we use it as a base for a simple method-jit
>> > compiler for python?

>>
>> > example:

>>
>> > def typer(f):
>> > def wrap(*args):
>> > a = f.func_code.co_varnames
>> > b = [type(i) for i in args]
>> > return dict(zip(a,b))
>> > return wrap

>>
>> > @typer
>> > def spam(a, b, c=3, d=4):
>> > pass

>>
>> >>>> spam(8,'hello',9.9, 10)
>> > {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
>> > 'int'>}

>>
>> > So by using this information, we record all the argument types used
>> > the first time each function/method is executed, and then we generate
>> > optimized code for them.
>> > >From this point on, a guard should check if all arguments remain the
>> > same and, if so, the optimized code is run.
>> > Otherwise, just fall back to the interpreter.

>>
>> > He! I have no idea how to implement it...

>>
>> Guido's been lending out his time machine
>> again:http://psyco.sourceforge.net/introduction.html
>>

>
> Well, psyco is a complex beast. Actually it's a Just-In-Time
> Specializer, not really a compiler at all (Guido's machine told me
> so).
> It's been superseded by pypy, which is a way more complex beast, and
> it's a tracing compiler (and many things more).
> What I mean is, why not using python's simple introspection
> capabilities (instead of partial evaluation, type inference or other
> complex mechanism) to gather type information?
> I can imagine that this, coupled with pyrex, cython or shedskin could
> give some results perhaps... by selectively compiling some function or
> all of them.


That *is* exactly what psyco does. A specialized function (or block) is a
pre-compiled block with a known combination of variable types, detected at
runtime.
psyco has not been superseded by pypy, they're two separate projects.
(psyco's author focus has migrated, though).

--
Gabriel Genellina

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
per-method jit compiler Luis M. GonzŠlez Python 1 04-06-2010 07:56 PM
Failed to load JIT compiler Ken ASP .Net 0 05-07-2007 07:28 PM
Why is a JIT compiler faster than a byte-compiler RickMuller Python 4 03-26-2005 04:30 PM
JIT Compiler drv ASP .Net 0 07-09-2003 01:35 PM
Variable JIT of ASPX files? Glen ASP .Net 2 06-26-2003 06:41 PM



Advertisments