Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Flython

Reply
Thread Tools

Flython

 
 
Terry Reedy
Guest
Posts: n/a
 
      08-16-2004

"Christopher T King" <(E-Mail Removed)> wrote in message
news(E-Mail Removed)...

>I'm using http://docs.python.org/ref/grammar.txt as my grammar
>reference.

....
> Is there at least some rationale why the AST trees don't match the
> grammar definitions?


The grammar in the Ref Manual, which is collected together in the above
file, is intended and used for human comsumption. The equivalent but not
identical grammar in a source code file is intended and used for
parser-generator comsumption. It consitutes its own documentation. You
aren't the first person to trip on the difference. The rationale is that
people and parser generators 'think' differently.

I don't know whether AST trees exactly match the parser grammarr or if they
effectively follow a third form, hopefully also exactly equivalent. You
have wandered into as-yet inadequately documented internals land.

Terry J. Reedy



 
Reply With Quote
 
 
 
 
Christopher T King
Guest
Posts: n/a
 
      08-16-2004
On Mon, 16 Aug 2004, Terry Reedy wrote:

> The grammar in the Ref Manual, which is collected together in the above
> file, is intended and used for human comsumption. The equivalent but not
> identical grammar in a source code file is intended and used for
> parser-generator comsumption.


Ahh... that makes sense. I assumed they were one in the same;
ref/grammar.txt was just more readily accessible at the time (read: I was
lazy).

> I don't know whether AST trees exactly match the parser grammarr or if they
> effectively follow a third form, hopefully also exactly equivalent.


A quick glance through Grammar/Grammar shows that it seems to match up
exactly to the AST trees, though I'll have to play with it to be sure.

Thanks!

 
Reply With Quote
 
 
 
 
Michael Hudson
Guest
Posts: n/a
 
      08-17-2004
Christopher T King <(E-Mail Removed)> writes:

> On Mon, 16 Aug 2004, Michael Hudson wrote:
>
> > Christopher T King <(E-Mail Removed)> writes:
> >
> > > (Unfortunately, AST trees don't match up exactly with Python grammar, so
> > > I'm having to learn the format by trial-and-error. Not much fun. Is
> > > there documentation on the AST forms anywhere?)

> >
> > Which AST are you talking about?

>
> Whatever is spit out by parser.suite().


Ah. This isn't, in any sense, an ABSTRACT syntax tree, despite what
bits and pieces of the documentation say, or at least used to say.

You'll almost certainly have a better time using the output of
Lib/compiler/transformer.py.

> Things like grammar.txt's "statement" being represented symbol.stmt,
> and grammar.txt's stmt_list not existing at all in the AST trees
> (though most/all of the other _lists exist).


As Terry said, grammar.txt is documentation only...

> > Which version of Python, etc?

>
> 2.3. I'm using http://docs.python.org/ref/grammar.txt as my grammar
> reference.


Grammar/Grammar in the source is much more accurate, but really, you
want to use compiler.transformer.

compiler.transformer spits out classes defined in compiler.ast, which
*claims* to be autogenerated from Tools/compiler/ast.txt, but I'm not
sure if that's actually the case any more.

Even without documentation, they are easier to work out than the raw
output of the parser module.

Cheers,
mwh

--
"Also, does the simple algorithm you used in Cyclops have a name?"
"Not officially, but it answers to "hey, dumb-ass!"
-- Neil Schemenauer and Tim Peters, 23 Feb 2001
 
Reply With Quote
 
Christopher T King
Guest
Posts: n/a
 
      08-17-2004
On Tue, 17 Aug 2004, Michael Hudson wrote:

> You'll almost certainly have a better time using the output of
> Lib/compiler/transformer.py.


Oh, wow! I didn't even notice that compiler could do that. Thanks!
These trees look much more manageable than the tuples parser spits out...
time for a code rewrite!

 
Reply With Quote
 
Dave Benjamin
Guest
Posts: n/a
 
      08-17-2004
In article <(E-Mail Removed)>, Peter Hansen wrote:
> Dave Benjamin wrote:
>
>> I just want to say, for the record, that if anyone can successfully build a
>> Python compiler that can produce Flash bytecode, I will personally allow
>> you to throw *any number of pies* in my face, in as public of a ceremony as
>> you desire. Furthermore, I will match you 50/50 in the cost of the pies.

>
> Why on earth (unless it's the liking pies thing or the really wanting
> this to be done thing) would you say that? Flash bytecode is pretty
> straightforward, and Python's not magic.


Because getting a pie (or twelve) in the face is a lot less work than
actually writing a parser and code generator. I have been writing
ActionScript for two years now, and I am so sick and tired of its lack of
useful data structures (like sets and dictionaries), god-awful
error-checking (any bad message results in the "undefined" value, which
accpets any message and returns the "undefined" value, and so on, until your
error surfaces someplace completely different from where it was caused),
lack of exceptions, lack of any standard for modularity... I could just go
on and on. I'm currently maintaining a 10,000 line ActionScript program, and
I would be so happy to rewrite it in Python right now.

And Python *is* magic. =)

> In any case, if this is a serious challenge, you'd better start
> identifying the constraints and minimum requirements, as I'm
> rather positive that it's possible to encode at least *one*
> Python statement (print) into Flash bytecode which would
> execute properly... shall I demonstrate for, say, one pie?


Good point. I expect at least the full syntax as of Python 2.1, which means
list comps but no "yield" or metaclasses or any fancy stuff like that.
Perhaps one pie per Bagley's Shootout example, or maybe one pie per fully
exposed Flash API (MovieClip, LocalConnection, etc.). I'm open to
negotiation.

But the irony here is that the amount of work that would go into a Python
port to SWF would be far greater than the amount of suffering I would endure
from a bunch of pies in the face. It's really on my side, because you do all
the work, and then I don't have to write ActionScript anymore. =)

Best wishes,
Dave

--
.:[ dave benjamin: ramen/[sp00] -:- spoomusic.com -:- ramenfest.com ]:.
"talking about music is like dancing about architecture."
 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      08-17-2004
Dave Benjamin wrote:
> I have been writing
> ActionScript for two years now, and I am so sick and tired of its lack of
> useful data structures (like sets and dictionaries), god-awful
> error-checking (any bad message results in the "undefined" value, which
> accpets any message and returns the "undefined" value, and so on, until your
> error surfaces someplace completely different from where it was caused),
> lack of exceptions, lack of any standard for modularity... I could just go
> on and on. I'm currently maintaining a 10,000 line ActionScript program, and
> I would be so happy to rewrite it in Python right now.
>
> And Python *is* magic. =)


So Dan was right on both counts. ('''Either Dave thinks it just
can't be done, or he really, /really/ wants this compiler.''')

What version of the SWF format are you stuck writing to? It
appears that even SWF 5 supports Javascript-style "dictionaries"
by using attributes on an object. For example, "ActionEnumerate"
will let you iterate over all properties on an object ala the
Python "for k in dict".

As for exceptions, they were added in SWF 7, so I'm guessing you're
stuck somewhere prior to that. Your life must really suck.

> Peter Hansen wrote:
>>In any case, if this is a serious challenge, you'd better start
>>identifying the constraints and minimum requirements, as I'm
>>rather positive that it's possible to encode at least *one*
>>Python statement (print) into Flash bytecode which would
>>execute properly... shall I demonstrate for, say, one pie?

>
> Good point. I expect at least the full syntax as of Python 2.1, which means
> list comps but no "yield" or metaclasses or any fancy stuff like that.
> Perhaps one pie per Bagley's Shootout example, or maybe one pie per fully
> exposed Flash API (MovieClip, LocalConnection, etc.). I'm open to
> negotiation.


Gosh you set a high target. Much as I like the taste of pie
(Mmmmm... piiiiie), and that cold clammy feeling as it drips down
inside my shirt, I don't think I'll be accepting the challenge any
time soon. Not, at least, until I learn a lot more about compilers
than I currently know (hint, it involves the word "sweet" and "all").

One thing that would help would be a little direction from
someone in the know as to how Jython handles things that are
simply not available in Java bytecode. For example, as far
as I know the main operators for basic primitive types in Java are
implemented with different bytecode instructions, while in
CPython things like "add" are implemented as a single bytecode
which happens to special case integers for performance, but
falls back to a dynamic lookup if the types being added are
not integers.

That looks to me like a bit of a conundrum. To turn the Python
code into the appropriate Java bytecodes, wouldn't you have to
know the types of the data during compilation (as in static
data types)? But Python doesn't have that (yet), so how is
it done? Are all "adds" sent to some library routine which
is made available to all applications, and that routine then
handles the Python-style dynamic add operation?

In some ways, I think the SWF bytecode is even closer to Python
than Java's bytecode is, but I'm pretty sure that I couldn't
begin to imagine how one would target either of them...

-Peter
 
Reply With Quote
 
Lucas Raab
Guest
Posts: n/a
 
      08-17-2004
Oh, Oh, it's magic!! You know!!!...

"Dave Benjamin" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> In article <(E-Mail Removed)>, Peter Hansen wrote:
> > Dave Benjamin wrote:
> >
> >> I just want to say, for the record, that if anyone can successfully

build a
> >> Python compiler that can produce Flash bytecode, I will personally

allow
> >> you to throw *any number of pies* in my face, in as public of a

ceremony as
> >> you desire. Furthermore, I will match you 50/50 in the cost of the

pies.
> >
> > Why on earth (unless it's the liking pies thing or the really wanting
> > this to be done thing) would you say that? Flash bytecode is pretty
> > straightforward, and Python's not magic.

>
> Because getting a pie (or twelve) in the face is a lot less work than
> actually writing a parser and code generator. I have been writing
> ActionScript for two years now, and I am so sick and tired of its lack of
> useful data structures (like sets and dictionaries), god-awful
> error-checking (any bad message results in the "undefined" value, which
> accpets any message and returns the "undefined" value, and so on, until

your
> error surfaces someplace completely different from where it was caused),
> lack of exceptions, lack of any standard for modularity... I could just go
> on and on. I'm currently maintaining a 10,000 line ActionScript program,

and
> I would be so happy to rewrite it in Python right now.
>
> And Python *is* magic. =)
>
> > In any case, if this is a serious challenge, you'd better start
> > identifying the constraints and minimum requirements, as I'm
> > rather positive that it's possible to encode at least *one*
> > Python statement (print) into Flash bytecode which would
> > execute properly... shall I demonstrate for, say, one pie?

>
> Good point. I expect at least the full syntax as of Python 2.1, which

means
> list comps but no "yield" or metaclasses or any fancy stuff like that.
> Perhaps one pie per Bagley's Shootout example, or maybe one pie per fully
> exposed Flash API (MovieClip, LocalConnection, etc.). I'm open to
> negotiation.
>
> But the irony here is that the amount of work that would go into a Python
> port to SWF would be far greater than the amount of suffering I would

endure
> from a bunch of pies in the face. It's really on my side, because you do

all
> the work, and then I don't have to write ActionScript anymore. =)
>
> Best wishes,
> Dave
>
> --
> .:[ dave benjamin: ramen/[sp00] -:- spoomusic.com -:- ramenfest.com ]:.
> "talking about music is like dancing about architecture."



 
Reply With Quote
 
Dave Benjamin
Guest
Posts: n/a
 
      08-18-2004
In article <(E-Mail Removed)>, Peter Hansen wrote:
> Dave Benjamin wrote:
>> I have been writing
>> ActionScript for two years now, and I am so sick and tired of its lack of
>> useful data structures (like sets and dictionaries), god-awful
>> error-checking (any bad message results in the "undefined" value, which
>> accpets any message and returns the "undefined" value, and so on, until your
>> error surfaces someplace completely different from where it was caused),
>> lack of exceptions, lack of any standard for modularity... I could just go
>> on and on. I'm currently maintaining a 10,000 line ActionScript program, and
>> I would be so happy to rewrite it in Python right now.
>>
>> And Python *is* magic. =)

>
> So Dan was right on both counts. ('''Either Dave thinks it just
> can't be done, or he really, /really/ wants this compiler.''')


I'm sure it can be done. I just don't think *I* can do it. =) At least, I
have very little experience in writing compilers or anything like that...

> What version of the SWF format are you stuck writing to? It
> appears that even SWF 5 supports Javascript-style "dictionaries"
> by using attributes on an object. For example, "ActionEnumerate"
> will let you iterate over all properties on an object ala the
> Python "for k in dict".


Flash MX, which is SWF 6 I think. It is true that you can (ab)use objects as
dictionaries, but this has several unforunate consequences:

- Keys must be strings. All objects will be toString()-ed.

- Keys are case-insensitive (until SWF 7, I suppose).

- Adding methods to the dictionary class.
(to support .keys(), .values(), etc.) requires modification of the Object
prototype, and thus the use of undocumented features (ASSetPropFlags) to
hide these methods from "for ... in ..." loops.

- There is no exception on a bad key, since there are no exceptions
(pre MX 2004, anyway), so the only way to even get an error report is
to use another undocumented feature, the "__resolve" method.

- Implementing dictionaries as nested arrays, to get around the case
insensitivity issue and string requirement for keys, has the most
awful performance I've ever seen in any language.

> As for exceptions, they were added in SWF 7, so I'm guessing you're
> stuck somewhere prior to that. Your life must really suck.


Well, I'm getting by. But there are so many times when I think "Wow...
Python really got this right". You don't realize how many ways things can go
wrong when you're spoiled by the good design decisions behind a language
like Python. I've had this feeling many times over.

> > Peter Hansen wrote:
>> Good point. I expect at least the full syntax as of Python 2.1, which means
>> list comps but no "yield" or metaclasses or any fancy stuff like that.
>> Perhaps one pie per Bagley's Shootout example, or maybe one pie per fully
>> exposed Flash API (MovieClip, LocalConnection, etc.). I'm open to
>> negotiation.

>
> Gosh you set a high target. Much as I like the taste of pie
> (Mmmmm... piiiiie), and that cold clammy feeling as it drips down
> inside my shirt, I don't think I'll be accepting the challenge any
> time soon. Not, at least, until I learn a lot more about compilers
> than I currently know (hint, it involves the word "sweet" and "all").


See, I shouldn't have blown my cover. I could've at least gotten a "print"
statement out of you before I opened my big mouth.

> One thing that would help would be a little direction from
> someone in the know as to how Jython handles things that are
> simply not available in Java bytecode. For example, as far
> as I know the main operators for basic primitive types in Java are
> implemented with different bytecode instructions, while in
> CPython things like "add" are implemented as a single bytecode
> which happens to special case integers for performance, but
> falls back to a dynamic lookup if the types being added are
> not integers.


Well, in ActionScript, and I'm guessing this applies even at the bytecode
level, things are very weakly typed. Numbers are transparently coerced to
Strings and back, all the time. There's a single "add" instruction that is
used for Numbers and Strings, performing addition, concatenation, and
coercion as necessary. At least, so it seems looking at the FLASM dump of my
program.

> That looks to me like a bit of a conundrum. To turn the Python
> code into the appropriate Java bytecodes, wouldn't you have to
> know the types of the data during compilation (as in static
> data types)? But Python doesn't have that (yet), so how is
> it done? Are all "adds" sent to some library routine which
> is made available to all applications, and that routine then
> handles the Python-style dynamic add operation?


I like key lime, but not banana cream. Apple and rhubarb are good, too.

> In some ways, I think the SWF bytecode is even closer to Python
> than Java's bytecode is, but I'm pretty sure that I couldn't
> begin to imagine how one would target either of them...


I'm sure it is, since it was designed with a dynamically typed language in
mind. They're similar though; they're both stack-based, and they both
support some notion of objects at the bytecode level.

Let me know when it's ready. Thanks.

Dave

--
.:[ dave benjamin: ramen/[sp00] -:- spoomusic.com -:- ramenfest.com ]:.
"talking about music is like dancing about architecture."
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
RE: Flython? Ben Last Python 1 08-18-2004 01:38 PM
Flython? Peter Hansen Python 27 08-18-2004 12:40 AM
RE: Flython? Ben Last Python 0 08-16-2004 07:06 PM



Advertisments