Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > flex: plex?

Reply
Thread Tools

flex: plex?

 
 
Peter Kleiweg
Guest
Posts: n/a
 
      08-17-2004

Hi,

I am looking for a Python equivalent of Flex. After some
browsing, it seems that Plex is my best bet, but I would like to
hear suggestions before I dive in.

I have been using Flex a lot in combination with C programming.
I never use Yacc (or Bison), though Flex and Yacc are supposed
to be used together. But I found Yacc too cumbersome, and
unnecessary. Yacc is about context-free grammars, nice if you
like to describe an input language as a set of context-free
rewrite rules. Flex is about finite-state automata, but can be
extended to push-down automata (equivalent to context-free
grammars) and beyond, as long as it stays deterministic. I found
this to be very productive. I prefer automata over rewrite rules.

So, what I am looking for is a Python version of Flex that
supports the following capabilities, which are also supported by Plex:

- assigning any Python code to a matched input pattern

- input pattern recognition depending on 'states', which can be
changed through actions

Things that seems to be missing from Plex:

- patterns like:
A/B : match A only if followed by B, give A as result, keep B
in the input stream

- actions like:
REJECT: ask the flexer to come up with the next best match for the
same input
LESS(n): push n characters back into the input stream

All these are not really necessary, but they make some things
simpler.

So, are there any other packages that do these things better
than Plex?

--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

 
Reply With Quote
 
 
 
 
Jeff Epler
Guest
Posts: n/a
 
      08-17-2004
If plex uses python regular expressions, you can use the zero-length
lookahead assertion instead of flex's A/B syntax: "A(?=B)"

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBInqKJd01MZaTXX0RAh5dAKCivbanEnAb1N7QShDi5U LE/+32DwCfTG34
cb0k64glgd54ryIoemvZBSo=
=l+Mj
-----END PGP SIGNATURE-----

 
Reply With Quote
 
 
 
 
David M. Cooke
Guest
Posts: n/a
 
      08-17-2004
At some point, Peter Kleiweg <(E-Mail Removed)> wrote:

> Hi,
>
> I am looking for a Python equivalent of Flex. After some
> browsing, it seems that Plex is my best bet, but I would like to
> hear suggestions before I dive in.
>
> I have been using Flex a lot in combination with C programming.
> I never use Yacc (or Bison), though Flex and Yacc are supposed
> to be used together. But I found Yacc too cumbersome, and
> unnecessary. Yacc is about context-free grammars, nice if you
> like to describe an input language as a set of context-free
> rewrite rules. Flex is about finite-state automata, but can be
> extended to push-down automata (equivalent to context-free
> grammars) and beyond, as long as it stays deterministic. I found
> this to be very productive. I prefer automata over rewrite rules.


If you've been using Flex, you might want to check out FlexModule at
<http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
which allows you to use Flex to generate a lexer callable from python.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
 
Reply With Quote
 
Peter Kleiweg
Guest
Posts: n/a
 
      08-17-2004
Jeff Epler schreef:

> If plex uses python regular expressions, you can use the zero-length
> lookahead assertion instead of flex's A/B syntax: "A(?=B)"


Alas, it doesn't.


--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

 
Reply With Quote
 
Peter Kleiweg
Guest
Posts: n/a
 
      08-17-2004
David M. Cooke schreef:


> If you've been using Flex, you might want to check out FlexModule at
> <http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
> which allows you to use Flex to generate a lexer callable from python.


I have tried it. (Had to edit the Makefile by hand, because
"make -f Makefile.pre.in boot" did not produce a valid Makefile,
but I did manage to build and run the example.) The problem is,
the Flex file (and Yacc file) is compiled into C, and then into
a shared lib. This means I must use C code in the Flex file.

What I want is something that lets me call arbitrary Python code
from the lexer. Ideal would be to have a version of Flex that
produces a Python file instead of a C file. I have kept looking,
and I think I found what I was looking for: PyLly, which is part
of PyGgy. http://www.lava.net/~newsham/pyggy/


--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments