«

Hello,

I'm having some problems with pyparsing, I could not find how to tell
it to view certain words as keywords, i.e. not as a possible variable
name (in an elegant way),
for example, I have this little grammar:

terminator = Literal(";")
expr = Word(alphas)
body = Forward();
ifstat = "if" + body + "fi"
stat = expr | ifstat
body << OneOrMore(stat + terminator)
program = body

I.e. some program which contains statements separated by semicolons. A
statement is either an if [....] fi statement or simply a word.

If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

How to fix this?

Thank you,

Berteun

"Berteun Damman" <> wrote in message
news: oups.com...
> Hello,
>
> I'm having some problems with pyparsing, I could not find how to tell
> it to view certain words as keywords, i.e. not as a possible variable
> name (in an elegant way),
> for example, I have this little grammar:
>
> terminator = Literal(";")
> expr = Word(alphas)
> body = Forward();
> ifstat = "if" + body + "fi"
> stat = expr | ifstat
> body << OneOrMore(stat + terminator)
> program = body
>
> I.e. some program which contains statements separated by semicolons. A
> statement is either an if [....] fi statement or simply a word.
>
> If I try however to parse the String "if test; testagain; fi;", it does
> not work, because the fi is interpreted as an expr, not as the end of
> the if statement, and of course, adding another fi doesn't solve this
> either.
>
> How to fix this?
>
> Thank you,
>
> Berteun
>
Berteun -

The simplest way I can think of for this grammar off the top of my head is
to use a parse action to reject keywords.

keywords = [ "if", "fi", "else", "return" ]
def rejectKeywords(string,loc,tokens):
if tokens[0] in keywords:
raise ParseException(string,loc,"found keyword %s" % tokens[0])
expr.setParseAction( rejectKeywords )

I took a different tack in the idl parser that is included in the pyparsing
examples directory, but it is more extensive.

-- Paul

On Tue, 14 Dec 2004 18:39:19 GMT, Paul McGuire
<._bogus_.com> wrote:
>> If I try however to parse the String "if test; testagain; fi;", it does
>> not work, because the fi is interpreted as an expr, not as the end of
>> the if statement, and of course, adding another fi doesn't solve this
>> either.
> The simplest way I can think of for this grammar off the top of my head is
> to use a parse action to reject keywords.

Thank you for your quick and good solution, that did the trick indeed.

But I'm wondering, isn't it possible to have some sort of lexing phase
which already identifies keywords as such? Or to provide a table with
keywords, which pyparsing is able to automatically recognize?

So you would be able to do IF = Keyword("if"), just as a Literal now is
created, but now the parser knows this word shouldn't be interpreted any
other way than as a keyword. Or would that be a bad idea?

Berteun

"Berteun Damman" <berteun@NO_SPAMdds.nl> wrote in message
news: nte.nl...
> On Tue, 14 Dec 2004 18:39:19 GMT, Paul McGuire
> <._bogus_.com> wrote:
> >> If I try however to parse the String "if test; testagain; fi;", it does
> >> not work, because the fi is interpreted as an expr, not as the end of
> >> the if statement, and of course, adding another fi doesn't solve this
> >> either.
> > The simplest way I can think of for this grammar off the top of my head
is
> > to use a parse action to reject keywords.
>
> Thank you for your quick and good solution, that did the trick indeed.
>
> But I'm wondering, isn't it possible to have some sort of lexing phase
> which already identifies keywords as such? Or to provide a table with
> keywords, which pyparsing is able to automatically recognize?
>
> So you would be able to do IF = Keyword("if"), just as a Literal now is
> created, but now the parser knows this word shouldn't be interpreted any
> other way than as a keyword. Or would that be a bad idea?
>
> Berteun
Berteun -

This is not a bad idea in and of itself, as I mentioned earlier, I had
similar problems with the idl parser. For example, if in the body of your
if block you had a statement:

ifThisWorksItsAMiracle;

The leading "if" would match the Literal("if"), although the statement is
actually the complete 'ifThisWorksItsAMiracle' method call. That is,
Literals are not automatically assumed to be whitespace-terminated. This is
part of pyparsing's philosophy of being whitespace-blind.

It's been a while since v 1.2.2 was released, I think adding a Keyword class
much as you describe (on top of the other bug-fixes and additions since
then) would be sufficient material for coming up with a 1.3 release. I'll
work on it over the holiday "break," assuming I get one!

-- Paul