Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > VHDL > Lexing the ' char

Reply
Thread Tools

Lexing the ' char

 
 
Ole Nielsby
Guest
Posts: n/a
 
      11-02-2007
I'm writing a lexer for VHDL and I don't know how to treat the ' char.
It can be used both for character literals and as an operator similar to ::
or .
in C++ if I understand correctly. How would a lexer decide?


 
Reply With Quote
 
 
 
 
kennheinrich@sympatico.ca
Guest
Posts: n/a
 
      11-02-2007
On Nov 2, 7:58 am, "Ole Nielsby" <ole.niel...@tekare-you-
spamminglogisk.dk> wrote:
> I'm writing a lexer for VHDL and I don't know how to treat the ' char.
> It can be used both for character literals and as an operator similar to ::
> or .
> in C++ if I understand correctly. How would a lexer decide?


This is where you need a few characters of lookahead in your lex
buffer. If you match ( TICK, char, TICK) you have a character literal.
Otherwise it's the TICK token (attribute or type qualifier).

- Kenn

 
Reply With Quote
 
 
 
 
Ole Nielsby
Guest
Posts: n/a
 
      11-02-2007
<(E-Mail Removed)> wrote:
> Ole Nielsby wrote:
>> I'm writing a lexer for VHDL and I don't know how to treat the ' char.
>> It can be used both for character literals and as an operator similar to
>> ::
>> or .
>> in C++ if I understand correctly. How would a lexer decide?

>
> This is where you need a few characters of lookahead in your lex
> buffer. If you match ( TICK, char, TICK) you have a character literal.
> Otherwise it's the TICK token (attribute or type qualifier).


Thanks. That's what I already implemented but I wasn't sure...


 
Reply With Quote
 
diogratia
Guest
Posts: n/a
 
      11-04-2007
On Nov 3, 12:58 am, "Ole Nielsby" <ole.niel...@tekare-you-
spamminglogisk.dk> wrote:
> I'm writing a lexer for VHDL and I don't know how to treat the ' char.
> It can be used both for character literals and as an operator similar to ::
> or .
> in C++ if I understand correctly. How would a lexer decide?


On Nov 3, 12:58 am, "Ole Nielsby" <ole.niel...@tekare-you-
spamminglogisk.dk> wrote:
> I'm writing a lexer for VHDL and I don't know how to treat the ' char.
> It can be used both for character literals and as an operator similar to ::
> or .
> in C++ if I understand correctly. How would a lexer decide?


case '\'': /* IR1045 check */

if ( last_token == DELIM_RIGHT_PAREN ||
last_token == DELIM_RIGHT_BRACKET ||
last_token == KEYWD_ALL ||
last_token == IDENTIFIER_TOKEN ||
last_token == STR_LIT_TOKEN ||
last_token == CHAR_LIT_TOKEN || !
(buff_ptr<BUFSIZ-2) )
token_flag = DELIM_APOSTROPHE;
else if (is_graphic_char(NEXT_CHAR) &&
line_buff[buff_ptr+2] == '\'') {
CHARACTER_LITERAL:
buff_ptr+= 3; /* lead,trailing \'
and char */
last_token = CHAR_LIT_TOKEN;
token_strlen = 3;
return (last_token);
}
else token_flag = DELIM_APOSTROPHE;
break;

See Issue Report IR1045:
http://www.eda-stds.org/isac/IRs-VHDL-93/IR1045.txt

As you can see from the above code fragment, the last token can be
captured and used to di"sambiguate something like:

foo <= std_logic_vector'('a','b','c');

without a large look ahead or backtracking.

Mind you you could try to argue that LRM 13.2:

...

"In some cases an explicit separator is required to separate adjacent
lexical elements (namely when, without separation, interpretation as a
single lexical element is possible). A separator is either a space
character (SPACE or NBSP),a format effector, or the end of a line. A
space character (SPACE or NBSP) is a separator except within a
comment, a string literal, or a space character literal."

could simply require the inclusion of disambiguating whitespace. The
accepted practice would be against you, however.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
question on java lang spec chapter 3.3 (unicode char lexing) Aryeh M. Friedman Java 34 01-07-2013 07:26 AM
(const char *cp) and (char *p) are consistent type, (const char **cpp) and (char **pp) are not consistent lovecreatesbeauty C Programming 1 05-09-2006 08:01 AM
/usr/bin/ld: ../../dist/lib/libjsdombase_s.a(BlockGrouper.o)(.text+0x98): unresolvable relocation against symbol `std::basic_ostream<char, std::char_traits<char> >& std::endl<char, std::char_traits<char> >(std::basic_ostre silverburgh.meryl@gmail.com C++ 3 03-09-2006 12:14 AM
simple lexing/parsing task Martin DeMello Ruby 4 02-10-2004 09:05 PM
Lexing and Parsing in Ruby. John Carter Ruby 2 11-19-2003 06:24 PM



Advertisments