Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > token pasting problem in K&R preprocessor

Reply
Thread Tools

token pasting problem in K&R preprocessor

 
 
Henry Townsend
Guest
Posts: n/a
 
      06-19-2006
I hope this is on-topic in c.l.c - it's about the C preprocessor more
than the language per se, more generally about the K&R behavior, and
most specifically about the Sun cpp which is why I've cross posted there.

The test case below is taken from an Imake setup (yes, old, I know).
There's a Concat() macro which uses old-style /**/ for token pasting (we
are not allowed to assume ANSI so ## isn't allowed). I'm trying to paste
AND expand the two macros X and Y. I cannot figure out why it doesn't
work in this scenario; the literal /**/ works but the macro doesn't.

% cat /tmp/X.c
#define X xxx
#define Y yyy
#define Concat(a,b)a/**/b
X/**/Y
Concat(X,Y)

% /usr/ccs/lib/cpp /tmp/X.c
# 1 "/tmp/X.c"
[blank space elided]
xxxyyy
XY

Thanks,
HT
 
Reply With Quote
 
 
 
 
Kenneth Brody
Guest
Posts: n/a
 
      06-19-2006
Henry Townsend wrote:
[...]
> % cat /tmp/X.c
> #define X xxx
> #define Y yyy
> #define Concat(a,b)a/**/b
> X/**/Y
> Concat(X,Y)
>
> % /usr/ccs/lib/cpp /tmp/X.c
> # 1 "/tmp/X.c"
> [blank space elided]
> xxxyyy
> XY


Using MSVC6 (cl /P usenet.c), I get:
==========
xxxyyy
xxx yyy
==========

Using gcc version "egcs-2.91.66" (gcc -E usenet.c), I get:

==========
# 1 "usenet.c"


xxx yyy
xxx yyy
==========

I can't imagine how you get "XY" for the second line. Doesn't the
preprocessor have to expand the X and Y into their #define'd values?

Can you run the C compiler, asking it to only preprocess the file,
rather than run cpp directly? What happens then?

Now, as to why MSVC gives "xxxyyy" for the first line, and gcc gives
"xxx yyy", I don't know. Can someone here tell me which is "right"
and which is "wrong"? (Or are they both "right"?)

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <(E-Mail Removed)>

 
Reply With Quote
 
 
 
 
Michael Mair
Guest
Posts: n/a
 
      06-19-2006
Henry Townsend schrieb:
> I hope this is on-topic in c.l.c - it's about the C preprocessor more
> than the language per se, more generally about the K&R behavior, and
> most specifically about the Sun cpp which is why I've cross posted there.
>
> The test case below is taken from an Imake setup (yes, old, I know).
> There's a Concat() macro which uses old-style /**/ for token pasting (we
> are not allowed to assume ANSI so ## isn't allowed). I'm trying to paste
> AND expand the two macros X and Y. I cannot figure out why it doesn't
> work in this scenario; the literal /**/ works but the macro doesn't.
>
> % cat /tmp/X.c
> #define X xxx
> #define Y yyy
> #define Concat(a,b)a/**/b
> X/**/Y
> Concat(X,Y)
>
> % /usr/ccs/lib/cpp /tmp/X.c
> # 1 "/tmp/X.c"
> [blank space elided]
> xxxyyy
> XY


I don't have a K&R compiler, so I am only guessing.

Note that you have the same problem for ##. There, you solve
it by an additional "level of indirection", i.e. you force
expansion of the macro arguments by wrapping "Concat" once:
#define X xxx
#define Y yyy
#define con_cat(a,b) a##b
#define CONCAT(a,b) con_cat(a,b)
con_cat(X,Y)
CONCAT(X,Y)
leads to
XY
xxxyyy

I'd try the same for K&R, too.

HTH
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.
 
Reply With Quote
 
Eric Sosman
Guest
Posts: n/a
 
      06-19-2006


Henry Townsend wrote On 06/19/06 14:52,:
> [...]
> There's a Concat() macro which uses old-style /**/ for token pasting (we
> are not allowed to assume ANSI so ## isn't allowed). [...]


ANSI C has been around for a little less than seventeen
years, so perhaps your caution in adopting it is justified.
Just as a point of idle interest, how many decades do you
think must elapse before it is safe to adopt a new standard?
How are you doing with the plans to move from FORTRAN II to
FORTRAN IV?

Challenge: Without reference to a newspaper, almanac,
Wikipedia, or other such source, name three people who were
heads of state of members of the UN Security Council at the
time ANSI C was adopted.

Challenge: Is an `int' large enough to count the number
of seconds since the adoption of ANSI C? The number of hours?
Of days?

Challenge: In twenty-five words or fewer, compare and contrast
your organization's rate of standards adoption with the rate of
proton decay.

(All right, all right -- I'm having some fun at your expense.
But in all seriousness, I urge you to consider moving forward to
within a decade of the leading edge. There were once good reasons
to accommodate pre-ANSI implementations, but their goodness has
diminished with the passage of time and is now approximately equal
to that of a funerary meal from a Pharaoh's tomb. You really,
really ought to take a hard look at your reasons for adhering to
outdated technologies. Observe: The thread at hand demonstrates
that this antiquarian romanticism is making trouble and hence
costing you money ...)

--
http://www.velocityreviews.com/forums/(E-Mail Removed)

 
Reply With Quote
 
CBFalconer
Guest
Posts: n/a
 
      06-19-2006
Henry Townsend wrote:
>

.... snip ...
>
> The test case below is taken from an Imake setup (yes, old, I
> know). There's a Concat() macro which uses old-style /**/ for
> token pasting (we are not allowed to assume ANSI so ## isn't
> allowed). I'm trying to paste AND expand the two macros X and Y.
> I cannot figure out why it doesn't work in this scenario; the
> literal /**/ works but the macro doesn't.


Then you just don't paste tokens. Even old K&R specified that any
comment was replaced by at least one blank. Some compilers were
too stupid to do this.

--
"I don't know where bin Laden is. I have no idea and really
don't care. It's not that important." - G.W. Bush, 2002-03-13
"No, we've had no evidence that Saddam Hussein was involved
with September the 11th." - George Walker Bush 2003-09-17


 
Reply With Quote
 
Henry Townsend
Guest
Posts: n/a
 
      06-20-2006
CBFalconer wrote:
> Henry Townsend wrote:
> ... snip ...
>> The test case below is taken from an Imake setup (yes, old, I
>> know). There's a Concat() macro which uses old-style /**/ for
>> token pasting (we are not allowed to assume ANSI so ## isn't
>> allowed). I'm trying to paste AND expand the two macros X and Y.
>> I cannot figure out why it doesn't work in this scenario; the
>> literal /**/ works but the macro doesn't.

>
> Then you just don't paste tokens. Even old K&R specified that any
> comment was replaced by at least one blank. Some compilers were
> too stupid to do this.


A) The example was not typed in, it was pasted verbatim from a shell
session. It's only five lines - feel free to try it yourself if you want
to see that there's no blank in the /**/ case.

B) This is an imake-based build system which has been in use for about
18 years (yes, before ANSI C was ratified), and has been running on
every commercially viable Unix platform during that time. Developers and
users of this very-well-known product will be surprised to hear that it
can't be built.

C) The Concat() macro was take directly from the X11 imake system, where
it survives to this day.

So regardless of what K&R may have _specified_, empirical evidence is
that no blank is inserted.
 
Reply With Quote
 
Eric Sosman
Guest
Posts: n/a
 
      06-20-2006
Henry Townsend wrote:

> CBFalconer wrote:
>
>> Henry Townsend wrote:
>> ... snip ...
>>
>>> The test case below is taken from an Imake setup (yes, old, I
>>> know). There's a Concat() macro which uses old-style /**/ for
>>> token pasting (we are not allowed to assume ANSI so ## isn't
>>> allowed). I'm trying to paste AND expand the two macros X and Y.
>>> I cannot figure out why it doesn't work in this scenario; the
>>> literal /**/ works but the macro doesn't.

>>
>>
>> Then you just don't paste tokens. Even old K&R specified that any
>> comment was replaced by at least one blank. Some compilers were
>> too stupid to do this.

>
>
> A) The example was not typed in, it was pasted verbatim from a shell
> session. It's only five lines - feel free to try it yourself if you want
> to see that there's no blank in the /**/ case.
>
> B) This is an imake-based build system which has been in use for about
> 18 years (yes, before ANSI C was ratified), and has been running on
> every commercially viable Unix platform during that time. Developers and
> users of this very-well-known product will be surprised to hear that it
> can't be built.
>
> C) The Concat() macro was take directly from the X11 imake system, where
> it survives to this day.
>
> So regardless of what K&R may have _specified_, empirical evidence is
> that no blank is inserted.


The biggest problem tackled by the ANSI Standard was not
inventing the token-pasting operator, or void, or prototypes,
or any of the other "new features" in the language. Larger
than all of these -- than all of these put together -- was
the problem of reconciling the multiple divergent versions of
C that had arisen.

Does sprintf() return a count or a pointer?

Does unsigned short promote to int or to unsigned int?

Does an extern declaration inside a block have block
scope or file scope?

If a macro definition contains a string literal that in
turn contains a substring identical to one of the macro's
arguments, does substitution occur?

... and, of course: What happens when a macro definition
expands two of its arguments with only a comment separating
them?

Actual, real implementations of pre-Standard C differed
on all of these points (and more). The whole reason for the
standardization effort in the first place was that there was
no consensus on these matters. There was no single definition,
either formal or de facto, about how to paste tokens.

Now: You say you are "not allowed to assume ANSI C," a
stance I've commented on elsethread. However, you mention
that all this token-pasting is done through a Concat() macro.
Can you not make a preprocessor test for __STDC__ and define
Concat() accordingly? If you are worried about pre-Standard
(or anti-Standard) compilers that define __STDC__ but don't
do token-pasting, I think you're worried about trifles.

--
Eric Sosman
(E-Mail Removed)lid
 
Reply With Quote
 
Henry Townsend
Guest
Posts: n/a
 
      06-20-2006
Eric Sosman wrote:
> Now: You say you are "not allowed to assume ANSI C," a
> stance I've commented on elsethread. However, you mention
> that all this token-pasting is done through a Concat() macro.
> Can you not make a preprocessor test for __STDC__ and define
> Concat() accordingly? If you are worried about pre-Standard
> (or anti-Standard) compilers that define __STDC__ but don't
> do token-pasting, I think you're worried about trifles.


No, if you go back to the original post I was trying to understand
specifically how to make a certain kind of pasting work with a certain
K&R cpp, with the idea that the answer would probably apply to the 6-8
other old K&R preprocessors I'll be encountering soon on other
platforms. But what I think I've learned here is that each cpp may
behave differently. Mind you, though, this Concat() macro has worked for
every platform X11 has been ported to over the last 20 years, which is
quite a lot.

As for whether to upgrade to ANSI, I personally am a big fan of getting
with the times but it's not necessarily part of my charter to do so in
this case. I may be able to get it on the table but at the moment the
referenced problem must be dealt with in the context of K&R. Luckily
it's not a huge difficulty; I can special-case this particular value. I
was just hoping for a solution short of ANSI C.

HT
 
Reply With Quote
 
Richard B. Gilbert
Guest
Posts: n/a
 
      06-20-2006
Henry Townsend wrote:

> Eric Sosman wrote:
>
>> Now: You say you are "not allowed to assume ANSI C," a
>> stance I've commented on elsethread. However, you mention
>> that all this token-pasting is done through a Concat() macro.
>> Can you not make a preprocessor test for __STDC__ and define
>> Concat() accordingly? If you are worried about pre-Standard
>> (or anti-Standard) compilers that define __STDC__ but don't
>> do token-pasting, I think you're worried about trifles.

>
>
> No, if you go back to the original post I was trying to understand
> specifically how to make a certain kind of pasting work with a certain
> K&R cpp, with the idea that the answer would probably apply to the 6-8
> other old K&R preprocessors I'll be encountering soon on other
> platforms. But what I think I've learned here is that each cpp may
> behave differently. Mind you, though, this Concat() macro has worked for
> every platform X11 has been ported to over the last 20 years, which is
> quite a lot.
>
> As for whether to upgrade to ANSI, I personally am a big fan of getting
> with the times but it's not necessarily part of my charter to do so in
> this case. I may be able to get it on the table but at the moment the
> referenced problem must be dealt with in the context of K&R. Luckily
> it's not a huge difficulty; I can special-case this particular value. I
> was just hoping for a solution short of ANSI C.
>
> HT


There are other advantages to coding in ANSI C, the nicest one is that
the compilers no longer complain about the sleazy coding practices that
were so prevalent in K&R. The other nice thing is that it tends to
uncover bugs that may have been making you itch for years. Doing it is
generally tedious but trivial.
 
Reply With Quote
 
Old Wolf
Guest
Posts: n/a
 
      06-21-2006
Eric Sosman wrote:
>
> Challenge: Without reference to a newspaper, almanac,
> Wikipedia, or other such source, name three people who were
> heads of state of members of the UN Security Council at the
> time ANSI C was adopted.


I'll bite: Elizabeth II, Mitterrand, Deng Xiaoping, Gorbachev, Reagan

When was the standard actually ratified? I've heard rumblings
that even though the "89" tag is attached, it wasn't actually
ratified by ANSI until sometime in 1990 (in which case I
may have failed your challenge)

> Challenge: Is an `int' large enough to count the number
> of seconds since the adoption of ANSI C? The number of hours?
> Of days?


Well, a 32-bit int is going to get us from 1 Jan 1970 to sometime
in 2039.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Token pasting problem jeff@muller.org C Programming 1 10-16-2008 05:33 PM
Token pasting and what does the result need to be? Mark Odell C Programming 7 05-01-2006 09:24 PM
Token-pasting trouble Mark Odell C Programming 8 04-06-2006 04:12 PM
Token pasting (## operator) - Add whitespace to a token Wessi C Programming 3 08-11-2005 01:02 PM
preprocessor, token concatenation, no valid preprocessor token Cronus C++ 1 07-14-2004 11:10 PM



Advertisments