«

Hi all,
From p 125, gives rise to this issue for me.

Is it true that a "token" in C ( philisophically ) is the least amount
of digits/chars/underscores/*s ( and other non blank space that I
have not thought of) that the compiler uses to derive useable
information. So, this would be a token

" ( ) "

but this

"( "

by itself would not?

Thanks as usual.

On Sun, 10 Aug 2008 16:20:02 -0700, mdh wrote:
> Hi all,
> From p 125, gives rise to this issue for me.
>
> Is it true that a "token" in C ( philisophically ) is the least amount
> of digits/chars/underscores/*s ( and other non blank space that I have
> not thought of) that the compiler uses to derive useable information.

In a way.

> So, this would be a token
>
> " ( ) "

If you mean including the quotation marks, then yes. Otherwise, no.

> but this
>
> "( "
>
> by itself would not?

So, no. A token is a word. A token is the largest string of characters
that cannot have whitespace inserted without breaking it into smaller
tokens, changing its meaning. () is not a token. It is two tokens, which
you can tell from the fact that you can separate the two by writing ( ).
If it were a single token, this would not be allowed. "" is a token,
because you cannot separate the quotation marks by writing " " -- at least
not without changing its meaning.

> Thanks as usual.

On Aug 10, 4:40 pm, Harald van D©¦k <true...@gmail.com> wrote:
> On Sun, 10 Aug 2008 16:20:02 -0700, mdh wrote:

> >...... a "token" in C ( philisophically ) is the least amount
> > of digits/chars/underscores/*s ( and other non blank space that I have
> > not thought of) that the compiler uses to derive useable information.
>
>
>
> ...... A token is a word. A token is the largest string of characters..........
> changing its meaning.

So,

(

is a token as it has some meaning to the compiler.... "get some more
characters, see if followed by another ) and if not, there is an
error" etc? or " found second ) so comma delineated list are
arguments" etc.


Is that why my example of ( ) is not a single token as the first "("
could be the start of a lot of different things?

On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
> Is that why my example of ( ) is not a single token as the first "("
> could be the start of a lot of different things?

No, your example of ( ) is not a single token because the two can be
separated.

On Aug 10, 5:11 pm, Harald van D©¦k <true...@gmail.com> wrote:
> On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
> > Is that why my example of ( ) is not a single token as the first "("
> > could be the start of a lot of different things?
>
> No, your example of ( ) is not a single token because the two can be
> separated.

Sorry...then I did not make myself clear. I am agreeing with you. ()
are 2 tokens, as each ( has a a meaning to the compiler.

The reason this is somewhat confusing is that on p125 of K&R2, they
define "tokens" ( and the quotation marks are theirs, not mine) as a
pair of parentheses, a pair of brackets perhaps including a number. I
assume that their definition is for the sake of the example, then.

On Sun, 10 Aug 2008 17:16:15 -0700, mdh wrote:
> On Aug 10, 5:11 pm, Harald van Dijk <true...@gmail.com> wrote:
>> On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
>> > Is that why my example of ( ) is not a single token as the first "("
>> > could be the start of a lot of different things?
>>
>> No, your example of ( ) is not a single token because the two can be
>> separated.
>
> Sorry...then I did not make myself clear. I am agreeing with you. () are
> 2 tokens, as each ( has a a meaning to the compiler.

I understood that you agreed that ( ) are two tokens, but I did not agree
with my understanding of your reasoning. It is possible I misunderstood
you, so I will give a different example. - can also be the start of a lot
of different things, such as -= or -- or ->, but those three are three
single tokens, each consisting of two characters. You cannot decrement a
variable using a- - or a- =b. You cannot dereference a pointer to a
structure using a- >b. In other words, you cannot separate the - from the
second character. At the same time, - in 3-2 is a token by itself.

> The reason this is somewhat confusing is that on p125 of K&R2, they
> define "tokens" ( and the quotation marks are theirs, not mine) as a
> pair of parentheses, a pair of brackets perhaps including a number.

This seems strange, but...

> I assume that their definition is for the sake of the example, then.

without knowing the context, I cannot be sure. I don't have K&R, so
hopefully someone else will comment.

On Aug 10, 5:44 pm, Harald van D©¦k <true...@gmail.com> wrote:
> On Sun, 10 Aug 2008 17:16:15 -0700, mdh wrote:
> > I am agreeing with you. () are

>
> I understood that you agreed that ( ) are two tokens, but I did not agree
> with my understanding of your reasoning. It is possible I misunderstood
> you, so I will give a different example.

ok... I see what you mean.


>
> This seems strange, but...
>
> > I assume that their definition is for the sake of the example, then.
>
> without knowing the context, I cannot be sure. I don't have K&R, so
> hopefully someone else will comment.

Thank you Harald.

On Sun, 10 Aug 2008 17:16:15 -0700 (PDT), mdh <>
wrote:

>On Aug 10, 5:11 pm, Harald van D©¦k <true...@gmail.com> wrote:
>> On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
>> > Is that why my example of ( ) is not a single token as the first "("
>> > could be the start of a lot of different things?
>>
>> No, your example of ( ) is not a single token because the two can be
>> separated.
>
>Sorry...then I did not make myself clear. I am agreeing with you. ()
>are 2 tokens, as each ( has a a meaning to the compiler.
>
>The reason this is somewhat confusing is that on p125 of K&R2, they
>define "tokens" ( and the quotation marks are theirs, not mine) as a
>pair of parentheses, a pair of brackets perhaps including a number. I
>assume that their definition is for the sake of the example, then.

On page 125 K&R use the word token to describe the unique processing
of a function they have provided. They enclosed it in quotes to
indicate the word is being used with a meaning other than its normal
one. In fact, it has at least two normal meanings within C.

One is its use to describe the processing of strtok(). In
this context, any string between the specified delimiters is a token.

The other is the meaning compiler writers use when describing
parse algorithms. In C, the expression a+++b is guaranteed to be
evaluated as
a++ + b
and not
a + ++b
because C uses a maximum munch rule for identifying tokens.

--
Remove del for email

"Harald van Dijk" <> wrote in message
news:9a55e$489f83bb$541dfcd3$ b.home.nl...
> On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
>> Is that why my example of ( ) is not a single token as the first "("
>> could be the start of a lot of different things?
>
> No, your example of ( ) is not a single token because the two can be
> separated.

+= is a single token, but it can be separated into two tokens + =

--
Bartc

On 2008-08-11, Bartc <> wrote:
>
> "Harald van Dijk" <> wrote in message
> news:9a55e$489f83bb$541dfcd3$ b.home.nl...
>> On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
>>> Is that why my example of ( ) is not a single token as the first "("
>>> could be the start of a lot of different things?
>>
>> No, your example of ( ) is not a single token because the two can be
>> separated.
>
> += is a single token, but it can be separated into two tokens + =
>

Yes, but that changes its meaning from a single "add and assign" to the
individual tokens "plus or positive" and "assignment", which is what
Harold wrote earlier in this thread.

--
Andrew Poelstra
To email me, use the above email addresss with .com set to .net