Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > perl identifier limits

Reply
Thread Tools

perl identifier limits

 
 
Alex Shinn
Guest
Posts: n/a
 
      02-06-2004
Got quite a surprise today when I encountered an "Identifier too long"
error message. Nothing in the FAQ, but the BUGS section of "perldoc
perl" does include:

While none of the built-in data types have any arbitrary size limits
(apart from memory size), there are still a few arbitrary limits: a
given identifier may not be longer than 255 characters

Not that I'd write such a long identifier, but I've got auto-generated
code that reaches twice that length. Any ideas apart from applying
compression algorithms to the id names? Any plans on fixing this?

--
Alex

 
Reply With Quote
 
 
 
 
Uri Guttman
Guest
Posts: n/a
 
      02-06-2004
>>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

AS> Got quite a surprise today when I encountered an "Identifier too long"
AS> error message. Nothing in the FAQ, but the BUGS section of "perldoc
AS> perl" does include:

AS> While none of the built-in data types have any arbitrary size limits
AS> (apart from memory size), there are still a few arbitrary limits: a
AS> given identifier may not be longer than 255 characters

AS> Not that I'd write such a long identifier, but I've got auto-generated
AS> code that reaches twice that length. Any ideas apart from applying
AS> compression algorithms to the id names? Any plans on fixing this?

fix your code. i can't see any possible reason to generate names that
long. you would have to come up with some amazing reasons to support
your claim that you need it.

uri

--
Uri Guttman ------ http://www.velocityreviews.com/forums/(E-Mail Removed) -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
 
Reply With Quote
 
 
 
 
Alex Shinn
Guest
Posts: n/a
 
      02-06-2004
At Fri, 06 Feb 2004 09:36:23 GMT, Uri Guttman wrote:
>
> >>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

>
> AS> Not that I'd write such a long identifier, but I've got auto-generated
> AS> code that reaches twice that length. Any ideas apart from applying
> AS> compression algorithms to the id names? Any plans on fixing this?
>
> fix your code. i can't see any possible reason to generate names that
> long. you would have to come up with some amazing reasons to support
> your claim that you need it.


You obviously don't write Perl with a Lisp mindset. If you
auto-generate code on the fly it is not always easy to design it such
that names won't conflict. In my case I'm working with an application
server which can have a *huge* base of dynamically generated code. A
potential workaround is to use only hashtables and store anonymous
subroutines in them, but this is far from an insignificant rewrite and
looses some flexibility. After googling I find I'm not the only one who
has had this problem:

http://www.gossamer-threads.com/arch...athnames_P480/

It's also a very silly & trivial bug in Perl, which is acknowledged as a
known bug. And Python does it right!

/me ducks and runs

--
Alex

 
Reply With Quote
 
A. Sinan Unur
Guest
Posts: n/a
 
      02-06-2004
Alex Shinn <(E-Mail Removed)> wrote in
news:(E-Mail Removed):

> At Fri, 06 Feb 2004 09:36:23 GMT, Uri Guttman wrote:
>>
>> >>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

>>
>> AS> Not that I'd write such a long identifier, but I've got
>> auto-generated AS> code that reaches twice that length. Any ideas
>> apart from applying AS> compression algorithms to the id names?
>> Any plans on fixing this?
>>
>> fix your code. i can't see any possible reason to generate names that
>> long. you would have to come up with some amazing reasons to support
>> your claim that you need it.

>
> You obviously don't write Perl with a Lisp mindset. If you
> auto-generate code on the fly it is not always easy to design it such
> that names won't conflict. In my case I'm working with an application
> server which can have a *huge* base of dynamically generated code. A
> potential workaround is to use only hashtables and store anonymous
> subroutines in them,


Complete shot in the dark: How about using the MD5 or SHA1 hash of the very
very very very long names you need?

--
A. Sinan Unur
(E-Mail Removed) (reverse each component for email address)
 
Reply With Quote
 
Rocco Caputo
Guest
Posts: n/a
 
      02-06-2004
On Fri, 06 Feb 2004 18:50:28 +0900, Alex Shinn wrote:
> At Fri, 06 Feb 2004 09:36:23 GMT, Uri Guttman wrote:
>>
>> >>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

>>
>> AS> Not that I'd write such a long identifier, but I've got auto-generated
>> AS> code that reaches twice that length. Any ideas apart from applying
>> AS> compression algorithms to the id names? Any plans on fixing this?
>>
>> fix your code. i can't see any possible reason to generate names that
>> long. you would have to come up with some amazing reasons to support
>> your claim that you need it.

>
> You obviously don't write Perl with a Lisp mindset. If you
> auto-generate code on the fly it is not always easy to design it such
> that names won't conflict. In my case I'm working with an application
> server which can have a *huge* base of dynamically generated code. A
> potential workaround is to use only hashtables and store anonymous
> subroutines in them, but this is far from an insignificant rewrite and
> looses some flexibility. After googling I find I'm not the only one who
> has had this problem:
>
> http://www.gossamer-threads.com/arch...athnames_P480/
>
> It's also a very silly & trivial bug in Perl, which is acknowledged as a
> known bug. And Python does it right!


But it's a very rare problem to run into. As such, it's not a pressing
issue for [wild guess] 99% of the people who use Perl. As you feel
strongly about it, you may want to address the problem yourself and
submit a patch.

Or you can do the damsel in distress routine ("OH! HELP! SOMEONE PLEASE
HELP ME!") until some shining knight patches it for you. For your sake,
I hope you're cute.

While you're holding your breath, consider rolling your own symbol
table: A hash of long identifiers mapped to computed short ones. As
your program writes Perl source, it can translate the too-long symbols
into the short ones.

Sure, nobody will understand the generated source code. You probably
don't want people editing it directly anyway, so the obfuscation acts as
a deterrent.

--
Rocco Caputo - (E-Mail Removed) - http://poe..perlorg/
 
Reply With Quote
 
Uri Guttman
Guest
Posts: n/a
 
      02-06-2004
>>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

AS> At Fri, 06 Feb 2004 09:36:23 GMT, Uri Guttman wrote:
>>
>> >>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

>>

AS> Not that I'd write such a long identifier, but I've got auto-generated
AS> code that reaches twice that length. Any ideas apart from applying
AS> compression algorithms to the id names? Any plans on fixing this?
>>
>> fix your code. i can't see any possible reason to generate names that
>> long. you would have to come up with some amazing reasons to support
>> your claim that you need it.


AS> You obviously don't write Perl with a Lisp mindset. If you

hell, i wouldn't do anything with a lisp mindset. i would rather toggle
in code by binary switches (done it) than have a lisp mindset.

AS> auto-generate code on the fly it is not always easy to design it such
AS> that names won't conflict. In my case I'm working with an application
AS> server which can have a *huge* base of dynamically generated code. A
AS> potential workaround is to use only hashtables and store anonymous
AS> subroutines in them, but this is far from an insignificant rewrite and
AS> looses some flexibility. After googling I find I'm not the only one who
AS> has had this problem:

AS> http://www.gossamer-threads.com/arch...athnames_P480/

that seems to be an asp problem as much as a perl one. why a path name
gets converted to a sub or identifier name is the question.

but the fact that is it tells me something.

the symbol table is not meant to be a general purpose hash structure. so
using it as such (via symrefs) is very dumb. you say you lose
flexibility by using hashes vs identifiers and that makes even less
sense than lisp mind. i would have done it with dispatch tables and
trees of them and had no issues with the names as i stay out of the
symtable unless i have to. you didn't have to do it but you chose
(wrongly) to use symbols for that. symbols are usually human written and
read so a limit of 255 chars is fine. hash keys have no length limit so
that is better for any auto generated stuff.

uri

--
Uri Guttman ------ (E-Mail Removed) -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
 
Reply With Quote
 
Alex Shinn
Guest
Posts: n/a
 
      02-07-2004
Rocco Caputo <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
>
> But it's a very rare problem to run into. As such, it's not a pressing
> issue for [wild guess] 99% of the people who use Perl. As you feel
> strongly about it, you may want to address the problem yourself and
> submit a patch.


I will probably end up doing so. The md5sum is an interesting idea,
but I don't like even insignificant probabilities of clashes, and at times
I need to debug the generated code so readable names are a plus.

> Or you can do the damsel in distress routine ("OH! HELP! SOMEONE PLEASE
> HELP ME!") until some shining knight patches it for you. For your sake,
> I hope you're cute.


How about I just say I'm cute and hide behind my gender-neutral
first name and race-neutral last name? Any brave knights out there?

> While you're holding your breath, consider rolling your own symbol
> table: A hash of long identifiers mapped to computed short ones. As
> your program writes Perl source, it can translate the too-long symbols
> into the short ones.


The more I think about this the uglier it gets. When you generate code like

$var1 = expr1;
$var2 = expr2;

sub func1 { <some-expr-of-var1> }
sub func2 { func1(<some-expr-of-var2>) }

replacing all of those with nested hash-tables gets really convoluted:

$hash = $globalhash{$modulename};

$hash{var1} = expr1;
$hash{var2} = expr2;

$hash{func1} = sub { <some-expr-of-$hash{var1}> }
$hash{func2} = sub { &{$hash{func1}}(<some-expr-of-$hash{var2}>) }

Maybe the above example doesn't look *too* horrible, but the more
variable references and subroutines you have the more cryptic it is.
And I do have to debug the generated code sometimes. That plus
all the places where I have to rewrite the code generators makes
patching Perl the easiest solution.

Thanks for your help,
Alex
 
Reply With Quote
 
Alex Shinn
Guest
Posts: n/a
 
      02-07-2004
Uri Guttman <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> >>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

>
> AS> You obviously don't write Perl with a Lisp mindset. If you
>
> hell, i wouldn't do anything with a lisp mindset. i would rather toggle
> in code by binary switches (done it) than have a lisp mindset.


I wasn't suggesting you do, nor was I suggesting there is anything
superior about Lisp. It just encourages another style of programming
called meta-programming. And the nice thing about Perl is TMTOWDI -
you can meta-program and write code generators if you want.

> the symbol table is not meant to be a general purpose hash structure. so
> using it as such (via symrefs) is very dumb.


I'm not using it as a hash table, I'm actually writing Perl *code* and so
the natural solution is to use identifiers. Using hash-tables is a clumsy
workaround. Regardless, I don't understand your animosity and don't
appreciate being called dumb. I can only assume you feel threatened by
something you don't understand and feel the need to put it down.

--
Alex
 
Reply With Quote
 
Uri Guttman
Guest
Posts: n/a
 
      02-07-2004
>>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

AS> The more I think about this the uglier it gets. When you generate
AS> code like

AS> $var1 = expr1;
AS> $var2 = expr2;

AS> sub func1 { <some-expr-of-var1> }
AS> sub func2 { func1(<some-expr-of-var2>) }

AS> replacing all of those with nested hash-tables gets really convoluted:

AS> $hash = $globalhash{$modulename};

AS> $hash{var1} = expr1;
AS> $hash{var2} = expr2;

AS> $hash{func1} = sub { <some-expr-of-$hash{var1}> }
AS> $hash{func2} = sub { &{$hash{func1}}(<some-expr-of-$hash{var2}>) }

this is cleaner code to generate IMO

$hash{func1} = sub { <some-expr-of-$hash{var1}> }
$hash{func2} = sub { $hash{func1}->(<some-expr-of-$hash{var2}>) }

AS> Maybe the above example doesn't look *too* horrible, but the more
AS> variable references and subroutines you have the more cryptic it
AS> is. And I do have to debug the generated code sometimes. That
AS> plus all the places where I have to rewrite the code generators
AS> makes patching Perl the easiest solution.

and you could do a global replace on all sub defs and sub calls to use
the hashes. in fact you could do this as a pass AFTER you generate all
the code. it would almost be as easy as:

s/sub\s*(\w+)/\$hash{$1} = sub/g ;
s/(\w+)\(/\$hash{$1}->(/g ;

the second one will probably need a tighter way to find your sub names
and not find perl funcs. but i leave that as an exercise to you. (hint:
use a /e and call a sub. in there check for the existance of the
generated sub name and only replace if found). (another hint: if all
your sub names are very long then just look for a minimum size to match)

see, simple. i will send you a bill.

uri

--
Uri Guttman ------ (E-Mail Removed) -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
 
Reply With Quote
 
Uri Guttman
Guest
Posts: n/a
 
      02-07-2004
>>>>> "AS" == Alex Shinn <(E-Mail Removed)> writes:

AS> Uri Guttman <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...

AS> You obviously don't write Perl with a Lisp mindset. If you
>>
>> hell, i wouldn't do anything with a lisp mindset. i would rather toggle
>> in code by binary switches (done it) than have a lisp mindset.


AS> I wasn't suggesting you do, nor was I suggesting there is anything
AS> superior about Lisp. It just encourages another style of programming
AS> called meta-programming. And the nice thing about Perl is TMTOWDI -
AS> you can meta-program and write code generators if you want.

well, you brought up lisp mindset. them's fighting words!

and i have generated code in several projects so i understand the
issues.

>> the symbol table is not meant to be a general purpose hash structure. so
>> using it as such (via symrefs) is very dumb.


AS> I'm not using it as a hash table, I'm actually writing Perl *code*
AS> and so the natural solution is to use identifiers. Using
AS> hash-tables is a clumsy workaround. Regardless, I don't
AS> understand your animosity and don't appreciate being called dumb.
AS> I can only assume you feel threatened by something you don't
AS> understand and feel the need to put it down.

but you are using it as a hash table in that you are creating names in
it. true they are simple (if long identifiers) but they are just
entries. the symtable has this max id restriction so you have to convert
to a regular hash table. my point was that assuming the symtable is a
normal hash with infinite length keys was wrong. and i have railed
against symrefs (which you aren't using) plenty of times so it carried
over here.

but see my other post just now for a solution that should work and be
very easy to do.

uri

--
Uri Guttman ------ (E-Mail Removed) -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
XSD Error: "Expecting '.', identifier or quoted identifier withmultiple tables. S_K ASP .Net 0 08-06-2008 09:22 PM
How can I fix the warning C4786 'identifier' : identifier wastruncated to 'number' characters in the debug information...? babakandme@yahoo.com C++ 2 12-11-2007 08:17 PM
identifier not found error, undeclared identifier aarthi28@gmail.com C++ 2 02-26-2007 02:11 AM
Trying to declare identifier locally and hide a file scope identifier Kobu C Programming 7 02-25-2005 05:19 AM
dhcp-client-identifier in CNR (Cisco Network Registrar) vigi98 Cisco 0 07-15-2003 06:18 PM



Advertisments