Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Javascript > Counting utf-8 characters -special characters

Reply
Thread Tools

Counting utf-8 characters -special characters

 
 
majna
Guest
Posts: n/a
 
      09-19-2007
I have character counter for textarea wich counting the characters.
Special character needs same place as two normal characters because of
16-bit encoding.
Counter is counting -2 when special character is added like some
language specific char.

How to count specials like 1 char?
tnx

 
Reply With Quote
 
 
 
 
Thomas 'PointedEars' Lahn
Guest
Posts: n/a
 
      09-19-2007
majna wrote:
> I have character counter for textarea wich counting the characters.
> Special character needs same place as two normal characters because of
> 16-bit encoding.


It doesn't.

> Counter is counting -2 when special character is added like some
> language specific char.


"".length === 1

> How to count specials like 1 char?


The same way. ECMAScript 3 implementations use UTF-16 encoded strings. RTFM.


PointedEars
--
Anyone who slaps a 'this page is best viewed with Browser X' label on
a Web page appears to be yearning for the bad old days, before the Web,
when you had very little chance of reading a document written on another
computer, another word processor, or another network. -- Tim Berners-Lee
 
Reply With Quote
 
 
 
 
Thomas 'PointedEars' Lahn
Guest
Posts: n/a
 
      09-19-2007
Thomas 'PointedEars' Lahn wrote:
> majna wrote:
>> I have character counter for textarea wich counting the characters.
>> Special character needs same place as two normal characters because of
>> 16-bit encoding.

>
> It doesn't.
>
>> Counter is counting -2 when special character is added like some
>> language specific char.


Should have been -1. But even if most implementations would not be
UTF-16 safe, that would not have sufficed. UTF-16 does not mean that
the representation of a glyph in that encoding requires always only
16 bits:

http://www.unicode.org/faq/utf_bom.html#6

> "€".length === 1


Windows(-1252). Hmpf. Make that "€" any Unicode glyph (such as "*")
and it is still true.


PointedEars
--
var bugRiddenCrashPronePieceOfJunk = (
navigator.userAgent.indexOf('MSIE 5') != -1
&& navigator.userAgent.indexOf('Mac') != -1
) // Plone, register_function.js:16
 
Reply With Quote
 
Thomas 'PointedEars' Lahn
Guest
Posts: n/a
 
      09-19-2007
Johannes Baagoe wrote:
> Thomas 'PointedEars' Lahn :
>> "".length === 1

>
> Should be, since '' (U+20AC) is represented as a single UTF-16 code
> point,


You mean code *unit*, _not_ code point. The latter is a completely
different thing, the *position* of a Unicode character in the definition tables.

Et non sequitur, as I have encoded my first followup accidentally with
Windows-1252, that is not the real code point of that character (it is
0x80). With UTF-16, you are correct, except that characters beyond
code point 63k, which would require more code units, are seldom used.

> but it is not, e.g., in spidermonkey, which obviously uses UTF-8:
>
> js> e = ""
>
> js> e.length
> 3
> js> for (i = 0; i < e.length; i++) {print(e.charCodeAt(i).toString(16))}
> e2
> 82
> ac


Probably due to your SpiderMonkey build. It works just fine since Mozilla/4.0.

> But then, OP mentions UTF-8 in the subject line.


Doesn't matter. The used document encoding is transparent to the
application. The `value' property of a HTMLTextAreaElement object is of
type DOMString, which is fully compatible to ECMAScript (UTF-16) strings.

>>> How to count specials like 1 char?

>> The same way. ECMAScript 3 implementations use UTF-16 encoded strings.
>> RTFM.

>
> Hmmm. Is there *any* implementation that actually respects the requirement
> of UTF-16?


Most would nowadays. Even Netscape 4.78 yields 1 for "".length.

> Besides, even assuming UTF-16, some "language specific" characters (whatever
> that means...) take up more than one code point. Some characters may even
> use one or more code points according to whether one uses decomposition
> or not, e.g., '' is either U+00E9 or U+0065 U+0301.


No unique Unicode glyph has more than one code point, that would be a major
flaw in the standard (that does not exist). However, a glyph may be
represented by more than one code unit, though, either due to the mere
necessity of its higher code point (position), surrogates or composition
(and in the latter case it consists of several glyphs with their own code
point, and their code units concatenated according to the used encoding).

However, that does not matter for implementations of ECMAScript 3.
Especially, glyph composition is transparent to the application, if it
supports it.

http://www.unicode.org/faq/char_combmark.html#2


PointedEars
--
Prototype.js was written by people who don't know javascript for people
who don't know javascript. People who don't know javascript are not
the best source of advice on designing systems that use javascript.
-- Richard Cornford, cljs, <f806at$ail$1$(E-Mail Removed)>
 
Reply With Quote
 
Thomas 'PointedEars' Lahn
Guest
Posts: n/a
 
      09-19-2007
Thomas 'PointedEars' Lahn wrote:
> Johannes Baagoe wrote:
>> Thomas 'PointedEars' Lahn :
>>>> How to count specials like 1 char?
>>> The same way. ECMAScript 3 implementations use UTF-16 encoded strings.
>>> RTFM.

>> Hmmm. Is there *any* implementation that actually respects the requirement
>> of UTF-16?

>
> Most would nowadays. Even Netscape 4.78 yields 1 for "€".length.


One might argue then that Netscape 4.78 evaluates the Windows-1252 encoded
version of the respective currency mark which is one byte, and that it does
not support Unicode. However, "€".charCodeAt() yields 8364 (not 12,
String.fromCharCode(8365) yields "*", and both "\u20AC".length and
String.fromCharCode(8365).length yield 1.


PointedEars
--
Anyone who slaps a 'this page is best viewed with Browser X' label on
a Web page appears to be yearning for the bad old days, before the Web,
when you had very little chance of reading a document written on another
computer, another word processor, or another network. -- Tim Berners-Lee
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
counting the number of characters that were matched in a regularexpression PugetSoundSylvia@gmail.com Perl Misc 2 04-16-2008 08:18 PM
Counting pairs of newline characters in a string celldee Ruby 2 04-15-2007 12:35 AM
counting up instead of counting down edwardfredriks Javascript 6 09-07-2005 03:30 PM
Counting characters in a string Bill ASP General 0 08-22-2003 05:16 PM
Counting lines/characters in an uploaded .DOC/.RTF file using ASP.NET j ASP .Net 1 07-22-2003 06:23 PM



Advertisments