Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > UTF-8 locales

Reply
Thread Tools

UTF-8 locales

 
 
saroj.yadav@gmail.com
Guest
Posts: n/a
 
      03-22-2005
As I understand it (correct me, if I am wrong) Unicode came into
picture so that a document containing multiple language characters can
be supported like somebody can write a document comparing Korean and
Chinese in French language.

Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
HP, Solaris) platforms support Unicode by supporting language/region
specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.

Now in order to use UTF-8 for Japanese, I have to set locale to
ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
ko_KR.UTF-8.

With this approach it's not possible to mix multiple language
characters. Doesn't this defeat the whole purpose of Unicode ?
Am I missing something ?

Thanks in advance for any insight you can provide.

 
Reply With Quote
 
 
 
 
Christian Bau
Guest
Posts: n/a
 
      03-22-2005
In article <(E-Mail Removed) .com>,
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> As I understand it (correct me, if I am wrong) Unicode came into
> picture so that a document containing multiple language characters can
> be supported like somebody can write a document comparing Korean and
> Chinese in French language.
>
> Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
> HP, Solaris) platforms support Unicode by supporting language/region
> specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.
>
> Now in order to use UTF-8 for Japanese, I have to set locale to
> ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
> ko_KR.UTF-8.
>
> With this approach it's not possible to mix multiple language
> characters. Doesn't this defeat the whole purpose of Unicode ?
> Am I missing something ?


A locale describes much more than just the characters.

There are for example the rules for formatting a number. Threethousand
twohundred and six hundredths is 3,200.06 in one locale and 3.200,06 in
another and 3'200.06 in a third locale.

There are rules how to display monetary values. There are rules how to
sort text. There are rules how to display dates. They all will be
different between locales.
 
Reply With Quote
 
 
 
 
Spacen Jasset
Guest
Posts: n/a
 
      03-23-2005
(E-Mail Removed) wrote:
> As I understand it (correct me, if I am wrong) Unicode came into
> picture so that a document containing multiple language characters can
> be supported like somebody can write a document comparing Korean and
> Chinese in French language.
>
> Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
> HP, Solaris) platforms support Unicode by supporting language/region
> specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.
>
> Now in order to use UTF-8 for Japanese, I have to set locale to
> ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
> ko_KR.UTF-8.
>
> With this approach it's not possible to mix multiple language
> characters. Doesn't this defeat the whole purpose of Unicode ?
> Am I missing something ?
>
> Thanks in advance for any insight you can provide.
>



I am not certain how this works on Unix platforms. However I suspect
that the locale has an effect on local conventions as the previous
poster mentioned. I.e date formats and what not.

You still should be able to display any language no matter what locale
you have set, i.e in ja_JP.UTF-8 you should still be able to display
French an Chinese and so on. Try it. ( You must have fonts capable of
mapping those characters of course, if it doesn't work try changing font
- look on the web for names of the fonts that support the glyphs that
are needed )
 
Reply With Quote
 
CBFalconer
Guest
Posts: n/a
 
      03-23-2005
(E-Mail Removed) wrote:
>
> As I understand it (correct me, if I am wrong) Unicode came into
> picture so that a document containing multiple language characters
> can be supported like somebody can write a document comparing
> Korean and Chinese in French language.
>
> Now, I am looking at all UNIX platforms and seems like all Unix
> (AIX, HP, Solaris) platforms support Unicode by supporting
> language/region specific UTF-8 locales like fr_FR.UTF-8,
> ja_JP.UTF-8, ko_KR.UTF-8 etc.


.... snip ...

This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
solaris, etc. in its name. c.l.c deals with the actual portable
language as defined in the C standard. As soon as you have to
specify a system or compiler, you become off-topic.

--
"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare


 
Reply With Quote
 
Kevin Bracey
Guest
Posts: n/a
 
      03-23-2005
In message <(E-Mail Removed)>
CBFalconer <(E-Mail Removed)> wrote:

> (E-Mail Removed) wrote:
> >
> > Now, I am looking at all UNIX platforms and seems like all Unix
> > (AIX, HP, Solaris) platforms support Unicode by supporting
> > language/region specific UTF-8 locales like fr_FR.UTF-8,
> > ja_JP.UTF-8, ko_KR.UTF-8 etc.

>
> ... snip ...
>
> This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
> solaris, etc. in its name. c.l.c deals with the actual portable
> language as defined in the C standard. As soon as you have to
> specify a system or compiler, you become off-topic.


Which is of course why the whole locale system and wchar_t are effectively
useless for portable programming. Everything about them is
implementation-defined, so you can't even talk about them in comp.*.c

Is there anywhere where one could discuss "best practice" or standardisation
for these facilities?

--
Kevin Bracey, Principal Software Engineer
Tematic Ltd Tel: +44 (0) 1223 503464
182-190 Newmarket Road Fax: +44 (0) 1728 727430
Cambridge, CB5 8HE, United Kingdom WWW: http://www.tematic.com/
 
Reply With Quote
 
Michael Mair
Guest
Posts: n/a
 
      03-23-2005
Kevin Bracey wrote:
> In message <(E-Mail Removed)>
> CBFalconer <(E-Mail Removed)> wrote:
>
>
>>(E-Mail Removed) wrote:
>>
>>>Now, I am looking at all UNIX platforms and seems like all Unix
>>>(AIX, HP, Solaris) platforms support Unicode by supporting
>>>language/region specific UTF-8 locales like fr_FR.UTF-8,
>>>ja_JP.UTF-8, ko_KR.UTF-8 etc.

>>
>>... snip ...
>>
>>This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
>>solaris, etc. in its name. c.l.c deals with the actual portable
>>language as defined in the C standard. As soon as you have to
>>specify a system or compiler, you become off-topic.

>
>
> Which is of course why the whole locale system and wchar_t are effectively
> useless for portable programming. Everything about them is
> implementation-defined, so you can't even talk about them in comp.*.c
>
> Is there anywhere where one could discuss "best practice" or standardisation
> for these facilities?


Ack.
I really would be interested in that, too.


Cheers
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.
 
Reply With Quote
 
CBFalconer
Guest
Posts: n/a
 
      03-23-2005
Michael Mair wrote:
> Kevin Bracey wrote:
>> CBFalconer <(E-Mail Removed)> wrote:
>>> (E-Mail Removed) wrote:
>>>
>>>> Now, I am looking at all UNIX platforms and seems like all Unix
>>>> (AIX, HP, Solaris) platforms support Unicode by supporting
>>>> language/region specific UTF-8 locales like fr_FR.UTF-8,
>>>> ja_JP.UTF-8, ko_KR.UTF-8 etc.
>>>
>>>... snip ...
>>>
>>> This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
>>> solaris, etc. in its name. c.l.c deals with the actual portable
>>> language as defined in the C standard. As soon as you have to
>>> specify a system or compiler, you become off-topic.

>>
>> Which is of course why the whole locale system and wchar_t are
>> effectively useless for portable programming. Everything about
>> them is implementation-defined, so you can't even talk about them
>> in comp.*.c
>>
>> Is there anywhere where one could discuss "best practice" or
>> standardisation for these facilities?

>
> Ack.
> I really would be interested in that, too.


I would consider comp.programming quite suitable for this sort of
general internationalization discussion.

--
Chuck F ((E-Mail Removed)) ((E-Mail Removed))
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!


 
Reply With Quote
 
websnarf@gmail.com
Guest
Posts: n/a
 
      03-27-2005
Kevin Bracey wrote:
> CBFalconer <(E-Mail Removed)> wrote:
> > (E-Mail Removed) wrote:
> > > Now, I am looking at all UNIX platforms and seems like all Unix
> > > (AIX, HP, Solaris) platforms support Unicode by supporting
> > > language/region specific UTF-8 locales like fr_FR.UTF-8,
> > > ja_JP.UTF-8, ko_KR.UTF-8 etc.

> >
> > ... snip ...
> >
> > This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
> > solaris, etc. in its name. c.l.c deals with the actual portable
> > language as defined in the C standard. As soon as you have to
> > specify a system or compiler, you become off-topic.

>
> Which is of course why the whole locale system and wchar_t are
> effectively useless for portable programming. Everything about
> them is implementation-defined, so you can't even talk about
> them in comp.*.c


Yeah, amazing he doesn't see the irony of his own statement. The
wchar_t types and functions are only portable in the very narrow scope
of "I can compile that source code to do something" sense. But since
the definition of wchar_t is so open ended, system dependent, and in
fact *NOT* universal in any real sense, there's no way you can actually
use it for any truly portable applications.

And there's no way to somehow "augment" the specification, or use these
types/function in some special way to *make* them portable. There are
no ifdef tricks or anything like that -- you're just screwed. The
whole wchar_t idea literally has no relevant functional content.

C's whole misconception of what it means to be portable, is in fact
what *prevents* it from being portable where it counts, on issues like
localization.

> Is there anywhere where one could discuss "best practice" or
> standardisation for these facilities?


Well more importantly, portability in general, really can't be
discussed in comp.lang.c. I lurk on comp.programming, so you can
always throw questions about portability there.

---
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Datagrid sorting from different locales. =?Utf-8?B?YWxhbg==?= ASP .Net 0 04-21-2005 11:21 AM
Bogus locales on Mac Ellen Herzfeld Python 3 07-16-2004 05:07 PM
Correct use of locales? William Payne C++ 3 12-13-2003 09:24 PM
Python Locales under windows Moof Python 1 10-28-2003 07:27 PM
Problem installing facets into locales explicitly David List C++ 0 08-06-2003 03:37 PM



Advertisments