Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Ruby (http://www.velocityreviews.com/forums/f66-ruby.html)
-   -   Decode/encode Unicode (http://www.velocityreviews.com/forums/t852568-decode-encode-unicode.html)

Kless 08-28-2008 08:51 AM

Decode/encode Unicode
 
How to decode a String type to Unicode?
And, to encode Unicode to String?

Thomas B. 08-28-2008 10:14 AM

Re: Decode/encode Unicode
 
Kless wrote:
> How to decode a String type to Unicode?
> And, to encode Unicode to String?


Unicode is not fully supported in Ruby 1.8.X, it will be in Ruby 1.9.
--
Posted via http://www.ruby-forum.com/.


James Gray 08-28-2008 12:11 PM

Re: Decode/encode Unicode
 
On Aug 28, 2008, at 3:51 AM, Kless wrote:

> How to decode a String type to Unicode?


$ ruby -KU -e 'p "R=E9sum=E9".unpack("U*")'
[82, 233, 115, 117, 109, 233]

> And, to encode Unicode to String?


$ ruby -KU -e 'p [82, 233, 115, 117, 109, 233].pack("U*")'
"R=E9sum=E9"

Hope that helps.

James Edward Gray II=


Stephen Boisvert 08-28-2008 05:19 PM

Re: Decode/encode Unicode
 
On Thu, Aug 28, 2008 at 9:51 AM, Kless <jonas.esp@googlemail.com> wrote:
> How to decode a String type to Unicode?
> And, to encode Unicode to String?
>
>


This is a trickier question than you probably realize.
You might need to know what encoding you want to transform your string
to and from. Sometimes you may not know - mp3 ID3 tag info is
supposed to be UTF8 for example but often isn't. If you don't specify
an encoding it may or may not end up being encoded based on your
locale settings.

Things I have seen mentioned:

iconv - a unix based library for translating between different
encodings. Requires you tell it what you decoding from and encoding
to.
http://wiki.rubyonrails.org/rails/pages/iconv

unidecode - I have used this translate Unicode strings to ASCII by
simple character mapping- it works about 99% of the time so you will
need error handling for when it goes bonk.

Setting KCODE:

$KCODE = 'UTF8'
or
$KCODE = 'u'
or
use -Ku at the command line. This tells Ruby that you want to be
using UTF8 encoding

require 'jcode'

will give you access to some code developed to deal with Japanese
Unicode encoding that address some of the problems with the String
class struggles with Unicode chars like jlength. Always keep in mind
that a lot of the string methods in Ruby do not work properly with
Unicode because they count letters wrong.

Good luck.

Stephen Boisvert
http://blog.ennuyer.net


Kless 08-28-2008 06:12 PM

Re: Decode/encode Unicode
 
Thanks! It will help until rb 1.9 been more extended.

On Aug 28, 1:11*pm, James Gray <ja...@grayproductions.net> wrote:
> On Aug 28, 2008, at 3:51 AM, Kless wrote:
>
> > How to decode *a String type to Unicode?

>
> $ ruby -KU -e 'p "Résumé".unpack("U*")'
> [82, 233, 115, 117, 109, 233]
>
> > And, to encode Unicode to String?

>
> $ ruby -KU -e 'p [82, 233, 115, 117, 109, 233].pack("U*")'
> "Résumé"
>
> Hope that helps.
>
> James Edward Gray II




All times are GMT. The time now is 04:15 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.