Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > how to print the GREEK CAPITAL LETTER delta under utf-8 encoding

Reply
Thread Tools

how to print the GREEK CAPITAL LETTER delta under utf-8 encoding

 
 
=?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7zS?=
Guest
Posts: n/a
 
      05-29-2007
I lookup the utf-8 form of delta from the link.
http://www.fileformat.info/info/unic...0394/index.htm

and then I want to print it in the python ( I work under windows)

#!/usr/bin/python
#coding=utf-8

print "\xce\x94"

but the result is not the 'delta' but an unknown character.

 
Reply With Quote
 
 
 
 
=?GB2312?B?Ik1hcnRpbiB2LiBMbyJ3aXMi?=
Guest
Posts: n/a
 
      05-29-2007
人言落日是天涯,望极天涯不见家 schrieb:
> I lookup the utf-8 form of delta from the link.
> http://www.fileformat.info/info/unic...0394/index.htm
>
> and then I want to print it in the python ( I work under windows)
>
> #!/usr/bin/python
> #coding=utf-8
>
> print "\xce\x94"
>
> but the result is not the 'delta' but an unknown character.


I assume you print to the terminal (cmd.exe). This cannot work;
the terminal (usually) does not interpret the characters in UTF-8.
Instead, you should print a Unicode string, e.g.

print u"\N{GREEK CAPITAL LETTER DELTA}"

or

print u'\u0394'

This should work as long as your terminal supports printing
the letter at all.

Regards,
Martin
 
Reply With Quote
 
 
 
 
=?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7zS?=
Guest
Posts: n/a
 
      05-29-2007
On 5月29日, 下午1时34分, "Martin v. Lo"wis" <(E-Mail Removed)> wrote:
> 人言落日是天涯,望极天涯不见家 schrieb:
>
> > I lookup the utf-8 form of delta from the link.
> >http://www.fileformat.info/info/unic...0394/index.htm

>
> > and then I want to print it in the python ( I work under windows)

>
> > #!/usr/bin/python
> > #coding=utf-8

>
> > print "\xce\x94"

>
> > but the result is not the 'delta' but an unknown character.

>
> I assume you print to the terminal (cmd.exe). This cannot work;
> the terminal (usually) does not interpret the characters in UTF-8.
> Instead, you should print a Unicode string, e.g.
>
> print u"\N{GREEK CAPITAL LETTER DELTA}"
>
> or
>
> print u'\u0394'
>
> This should work as long as your terminal supports printing
> the letter at all.
>
> Regards,
> Martin


yes, it could print to the terminal(cmd.exe), but when I write these
string to file. I got the follow error:

File "E:\Tools\filegen\filegen.py", line 212, in write
self.file.write(data)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
position 0
: ordinal not in range(12

but other text, in which include "chinese characters" got from
os.listdir(...), are written to the file OK. why?

 
Reply With Quote
 
=?GB2312?B?Ik1hcnRpbiB2LiBMbyJ3aXMi?=
Guest
Posts: n/a
 
      05-29-2007
> yes, it could print to the terminal(cmd.exe), but when I write these
> string to file. I got the follow error:
>
> File "E:\Tools\filegen\filegen.py", line 212, in write
> self.file.write(data)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
> position 0
> : ordinal not in range(12


Yes, when writing to a file, you need to define an encoding, e.g.

self.file.write(data.encode("utf-8"))

You can use codecs.open() instead of open(),
so that you can just use self.file.write(data)

Alternatively, you can find out what sys.stdout.encoding is,
and use that when encoding data for the terminal (falling back
to "utf-8" when .encoding is not available on the file).

> but other text, in which include "chinese characters" got from
> os.listdir(...), are written to the file OK. why?


Your version of Windows uses a code page that supports Chinese
characters in the byte-oriented character set. These are normally
encoded using the "mbcs" encoding (except that the terminal likely
uses a different encoding). So if you use "mbcs" instead of "utf-8",
you might be able to read the text as well.

Regards,
Martin
 
Reply With Quote
 
=?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7zS?=
Guest
Posts: n/a
 
      05-29-2007
On 5月29日, 下午3时05分, "Martin v. Lo"wis" <(E-Mail Removed)> wrote:
> > yes, it could print to the terminal(cmd.exe), but when I write these
> > string to file. I got the follow error:

>
> > File "E:\Tools\filegen\filegen.py", line 212, in write
> > self.file.write(data)
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
> > position 0
> > : ordinal not in range(12

>
> Yes, when writing to a file, you need to define an encoding, e.g.
>
> self.file.write(data.encode("utf-8"))
>
> You can use codecs.open() instead of open(),
> so that you can just use self.file.write(data)
>
> Alternatively, you can find out what sys.stdout.encoding is,
> and use that when encoding data for the terminal (falling back
> to "utf-8" when .encoding is not available on the file).
>
> > but other text, in which include "chinese characters" got from
> > os.listdir(...), are written to the file OK. why?

>
> Your version of Windows uses a code page that supports Chinese
> characters in the byte-oriented character set. These are normally
> encoded using the "mbcs" encoding (except that the terminal likely
> uses a different encoding). So if you use "mbcs" instead of "utf-8",
> you might be able to read the text as well.
>
> Regards,
> Martin


Thanks a lot!
I want to just use the utf-8. how could I convert my 'mbcs' encoding
to the utf-8 and write it to the file?
I have replaced the open() to codecs.open()

but it still can not decode the 'mbcs', the error is as follow:

File "E:\Tools\filegen\filegen.py", line 213, in write
self.file.write(data)
File "C:\Python25\lib\codecs.py", line 638, in write
return self.writer.write(data)
File "C:\Python25\lib\codecs.py", line 303, in write
data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position
32: ordinal
not in range(12



 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      05-29-2007
En Tue, 29 May 2007 04:24:15 -0300, 浜鸿█钀芥棩鏄ぉ娑紝鏈涙瀬澶╂动涓嶈瀹
<(E-Mail Removed)> escribi贸:

> On 5鏈29鏃, 涓嬪崍3鏃05鍒, "Martin v. Lo"wis" <(E-Mail Removed)> wrote:


>> > yes, it could print to the terminal(cmd.exe), but when I write these
>> > string to file. I got the follow error:

>>
>> > File "E:\Tools\filegen\filegen.py", line 212, in write
>> > self.file.write(data)
>> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
>> > position 0
>> > : ordinal not in range(12

>>
>> Yes, when writing to a file, you need to define an encoding, e.g.
>>
>> self.file.write(data.encode("utf-8"))
>>
>> You can use codecs.open() instead of open(),
>> so that you can just use self.file.write(data)
>>

> Thanks a lot!
> I want to just use the utf-8. how could I convert my 'mbcs' encoding
> to the utf-8 and write it to the file?
> I have replaced the open() to codecs.open()
>
> but it still can not decode the 'mbcs', the error is as follow:
>
> File "E:\Tools\filegen\filegen.py", line 213, in write
> self.file.write(data)
> File "C:\Python25\lib\codecs.py", line 638, in write
> return self.writer.write(data)
> File "C:\Python25\lib\codecs.py", line 303, in write
> data, consumed = self.encode(object, self.errors)
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position
> 32: ordinal
> not in range(12


Just to provide an example of what MvL already said:

py> line = u"Delta=\u0394"
py> f = open("data.txt","w")
py> f.write(line.encode("utf8"))
py> f.close()
py> print repr(open("data.txt").read())
'Delta=\xce\x94'

py> import codecs
py> f = codecs.open("data2.txt","w","utf8")
py> f.write(line)
py> f.close()
py> print repr(open("data2.txt").read())
'Delta=\xce\x94'

--
Gabriel Genellina

 
Reply With Quote
 
Ragnar Ouchterlony
Guest
Posts: n/a
 
      05-29-2007
tis 2007-05-29 klockan 09:05 +0200 skrev "Martin v. Lo"wis":
> Yes, when writing to a file, you need to define an encoding, e.g.
>
> self.file.write(data.encode("utf-8"))
>
> You can use codecs.open() instead of open(),
> so that you can just use self.file.write(data)


If I for some reason can't open the object myself or needs encoding on
other file-like objects, I think the following wrapper function is of
use (it essentially does what codecs.open() does but takes a file-object
instead of a filename):

def filewrapper(f, encoding=None, errors='strict'):
if encoding is None:
return f

info = codecs.lookup(encoding)
srw = codecs.StreamReaderWriter(f, info.streamreader,
info.streamwriter, errors)
# Add attributes to simplify introspection
srw.encoding = encoding
return srw

I find this especially useful for changing how stdout and friends does
it's encoding, e.g:

>>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
>>> print u"邃 \N{GREEK CAPITAL LETTER DELTA}"


Useful if you don't want to append .encode() to everything you print out
that potentially can contain non-ascii letters.

/Ragnar

 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      05-29-2007
En Tue, 29 May 2007 15:16:52 -0300, Ragnar Ouchterlony
<(E-Mail Removed)> escribi:

> If I for some reason can't open the object myself or needs encoding on
> other file-like objects, I think the following wrapper function is of
> use (it essentially does what codecs.open() does but takes a file-object
> instead of a filename):
>
> def filewrapper(f, encoding=None, errors='strict'):
> if encoding is None:
> return f
>
> info = codecs.lookup(encoding)
> srw = codecs.StreamReaderWriter(f, info.streamreader,
> info.streamwriter, errors)
> # Add attributes to simplify introspection
> srw.encoding = encoding
> return srw
>
> I find this especially useful for changing how stdout and friends does
> it's encoding, e.g:
>
>>>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
>>>> print u"邃 \N{GREEK CAPITAL LETTER DELTA}"

>
> Useful if you don't want to append .encode() to everything you print out
> that potentially can contain non-ascii letters.


Isn't the same as codecs.EncodedFile?

--
Gabriel Genellina

 
Reply With Quote
 
Ragnar Ouchterlony
Guest
Posts: n/a
 
      05-29-2007
tis 2007-05-29 klockan 16:08 -0300 skrev Gabriel Genellina:
> >>>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
> >>>> print u"邃 \N{GREEK CAPITAL LETTER DELTA}"

> >
> > Useful if you don't want to append .encode() to everything you print out
> > that potentially can contain non-ascii letters.

>
> Isn't the same as codecs.EncodedFile?
>


No, codecs.EncodedFile() doesn't do exactly the same.

If I understand it correctly, EncodedFile() takes a byte stream as input
and produces another bytestream (with possibly different encoding) as
output in both read and write.

My function (as well as codecs.open()) decodes a byte stream for read
and produces a unicode object or encodes a unicode object for write and
produces a byte stream.

At least, I were unable to create the same behaviour as my filewrapper()
using EncodedFile(). If you are able to do that, I'm interested in how
you do it.

/Ragnar

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
File names are changed into capital letter in Python CE and WM5 mujunshan@gmail.com Python 1 11-09-2008 12:35 AM
delta cycle?? (delta delay) srikanth.padava VHDL 0 02-28-2008 09:01 AM
How Do I Put the Greek Letter Pi on a Button? kvnsmnsn@hotmail.com HTML 9 05-10-2007 10:50 PM
Inputting Greek letter in textarea dungping HTML 27 09-02-2005 09:02 PM



Advertisments