Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Encoding conundrum

Thread Tools

Re: Encoding conundrum

Dave Angel
Posts: n/a
On 11/20/2012 04:49 PM, Daniel Klein wrote:
> With the assistance of this group I am understanding unicode encoding
> issues much better; especially when handling special characters that are
> outside of the ASCII range. I've got my application working perfectly now
> However, I am still confused as to why I can only use one specific encoding.

Who says you can only use one? You need to use the right encoding for
the device or file you're talking with, and if different devices want
different encodings, then you must use multiple ones. Only one can be
the default, however, and that's where some problems come about.

> I've done some research and it appears that I should be able to use any of
> the following codecs with codepoints '\xfc' (chr(252)) '\xfd' (chr(253))
> and '\xfe' (chr(254)) :
> ISO-8859-1 [ note that I'm using this codec on my Linux box ]
> cp1252
> cp437
> latin1
> utf-8
> If I'm not mistaken, all of these codecs can handle the complete 8bit
> character set.

What 8 bit character set? This is a nonsense statement. If you mean
all of them can convert an 8 bit byte to SOME unicode character, then
fine. But they won't convert each such byte to the SAME unicode
character, or they'd be the same encoding.

> However, on Windows 7, I am only able to use 'cp437' to display (print)
> data with those characters in Python. If I use any other encoding, Windows
> laughs at me with this error message:
> File "C:\Python33\lib\encodings\", line 19, in encode
> return codecs.charmap_encode(input,self.errors,encoding_m ap)[0]
> UnicodeEncodeError: 'charmap' codec can't encode character '\xfd' in
> position 3: character maps to <undefined>
> Furthermore I get this from IDLE:
>>>> import locale
>>>> locale.getdefaultlocale()

> ('en_US', 'cp1252')
> I also get 'cp1252' when running the same script from a Windows command
> prompt.
> So there is a contradiction between the error message and the default
> encoding.
> Why am I restricted from using just that one codec? Is this a Windows or
> Python restriction? Please enlighten me.

I don't know much about Windows quirks anymore. I haven't had to use it
much for years.



Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Encoding conundrum Ian Kelly Python 4 11-21-2012 01:02 PM
Complex Custom Control Conundrum Iain ASP .Net 0 02-17-2006 08:38 PM
Provider MPLS IP-Enabled Frame Relay Cloud Routing Conundrum Scott Cisco 0 12-23-2004 02:13 AM
Bookmark Filing Conundrum mapmaker Firefox 2 12-21-2004 03:22 AM
Clunky Cache Code Conundrum? Jim Owen ASP .Net 4 07-03-2003 04:49 PM