Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > getdefaultencoding - how to change this?

Reply
Thread Tools

getdefaultencoding - how to change this?

 
 
Helmut Jarausch
Guest
Posts: n/a
 
      01-20-2011
Hi,
I've searched the net but didn't find the information I need.
Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec.
How can I recompile Python (itself) to change this to iso8859-1 ?
(My favourite editor cannot be told to use unicode.)

Many thanks for a hint,
Helmut.

 
Reply With Quote
 
 
 
 
Helmut Jarausch
Guest
Posts: n/a
 
      01-20-2011
On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:

> Hi,
> I've searched the net but didn't find the information I need. Using
> Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
> even ignores
> export PYTHONIOENCODING=ISO8859-1
>
> locale.getdefaultlocale()[1]
> returns
> 'ISO8859-1'
>
> still sys.stdout is using the ascii codec. How can I recompile Python
> (itself) to change this to iso8859-1 ? (My favourite editor cannot be
> told to use unicode.)
>


Sorry for answering myself. One last trial did succeed.
My locale as root differed from my locale as non-root user.
After changing the root locale and recompiling Python, it works now.





--
 
Reply With Quote
 
 
 
 
Philip Semanchuk
Guest
Posts: n/a
 
      01-20-2011

On Jan 20, 2011, at 10:39 AM, Helmut Jarausch wrote:

> On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:
>
>> Hi,
>> I've searched the net but didn't find the information I need. Using
>> Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
>> even ignores
>> export PYTHONIOENCODING=ISO8859-1
>>
>> locale.getdefaultlocale()[1]
>> returns
>> 'ISO8859-1'
>>
>> still sys.stdout is using the ascii codec. How can I recompile Python
>> (itself) to change this to iso8859-1 ? (My favourite editor cannot be
>> told to use unicode.)
>>

>
> Sorry for answering myself. One last trial did succeed.
> My locale as root differed from my locale as non-root user.
> After changing the root locale and recompiling Python, it works now.


I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


Cheers
Philip
 
Reply With Quote
 
Robert Kern
Guest
Posts: n/a
 
      01-20-2011
On 1/20/11 8:31 AM, Helmut Jarausch wrote:
> Hi,
> I've searched the net but didn't find the information I need.
> Using Python-2.7.1, I know, I can't modify defaultencoding at run time.


You're not supposed to. It must remain 'ascii'. Otherwise, you will break dict
lookups among other things. Can you be specific about why you think you want to
change this? What are you trying to achieve? It looks like you are conflating a
variety of different behaviors below.

> Python even ignores
> export PYTHONIOENCODING=ISO8859-1
>
> locale.getdefaultlocale()[1]
> returns
> 'ISO8859-1'


This does not introspect the same thing as sys.getdefaultencoding().

> still sys.stdout is using the ascii codec.


This reflects your terminal settings, not sys.getdefaultencoding(). I'm not sure
why the PYTHONIOENCODING variable isn't affecting that. It works fine for me.

> How can I recompile Python (itself) to change this to iso8859-1 ?
> (My favourite editor cannot be told to use unicode.)


You wouldn't want to change sys.getdefaultencoding() for this. That parameter
affects how str and unicode objects are converted between each other without an
explicit .encode()/.decode() call, not what encoding Python assumes for the
parsing of the code.

Instead, you want to use an encoding declaration in each file:

http://docs.python.org/reference/lex...g-declarations

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 
Reply With Quote
 
Robert Kern
Guest
Posts: n/a
 
      01-20-2011
On 1/20/11 9:47 AM, Philip Semanchuk wrote:

> I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


Never do that. It breaks dicts and makes your code non-portable.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 
Reply With Quote
 
Philip Semanchuk
Guest
Posts: n/a
 
      01-20-2011

On Jan 20, 2011, at 11:47 AM, Robert Kern wrote:

> On 1/20/11 9:47 AM, Philip Semanchuk wrote:
>
>> I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.

>
> Never do that. It breaks dicts and makes your code non-portable.


I've never been tempted for the very non-portability reason you cite. I didn't know that it would break dicts, though. Thanks for the education.

bye
Philip
 
Reply With Quote
 
Helmut Jarausch
Guest
Posts: n/a
 
      01-20-2011
On Thu, 20 Jan 2011 10:46:37 -0600, Robert Kern wrote:

> On 1/20/11 8:31 AM, Helmut Jarausch wrote:
>> Hi,
>> I've searched the net but didn't find the information I need. Using
>> Python-2.7.1, I know, I can't modify defaultencoding at run time.

>
> You're not supposed to. It must remain 'ascii'. Otherwise, you will
> break dict lookups among other things. Can you be specific about why you
> think you want to change this? What are you trying to achieve? It looks
> like you are conflating a variety of different behaviors below.
>
>> Python even ignores
>> export PYTHONIOENCODING=ISO8859-1
>>
>> locale.getdefaultlocale()[1]
>> returns
>> 'ISO8859-1'

>
> This does not introspect the same thing as sys.getdefaultencoding().
>
>> still sys.stdout is using the ascii codec.

>
> This reflects your terminal settings, not sys.getdefaultencoding(). I'm
> not sure why the PYTHONIOENCODING variable isn't affecting that. It
> works fine for me.
>
>> How can I recompile Python (itself) to change this to iso8859-1 ? (My
>> favourite editor cannot be told to use unicode.)

>
> You wouldn't want to change sys.getdefaultencoding() for this. That
> parameter affects how str and unicode objects are converted between each
> other without an explicit .encode()/.decode() call, not what encoding
> Python assumes for the parsing of the code.
>
> Instead, you want to use an encoding declaration in each file:
>
> http://docs.python.org/reference/lex...html#encoding-

declarations

Thanks Robert,
probably I wasn't too clear about my issue.
I couldn't "print" any non-ascii character to my console although
my console has an en_US.iso88591 locale.

I didn't modfiy anything in Python's source code.
I noticed that my root account still had an ASCII locale.
Now I have changed it such that the root account has the same locale
as non-root accounts. Recompiling Python under such a root account
has rectified things. Now I can print non-ascii characters if they are
properly encoded.

Thanks,
Helmut.




--
 
Reply With Quote
 
Antoine Pitrou
Guest
Posts: n/a
 
      01-20-2011
On 20 Jan 2011 17:20:14 GMT
Helmut Jarausch <(E-Mail Removed)> wrote:
> Thanks Robert,
> probably I wasn't too clear about my issue.
> I couldn't "print" any non-ascii character to my console although
> my console has an en_US.iso88591 locale.


Well, if you want a correct answer, you should paste actual code as
well as its result on your system, rather than start by asking how to
change getdefaultencoding (which you shouldn't do as explained by
others).

> I didn't modfiy anything in Python's source code.
> I noticed that my root account still had an ASCII locale.
> Now I have changed it such that the root account has the same locale
> as non-root accounts. Recompiling Python under such a root account
> has rectified things.


Recompiling Python doesn't change its behaviour wrt. locales and
encodings. All this is done at runtime.

> Now I can print non-ascii characters if they are
> properly encoded.


You can *always* print characters if they are properly encoded. What
you are asking is for Python to guess and do the encoding by itself,
which is a different matter (and a poorly supported one under 2.x;
Python 3 behaves much better in that regard).

Regards

Antoine.


 
Reply With Quote
 
John Pinner
Guest
Posts: n/a
 
      01-21-2011
On Jan 20, 4:46*pm, Robert Kern <(E-Mail Removed)> wrote:

<snips>

>
> Instead, you want to use an encoding declaration in each file:
>
> http://docs.python.org/reference/lex...encoding-decla...


All that this does is tell the interpreter how the source file is
encoded, it does not affect default encodings etc.

John
--


 
Reply With Quote
 
John Pinner
Guest
Posts: n/a
 
      01-21-2011
To answer the OP's original question:

On Jan 20, 2:31*pm, Helmut Jarausch <(E-Mail Removed)> wrote:
> Hi,
> I've searched the net but didn't find the information I need.
> Using Python-2.7.1, I know, I can't modify defaultencoding at run time.


I think you can. There is a function setdefaultencoding in the sys
module, but at startup when site.py runs the function gets deleted
after it has been used, because as others have said it is a *bad* idea
to change the default encoding (although I *think* changing it to utf8
should cause no harm).

so if you reload sys, setdefaultencoding() becomes available again,
and you can use it, with all the caveats mentioned:

import sys
reload(sys)
sys.setdefaultencoding( 'utf-8' )

This works on a Unicode build of Python only.

As has been said, you can change the default encoding in site.py, but
this means that it gets changed for everyone/every Python program on
the system, using setdefaultencoding() as above only changes it for
the running program, and hopefully the change will have been made by
someone who knows what they are doing and is aware of the possible
consequences.

> Python even ignores
> export PYTHONIOENCODING=ISO8859-1
>
> locale.getdefaultlocale()[1]
> returns
> 'ISO8859-1'
>
> still sys.stdout is using the ascii codec.
> How can I recompile Python (itself) to change this to iso8859-1 ?


You don't need to, nor should you.

> (My favourite editor cannot be told to use unicode.)


Maybe you need a new editor. scite works well with Python, and is
cross-platform.

John
--


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
"Change your language and you change your thoughts." Suganya C Programming 0 04-29-2008 01:35 PM
Change the master GridView after detail change? Q. John Chen ASP .Net 0 11-15-2006 05:31 PM
Change the master GridView after detail change? Q. John Chen ASP .Net 0 11-15-2006 05:30 PM
A Paradise DNS address change? What change? There was no change. Tony Neville NZ Computing 7 09-22-2006 01:02 PM



Advertisments