Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Missing codecs in Python 3.0

Reply
Thread Tools

Missing codecs in Python 3.0

 
 
samwyse
Guest
Posts: n/a
 
      06-03-2009
I have a Python 2.6 program (a code generator, actually) that tries
several methods of compressing a string and chooses the most compact.
It then writes out something like this:
{ encoding='bz2_codec', data = '...'}

I'm having two problems converting this to Py3. First is the absence
of the bz2_codec, among others. It was very convenient for my program
to delay selection of the decoding method until run-time and then have
an easy way to load the appropriate code. Is this gone forever from
the standard libraries?

Second, I would write my data out using the 'string_escape' codec.
It, too, has been removed; there's a 'unicode_escape' codec which is
similar, but I would rather use a 'byte_escape' codec to produce
literals of the form b'asdf'. Unfortunately, there isn't one that I
can find. I could use the repr function, but that seems less
efficient. Does anyone have any ideas? Thanks.
 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      06-03-2009
On Tue, Jun 2, 2009 at 7:15 PM, samwyse <(E-Mail Removed)> wrote:
> I have a Python 2.6 program (a code generator, actually) that tries
> several methods of compressing a string and chooses the most compact.
> It then writes out something like this:
> ┬*{ encoding='bz2_codec', data = '...'}
>
> I'm having two problems converting this to Py3. ┬*First is the absence
> of the bz2_codec, among others. ┬*It was very convenient for my program
> to delay selection of the decoding method until run-time and then have
> an easy way to load the appropriate code. ┬*Is this gone forever from
> the standard libraries?


That appears to be the case. "bz2" is not listed on
http://docs.python.org/3.0/library/codecs.html , but it is listed on
the counterpart 2.6 doc page.
You can always use the `bz2` module instead. Or write your own
encoder/decoder for bz2 and register it with the `codecs` module.

> Second, I would write my data out using the 'string_escape' codec.
> It, too, has been removed; there's a 'unicode_escape' codec which is
> similar, but I would rather use a 'byte_escape' codec to produce
> literals of the form b'asdf'. ┬*Unfortunately, there isn't one that I
> can find. ┬*I could use the repr function, but that seems less
> efficient. ┬*Does anyone have any ideas? ┬*Thanks.


Well, if you can guarantee the string contains only ASCII, you can
just unicode_escape it, and then prepend a "b".
On the other hand, I don't see any reason why repr() would be
inefficient as compared to the codec method.

Cheers,
Chris
--
http://blog.rebertia.com
 
Reply With Quote
 
 
 
 
Carl Banks
Guest
Posts: n/a
 
      06-03-2009
On Jun 2, 7:35*pm, Chris Rebert <(E-Mail Removed)> wrote:
> On Tue, Jun 2, 2009 at 7:15 PM, samwyse <(E-Mail Removed)> wrote:
> > I have a Python 2.6 program (a code generator, actually) that tries
> > several methods of compressing a string and chooses the most compact.
> > It then writes out something like this:
> > *{ encoding='bz2_codec', data = '...'}

>
> > I'm having two problems converting this to Py3. *First is the absence
> > of the bz2_codec, among others. *It was very convenient for my program
> > to delay selection of the decoding method until run-time and then have
> > an easy way to load the appropriate code. *Is this gone forever from
> > the standard libraries?

>
> That appears to be the case. "bz2" is not listed onhttp://docs.python.org/3.0/library/codecs.html, but it is listed on
> the counterpart 2.6 doc page.
> You can always use the `bz2` module instead. Or write your own
> encoder/decoder for bz2 and register it with the `codecs` module.


IIRC, they decided the codecs would only be used for bytes<->unicode
encodings in Python 3.0 (which was their intended use all along),
moving other mappings (like bz2) elsewhere. Not sure where they all
went, though.

It was convenient, admittedly, but also confusing to throw all the
other codecs in with Unicode codecs.



Carl Banks
 
Reply With Quote
 
Martin v. L÷wis
Guest
Posts: n/a
 
      06-03-2009
samwyse wrote:
> I have a Python 2.6 program (a code generator, actually) that tries
> several methods of compressing a string and chooses the most compact.
> It then writes out something like this:
> { encoding='bz2_codec', data = '...'}
>
> I'm having two problems converting this to Py3. First is the absence
> of the bz2_codec, among others. It was very convenient for my program
> to delay selection of the decoding method until run-time and then have
> an easy way to load the appropriate code. Is this gone forever from
> the standard libraries?


bz2 compression is certainly not gone from the standard library; it
is still available from the bz2 module.

I recommend that you write it like

{ decompressor = bz2.decompress, data = '...'}

Then you can still defer invocation of the decompressor until you
need the data.

> Second, I would write my data out using the 'string_escape' codec.
> It, too, has been removed; there's a 'unicode_escape' codec which is
> similar, but I would rather use a 'byte_escape' codec to produce
> literals of the form b'asdf'. Unfortunately, there isn't one that I
> can find. I could use the repr function, but that seems less
> efficient. Does anyone have any ideas?


Why does the repr() function seem less efficient? Did you measure
anything to make it seem so?

I would recommend to use repr() exactly.

Regards,
Martin

 
Reply With Quote
 
Benjamin Peterson
Guest
Posts: n/a
 
      06-03-2009
samwyse <samwyse <at> gmail.com> writes:

>
> I have a Python 2.6 program (a code generator, actually) that tries
> several methods of compressing a string and chooses the most compact.
> It then writes out something like this:
> { encoding='bz2_codec', data = '...'}


In 3.x, all codecs which don't directly map between unicode and bytestrings have
been removed.

>
> I'm having two problems converting this to Py3. First is the absence
> of the bz2_codec, among others. It was very convenient for my program
> to delay selection of the decoding method until run-time and then have
> an easy way to load the appropriate code. Is this gone forever from
> the standard libraries?


No, just use the bz2 module in the stdlib.

>
> Second, I would write my data out using the 'string_escape' codec.


Why does the repr seem less efficient?


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Issues with `codecs.register` and `codecs.CodecInfo` objects Karl Knechtel Python 2 07-10-2012 02:49 PM
Missing codecs on WMP Billy Dee Computer Support 1 06-24-2006 07:31 PM
Missing C-Media Codecs PETER NIELSEN Computer Support 6 05-26-2005 02:20 AM
IP2006 SIP Phone codecs missing! Martin 53N 1W VOIP 4 11-22-2004 07:46 PM
how to register private python codecs? Radovan Garabik Python 1 07-01-2003 07:35 PM



Advertisments