Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > byte count unicode string

Reply
Thread Tools

byte count unicode string

 
 
willie
Guest
Posts: n/a
 
      09-20-2006
John Machin:

>You are confusing the hell out of yourself. You say that your web app
>deals only with UTF-8 strings. Where do you get "the unicode string"
>from??? If name is a utf-8 string, as your comment says, then len(name)
>is all you need!!!



# I'll go ahead and concede defeat since you appear to be on the
# verge of a heart attack
# I can see that I lack clarity so I don't blame you.

# By UTF-8 string, I mean a unicode object with UTF-8 encoding:

type(ustr)
<type 'unicode'>
>>> repr(ustr)

"u'\\u2708'"

# The database API expects unicode objects:
# A template query, then a variable number of values.
# Perhaps I'm a victim of arbitrary design decisions
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      09-20-2006

willie wrote:
> John Machin:
>
> >You are confusing the hell out of yourself. You say that your web app
> >deals only with UTF-8 strings. Where do you get "the unicode string"
> >from??? If name is a utf-8 string, as your comment says, then len(name)
> >is all you need!!!

>
>
> # I'll go ahead and concede defeat since you appear to be on the
> # verge of a heart attack
> # I can see that I lack clarity so I don't blame you.


All you have to do is use terminology like "Python str object, encoded
in utf-8" and "Python unicode object".

>
> # By UTF-8 string, I mean a unicode object with UTF-8 encoding:


There is no such animal as a "unicode object with UTF-8 encoding".
Don't make up terminology as you go.

>
> type(ustr)
> <type 'unicode'>
> >>> repr(ustr)

> "u'\\u2708'"


Sigh. I suppose we have to infer that "ustr" is the same as the "name"
that you were getting as post data. Is that correct?

>
> # The database API expects unicode objects:
> # A template query, then a variable number of values.
> # Perhaps I'm a victim of arbitrary design decisions


And the database will encode those unicode objects as utf-8, silently
truncating any that are too long -- just as Duncan feared? "Arbitrary"
is not the word for it.

Good luck!

Cheers,
John

 
Reply With Quote
 
 
 
 
Diez B. Roggisch
Guest
Posts: n/a
 
      09-20-2006
willie wrote:

> John Machin:
>
> >You are confusing the hell out of yourself. You say that your web app
> >deals only with UTF-8 strings. Where do you get "the unicode string"
> >from??? If name is a utf-8 string, as your comment says, then len(name)
> >is all you need!!!

>
>
> # I'll go ahead and concede defeat since you appear to be on the
> # verge of a heart attack
> # I can see that I lack clarity so I don't blame you.


Could you please change your style of quoting/posting? It is extremely
confusing - not only using a different character than > for citations, but
also appearing to cite yourself while in fact it is your answer one reads.

I'm all for expressing oneself and proving to be an individual - but
communication can get tricky even with standardized manners of doing so,
and there is no need to add more confusion.

> # By UTF-8 string, I mean a unicode object with UTF-8 encoding:
>
> type(ustr)
> <type 'unicode'>
> >>> repr(ustr)

> "u'\\u2708'"


You ARE confusing the hell out of yourself. There is no such thing as a
unciode object with UTF-8 encoding. There are unicode objects. And there
are byte-strings, which may happen to represent text encoded in utf-8.

What you see above is a unicode code point literal - which is translated to
a certain utf-8 string, that looks suspiciously alike because of the way
utf-8 defines the mapping between the code-points of unicode to utf-8.

But it still remains true: a unicode object is a unicode object. And has no
encoding whatsoever!

> # The database API expects unicode objects:
> # A template query, then a variable number of values.
> # Perhaps I'm a victim of arbitrary design decisions


The same happens in java all the time, as java only deals with unicode
strings. And for dealing with it, you also need to explicitly convert them
to the proper encoded byte array. Unfortunate, but true.

Diez
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
byte count unicode string willie Python 2 09-22-2006 11:10 AM
byte count unicode string willie Python 7 09-20-2006 11:45 PM
byte count unicode string willie Python 2 09-20-2006 05:29 PM
byte count unicode string willie Python 1 09-20-2006 09:15 AM
byte count unicode string willie Python 2 09-20-2006 06:43 AM



Advertisments