«

hello,

my script selects a comment saved as VARCHAR in MySQL and displays it
inside an html page.

the problem is, that the comment contains several special characters, as
mysterious utf-8 hyphens, german umlauts, etc.

i could write a function to parse the comment and substitute special
chars with the relevant html code, but maybe this already exists in some
module?

if not, it'll be hard work, as i've to consider many special chars, and
at least iso-8859-1* and utf-8 as charmaps.

bye
jonas

Jonas Meurer <> wrote:
> hello,
>
> my script selects a comment saved as VARCHAR in MySQL and displays it
> inside an html page.
>
> the problem is, that the comment contains several special characters, as
> mysterious utf-8 hyphens, german umlauts, etc.
>
> i could write a function to parse the comment and substitute special
> chars with the relevant html code, but maybe this already exists in some
> module?

just make the page in utf-8, and you'll save you a lot of troubles


>
> if not, it'll be hard work, as i've to consider many special chars, and
> at least iso-8859-1* and utf-8 as charmaps.

if you insist...
a = u'\u010c'
a.encode('ascii', 'xmlcharrefreplace')


--
-----------------------------------------------------------
| Radovan GarabÃ*k http://melkor.dnp.fmph.uniba.sk/~garabik/ |
| __..--^^^--..__ garabik @ kassiopeia.juls.savba.sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!

On 22/02/2005 Radovan Garabik wrote:
> > i could write a function to parse the comment and substitute special
> > chars with the relevant html code, but maybe this already exists in some
> > module?
>
> just make the page in utf-8, and you'll save you a lot of troubles

ok, how do i do this? simply add a second line with this
# -*- encoding: utf-8 -*-

i use utf8 locales on my machine anyway.

> > if not, it'll be hard work, as i've to consider many special chars, and
> > at least iso-8859-1* and utf-8 as charmaps.
>
> if you insist...
> a = u'\u010c'
> a.encode('ascii', 'xmlcharrefreplace')

this fails as the comment contained several chars that couldn't be
converted.

i've changed my plans, and now will transform the comments to html
before saving them in mysql. this way, the comment never contains
special chars except they weren't filtered out when safed in mysql.

do any filters exist, to transform plain text to html? otherwise i might
use third-party products, as text2html.

what do you think?

bye
jonas

Jonas Meurer wrote:

> On 22/02/2005 Radovan Garabik wrote:
>
>>>i could write a function to parse the comment and substitute special
>>>chars with the relevant html code, but maybe this already exists in some
>>>module?
>>
>>just make the page in utf-8, and you'll save you a lot of troubles
>
>
> ok, how do i do this? simply add a second line with this
> # -*- encoding: utf-8 -*-
>
> i use utf8 locales on my machine anyway.
>
>
>>>if not, it'll be hard work, as i've to consider many special chars, and
>>>at least iso-8859-1* and utf-8 as charmaps.
>>
>>if you insist...
>>a = u'\u010c'
>>a.encode('ascii', 'xmlcharrefreplace')
>
>
> this fails as the comment contained several chars that couldn't be
> converted.
>
> i've changed my plans, and now will transform the comments to html
> before saving them in mysql. this way, the comment never contains
> special chars except they weren't filtered out when safed in mysql.
>
> do any filters exist, to transform plain text to html? otherwise i might
> use third-party products, as text2html.
>
> what do you think?
>
I think you should store your data with a known encoding, then encode it
as necessary for transmission. That way you can provide it in the forms
most relevant to different clients.

regards
Steve

Jonas Meurer wrote:
> hello,
>
> my script selects a comment saved as VARCHAR in MySQL and displays it
> inside an html page.
>
> the problem is, that the comment contains several special characters, as
> mysterious utf-8 hyphens, german umlauts, etc.
>
> i could write a function to parse the comment and substitute special
> chars with the relevant html code, but maybe this already exists in some
> module?
>
> if not, it'll be hard work, as i've to consider many special chars, and
> at least iso-8859-1* and utf-8 as charmaps.
>
> bye
> jonas
If I understand you correctly, just put

<meta http-equiv="CONTENT-TYPE" content="text/html; charset=utf-8">

somewhere in the <head>-section of you HTML-Page.

HTH,
Wolfram

Wolfram Kraus <> wrote:
> Jonas Meurer wrote:
>> hello,
>>
>> my script selects a comment saved as VARCHAR in MySQL and displays it
>> inside an html page.
>>
>> the problem is, that the comment contains several special characters, as
>> mysterious utf-8 hyphens, german umlauts, etc.
>>
>> i could write a function to parse the comment and substitute special
>> chars with the relevant html code, but maybe this already exists in some
>> module?
>>
>> if not, it'll be hard work, as i've to consider many special chars, and
>> at least iso-8859-1* and utf-8 as charmaps.
>>
>> bye
>> jonas
> If I understand you correctly, just put
>
> <meta http-equiv="CONTENT-TYPE" content="text/html; charset=utf-8">
>
> somewhere in the <head>-section of you HTML-Page.

.... and make sure the deault charset of your HTTP server is *OFF* (or
UTF-, since it overrides the per-page setting (most unfortunate).

--
-----------------------------------------------------------
| Radovan GarabÃ*k http://melkor.dnp.fmph.uniba.sk/~garabik/ |
| __..--^^^--..__ garabik @ kassiopeia.juls.savba.sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!

Jonas Meurer wrote:
(...)
> i've changed my plans, and now will transform the comments to html
> before saving them in mysql. this way, the comment never contains
> special chars except they weren't filtered out when safed in mysql.
>
> do any filters exist, to transform plain text to html? otherwise i might
> use third-party products, as text2html.
>
> what do you think?

as you may known mysql 4.1 offers utf-8 support. il would be
wise to keep everything as utf-8: db, html generation and finally
serve, with correct HTTP headers, pages encoded as utf-8.

to do this you might have to fiddle with mysql settings and make
sure that issuing a:

show varibles;

almost all of these settings:

character_set_client latin1
character_set_connection latin1

character_set_database latin1

character_set_results latin1
character_set_server latin1
character_set_system utf8


use utf-8 (as you can see my copy of mysql does not), otherwise i
think bad things will occur.

if you prefer to filter out weird characters and
encode as html &#xxxx entities textile[1] does the job
just fine, you can specify input and output encoding.

cheers,
deelan.

[1] http://dealmeida.net/en/Projects/PyTextile/

--
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
<#me> a foaferson ; foaf:nick "deelan" ;
foaf:weblog <http://blog.deelan.com/> .