Dealing with accented characters
 

I don't do a lot of perl, but what I do usually always involves working
with data in LDAP directories. I have a script now that I run every
Monday to produce a CSV file containing a dump of some key user data
from my directory. I pass that CSV on to some other people and they pull
it in to Excel for viewing.

We recently started allowing accented characters in the directory and
I've noticed that the resulting CSV file ends up with some garbage in
the fields where accepted chars are being used. I'm trying to figure out
what I need to do to handle the accented characters properly.

My script runs from my WinXP laptop, using ActivePerl 5.8.x with the
latest perl-ldap module, connecting to a Sun directory server running on
a Solaris/SPARC box. It simply retrieves the attributes of interest for
all users in one branch of my DIT and outputs those attributes in CSV
format to a file on my laptop. I then email that file to the people who
need it.

Any help is greatly appreciated!

Mark

Re: Dealing with accented characters
 

Quoth Mark Drummond <mark@gangwarily.ca>:
> I don't do a lot of perl, but what I do usually always involves working
> with data in LDAP directories. I have a script now that I run every
> Monday to produce a CSV file containing a dump of some key user data
> from my directory. I pass that CSV on to some other people and they pull
> it in to Excel for viewing.
>
> We recently started allowing accented characters in the directory and
> I've noticed that the resulting CSV file ends up with some garbage in
> the fields where accepted chars are being used. I'm trying to figure out
> what I need to do to handle the accented characters properly.
>
> My script runs from my WinXP laptop, using ActivePerl 5.8.x with the
> latest perl-ldap module, connecting to a Sun directory server running on
> a Solaris/SPARC box. It simply retrieves the attributes of interest for
> all users in one branch of my DIT and outputs those attributes in CSV
> format to a file on my laptop. I then email that file to the people who
> need it.

That's all good, but we can't help without

1. A minimal script that reproduces the problem
2. An example of input that fails
3. What output that input should produce
4. What output it, in fact, does produce

I know nothing about how LDAP handles i18n, but, at a guess, your
accented characters are going into the CSV as UTF8 and Excel is reading
them as Windows-1251 (or whatever). The way forward is probably to write
your CSV file in UTF16, which Windows seems to like. Without knowing how
you write it, I can't tell you how to do that.

Ben

--
All persons, living or dead, are entirely coincidental.
benmorrow@tiscali.co.uk Kurt Vonnegut