Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > returning XML as UTF-8 from a servlet

Reply
Thread Tools

returning XML as UTF-8 from a servlet

 
 
Andy Fish
Guest
Posts: n/a
 
      03-02-2004
Hi,

I have a servlet (running under tomcat 4.1, java 1.4.2) that sends XML in
the HTTP body from a servlet. The I want the XML to be encoded in UTF-8.

when I run Tomcat on windows 2000, the XML appears fine on the client end,
but running Tomcat on debian woody linux, accented characters don't appear
correctly. In the XML output stream, each accented character comes out as
two characters, so obviously the fact that it's supposed to be UTF-8 is
being lost.

here's how I'm streaming the XML:

response.setContentType("text/xml");
OutputStream os = response.getOutputStream();
OutputStreamWriter osw = new OutputStreamWriter(os , "UTF-8");
PrintWriter pw = new PrintWriter(osw);
pw.print("..all the xml..")

If, instead of writing to the response object, I write to a
FileOutputStream, the accented characters appear OK in the file.

I'm a bit stuck here because when I wrote this code, I read up all about
character encoding and did what I thought was right, and it all worked on my
Win2000 test system. I can't figure out what could be going wrong on the
linux box.

many thanks for any advice on hints,

Andy


 
Reply With Quote
 
 
 
 
Johannes Koch
Guest
Posts: n/a
 
      03-02-2004
Andy Fish wrote:
> Hi,
>
> I have a servlet (running under tomcat 4.1, java 1.4.2) that sends XML in
> the HTTP body from a servlet. The I want the XML to be encoded in UTF-8.
>
> when I run Tomcat on windows 2000, the XML appears fine on the client end,
> but running Tomcat on debian woody linux, accented characters don't appear
> correctly. In the XML output stream, each accented character comes out as
> two characters, so obviously the fact that it's supposed to be UTF-8 is
> being lost.


How do you check the XML? With a browser?

>
> here's how I'm streaming the XML:
>
> response.setContentType("text/xml");


Maybe you can add the encoding to the HTTP header:
response.setContentType("text/xml;charset=utf-8");

f'up2 c.t.x
--
Johannes Koch
In te domine speravi; non confundar in aeternum.
(Te Deum, 4th cent.)
 
Reply With Quote
 
 
 
 
John C. Bollinger
Guest
Posts: n/a
 
      03-02-2004
Andy Fish wrote:

> Hi,
>
> I have a servlet (running under tomcat 4.1, java 1.4.2) that sends XML in
> the HTTP body from a servlet. The I want the XML to be encoded in UTF-8.
>
> when I run Tomcat on windows 2000, the XML appears fine on the client end,
> but running Tomcat on debian woody linux, accented characters don't appear
> correctly. In the XML output stream, each accented character comes out as
> two characters, so obviously the fact that it's supposed to be UTF-8 is
> being lost.


No, that's not obvious at all. Not from the information you have given.
Unicode provides for logical characters to be composed of two or more
characters; for instance, a lowercase u with an umlaut could be
represented as the latin lowercase 'u' followed by the umlaut "combining
character". Many of the more common combinations also have
single-character representations, including the u-umlaut example, and
pretty much all the "diacriticalized" characters used in Western
European languages. The alternative representations are equivalent as
far as Unicode is concerned, and Unicode processors are permitted to
freely substitute one for another. They should be displayed or printed
the same by a conformant processor.

Moreover, the fact that you are making judgements about the "UTF-8ness"
of the stream based on the character count leads me to wonder whether
perhaps you are confusing characters with bytes / octets, or whether you
misunderstand the nature of character encodings. The character count
has little to do with whether the characters are encoded in UTF-8;
rather it has everything to do with which character or characters have
been encoded. The byte count has more relation to the encoding, but is
still closely tied to the characters that have been encoded.

> here's how I'm streaming the XML:
>
> response.setContentType("text/xml");


Better would probably be "text/xml; charset=UTF-8".

> OutputStream os = response.getOutputStream();
> OutputStreamWriter osw = new OutputStreamWriter(os , "UTF-8");
> PrintWriter pw = new PrintWriter(osw);
> pw.print("..all the xml..")
>
> If, instead of writing to the response object, I write to a
> FileOutputStream, the accented characters appear OK in the file.


As judged how?

> I'm a bit stuck here because when I wrote this code, I read up all about
> character encoding and did what I thought was right, and it all worked on my
> Win2000 test system. I can't figure out what could be going wrong on the
> linux box.


The output part looks okay to me. I suspect you have a different
problem than you think you have.


John Bollinger
http://www.velocityreviews.com/forums/(E-Mail Removed)

 
Reply With Quote
 
Jon A. Cruz
Guest
Posts: n/a
 
      03-02-2004
Andy Fish wrote:
> correctly. In the XML output stream, each accented character comes out as
> two characters, so obviously the fact that it's supposed to be UTF-8 is
> being lost.


No. Not "obviously"

Capture and list the actual *bytes* going across the wire. Inspect them
and then you can say one way or another.


>
> here's how I'm streaming the XML:
>
> response.setContentType("text/xml");
> OutputStream os = response.getOutputStream();


IIRC, you need to set the encoding before the call to getOutputStream().

 
Reply With Quote
 
Hiran Chaudhuri
Guest
Posts: n/a
 
      03-04-2004

"Jon A. Cruz" <(E-Mail Removed)> schrieb im Newsbeitrag
news:(E-Mail Removed)...
> >
> > response.setContentType("text/xml");
> > OutputStream os = response.getOutputStream();

>
> IIRC, you need to set the encoding before the call to getOutputStream().


The encoding needs to be specified on several levels. One is the HTTP
Response header, one is in the XML header ( <?xml version="1.0"
encoding="..."?> ), and finally the output sent to the response's
outputstream need to use the very same encoding as well.

The background is that outputstream just handles bytes. You must ensure
these bytes are in the above mentioned encoding. This can be done by using a
OutputStreamWriter and setting the encoding in the constructor. Now you can
output characters and OutputStreamWriter will ensure that the outputstream
gets the correct bytes.

Hiran


 
Reply With Quote
 
Jon A. Cruz
Guest
Posts: n/a
 
      03-04-2004
Hiran Chaudhuri wrote:
>
> The background is that outputstream just handles bytes. You must ensure
> these bytes are in the above mentioned encoding. This can be done by using a
> OutputStreamWriter and setting the encoding in the constructor. Now you can
> output characters and OutputStreamWriter will ensure that the outputstream
> gets the correct bytes.


My point is that the order of things is very important. In order to get
the response headers to properly reflect what you're going to send, you
need to set things *before* getOutputStream() or getWriter().

That's a point that trips up a lot of people.

 
Reply With Quote
 
Hiran Chaudhuri
Guest
Posts: n/a
 
      03-05-2004

"Jon A. Cruz" <(E-Mail Removed)> schrieb im Newsbeitrag
news:(E-Mail Removed)...
> Hiran Chaudhuri wrote:
> My point is that the order of things is very important. In order to get
> the response headers to properly reflect what you're going to send, you
> need to set things *before* getOutputStream() or getWriter().
>
> That's a point that trips up a lot of people.


That's right.

It should be easy to handle as I have seen servlet containers complaining
about attempts to set headers after the response has been committed. This is
exactly when you first fill the HTTP response body and afterwards care for
the headers.

Hiran


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Offending class: javax/servlet/Servlet.class chamikara Java 1 06-19-2004 05:42 AM
Servlet question(Tomcat, web.xml, servlet-class, servlet-name) circuit_breaker Java 2 04-04-2004 03:26 AM
returning XML as UTF-8 from a servlet Andy Fish XML 7 03-05-2004 01:25 PM
Tomcat: POST from one servlet or internal class to another servlet Sean Clarke Java 1 01-07-2004 02:22 PM
how to get my servlet configuration before the servlet is initialised Andy Fish Java 4 12-17-2003 09:47 PM



Advertisments