Java Development

[Karthi_rs]

No peter can't get you clearly there is no way to resolve this ah??

FYI today only i tried this

 I just put the given feed in my webserver and through blackberry when i hit the url and get the feed contents asusual there are three special characters appended in the beginning and also the characters like ä, ö (which are in the feed contents) are get collapsed.

But if i open the given feed and copy all the contents to a notepad and save it in the same name and hit the url then there are no special characters appended and also the characters like ä, ö are also displayed correctly, there is nothing problem with this feed.

My system in windows7, also when i checked both these feeds with the beyond compare it tells that the feed which works correctly is in ANSI PC and the given feed is in UTF-8 BOM Unix. From this i think they are encoded the content right? if so then there should be a way to decode the content?? is there any way??

I have tried this also

httpConnection = (HttpConnection)Connector.open(url);

httpConnection.setRequestProperty("Content-Type", "text/plain; charset=UTF-8"), but it dint work for me

[Karthi_rs]

If the BOM is present you said that using it to determine the conversion means what type of conversion do i need to do and which part of code??

[peter_strange]

I was hoping you would do a little more investigation yourself.  I can answer your question and explain why the characters are collapsed, but I think you can too.  So I'd like you to try.  Because you will learn more if you figure this out yourself. 

Here is the start of the trail you need to follow:

EFBBBF is a Beginning of Message (BOM) indicator.  I don't know for sure, but I suspect all Browsers and even some text viewing packages will interpret this correctly.  So what does this mean?

The Wikipedia link I gave you says this:

The UTF-8 representation of the BOM is the byte sequence 0xEF,0xBB,0xBF, which appears as the ISO-8859-1 characters  in most text editors and web browsers not prepared to handle UTF-8.

These characters are telling you you need to treat the data as UTF-8.  So what does that mean and how do you do it?

Your mission, should you choose to accept it, is to find out.  This tape will self destruct in 5 seconds. 

BTW that was a reference to an old TV show called Mission Impossible.  If you have not seen it, or the films, you will not understand.  Apologies...   And the post won't self destruct. 

And the BlackBerry default character encoding is ISO-8859-1, which explains what you see when you process these bytes.  . 

[Karthi_rs]

Thanks for your explanations and ideas peter and i will do some more investigation by myself to resolve this

[Karthi_rs · Accepted solution]

Single mild change in the code resolves my problem Peter

Yes it needs to specify the encoding type to convert the array of bytes using the specified character encoding

while (-1 != (length = inputStream.read(responseData)))  {
                        Session.response.append(new String(responseData, 0, length, "UTF-8"));
                    }

[peter_strange]

Yep that was it!  Well done finding that. 

Can you mark this as solved?

[italobb]

This problem is explained in this article "Support - HTTP transmits incorrectly when User-Agent includes "BlackBerry"". The solution is to use:

HttpConnection conn = (HttpConnection) Connector.open(url);

conn .setRequestProperty("x-rim-transcode-content", "none");