08-31-2010 05:43 PM
Yes, it could be UTF-16BE, UTF-16LE, or UTF-16 (platform-dependent). But why not UTF-8? It's just as much Unicode as the other encodings, and I think a server is much more likely to transmit UTF-8 than anything else.
08-31-2010 10:01 PM
UTF-8 is stored per-byte, UTF-16 is stored in a similar manner but uses shorts instead.
Now I was going to do an example but noticed something, this isn't an encoded String, this binary. UTF-8 and UTF-16 won't convert this to any proper format, the binary needs to be parsed.
Infact I just read BinaryReader and BinaryWriter, (my primary language is C#) these are binary readers and writers. They are the .Net equivilants of DataInput and DataOutput.
08-31-2010 11:31 PM
The original post said that this was a response from a server, not stored data. While the response might theoretically have been UTF-16, I wouldn't expect it. The IANA says that for transmission, the preferred encoding for Unicode text is UTF-8.
Now that I think about it, with all those nulls (\u0000) in the text stream, the server might have been responding with BOCU-1 rather than with any of the UTF-* encodings. If that's the case, then I think OP is going to be stuck with writing a decoder, because to my knowledge, BlackBerry does not support the BOCU-1.
08-31-2010 11:45 PM
True but as I said, BinaryWriter is used to write binary data so I think it is a binary response, not some encoded String.
09-01-2010 12:11 AM
You're right. I somehow glossed over the fact that the server was writing with a BinaryWriter. There's a Java implementation of BinaryReader at Koders.com. I've never used it and don't know anything about it other than it is there.
09-01-2010 09:40 AM
This is the BinaryReader they are using:
It is like java.io.DataInput.
09-01-2010 09:50 AM
In used all UTF encodings that didn't work.
They used following methos in C# for writing data
public override void StartWrite(byte buffer)
and following are the datatypes they are using:
public double BidPx_;
public long BidSize_;
public double AskPx_;
public long AskSize_;
public DateTime TradeDate_;
public string Symbol_;
public string LPName_;
public double HighPx_;
public double LowPx_;
public double PreviousClosePx_;
And i am getting response in bytes..and i want the values they are sending me.
09-01-2010 10:34 AM
The code that Ted pointed you at looks like it should help you with this. Turn your bytes into an input stream (ByteArrayInputStream), then feed it in, and issue readLong, readString etc to match your data structure.
09-01-2010 05:04 PM - edited 09-01-2010 06:36 PM
There we go.
You can use a DataInputStream to read all (.Net, at least the Windows version, is Little-endian and so is BlackBerry so byte order isn't a problem) data types except the String and DateTime.
In .Net, BinaryWriter uses an encoded integer to prefix the String data. This means if your String is only 128 ASCII chars long it will only use 1 byte. If it is 1000 ASCII chars then it will take 2 bytes, etc. There are other names for this and you can figure them out easily. The next issue is encodeing, if you used new BinaryReader(Stream) then the encoding is UTF-8. otherwise you need to check what encoding is used to write the data.
As for the DateTime, this will be your biggest problem. First off it is written as a String so the same operation as above applies, second it is a formatted date. As of 6.0 there is no Date/Calendar parsers in the BB API, that means you need to parse the string manually or write the date out in something easier to read in, like a UTC long. This would be a lot easier to read in and process.
Encoded int/variable-length value: http://en.wikipedia.org/wiki/Variable-length_quant
Date format that you are using is:
dddd, MMMM dd, yyyy
If converted to BlackBerry SimpleDateFormat it would be:
EEEE, MMMM dd, yyyy