views:

336

answers:

6

Hi, I'm working on an embedded system, and i'm having dramas getting it to send a certain chunk of data across the serial port. I narrowed it down and found that if a 0x9B is present in the message, it corrupts the message.

So i then look up 0x9b (155) on http://www.asciitable.com/, and it's missing! Isn't that a bizarre coincidence!

Any ideas, is this a special character or something?

edit: Here's my serial port settings (8N1):

alt text

-edit- Okay sorry guys, it wasn't the 0x9b causing this, it was an 0x11 character. Which...drumroll... is an XON/XOFF character. I mistakenly had flow control as xon/xoff on the computer, and no flow control on the device! Thanks for the help anyway.

+1  A: 

I'm guessing this is a 0x1B, i.e. the ASCII escape character, with a parity bit in the 8th bit position (it coming from serial communication and all).

Technically, the characters in the ASCII set are all smaller or equal to 0x7F, characters between 0x80 to 0xFF are part of the extended ASCII. The meaning of the codes above 0x7F typically varies, allowing one of multiple characters sets to be handled with codes exactly one byte in size. This capability unfortunately introduces ambiguity for one needs to know the particular extra character set in use (the "code page" if you will).
For example the "ASCII" table referenced in the question doesn't appear to have any character associated with 0x9B, while many other extended ASCII set use this for a "plain" / displayable character (ex: a > looking character in ISO-8859-1, the cent sign (a c-like character) with another set etc.

The possible meaning of the 0x9B character could therefore depend on the [implied] character set in use with the underlying application. But as said, earlier, it looks more like the characters are encoded on 7 bits (hence are likely "pure" ASCII characters) with one parity bit.

mjv
Hmm, i'm not so sure: my comms setting are 8N1 - 8 bits, no parity, 1 stop bit
Chris
@Chris, it could be a _logical_ parity bit, i.e. one managed at the level of the file/content, not one added with the serial communication device. (Kind of a "belt + suspenders" setup if the serial com also includes a parity bit.) You could verify this by asserting that for example you never have 0x41 or 0x42 (but instead 0xC1 and 0xC2 respectively) and that conversely, you always have 0x43 (and never 0xC3). (Guesses...)
mjv
@Chris: also, remember that it is not because one side of the serial communication is set as 8 bits no parity that the other side couldn't be 7 bit plus parity... (on second thought, possibly odd, _literally_ since typically serial coms use an even parity bit, not an odd parity...)
mjv
However i suspect you're right: this message has to go through some vendor API's before getting out the com port, i'll bet they're dropping the bit 8 and getting this character confused somehow... frigging embedded programming with buggy obscure api's!
Chris
It's just that plenty of other >0x80 characters get through just fine...
Chris
@Chris, I see... I'm just making suggestions; educated guesses of sorts... You may need to manually analyze some of these packets; What do these > 0x7F character "mean"? Is this generally textual data, is there some binary mixed amid? What does the "message corruption" indicated in the question effectively means (does it break the protocol, or merely makes a packet which doesn't match the expected pattern and can't be parsed etc...
mjv
Okay sorry, it was a stupid mistake on my part. I had flow control off on the device, and xon/xoff on the computer, so the computer was eating 0x11's. The 0x9b was a red herring.
Chris
NP. Glad you sorted this out!
mjv
+4  A: 

Careful, friend. You have come across the ASCII symbol of the beast. It is not to be displayed, transmitted or encoded in any way.

Those who even dare to discuss it on Stack Overflow will soon find their internet connection has been termi

Assaf Lavie
Also http://www.theasciicode.com.ar/ascii-codes-table/ascii-codes-155.html
Assaf Lavie
hahaha... ;) we were joking at work saying i've discovered some secret nazi ascii symbol.
Chris
We enter the realm of BLOCK ASCII!
ItzWarty
+3  A: 

In ANSI escape sequences, 0x9B is the one-character Control Sequence Introducer (the multi-character version which is more familiar is ESC-[.

caf
+2  A: 

0x9B is the CSI or "Control Sequence Introducer" its part of the C1 set of control codes, see here: http://www.search.com/reference/C0_and_C1_control_codes

Assuming the data is passing through a layer that processes C1 control codes it not surprising that there are a few bytes missing after this character as it is used to indicate the beginning of an ansi escape sequence. The bytes are disappearing because some layer is pealing them off as part of an instruction. More info on that here: http://en.wikipedia.org/wiki/Control_Sequence_Introducer

Obviously can't guarantee this is your problem but its where I would start digging in the api documentation based on the symptoms you described.

Mark
+2  A: 

If the character before 0x9B is 0x10 (DLE - data link escape character) that might explain the lost characters you see. Some devices use DLE as a control command indicator and the subsequent character is the command. If DLE characters aren't escaped, the usual sign is a loss of 2 characters in the stream, or strange behavior from the device. Escape DLE characters with DLE. So, in your case if your data stream includes:

... 0x10 0x9b ...

you would have to write

... 0x10 0x10 0x9b ...

Misk
A: 

For anyone coming to this question just because of the title: 0x9B / 155 is missing from ASCII tables because it isn't an ASCII character. ASCII characters are only 7 bits wide, meaning there are only 128 of them, there simply is no character 155.

[Community Wiki because it doesn't actually answer the question, only the title.]

Jörg W Mittag
Well, 154 and 156 are in the ascii table i was looking at, just 155 was missing!Anyway i figured out the problem, it was an xon/xoff config error.
Chris