Can't read Integer in Stream after encoding it as UTF-8 instead of ASCII

Question

I had problems with Umlauts in ASCII so I encode my Stream as UTF-8 now, which works, but it brings up a problem. I normally read 4 Bytes before ARTIST to determine the length of ARTIST=WHOEVER using

UTF8Encoding enc = new UTF8Encoding();
string response = enc.GetString(message, 0, bytesRead);
int posArtist = response.IndexOf("ARTIST");
BitConverter.ToInt32(message, posArtist - 4);

This works for ASCII perfectly.

The hex-editor examples are just to illustrate that reading the length doesn't work anymore like with ASCII

Here is an example-screenshot from a hex-editor:

"ARTIST=M.A.N.D.Y. vs. Booka Shade" Length = 21

However that doesn't work for the UTF8-encoded stream. Here is a screenshot:

"ARTIST=Paulseq" Length = E but in the picture its 2E.

What am I doing wrong here?

Answer 1

It is an utter mystery how you got 21 out of the ASCII data. The shaded byte is in hex, its real value is 33. There's no way you can get 21 out of BitConverter.ToInt32, that requires bytes values (in hex) 15 00 00 00.

This must have worked by accident but no idea what that accident might look like. Post more code, including the code that writes this.

Answer 2

Only the strings should be UTF-8 encoded/decoded. If you're passing other (non-string) values in binary, the encoders they will destroy them.

Answer 3

My guess is that you are mixing tools. That is a binary stream. It should be read with a BinaryReader and written with a BinaryWriter. When writing text, use Encoder.GetBytes to get the raw bytes to write, and when reading use Encoder.GetString on the raw bytes read. BinaryWriter/Reader have methods for values (like lengths) directly.

Answer 4

your data is wrong - you actually have the character '\0' in the data where there should be binary zeroes

The problem lies in how you created this data, not in the reading of it