Converting string from memorystream to binary[] contains leading crap

Question

--Edit with more bgnd information--

A (black box) COM object returns me a string. A 2nd COM object expects this same string as byte[] as input and returns a byte[] with the processed data. This will be fed to a browser as downloadable, non-human-readable file that will be loaded in a client side stand-alone application.

so I get the string inputString from the 1st COM and convert it into a byte[] as follows

BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, inputString);
obj = ms.ToArray();

I feed it to the 2nd COM and read it back out. The result gets written to the browser.

Response.ContentType = "application/octet-stream";
Response.AddHeader("content-disposition", "attachment; filename="test.dat");
Response.BinaryWrite(obj);

The error occurs in the 2nd COm because the formatting is incorrect. I went to check the original string and that was perfectly fine. I then pumped the result from the 1st com directly to the browser and watched what came out. It appeared that somewhere along the road extra unreadable characters are added. What are these characters, what are they used for and how can I prevent them from making my 2nd COM grind to a halt?

The unreadable characters are of this kind:

NUL/SOH/NUL/NUL/NUL/FF/FF/FF/FF/SOH/NUL/NUL/NUL etc

Any ideas?

--Answer--
Use

System.Text.Encoding.UTF8.GetBytes(theString)

rather then

BinaryFormatter.Serialize()

Answer 1

I think you are missing the point of BinarySerialization.

For starters, what Type is formulaXml?

Binary serialization will compact that into a machine represented value, NOT XML! The content will look like:

    ÿÿÿÿ          AIronScheme, Version=1.0.0.0, Culture=neutral, Public

Perhaps you should be looking at the XML serializer instead.

Update:

You want to write out some XML as a 'content-disposition' stream.

To do this, do something like:

byte[] buffer = Encoding.Default.GetBytes(formulaXml);
Response.BinaryWrite(buffer);

That should work like you hoped (I think).

Answer 2

Is the crap in the beginning two bytes long?

This could be the byte order mark of a Unicode encoded string.

http://en.wikipedia.org/wiki/Byte-order_mark

Answer 3

BinaryFormatter is almost certainly not what you want to use.

If you just need to convert a string to bytes, use Encoding.GetBytes for a suitable encoding, of course. UTF-8 is usually correct, but check whether the document specifies an encoding.

Answer 4

The BinaryFormatter's job is to convert the objects into some opaque serialisation format that can only be understood by another BinaryFormatter at the other end.

(Just about to mention Encoding.GetBytes as well, but Jon beat me to it.)

You'll probably want to use System.Text.Encoding.UTF8.GetBytes().

Answer 5

Okay, with your updated information: your 2nd COM object expects binary data, but you want to create that binary data from a string. Does it treat it as plain binary data?

My guess is that something is going to reverse this process on the client side. If it's eventually going to want to reconstruct the data as a string, you need to pick the right encoding to use, and use it on both sides. UTF-8 is a good bet in most cases, but if the client side is just going to write out the data to a file and use it as an XML file, you need to choose the appropriate encoding based on the XML.

You said before that the first few characters of the string were just "<foo>" (or something similar) - does that mean there's no XML declaration? If not, choose UTF-8. Otherwise, you should look at the XML declaration and use that to determine your encoding (again defaulting to UTF-8 if the declaration doesn't specify the encoding).

Once you've got the right encoding, use Encoding.GetBytes as mentioned in earlier answers.