Creating a binary file from an IntelHex in C#

Question

I'm trying to create a binary file from a intelHex file. Iside the intelHex file I have data and address to which I should write the data inside the binary file. IntelHex file looks like that

:10010000214601360121470136007EFE09D2190140
:100110002146017EB7C20001FF5F16002148011988
:10012000194E79234623965778239EDA3F01B2CAA7
:100130003F0156702B5E712B722B732146013421C7
:00000001FF

So I have 4 lines here with data since the last one tells us thats the end of file. Here is what I'm doing to create the file

        while (!streamReader.EndOfStream)
        {
            string temp = String.Empty;
            int address = 0;
            line = streamReader.ReadLine();
            // Get address for each data
            address = Convert.ToInt32(line.Substring(3, 4), 16);
            // Get data from each line
            temp = line.Substring(7, 2);
            if (temp == "01")
                break;
            else
            {
                temp = line.Substring(9, line.Length - 11);
                string[] array = new string[(temp.Length / 2)];
                int j = 0;
                for (int i = 0; i < array.Length; ++i)
                {
                    array[i] = temp[j].ToString() + temp[j + 1].ToString();
                    j = j + 2;
                }
                temp = String.Empty; 
                for (int i = 0; i < array.Length; ++i)
                {
                    temp = temp + Convert.ToChar(Convert.ToInt32(array[i], 16));

                }
            }
            binaryWriter.Seek(address, SeekOrigin.Begin);
            binaryWriter.Write(temp);
            binaryWriter.Flush();
        }
        Console.WriteLine("Done...\nPress any key to exit...");

The problem here is, that data in binary file in some places is not equal to data from the intelHex file. Looks like there is some random data added to the file and I do not know from where. First time I saw that there is an additional data before the data from the intelHex file. For instance first data line starts with 21, but in binary file I have a number 12 before the 21. I do not know what is wrong here. Hope someone can help me or guide me where I can find some usefull informations about creating binary files in C#

Answer 1

<Generic answer pointing out that a Unicode character (char) is not an octet (byte), and that the code produces the wrong output because binary data is written as Unicode string to the file.>

Hint: use a byte[] for binary data, not a string.

Also: In before answers suggesting to use a StringBuilder for the loop.