C# XmlDocument modifies data when you load it?

Question

I'm recieving a Base64 encoded saml assertion from a 3rd party (its signed xml). I cant verify the signature. After close inspection, it appears that when i read in the SAML xml that XmlDocument changes the data. See this code:

string rawXml = Util.DecodeBase64(SAMLResponse);
byte[] b = Encoding.UTF8.GetBytes(rawXml);
MemoryStream s = new MemoryStream();

s.Write(b, 0, b.Length);
s.Flush();
s.Position = 0;
XmlDocument xml = new XmlDocument();
xml.PreserveWhitespace = true;
xml.Load(s);


MemoryStream s2 = new MemoryStream();
xml.Save(s2);
s2.Position = 0;
byte[] b2 = new byte[s2.Length];
s2.Read(b2, 0, b2.Length);

b has a length of 6410 bytes where b2 has a length of 6416 further when i add

int diffcount = 0;
for (int x = 0; x < b2.Length; x++)
{
    if(x > b.Length-1 || b[x]!=b2[x])
    diffcount++;
}

difffcount turns out to be 5572

any ideas how i can get xmldocument to preserve the byte values? Thanks