F# Serialize Discriminated Union why so many bytes?

Question

I'm trying to serialize some data for a UDP packet stream and I'm getting a huge overhead from serialization. If I encode a FileData with a 1k Byte array I get back 2312 bytes. How would I reduce this overhead without encoding and decoding everything myself?

[<Serializable>]
type Response =
    | FileSize of String * int64
    | FileData of int64 * byte[]
with
    static member Decode(packet : byte[]) =
        use ms = new MemoryStream(packet)
        let bf = new BinaryFormatter()
        bf.Deserialize(ms) 
        |> unbox<Response>

    member this.Encode() =
        use ms = new MemoryStream()
        let bf = new BinaryFormatter()
        bf.Serialize(ms, this)
        ms.GetBuffer()

Answer 1

BinaryFormatter is probably the most concise formatter out of the box, so the only option would be to "do it yourself".

The reason you're getting the extra overhead has to do with all of the other information saved with serialization. Serializing doesn't just save the data, it also stores the meta data (ie: all of the types, etc) in a way that the entire object can be reconstructed safely. This adds overhead.

Fortunately, the overhead doesn't really increase as the data gets larger. If you saved a 2k byte array, you'd probably get back ~3300 bytes instead of the ~2300 bytes - since the overhead should be near constant (provided the type information doesn't change).