views:

57

answers:

2

I am faced with the following problem. I need to (de)serialize (binary) a stream of objects to a single file on disk. Serialization part is not an issue, just open a stream in append mode and use .Net's BinaryFormatter Serialize method and you are done. The problem with this approach is that I can't just give this stream to BinaryFormatter's deserialize function, what it contains is not a single instance of the object I've serialized.

Does a common solution to this problem exists? All objects serialized to a given stream are of the same type, so at least we don't need to figure out what is to be deserialized, that's a given, but it doesn't seem to suggest a way out of this to me.

Clarification based on replies: The number of objects sent in is expected to be large and it is therefore infeasible to hold them all in a wrapper collection (as flushing to disk would require to load them all into memory -> add the new ones -> flush to disk).

  • Normally when you serialize a single object you get a file that contains:

[Object]

  • What I am creating is a file that contains:

[Object][Object][Object][Object]...[Object]

And I need to deserialize the individual Object instances.

Thanks in advance!

Answer: Since the answer is alluded to in this thread (with sufficient clarity), but never explicitly stated, I thought I'll state it here:

while (fileStream.Position < fileStream.Length)
    messages.Add((Message)formatter.Deserialize(fileStream));

The BinaryFormatter will deserialize one object at a time as desired :) You might want to cache the fileStream.Length property, since the length appears to be re-computed every time you call the property, slowing things down. I've got no clue why that didn't work the first time I tried it before posting this question, but it does work flawlessly now.

+1  A: 

Try putting your objects into a serializable collection (I believe List is serializable), then (de)serializing that object

EDIT in response to clarification: I just realized that this question has the same answer as this question. Rather than try and reinvent an answer, I'd just take a look at Mark Gravell's answer, or to this one

Josh E
the objects are streamed in during the life time of the system and there is a very large number of them expected, I need to flush. With holding them in wrapper-collection I will have to hold them all in memory and re-write the entire file every time I want to flush to disk.
United
the first suggestion doesn't work, on account of needing another library (aka 6month+ approval process through legal, if I'm lucky). The second one is intriguing. For the second answer, it appears they've got the normal serialization to deserialize the series of objects from one file without those objects being in the collection. In my tests that didn't work, so I guess I'll try to look into it further + maybe follow up in that SO question.
United
A: 

A file is a serialization, so I would think sending your stream directly to a file would do what you seem to want. A more specific explanation of your situation would help, to provide a more useful answer. (I wish I could enter this as a 'comment', but somehow the comment button is not available to me.)

R. Hill
Serialization is not the problem. The problem is that if I serialize one object at a time (which I must) and append them to one stream, I cannot deserialize the resulting stream, because the entire stream does not represent a single object.
United
Basically, normally when you serialize an object to a file, the file will contain that one object (i.e. [Object]) so to deserialize you do Object obj = binaryFormatter.Deserialize(fs, MyObjType); What I have is file of the form [Object][Object]...[Object] so I can't just call the deserialize function on this file, since what I provided the deserializer with is not a serialization of a single object.
United
What I was thinking is put some delimiters in it (provided there exists a combination of symbols that a serializer can never produce). That way I can manually parse the file into chunks (representing single serialized object) in one stream and give chunks to another stream, which in turn is given to the deserializer.
United