Why isn't every type of object implicitly serializable?
In my limited understanding, are objects not simply stored on the heap and pointers to them on the stack?
Shouldn't you be able to traverse them programatically, store them in a universal format and also be able to reconstruct them from there?
Some objects encapsulate resources like file pointers or network sockets that can't be deserialized to the state they were in when you serialized the object that contained them.
Example: you shouldn't deserialize an object that serves as an authenticated database connection, because to do so, you'd need the serialized form to contain a plaintext password. This would not be a good practice, because someone might get a hold of the saved serialized form. You also have no idea when you deserialize that the database server is still running, can be accessed, the authentication credentials still valid, etc.
No, because sometimes you don't have all the information in the place that you reconstruct them. Remember that you may not be reconstructing the object in the same context as where you had it; it may be a different machine or even different language.
Even if you only consider objects that don't include OS state, the problem is harder than it looks at first glance. The graph may have cycles. Entities may be referenced from multiple top-level entities.
I tried to outline a universal serialization library in c in a previous answer, and found that there are some hard cases.
Technically, any object in a memory space can be serialized and persisted to a durable medium like a hard drive. After all most OSes page active memory to and from disk and many also have a hibernate style feature. The problem is one of scope, for example: you create a string object in your memory space, its yours to serialize and deserialize as you see fit. When you open a file, the OS gives you a file handle, but the OS still owns the file system containing the actual file object you have a handle to. A file system driver on the other hand has to maintain a persistent database of file handles and other file related metadata.
How much sense would it make to serialize an object that contains a network connection and is responsible for streaming data back from a web server?
What about deserializing it, how would that work? Should it reopen the connection, redownload the file?
You are right in your assumptions, in a way.
It must be possible to partition the set of all objects in the program into groups
1) You have complete information which allows complete deconstruction and reconstruction of the object. Arrays of numbers or strings, structs are good examples.
2) You have construction information. You can reconstruct the object by calling external code. A file is a good example, but it requires that your program has a file abstraction that remembers the construction and state parameters. We can for example save the path to the file and the position in the file. However reconstruction might fail. (For example, the file was deleted or changed)
3) You have no construction information, the object was somehow randomly received.
Here, to be able to serialize the objects completely, we have to go from 3) to 2) to 1). Objects in 3) can be attributes of an object of type 2), and can be retrieved by successfully reconstructing a type 2) object.
A type 2) object however, must be reconstructed by serializing just construction information, which has to be of type 1), for example numbers and strings, true data.
This whole scheme seems costly since if we want to reconstruct the whole program state, we have to work with abstractions that encapsulate objects of type 2). And we have to know what we do when an object cannot be reconstructed. Also, we must be sure that we don't mix objects of these types, that we don't mix in objects of type 3 or 2 where we expect to collect just objects of type 1.