XmlSerializer works by generating code to do the serialization/deserialization, and storing this in a temporary assembly. (This approach gives good performance for repeated serialization/deserialization but (traditionally) shockingly awful performance for the first run)
To help mitigate the shocking performance, from VS2005 onwards (and earlier using less well known techniques), you can explicitly create the serialization assembly at build time and ship it with your main assembly.
If you don't create/ship the serialization assembly, then the framework tends to throw an exception when it's looking for it, though normally it catches the exception, builds the assembly on the fly, and gets on with things. If you're running under a debugger, with 'break-on-throw' set though, it can be a bit alarming to have the FileNotFound exception being thrown deep in the bowels of the framework.
Are you sure that the FileNotFound exception is directly associated with the serialization failure you're seeing? Have you tried including the serialization assembly explicitly?