What are the ramifications of null bytes and multipart/form-data?

Question

A third party is sending us a flat file that is supposed to contain exclusively printable ASCII characters. However, we've discovered that there's a string of about 50 0x00 bytes in the middle of the file.

We want to be able to upload the file to our web application, but I've discovered that Django doesn't seem to like the null characters in the multipart/form-data. If I remove the null characters, the upload succeeds. (Sorry I don't have the stack trace available at the moment, but will produce one if necessary)

We can pre-process the file to remove the null characters and/or work with our third party to fix their file generator, but I don't like to leave mystical problems like this.

Does this sound like a bug in Django or is there some aspect of multipart/form-data that I don't fully understand? Do I need to set a transfer encoding of some sort so Django doesn't get hung up on the null characters?

Answer 1

Nope, no transfer-encoding is needed (or ever used by browsers) on form-data. It's perfectly valid to include a run of 50 null bytes in a multipart/form-data value... indeed given that most binary files contain a lot of nulls that situation should arise as often as not with file uploads!

Which makes me question whether it's really a Django bug, or whether there's not something else going on. Let's have that stacktrace!