Determining binary/text file type?

Question

Namely, how would you tell an archive (jar/rar/etc.) file from a textual (xml/txt, encoding-independent) one?

Answer 1

There's no guaranteed way, but here are a couple of possibilities:

1) Look for a header on the file. Unfortunately, headers are file-specific, so while you might be able to find out that it's a RAR file, you won't get the more generic answer of whether it's text or binary.

2) Count the number of character vs. non-character types. Text files will be mostly alphabetical characters while binary files - especially compressed ones like rar, zip, and such - will tend to have bytes more evenly represented.

3) Look for a regularly repeating pattern of newlines.

Answer 2

You could try the DROID tool.

Answer 3

See http://en.wikipedia.org/wiki/Magic_number_(programming)

Answer 4

If the file consists of the bytes 0x09 (tab), 0x0A (line feed), 0x0C (form feed), 0x0D (carriage return), or 0x20 through 0x7E, then it's probably ASCII text.

If the file contains any other ASCII control character, 0x00 through 0x1F excluding the three above, then it's probably binary data.

UTF-8 text follows a very specific pattern for any bytes with the high order bit, but fixed-length encodings like ISO-8859-1 do not. UTF-16 can frequently contain the null byte (0x00), but only in every other position.

You'd need a weaker heuristic for anything else.

Answer 5

Have a look at the JMimeMagic library.

jMimeMagic is a Java library for determining the MIME type of files or streams.

Answer 6

Just to let you know, I've chosen quite a different path. I my case, there are only 2 types of files, chances that any given file will be a binary one are high. So

1. presume that file is binary, try doing what's supposed to be done (e.g. deserialize)
2. catch exception
3. treat file as textual
4. if that fails, something is wrong with file itself

Answer 7

Run file -bi {filename}. If whatever it returns starts with 'text/', then it's non-binary, otherwise it is. ;-)