Best compression algorithm for XML?

Question

Hello, I barely know a thing about compression, so bear with me (this is probably a stupid and painfully obvious question).

So lets say I have an XML file with a few tags.

<verylongtagnumberone>
  <verylongtagnumbertwo>
    text
  </verylongtagnumbertwo>
</verylongtagnumberone>

Now lets say I have a bunch of these very long tags with many attributes in my multiple XML files. I need to compress them to the smallest size possible. The best way would be to use an XML-specific algorithm which assigns individual tags pseudonyms like vlt1 or vlt2. However, this wouldn't be as 'open' of a way as I m trying to go for, and I want to use a common algorithm like DEFLATE or LZ. It also helpes if the archive was a .zip file.

Since I'm dealing with plain text (no binary files like images), I'd like an algorithm that suits plain text. Which one produces the smallest file size (lossless algorithms are preferred)?

By the way, the scenario is this: I am creating a standard for documents, like ODF or MS Office XML, that contain XML files, packaged in a .zip.

EDIT: The 'encryption' thing was a typo; it should ave ben 'compresion'.

Answer 1

It seems like you're more interested in compression rather than encryption. Is that the case? If so, this might prove an interesting read even though is not an exact solution.

Answer 2

Hey, I hope I understood correctly what you need to do... First thing I would like to say is that there are no good or bad compression algorithmss for text - zip, bzip, gzip, rar, 7zip are good enough to compress anything that has a low entrpy - i.e. large file with small character set. If I would have to use them I would choose 7zip at my first choice, rar as a second and zip as third. But the difference is very small so you should try whatever easier for you. Second - I could not understand what you are trying to encrypt. Suppose that this is an XML file then you should first compress it using your favourite compression algorithm and then encrypt it using your favourite encryption algorithm. In most cases any modern algorithm implemented for instance in PGP will be secure enough for anything. Hope that helps,

       Jack David

Answer 3

Your alternatives are:

• Use a webserver that supports gzip compression. It'll auto compress all outgoing html. There's a small CPU penalty though.
• Use something like JSON. It'll drastically reduce the size of the message
• There's also a binary XML but I have not tried it myself.

Answer 4

By the way, the scenario is this: I am creating a standard for documents, like ODF or MS Office XML, that contain XML files, packaged in a .zip.

then I'd suggest you use .zip compression, or your users will get confused.

Answer 5

There is a W3 (not-yet-released) standard named EXI (Efficient XML Interchange).

Should become THE data format for compressing XML data in the future (claimed to be the last necessary binary format). Being optimized for XML, it compresses XML more ways more efficient than any conventional compression algorithm.

With EXI, you can operate on compressed XML data on the fly (without the need to uncompress or re-compress it).

EXI = (XML + XMLSchema) as binary.

And here you go with the opensource implementation (don't know if it's already stable):
Exificient

Answer 6

Another alternative to "compress" XML would be FI (Fast Infoset).

XML, stored as FI, would contain every tag and attribute only once, all other occurrences are referencing the first one, thus saving space.

See:

Very good article on java.sun.com, and of course
the Wikipedia entry

The difference to EXI from the compression point of view is that Fast Infoset (being structured plaintext) is less efficient.

Other important difference is: FI is a mature standard with many implementations.
One of them: Fast Infoset Project @ dev.java.net