emacs working with big xml files

Question

Often I need to read big XML files (> 100 MB) which have a fairly simple structure. I would like to use emacs, but it does not support big files very well. I am thinking maybe I can use a java application which can do SAX or StAX parsing for me and provide me with all necessary operations (e.g. search). What is the "right way" to communicate between emacs and an external java application?

Answer 1

I don't think there is a single "right way", but here are some ideas:

• make your Java app present the XML document as a bunch of threaded articles in groups; have it speak NNTP and use Gnus, or come up with your own protocol and write a Gnus backend for it (see the Web-browsing backends for inspiration)
• think of the XML document as a Lisp program (S-expressions, XML elements, what is the difference?) and write a Swank backend that communicates with SLIME in Emacs, then use SLIME commands to make queries
• write a FUSE driver so you can mount your XML document as a file system and use Dired to explore it.

Answer 2

You can try disable font-lock-mode for that particular buffer. I have opened much bigger files than that using emacs. I noticed slow down only when lines in the file are bigger than my screen and they start to wrap. Letting them go out of the screen and disabling font-lock accelerates performence.