I'm trying to run Selenium headless (without the browser appearing). Other questions have pointed to xvfb as the tool to do this. However, it appears highly unstable, crashing all the time, so I'm looking for another alternative.
Is there a non-xvfb way of running Selenium headless?
I'm surprised. I've used Selenium and Xvfb several times without any problems and many others users are doing so too. Can you be more specific about your setup and the problems you are facing? How do you start Xvfb? Can you provide xvfb.log?
However, to answer your question, it is possible to use an X VNC server. See for example this page for some instructions. It's actually hard to be more precise without any details about your configuration.