Best way to use an InputStream regarding persistance and XML

Question

Hi,

I have a REST webservice that listens to POST requests and grabs hold of an XML payload from the client and stores it initially as an InputStream i.e. on the Representation object you can call getStream().

I want to utilise the XML held in the InputStream and I am begining to think it would be wise to persist it, so I can interrogate the data multiple times - as once you read through it, the object becomes null. So I thought about converting the InputStream to a string. This is not a good idea as DocumentBuilder.parse() from javax.xml.parsers library will only allow you to pass:

• InputStreams
• Files
• URLs
• SAX InputSources

not strings.

What should I really be doing here with InputStreams in relation to parsing XML out of it? Bearing in mind I will want to re-interrogate that XML in future processes by the code.

Answer 1

java.io.StringReader will allow you to use InputSource.

You might want to store the data in a byte[] and then read with ByteArrayInputStream. If it's particular large, you might might want to consider compression. This can be read out iwth GzipInputStream, which should often be wrapped in a BufferedInputStream.

Answer 2

I think you should look into some structures better suited for preserving encodings (ie. more encoding agnostic). For low-level structures, consider byte[] (but be careful with memory deallocation!) or you could try to design a data type that fits your needs.

You could read the InputStream into a ByteArrayOutputStream (using one of the read() methods) and extract the byte[] from there.

Answer 3

Generally, when we're talking persistence, we're talking about writing it to disk or other media. There's a performance hit there, and you have to think about disk space concerns. You'll want to weigh that against the value of having that XML around for the long term.

If you're just talking about holding it in memory (which sounds like what you're asking), then you could allocate a byte array, and read the whole thing into the byte array. The you can use ByteArrayInputStream to read and re-read that stream.

The cost with that is two-fold. First, you're holding a copy in memory, and you need to weigh that against your scalability requirements. Second, parsing XML is somewhat expensive, so it's best to parse it once only, if possible, and save the result in an object.

Edit:

To allocate and read the byte array, you can often (but not always) rely on InputStream's available() method to tell you how much to allocate. and wrap the InputStream with a DataInputStream so that you can call readFully() to suck the whole thing into the byte array with one call.

Edit again:

Read Steen's comment below. He's right that it's a bad idea to use available() in this case.

Answer 4

I would advise to use the Apache Commons IO library. The IOUtils class contains many convenience methods to convert InputStreams to String and vice versa.

Answer 5

If you want to use the XML multiple times, why not parse it once from the InputStream (which is the heavy work), and then hold on to the Document returned?

Answer 6

If you have an InputStream, and want to use it as an XML document, then why aren't you simply parsing it and passing around the Document object? If you want to persist this object, then use a serializers to write it back out as text.

As I noted in my comment to Tom Hawtin, encoding is very important when dealing with XML. Rather than write a long posting here that may miss your specific situation, here's an article that I wrote.

Edit: actually, since my article doesn't specifically talk about web services, I should dive into it a little here. There are two places where the content encoding can be specified: in the XML prologue, or in the Content-Type response header. According to the XML spec, the former is the one that you want to use, and it's what the parser will use. In most cases, that doesn't matter: a webservice set up by a person who doesn't know the spec will typically use a text/xml without a character set specification (which is incorrect but probably not going to cause harm). If they do things correctly, they'll specify application/xml, with utf-8 encoding. However, you should verify what you're getting, so that you don't end up with some strange encoding that the parser can't handle.