Best way to handle light weight (skeletal) representation in web service XML schema?

Question

I'm new to the whole web service space, so pardon the question if it seems stupid or obvious!

I have a number of entities that can be provided by a new web service that I am creating. Some of these entities are composed of very large graphs of objects.

However, at times the client will want to search for some of these entities and get back a (potentially large) result set. I am thinking that this result set should contain a lighter weight representation of these huge entities, rather than the entire entity. After all they are only going to be displayed in a 'entities found' list on a GUI...and then individually retrieved.

What's a good pattern for handling this problem? I am particularly concerned in how it relates to my XML schema? If I use the same XML element type as the heavy weight representation (say with an isSkeleton boolean switch), then all of my XML constraints are rendered useless (because child elements won't be there).

On the other hand, if I make a specialized "lightweight" DTO for every potentially large entity (with its own XML definition) then I need to hassle with the conversion to and from these DTOs and my entities.

Answer 1

Not sure if I'm following what you're describing, but it sounds like you need to do the following:

• In the context of a search, represent different types of entities in a common way
• There are potentially large numbers of entities returned in a result set
• Only need to provide further depth when individual entities are accessed

This sounds like an ideal use for Solr. Solr is the open-source search framework built on top of Lucene. It provides access over HTTP to a Lucene-based search index, using XML as the payload basis. It is REST-oriented, speaks JSON as well, so it's language-agnostic.

Most implementations of Solr run behind-the-scenes, with access to results served up behind a website's search results page. Your requirements might actually be a great fit for exposing access to a Solr server's query directly. All the elements of working with the results are already handled -- paging, filtering, sorting, etc. All the logistics work that you really don't want to do yourself.

As the implementor, your real work is determining what goes in your search index. The index could be structured to hold the common elements of your variant entities, and the search speed would be blazingly fast. Upon retrieval of a given resultset, your application could retrieve deeper details of an entity from its original source (if you didn't want to push the entire entity into the search index, which is another possibility.)