views:

316

answers:

5

Imagine we need to pass a a number structured objects to the web application - for instance, locale, layout settings and a definition of some query. This can be easily done with JSON or XML similar to the following fragment:

<Locale>en</Locale>  
<Layout>  
  <Block id="header">hide</Block>  
  <Block id="footer">hide</Block>  
  <Block id="navigation">minimize</Block>  
</Layout>  
<Query>  
  <What>water</What>  
  <When>  
    <Start>2010-01-01</Start>  
  </When>  
</Query>

However, passing such structures with HTTP implies (roughly speaking) HTTP POST.

Now assume we're limited to HTTP GET. Is there some kind of a standard solution for encoding structured data in HTTP GET request parameters?

I can easily imagine something like:

Locale=en&
Layout.Block.header=hide&
Layout.Block.footer=hide&
Layout.Block.navigation=minimize&
Query.What=water&
Query.When.Start=2010-01-01

But what I'm looking for is a "standard" syntax, if there's any.

ps. I'm surely aware of the problem with URL length. Please assume that it's not a problem in this case.

pps. I'd be also greatful for links to key-value pair URL APIs (like Paypal NVP) which you think to be worth taking a look at.

ppps. We're certainly forseeing the callback URLs but we need HTTP GET key-value pairs as well. The question focuses on the latter.

+2  A: 

It depends on how you want to decode it. Your solution works if you have a system in place that exclusively knows how to split parameters and values along & and = (i.e., PHP), but if that restriction is removed, then it can become much more concise:

Locale{en}Layout{Block(id:header){hide}Block(id:footer){hide}Block(id:navigation){minimize}}Query{What{water}When{Start{2010-01-01}}}

In short, there is no standard (as far as I can find), so be creative according to your needs. You could even use JSON, omitting or encoding whitespace as necessary.

Jon Purdy
Is the syntax you're quoting something standard? JSON is an option indeed, however its elegancy will be completely destoroyed by the URL encoding.
lexicore
Nope. I was just winging it based on what I thought would be concise and look nice.
Jon Purdy
Hah, +1 for winging it
webdestroya
+3  A: 

This idea would only work if you have control on both the server and the client and both are mutually accessible over the network.

Client-end:

The client calling the server will need to pass a call-back URL within HTTP-HEADER of the HTTP-REQUEST. The URL will also have a unique-id and a sample REQUEST may look like this.

//HTTP Request Header

GET / HTTP/1.1[CRLF]
Host: www.your-server.com/server-end.aspx[CRLF]
Connection: close[CRLF]
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.2; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: some referrer[CRLF]
Input-callback-Url: www.your-client.com/inputprovider.aspx?requestid=xxxxxxx[CRLF]
[CRLF]

Server-end:

At the server-endpoint, the server will read the headers of the Http-Request from the client and read the Url as received in "Input-callback-Url". The server then will intiate the Http-Request GET on www.your-client.com/inputprovider.aspx?requestid=xxxxxxx. This URL when called by the server, will render output in JASON or XML form.

<Locale>en</Locale>          
<Layout>          
  <Block id="header">hide</Block>          
  <Block id="footer">hide</Block>          
  <Block id="navigation">minimize</Block>          
</Layout>          
<Query>          
  <What>water</What>          
  <When>          
    <Start>2010-01-01</Start>          
  </When>          
</Query>

Server can receive this response from "Input-callback-Url" and proceed ahead serving original client-request.

this. __curious_geek
We certainly forsee the callback URL (one of the method for handling longer URLs). However this requires significant effort on the client side therefore we don't want it to be the **only** method. We do need the key-value-pair encoding to allow simple links which can be, for instance inserted in a CMS/Wiki/forum. In any case +1 from me for a sensible idea.
lexicore
+1  A: 

I understand you question very good. I like and have been using XML and XML schema since many years. Since a half of year I use JavaScript and jQuery together with RESTfull WFC services, ASP.NET MVC technologies with JSON encoded data for transfer. So I had to ask me the same question some time ago. Here is my answer with a short explanation.

The XML structure which you define can represent one input parameter of our Web Service. If you prefer divide XML structure to different input parameters (like Locale, Layout and Query) all what I write here can be easy modified to this case of parameter coding.

The most native way in my opinion is the way which we implement using out-of-the-box tools. So let us we have our Web Service with a method like MyMethod and an input parameter param1 with the type MyData (something like public int MyMethod (MyData param1)). Class MyData represent our all data from the XML structure. Then the corresponding HTTP GET request should look like

http://server/Service/MyMethod?param1=JsonEncodedData

where JsonEncodedData:

%7B%22locale%22%3A%22en%22%2C%22layout%22%3A%5B%7B%22id%22%3A%22header%22%2C%22value%22%3A%22hide%22%7D%2C%7B%22id%22%3A%22footer%22%2C%22value%22%3A%22hide%22%7D%2C%7B%22id%22%3A%22navigation%22%2C%22value%

or the same without HTML encoding:

{"locale":"en","layout":[{"id":"header","value":"hide"},{"id":"footer","value":"hide"},{"id":"navigation","value":"minimize"}],"query":{"what":"water","when":{"Start":{"Year":2010,"Month":1,"Day":1}}}

This data is a JSON encoded JavaScript object

var myInput = {
  locale: "en",
  layout:[
    {id: "header", value: "hide"},
    {id: "footer", value: "hide"},
    {id: "navigation", value: "minimize"}
  ],
  query:{
    what: "water",
    when: {
      Start:{Year:2010,Month:1,Day:1}
    }
  }
};

REMARK: Because Date class of JavaScript can be serialized in a little different ways I changed a little the representation of start date here. In the "Microsoft world" we can use Sys.Serialization.JavaScriptSerializer.serialize to serialize DateTime .NET type which will encode 2010-01-01 like "\/Date(1262300400000)\/".

We can write in C# an easy Web Method of Web Service Service1.asmx like

[WebMethod]
[ScriptMethod (UseHttpGet = true, ResponseFormat = ResponseFormat.Json)]
public int MyMethod (MyData param1) {
    return param1.query.when.Start.Year;
}

where

public enum BlockType {
    hide,
    minimize
}
public class Block {
    public string id { get; set; }
    //public BlockType value { get; set; }
    public string value { get; set; }
}
public class MyDate {
    public int Year { get; set; }
    public int Month { get; set; }
    public int Day { get; set; }
}
public class When {
    public MyDate Start { get; set; }
}
public class Query {
    public string what { get; set; }
    public When when { get; set; }
}
public class MyData {
    public string locale;
    public List<Block> layout;
    public Query query;
}

And the corresponding client code will looks like following

var myInput = {
  locale: "en",
  layout:[
    {id: "header", value: "hide"},
    {id: "footer", value: "hide"},
    {id: "navigation", value: "minimize"}
  ],
  query:{
    what: "water",
    when: {
      Start:{Year:2010,Month:1,Day:1}
    }
  }
};

$.ajax({
    type: "GET",
    url: "/Service1.asmx/Method2",
    data: {param1: JSON.stringify(myInput)},
    contentType: "application/json; charset=utf-8",
    success: function(data, textStatus, XMLHttpRequest) {
        alert(XMLHttpRequest.responseText);
    },
    error: function(res, status) {
        if (status ==="error") {
            // errorMessage can be an object with 3 string properties:
            //      ExceptionType, Message and StackTrace
            var errorMessage = $.parseJSON(res.responseText);
            alert(errorMessage.Message);
        }
    }
});

For JSON encoding I use in the example json2.js (see http://www.json.org/json2.js and http://www.json.org/js.html). One can use instead JSON jQuery (from http://code.google.com/p/jquery-json/), then JSON.stringify should be replaces to $.toJSON.

Initializing of data of the type MyData can look like

new MyData {
        locale = "en",
        layout = new List<Block> {
            new Block() { id="header", value=BlockType.hide.ToString()},
            new Block() { id="footer", value=BlockType.hide.ToString()},
            new Block() { id="navigation", value=BlockType.minimize.ToString()}
        },
        query = new Query {
            what = "water",
            //when = new When() {Start = new DateTime(2010,1,1)}
            when = new When () {
                Start = new MyDate () {
                    Year = 2010,
                    Month = 1,
                    Day = 1
                }
            }
        }
    };
}

If you use WCF with webHttpBinding (RESTfull endpoints) instead of asmx based web services you can receive more flexible solution.

If use use JSON with HTTP GET you should take in consideration existence of the JSON Hijacking problem (see http://weblogs.asp.net/scottgu/archive/2007/04/04/json-hijacking-and-how-asp-net-ajax-1-0-mitigates-these-attacks.aspx, http://haacked.com/archive/2009/06/25/json-hijacking.aspx). This problem can be solved (or dramatically reduced) in the different ways and it don't stop me from the usage of JSON in HTTP requests, which can be perfect cached.

For more information about the same subject you can look through my answer to a close questions: http://stackoverflow.com/questions/2737525/how-do-i-build-a-json-object-to-send-to-an-ajax-webservice/2738086#2738086 http://stackoverflow.com/questions/2670147/can-i-return-json-from-an-asmx-web-service-if-the-contenttype-is-not-json/2671583#2671583 http://stackoverflow.com/questions/2651091/jquery-ajax-call-to-httpget-webmethod-c-not-working/2656543#2656543

Oleg
Thanks for the verbose post. The problem with JSON encoding is that it is very hard to handle it manually. Controls symbols are URL-encoded, the brackets must match and so on. This syntax is surely suitable for JS-programmers, but it's not friendly for "end"-users which are our target group in this case.We've finally come to the conclusion that there's no "standard" encoding, so we've developed our own syntax. I'll describe it when we're ready with the parser.
lexicore
Sorry but It seems to me nobody (as a person) will be manually build and call an URL with more then 10 parameters. So you should write a program from both side (client and server) which encode and decode some structures represented the data. I try to suggest a native way for both JavaScript and .NET Web Service with a simplest de- and JSON encoding. Disadvantage of encoding from your question is that both client and server have to hold not a structures, but many separated variables. What is better?
Oleg
The requirements are present and not imaginary. This is what our clients need. Actually, there's a lot of APIs like this - there's PayPal NVP, there's Google Charts, there are OGC things like WMS and so on. Th "end-user" is a someone like professional system administrator or application maintainer who needs a simple URL API to integrate our application in his/her environments. Yes, we can provide additional instruments like "here's a thingy you'll have to build your URLs with", but this must not be exclusive since it lowers the acceptance (which influences the sales numbers).
lexicore
"What is better?" - well, I do agree that KVP->object structures is a bit of challenge, and that JSON or XML would be easier to handle here. But it's nothing but a techical challenge which we gladly accept.
lexicore
Oleg
+3  A: 

Well, the standard way to encode data in the URI, is application/x-www-form-urlencoded. The way that most applications handle hierarchical data, is with square brackets in the key-part. Eg.:

Locale=en&
Layout[Block][header]=hide&
Layout[Block][footer]=hide&
Layout[Block][navigation]=minimize&
Query[What]=water&
Query[When][Start]=2010-01-01

This is somewhat application specific, but it's pretty close to being a de-facto standard

troelskn
Is this "close to being a de-facto standard" documented anywhere? I can't recall seeing this syntax before. I'd be grateful for references. Thank you!
lexicore
No, it's anecdotal. Both PHP and Ruby on Rails use that syntax, which is what I based my statement on.
troelskn
You should link to a document that shows this for PHP or RoR...
Frank V
@frank I'm not actually sure such a document exists. I could write one, but that would sort of defeat the purpose.
troelskn
@troelskn: this helps for RoR http://dizzy.co.uk/ruby_on_rails/cheatsheets/form-helpers#example - the html is in section 3.3
Stobor
And for php: http://au.php.net/manual/en/language.variables.external.php
Stobor
A: 

To my knowledge, there are no standards for passing structured data using GET. There are a handful of conventions (the PHP/RoR example above possibly the most prevalent), but nothing that has reached any level of maturity, documentation, reference implementation etc... to be considered a standard.

And it's not too surprising a simple standard hasn't emerged. There are open questions whose answers depend upon specifics of the situation. Should the URL be transparent (as visible query parameters) or opaque (e.g. base64 encoded). Can compression be used? Can/should/must other representations be used/supported, e.g. JSON, binary XML or even Google's protocol buffers). How is URL overflow handled? 10 different people might all give different answers about what needed based on their situation. A standard could attempt to address all these concerns, but it would then be far from simple.

Even if there were a standard, would it make sense to follow it?

It's desirable, comforting even, to follow a standard, knowing that your work is building on the road-tested work of respected field-experts and engineers, and knowing that we are invited to the party and play with other systems that follow the same standard. But when the requiremtents are diverse, as they are here, a standard will typically only give you a middle-ground solution, a medium of comprimise. And when you've only got 2K to play with, comprimising is probably last on your list!

Roll your own and you'll be sure to get exactly what you want. I'd go for representing as binary XML or protocol buffers, gzipped and base64 encoded, but you're millage may vary.

Either way, good luck!

mdma
This is the conclusion I've come to as well. We're on our way already.
lexicore