Bill the Plumber

I’ve been getting a lot of concern internally at Red Hat as I’ve been pushing for all our products and projects to have a RESTful interface.  Some scorn that there’s no way you can model such complex interactions with PUT, POST, GET, and DELETE.  Many see it as yet another bullshit rehash of distributed protocols.  This is what I promise them:

• Simplicity
• Low footprint requirements.
• True interoperability.  (SOAP interoperability is a joke).
• Flexibility.

If REST and our RESTful interfaces do not have these properties, then I’ve failed.  I really don’t care in the end if any of the architectural principles of Roy’s thesis are broken as long these requirements are met.  Pragmatism has to be the most important thing here.  We can’t fall into religious and academic debates on the purity of a distributed interface.

I edited this to be correct.  I incorrectly assumed that WADL didn’t support HATEOAS.  Apologies Marc Hadley

A friend, colleague, and old mentor of mine, Steve Vinoski (he also introduced me to REST) talks a little bit about his RESTful experiences with his current development team.  It is cool to hear that this stuff is working in action specifically:

For example, I just finished a meeting a couple of hours ago where some client code needs to interact with a particular part of my system via HTTP, but wants XML instead of JSON currently provided.  Simple — it’s just a different representation of the same resources…..[snip] Can you imagine the hoops one would have to jump through with typical RPC-oriented systems [snip]?

He also goes on to talk about how client developers asked for a “REST-like-interface”:

[snip] A document listing all resource URIs, and for each one, the HTTP verbs that apply to it, the representations available from it, and what status codes to expect from invoking operations on it….[snip]for a proper RESTful system, you don’t need a document like that, at least not the type of document they’re asking for.

My guess is that his developers are looking for something a WSDL-like document, or something like WADL. What Steve is getting at is that you don’t need a WADL because HATEOAS allows you to discover how to interact with your services.  In abrowser-based, HTML-based world, this is definately true.  HTML documents are self-describing and renderable by your browser.  Humans can look at a rendered page and now how to interact, serf, and fill and post forms.  Machine-based clients are different though.  They usually exchange more machine-friendly formats like XML and JSON.  They can’t make dynamic decisions and need to know ahead of time what interactions are available and how to perform the interactions.  The de facto way of doing this has been to embed somethign like an Atom link into a document:

<order id="123">
   <amount>$123.45</amount>
   <atom:link rel="CANCEL" href="http://orders.com/orders/123/cancelled"/>
   <atom:link rel="PAYMENT" href="http://orders.com/orders/123/payment" type="application/xml"/>
   ...
</order>

In Atom land, interactions are defined as named, logical, relationships which provide a URI and expected type.  While machine-based clients are not going to be able to make many dynamic decisions on a self-describing document, there are a lot of advantages to HATEOAS that Craig McClanahan talks about.

The thing is, I think Steve’s developers are right.  They need some kind of “RESTful like interface”.  I do agree that you don’t need a document that describes status codes.  The status codes exchanged between a RESTful client and server for a given HTTP operation is well defined by the HTTP specification.  Users should not try to override the meaning of a given status code.  The thing is, even with links, you still need to document which relationships will be available, what they do, and what formats the linked URIs expect.  As a client developer, you need to know this information ahead of time.  For state-changing interactions, you also need to know whether you need to do a PUT or POST.  This is extremely important information because PUT is idempotent and POST is not.  For those URI’s that take query parameters, you will need to define what query parameters there are and what values they expect.  Finally, on error status codes, you will need to know whether or not the server will return a representation that describes the error and what type it will be.

WADL allows you to define all this stuff.  It allows you to define URIs, their media types, what methods they expose, what query parameters,  what, if any representations are returned on failures.  It also allows you to define links and how they relate to other resources.

<resource_type id="vdc">
  <method name="GET">
    <response status="200">
      <representation mediaType="application/vnd.order+xml">
        <parameter name="CANCEL" path="/order/link[@rel='CANCEL']" style="plain">
          <link resource_type="cancel"/>
        </parameter>
        <parameter name="PAYMENT" path="/order/link[@rel='PAYMENT']" style="plain">
          <link resource_type="payment"/>
        </parameter>
      </representation>
    </response>
  </method>
</resource_type>
<resource_type id="cancel">
   <method name="PUT">
      <failure status="400" mediaType="aplication/vnd.error+xml"/>
   </method>
</resource_type>
...

I’ll admit, I am squeamish about WADL.  One of the things I liked about REST is that I wasn’t stuck writing WSDL documents and being dependent on a bloated SOAP stack.  WADL is just as verbose as WSDL.  I like how you can have any link structure you want in your documents and map it using XPath or some JSON equivalent.  Its just so damn verbose!  I still think something like this encourages an RPC-like approach to system design.  So what do we do instead?    Since HATEOAS is so close to the dataformat, awhile back, I talked about using XML schema to define your interface and link relationships.  Subbu also talks about expanding Atom links with a template, which should satisfy the query parameter problem.  I would also suggest adding allow and failure-type attributes to Atom links.  Allow being what HTTP methods are allowed, and failure-type specifying the media type that would be used with any failure responses.  So, here would be an example:

<order id="123">
   <amount>$123.45</amount>
   <atom:link rel="CANCEL" href="http://orders.com/orders/123/cancelled"
                           allow="GET, PUT" failure-type="application/vnd.failures+xml"/>
   <atom:link rel="PAYMENT" href="http://orders.com/orders/123/payment" type="application/xml"
                            allow="GET, PUT" failure-type="text/plain"/>
   ...
</order>

I think this covers the bases.  Thoughts?  Viable?

I was thinking alot about REST and service versioning again today.  In my previous blog I talked a bit about using the media type and probably XML schema to guarantee a contract between the client and server.  The client would ask for “application/vnd1+xml” and be all set for its conversation to begin.

The big problem I see with this is a service maintenance problem.  As you rev your service and your schema you’re going to have to support a lot of different media types.  What would make things easier is if XML Schema supported polymorphism.  That way a client could ask for “application/vnd1+xml” format and still receive and validate a newer version of the schema.  Please correct me if I’m wrong, but I don’t think you can do this in XML schema.  Yes you could extend various types from the parent to create a newer version of a schema, but a validator would not be able to enforce a polymorphic relationship with its parent AFAIK.

Clients and servers could get around this XSD problem by not doing validation and hoping that the schema developers are disciplined enough to make future versions of the format backward compatible.  It just seems incomplete to me to not be able to validate.

I was reading recently a very excellent article by Subbu about Describing RESTFul Applications.  I highly recommend the article as it very clearly illustrates the RESTful principle of HATEOAS.  One of the profound questions the article asks is does REST need a description language like IDL or WSDL? I guess the main arguments for this would be:

• Tooling.  Would be nice to be able to generate some client and server code to avoid all the boilerplate nonsense you usually have to do.
• Define an enforceable contract that can be validated.

There is something called WADL out there, but, as Subbu sort of hints at in his article, such an approach encourages an RPC-like approach for defining interactions with your application.  He goes on to describe what he feels like might be a good definition language while, at the end, asking the question of: Is it Practical? with the biggest roadblock being:

How about software tools that want to test or enforce the contract? It is possible to create such software that reads the above machine-readable description to do the following at runtime.

• Check that the media types of representations are predefined.
• Check that the representations match the predefined descriptions of media types.
• Check that all links contained in representations have predefined relations and media types, and that the URIs included follow predefined URI patterns.

I am not aware of any software that can validate these in this manner.

My answer to this question and roadblock is, why not have your media type define your entire contract?  Require that certain links and relations be a part of your document and define them explicitly in your XSD.  Maybe even the atom link’s media type should be required as well.  Then, simple XML validation can guarantee the contract.  The client application locks in the contract simply by specifying an Accept header containing a custom media type with its initial URI traversal.  We still get a lot of decoupling because the endpoint URIs are discovered at runtime yet we have something we can both generate code from and enforce a contract through.  Add in URI Templates to your links and all you’re missing is what HTTP methods are allowed to be invoked.  Using schema to enforce your contract doesn’t have to stop at XML as mappings like BadgerFish and tools like Jettison have shown.

All and all, the more I think about it, the more I believe that Human Documentation plus a schema is all that is needed in the RESTful world.  I’m glad that as more people like Subbu think and analyze through the IDL problem there are less and less reasons to have an IDL for REST.