Bill the Plumber

Edited 10/28/2010: Since I wrote this blog, we’ve released the HornetQ REST Interface.  Check it out and tell me what you think.

A few months ago after my first release of RESTEasy JAX-RS, I realized that I had made a cardinal mistake with our JAX-RS implementation. I had yet to build any real application with it. I had yet to test drive it. In fact, I had actually never written a RESTful application period. So, with that in mind, I decided to build something so I could get a feel how good JAX-RS and REST actually was. But what to build? I thought of redoing Pet Store with AJAX and RESTEasy as the backend, but then I’d actually have to learn AJAX or some AJAX framework. Then I thought, I’m a middleware developer, why not make some piece of middleware RESTful? Thus, RESTEasy MOM was born. This blog entry is about both an exercise in RESTful Web Service design as well as its Java counterpart, JAX-RS.

RESTEasy MOM is a RESTful interface over any generic JMS implementation. I’ve only tested it with JBoss MQ on JBoss AS 4.2, but I’m sure it will work just fine with other application servers and/or JMS implementations with a little tweaking. It is a little different than vanilla JMS in that it deals with sending and receiving of HTTP messages and thus is a little more HTTP aware than regular JMS.

RESTful Queues

My first prototype with RESTEasy MOM was to build a publisher and subscriber interface to a JMS queue. I thought I’d start off simple and get more complicated from there. I’ll describe the interface as both an HTTP request/URI template as well as a JAX-RS interface so you can get two perspectives, a client one, and a server-side implementation perspective.

POST /queue/{queue-name}?persistent=[true/false]
GET /queue/{queue-name}

Publishing to the queue would be a POST HTTP request with an optional persistent flag that would tell the JMS provider whether or not the publisher wants the message stored and forwarded. Queue subscribers would pull the queue via a simple GET and they would get the same content posted by the original POST request. Here’s what the server-side JAX-RS implementation would look like:

@Path("/queues")
public interface QueueManager {
   @POST
   @Path("/{name}")
   public void send(@PathParam("name") String destination,
                          @QueryParam("persistent") @DefaultValue("true") boolean persistent,
                          @Context HttpHeaders headers,
                          InputStream body);

   @GET
   @Path(("/{name}")
   public Response receive(@PathParam("name" String destination);
}

We want the persistent parameter to the send() method to default to true so we use the @DefaultValue parameter annotation. We want access to all of the HttpHeaders sent with the request because we may want to forward certain non-request headers to the receiver of the message. We represent the body of the HTTP request as an InputStream because we’re going to deal with the request generically. Basically all we’re really going to do is store the HTTP request and send it as-is as an HTTP response when a client wants to pull a message from the queue.

What’s cool about RESTful messaging here?

*sarcasm* Yay team! We’ve created a RESTful messaging interface over JMS. *end sarcasm* What’s so interesting about it? What do we get out of making something RESTful?

1. Addressability

Because we are representing our queue as a URI we have a standard way to identify where our queues live on the network. The URI specification is old, well understood, and stable. Believe it or not the JMS specification does not specify where a client is supposed to connect to a remote queue. It is up to the vendor. Many vendors use JNDI to do this, but they are not required to.

2. A uniform, constrained interface

We are constraining ourselves to interact with our queue through the methods provided by the HTTP protocol. If we want to evolve the functionality of our RESTful messaging system, we are constrained to only 4-6 HTTP methods (GET, POST, DELETE, PUT, HEAD, OPTIONS). This forces us architecturally to define new functionality as new resources (different URI paths). Why is this a good thing? Well, think of versioning over time. If new functionality is represented as different URI paths we do not have to worry about breaking existing older clients of our messaging system. Even if we add a new attribute to a specific URI request, things like Query parameters are always optional to a request and we won’t break older clients that are not aware of the interface change.

3. A stable protocol

HTTP is an old and well defined protocol. In the JMS specification there really is no remote protocol defined. It is hidden by the implementation. In the distributed world, distributed messaging specifications are either dead or dieing (DCE, CORBA) or a moving target (WS-*) or out of style (JAX-RPC vs. design by contract). HTTP on the otherhand is stable. I’ll also gander that there are a lot more people that know how to pass HTTP messages around than there are people that know CORBA or WS-*.

Points 1-3 are interesting, but honestly, not really *that* compelling. But, when you start analyzing the side-effects of points 1-3 you really start to see why a RESTful messaging system is a great idea:

4. Interoperability

JMS has no interoperable protocol defined between vendors. So there is really no way a IBM client could talk to a JBoss one. If you had an application that needed to interact remotely with both a JBoss MQ and IBM MQSeries, you’d have to install both libraries on your system. Both systems would even probably have different ways to configure, connect to, and bootstrap themselves. With HTTP it is all well understood and simple. Because JMS is not interoperable, what happens if you want to ditch your ultra-expensive IBM MQSeries implementation and slide in the ultra-reliable, ultra-cheaper, JBoss solution? You have to upgrade both your client and server. Not too bad if you have one client, but what if you have a 1000 clients? Not very fun…

With REST over HTTP all you have to worry about is that your server and receivers understand the data you are sending them, which is really no different than what you have to worry about with sending regular JMS messages.

5. Ubiquity == platform/language independence

Most (all?) credible programming languages have an HTTP client stack. Because we have created a RESTful interface to our messaging system, virtually any client from any language can use it. You just cannot say the same about JMS (Java Messaging Service), or really any WS-* or CORBA equivalent (or even AMQP). Because we are using a standard, ubiquitous, messaging protocol (HTTP), we can have a Java server in which a C++ client is posting XML or JSON messages and a Ruby client receiving and processing them. We could have a RESTFul Messaging Service written in PHP in which Java clients are exchanging Java objects (No reason the HTTP body can’t be a Java object!). IMO, this is the most compelling reason to use REST and HTTP.

6. Familiarity

Software developers are usually a curious animal. They don’t mind learning new technologies, its fun to the. I hope I don’t insult anybody too much with the next statement, but, IT operations aren’t as open to change and new things. Usually, its not because they don’t want to learn new technologies. Its just that they just have one more complication to worry about. For the app developer, putting their project into production, they only have to worry about their application, their world. For ops, they have to worry about the whole organization. So, what does this have to do with REST? Well, take security for instance. In a Java EE, WS-*, or CORBA world, the ops guy is going to have to learn how that particular platform configures security. For REST? Well, securing HTTP requests is a well known and practiced trade. All the ops professional needs is a set of URI’s and HTTP operations the application exposes. Since we’re contrained by the HTTP uniform interface, its also intuitive on what is being secured. GET is read access. PUT/POST update/insert access. DELETE delete access. Most admins know how to define authentication and authorization for URI patterns and HTTP methods. Many times it is as simple as configuring the good, old reliable Apache HTTPD.

What’s cool about JAX-RS?

The easiest way to appreciate JAX-RS is just to write the same application as a raw servlet and compare the two. I think if you do this, you’ll find that JAX-RS is simpler, more elegant, and just plain fun. It doesn’t get in your way and you can focus on writing your application rather than struggling with an API. Details?

1. Easy access to URI parameters

If we were implementing our messaging service as a servlet, we’d have to do a lot of URI parsing. @PathParam makes things so much easier

2. WYSIWYG Development

Another thing I like about JAX-RS, is that, because of the annotations, you can look at a servler class and know exactly how to invoke on it. You know what headers are being used, params, and the base URI.

3. A la carte HTTP

With servlet development, you have to extend a specific class and follow a specific contract. You get access to the entire HTTP request and response, but, you have to write a bit of gluecode to pull that information out. With JAX-RS, you can pull in exactly what you want from the HTTP request and have JAX-RS do any String->to->primitive conversions you need.

4. Pluggable message body mappings

RESTEasy MOM does not take advantage of this feature of JAX-RS because it is dealing with HTTP message generically and is really just a router of messages. It doesn’t really care about the content. But, this will not be true in other, less-generic, business oriented applications. Through the @Provider contract, developers can write plugins that know how to marshal and/or unmarshall a specific MIME type to a specific Java class. JAX-RS itself requires built-in support for JAXB. Other mappings like JSON->Java could easily be plugged in as well.

Evolving our RESTFul Messaging interface

There are few things that need to be tweeked in this simple RESTful interface.

GET is idempotent

Our interface for receiving messages has broken our constrained uniform interface. How you ask? Well, the HTTP GET operation is supposed to idempotent. What that means is no matter how many times you invoke it it is not supposed to change the state of the underlying resource. It is supposed to be repeatable with no side affects. This is not the case here as when we GET from the queue, we are consuming a message, sending it back to the receiving client, and removing the message from the queue. This is changing the state of the resource and using GET in the manner is poor RESTful design.

This may sound like nit-picking, but its not. Consider if the Unix file system allowed users to change the state of a file or device with a read access? It would break file permissions and our system wouldn’t be consistent. Later, in another blog , (this article is already too long) when I talk about caching and Cache Control headers, I’ll show you why this might be important for a RESTful messaging system. But for now, I hope you get what I’m saying.

What HTTP method should we use then?

So, we have to replace GET. Your first insite might be, what about DELETE? We are consuming/deleting a message from the queue when we send it back to the client and DELETE is allowed to send back content as a response. Unfortunately, again, DELETE is supposed to be an idempotent method. It is supposed to be repeatable. The only logic choice is POST because POST allows you to change the state of your resource. Unfortunately, we already have POST assigned to an operation so what should we do?

I don’t like overloaded POST

What some people like to do is overload a POST operation by adding an action query parameter:

POST /queues/{name}?action=[send|receive]

I don’t like this because what you’ve done is provided multiple operations for what is supposed to be a uniform operation. You’ve created a mini-rpc framework. Beyond that, it doesn’t map very well to JAX-RS. You’d have to do some mini dispatching in a specialized generic method. Might as well just write a servlet :).

Just create a new resource

Instead, a better plan is to just create a different resource, a.k.a. just extend the URI path.

POST /queues/{name}/receiving

Now that receiving is its own resource we use POST for getting and consuming the message. We could, in the future, use GET to peek at the message. In much the same way in programming we’d want to separate the sender and receiver into two different interfaces and/or objects, we can do the same with URI resources.

PUT instead of POST for sending

When we send a message via POST, POST does not guarantee any idempotency. This kinda sucks because we could possibly send a duplicate message. For example, lets say we POST a message, the server locally sends it over JMS successfully, but crashes before it can send the acknowledgement. The sending client has no idea whether the message posted to the internal JMS queue or not, so, it might resend the message. So can we solve this problem? We can make sending a message idempotent by writing to a specific URI that is dedicated to that particular message. The client would be responsible for generating a unique identifier for that and PUTing it at a specific URI:

PUT /queues/{name}/messages/{message-id}

Using PUT in this way makes sending the message idempotent. Why? Well, if, on the initial send, there is a failure, the sending client can just retransmit. When the server processes a send, it has the identifier of the message. It can look in its persistent store to see if the message already exists and has been posted, or look in memory. If a message of that id doesn’t exist, just post it, if it does exist then do nothing. Our logic doesn’t even have to be that complicated. Anytime a message comes in, just log a message of that id and overwrite any existing one. Since they should be the same data, no harm done. We would use a PUT instead of POST here because the standard HTTP 1.1 definition of PUT tells the user of our messaging system that sending a message is idempotent and they don’t have to worry about performing resends.

Unfortunately, we cannot implement this with our JAX-RS facade over JMS. While JMS does let you specify a message id, it is really undefined on what happens if you post a message of the same id. We could browse the queue to see if the message is there, but, if the message gets processed before we can’t check to see if it already exists. We would have to extend the functionality of an open source JMS implementation like JBoss Messaging.

Improvements to JAX-RS design

So, after our RESTful design improvements we have the follow JAX-RS design:

@Path("/queues")
public interface QueueManager {
   @POST
   @Path("/{name}")
   public void send(@PathParam("name") String destination,
                          @QueryParam("persistent") @DefaultValue("true") boolean persistent,
                          @Context HttpHeaders headers,
                          InputStream body);

   @POST
   @Path(("/{name}/receiving")
   public Response receive(@PathParam("name" String destination);
}

One of the problems with defining our JAX-RS interface in this way is that we now have two things coupled together when they shouldn’t be. Any implementation of our QueueManager interface would have to both locate a JMS ConnectionFactory and Destination as well as process the request and interact with a Connection and JMS Session. The problem is that, locating the ConnectionFactory and Destination is going to be implementation specific. Each JMS implementation could have their own specific way of finding these items. So, it would be better if we could separate these concerns into two separate classes. What’s cool is that JAX-RS allows us to model such a scenario with Subresources and Subresource Locators.

JAX-RS Subresources and Subresource Locators

A cool feature of JAX-RS is that you can divide the processing of an individual requests between multiple objects. A top-level resource could gather some information from part of the HTTP request and hand back to the JAX-RS runtime an object that can handle the remainder of the request. These are called Subresource Locators and Subresources. What we are going to use them for in our messaging system is to break up finding a JMS destination and processing a sending or receiving of a message into two separate classes/objects/resources.

@Path("/queues")
public class RestfulJBossQueueManager {

   @Path("/{name}")
   public RestfulQueue locate(@PathParam("name") String destination) {
       ConnectionFactory connectionFactory = (ConnectionFactory)jndi.lookup("java:/ConnectionFactory");
       Destination queue = (Destination)jndi.lookup("queue/" + destination);
       return new RestfulQueue(connectionFactory, queue);
   }
}

The sole purpose of the RestfulJBossQueueManager class is to find the target destination through the destination @PathParam parameter. It must not have a HTTP method annotation like @POST and must return an object that can process the rest of the request. The locate() method does just this. It finds a ConnectionFactory and Destination and uses these variables to construct an instance of the subresource class, RestfulQueue.

public class RestfulQueue {
   public RestfulQueue(ConnectionFactory factory, Destination destination) {
     ...
   }

   @POST
   public void send(@QueryParam("persistent") @DefaultValue("true") boolean persistent,
                    @Context HttpHeaders headers,
                    InputStream body) {...}

   @POST
   @Path(("/receiving")
   public Response receive(@PathParam("name") String destination) {...}
}

The JAX-RS runtime dynamically scans the returned RestfulQueue class for annotated methods that could be used to finish processing the HTTP request. This is a simple example of Subresources and their locators, but imagine if your Hibernate object/rational model served the dual purpose of a data and restful interface. Just some thoughts… 🙂

More to come…

I had a lot more to write but I’m already at 3000 words, a little bit long for a blog entry. I plan on writing one or two more entries diving into some more complexities and features you could add to RESTEasy MOM including content negotiation, listeners, tighter reliability, and overall better RESTFul design, applying more RESTFul techniques and architecture to a distributed messaging system. For now, you can check out the RESTEasy MOM project, download the code, read the WIKI and see what I’ve experimented with so far. If you like what you see, post us an email on resteasy-developers@lists.sourceforge.net