Buggy/Broken Tomcat 6 Comet NIO APIs

November 12, 2008

billburke java 23 Comments

I’ve had some bad experiences lately with the Tomcat 6 Comet APIs with the NIO Connector.  I have a simple COMET servlet that spawns a thread and waits 5 seconds to respond to a Firefox browser.  I also call event.setTimeout(10000) (timeout on the event of 10 seconds).  After the response is finished writing I call event.close();  When using the Tomcat NIO, I don’t see a response until after event.setTimeout() time.  So, if I set the timeout to 100 seconds I don’t see a response in the browser for 100 seconds.

Maybe I’m not adding some undocumented secret sauce, but when I test this out with JBoss Web and our APR connector, it works as expected: I see a response in the browser after 5 seconds.  I also had a go with Servlet 3.0 through Jetty 7.0 pre3.  Works as expected there as well.

This isn’t the only problems I’ve had.  I also had a ClassCastException when I add a ServletFilter in front of a COMET Servlet.  The APIs assume that all your filters implement CometFilter.  Granted, I experienced this particular problem in the version of Tomcat 6 that ships with JBoss 4.2.3.GA, but after trying out the timeout problem described above with Tomcat 6 binaries from apache.org, I didn’t bother going any further to see if this was a JBoss specific problem or not.  I did test to see if JBossWeb has the same problem…As expected it DID NOT.

So what’s the deal here?

  1. I’m just stupid?
  2. Nobody uses Tomcat 6 COMET + NIO APIs?
  3. Everybody who is doing asynchronous HTTP is using Jetty?
  4. Asynchronous HTTP is just hype generated by Greg Wilkins to breath new life into his increasingly irrelevant company?

My personal guess is that 2-3 are the main problems, with #1 a high possibility, with a sprinkle of 4.  Anybody out there have any experiences with Tomcat 6 + COMET + NIO?

As a result of this, I’m not going to support COMET + Tomcat NIO with RESTEasy’s Async HTTP APIs.  Only Jetty and JBossWeb.

JAX-RS Atom Support with Resteasy

November 10, 2008

billburke java, JAX-RS, jboss, REST, Webservices 3 Comments

I have implemented Atom support which will be available with the next release of RESTEasy.

Although the Atom format is used primarily for the syndication of blogs and news, many are starting to use this format as the envelope for Web Services, for example, distributed notifications, job queues, or simply a nice format for sending or receiving data in bulk from a service.

RESTEasy has defined a simple object model in Java to represent Atom and uses JAXB to marshal and unmarshal it. The main classes are in the org.jboss.resteasy.plugins.providers.atom package and are Feed, Entry, Content, and Link. If you look at the source, you’d see that these are annotated with JAXB annotations. The distribution contains the javadocs for this project and are a must to learn the model. Here is a simple example of sending an atom feed using the Resteasy API.

import org.jboss.resteasy.plugins.providers.atom.Content;
import org.jboss.resteasy.plugins.providers.atom.Feed;
import org.jboss.resteasy.plugins.providers.atom.Link;
import org.jboss.resteasy.plugins.providers.atom.Person;

@Path("atom")
public class MyAtomService
{

   @GET
   @Path("feed")
   @Produces("application/atom+xml")
   public Feed getFeed()
   {
      Feed feed = new Feed();
      feed.setId(new URI("http://example.com/42"));
      feed.setTitle("My Feed");
      feed.setUpdated(new Date());
      Link link = new Link();
      link.setHref(new URI("http://localhost"));
      link.setRel("edit");
      feed.getLinks().add(link);
      feed.getAuthors().add(new Person("Bill Burke"));
      Entry entry = new Entry();
      entry.setTitle("Hello World");
      Content content = new Content();
      content.setType(MediaType.TEXT_HTML_TYPE);
      content.setText("Nothing much");
      feed.getEntries().add(content);
      return feed;
   }
}

Because Resteasy’s atom provider is JAXB based, you are not limited to sending atom objects using XML. You can automatically re-use all the other JAXB providers that Resteasy has like JSON and fastinfoset. All you have to do is have “atom+” in front of the main subtype. i.e. @Produces(“application/atom+json”) or @Consumes(“application/atom+fastinfoset”)

Using JAXB with the Atom Provider

The org.jboss.resteasy.plugins.providers.atom.Content class allows you to unmarshal and marshal JAXB annotated objects that are the body of the content. Here’s an example of sending an Entry with a Customer object attached as the body of the entry’s content.

@XmlRootElement(namespace = "http://jboss.org/Customer")
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer
{
   @XmlElement
   private String name;

   public Customer()
   {
   }

   public Customer(String name)
   {
      this.name = name;
   }

   public String getName()
   {
      return name;
   }
}

@Path("atom")
public static class AtomServer
{
   @GET
   @Path("entry")
   @Produces("application/atom+xml")
   public Entry getEntry()
   {
      Entry entry = new Entry();
      entry.setTitle("Hello World");
      Content content = new Content();
      content.setJAXBObject(new Customer("bill"));
      entry.setContent(content);
      return entry;
   }
}

The Content.setJAXBObject() method is used to tell the content object you are sending back a Java JAXB object and want it marshalled appropriately. If you are using a different base format other than XML, i.e. “application/atom+json”, this attached JAXB object will be marshalled into that same format.

If you have an atom document as your input, you can also extract JAXB objects from Content using the Content.getJAXBObject(Class clazz) method. Here is an example of an input atom document and extracting a Customer object from the content.

@Path("atom")
public class AtomServer
{
   @PUT
   @Path("entry")
   @Produces("application/atom+xml")
   public void putCustomer(Entry entry)
   {
      Content content = entry.getContent();
      Customer cust = content.getJAXBObject(Customer.class);
   }
}

Double Check does work!

October 30, 2008

billburke java 1 Comment

I was reading an interview with Josh Bloch about the 2nd edition of his great book Effective Java on Java.net yesterday and was pleasantly surprised to find out that if you use volatile within the Double Check pattern it is now safe in JDK 5 and above.  I did not know this, the rumor mill was that Double Check did not work even in JDK 5.  Here’s an example of Double Check.  Josh suggests that you copy it as is and make no modifications.

// Double-check idiom for lazy initialization of instance fields.
private volatile FieldType field;
FieldType getField() {
    FieldType result = field;
    if (result == null) { // First check (no locking)
        synchronized(this) {
            result = field;
            if (result == null) // Second check (with locking)
                field = result = computeFieldValue();
        }
    }
     return result;
}

Just this revelation alone makes me want to buy his book.  I wonder what other goodies are in there.  Seriously though, I’ve had need for the double-check pattern a lot over the years as avoiding synchronization can be the biggest scalability  increase in writing middleware.  I’m psyched that it actually works!

JAX-RS Multipart support with RESTEasy

October 23, 2008

billburke java, JAX-RS, jboss, REST, Webservices 7 Comments

I just finished implementing some “multipart/*” and multipart/form-data support within RESTEasy.  It will be out with the next release.  Highlights?

A JAX-RS compatable API

Our current support for multipart is through the MimeMultipart classes within the Java Mail library.  They are limited because they do not use the MessageBodyReader/Writers available in JAX-RS.  I have written two parallel APIs that provide multipart support in a JAX-RS way:

package org.jboss.resteasy.plugins.providers.multipart;

public interface MultipartInput {
   List<InputPart> getParts();
   String getPreamble();
}

public interface InputPart {
   MultivaluedMap<String, String> getHeaders();
   String getBodyAsString();
   <T> T getBody(Class<T> type, Type genericType) throws IOException;
   <T> T getBody(org.jboss.resteasy.util.GenericType<T> type) throws IOException;
   MediaType getMediaType();
}

MultipartInput is a simple interface that allows you to get access to each part of the multipart message. Each part is represented by an InputPart interface. Each part has a set of headers associated with it.  You can unmarshall the part by calling one of the getBody() methods. The Type genericType parameter can be null, but the Class type parameter must be set. Resteasy will find a MessageBodyReader based on the media type of the part as well as the type information you pass in. The following piece of code is unmarshalling parts which are XML into a JAXB annotated class called Customer.

   @Path("/multipart")
   public class MyService   {
      @PUT
      @Consumes("multipart/mixed")
      public void put(MultipartInput input)
      {
         List<Customer> customers = new ArrayList...;
         for (InputPart part : input.getParts())
         {
            Customer cust = part.getBody(Customer.class, null);
            customers.add(cust);
         }
      }
   }

There’s a similar API for outputing multipart.

Multipart with vanilla Lists and Maps

Another feature I’ve added is the ability to use regular instances of java.util.List(any multipart format) or Map (form-data only) to represent multipart data.  Its only usable when your parts are uniform though.

Here’s an example of using it:

parameter of the List type declaration. Here’s an example again of unmarshalling a list of customers.

   @Path("/multipart")
   public class MyService
   {
      @PUT
      @Consumes("multipart/mixed")
      public void put(List<Customer> customers)
      {
         ...
      }
   }

That’s using input.  The problem with output is that RESTEasy has no idea what mime type you want to marshal or List or Map into.  So, the @PartType annotation is used.  Here’s an example of outputing multipart/form-data with a map:

@Path("/multipart")
   public class MyService
   {
      @GET
      @Produces("multipart/form-data")
      @PartType("application/xml")
      public Map<String, Customer> get()
      {
         ...
      }

Multipart-Formdata with POJOs

Finally, I added the ability to map POJO form classes to multipart/form-data.  You re-use @FormParam on fields and/or properties of a user-provided POJO.  Here’s an example:

   public class CustomerProblemForm {
      @FormData("customer")
      @PartType("application/xml")
      private Customer customer;

      @FormData("problem")
      @PartType("text/plain")
      private String problem;

      public Customer getCustomer() { return customer; }
      public void setCustomer(Customer cust) { this.customer = cust; }
      public String getProblem() { return problem; }
      public void setProblem(String problem) { this.problem = problem; }
   }

After defining your POJO class you can then use it to represent multipart/form-data. Here’s an example of sending a CustomerProblemForm using the RESTEasy client framework

   @Path("portal")
   public interface CustomerPortal {

      @Path("issues/{id}")
      @Consumes("multipart/form-data")
      @PUT
      public void putProblem(@MultipartForm CustomerProblemForm,
                             @PathParam("id") int id);
   }

   {
       CustomerPortal portal = ProxyFactory.create(CustomerPortal.class, "http://example.com");
       CustomerProblemForm form = new CustomerProblemForm();
       form.setCustomer(...);
       form.setProblem(...);

       portal.putProblem(form, 333);
   }

You see that the @MultipartForm annotation was used to tell RESTEasy that the object has @FormParam and that it should be marshalled from that. You can also use the same object to receive multipart data. Here is an example of the server side counterpart of our customer portal.

   @Path("portal")
   public class CustomerPortalServer {

      @Path("issues/{id})
      @Consumes("multipart/form-data")
      @PUT
      public void putIssue(@MultipartForm CustoemrProblemForm,
                           @PathParm("id") int id) {
         ... write to database...
      }
   }

JAX-RS + Asynch HTTP

October 9, 2008

billburke java, JAX-RS, jboss, REST, RESTEasy, Webservices , 10 Comments

A RESTEasy user (and JBoss customer) asked about 6 weeks ago if we were planning on supporting Resteasy with Asynchronous HTTP.  For those of you not familiar with Asynchronous HTTP, take a look at Servlet 3.0, Tomcat Comet APIs, or Jetty’s continuations.  I decided to add some limited support after doing some research.

The primary (and IMO, the only) usecase for Asynchronous HTTP is in the case where the client is polling the server for a delayed response.  The usual example is an AJAX chat client where you want to push/pull from both the client and the server.  These scenarios have the client blocking a long time on the server’s socket waiting for a new message.  What happens in synchronous HTTP is that you end up having a thread consumed per client connection.  This eats up memory and valuable thread resources.  No such a big deal in 90% of applications, but when you start getting a lot of concurrent clients that are blocking like this, there’s a lot of wasted resources.

As for Resteasy and asynchronous HTTP, I don’t want to create a COMET server.  Maybe I’m totally wrong, but it seems that the idea of COMET is to use HTTP solely for the purpose of initiating a dedicated socket connection to your server.  After the initial connection is made, the client and server just tunnel their own message protocol across the socket for any number of requests.

Now, this isn’t necessarily a bad idea.  Its actually a good one for performance reasons.  As a JBossian David Lloyd explained to me in private email, what this API allows you to do is create a direct connection to your service so that you don’t have to redispatch with every message that is exchange from the client and server.  Sure, HTTP Keep Alive allows you to avoid re-establishing the socket connection, but you don’t have to worry about all the other path routing logic to get the message to the Java object that services the request.

Still, this is something I don’t want or need to provide through RESTEasy.  Its really using HTTP to tunnel a new protocol and something that is very unRESTful to me.  Basically it requires that the client knows how to communicate the COMET protocol, which IMO, makes ubiquity very hard.  Besidesk, Tomcat, JBoss Web, and Jetty will probably do a fine job here.  There are also already a number of COMET servers available on the web.  No, I will focus on giving asynchronous HTTP features to a pure HTTP and JAX-RS request.

What I will initially provide through RESTEasy is a very simple callback API.

@Path("/myresource")
@GET
public void getMyResource(@Suspend AsynchronousResponse response) {

   ... hand of response to another thread ...
}

public interface AsynchronousResponse {

   void setResponse(javax.ws.rs.core.Response response);

}

The @Suspend annotation tells Resteasy that the HTTP request/response should be detached from the currently executing thread and that the current thread should not try to automaticaly process the response.  The AsynchronousResponse is the callback object.  It is injected into the method by Resteasy.  Application code hands off the AsynchronousResponse to a different thread for processing.  The act of calling setResponse() will cause a response to be sent back to the client and will also terminate the HTTP request.

Servlet 3.0 has asynchronou APIs.  When you suspend a request, the request may be redelivered when a timeout occurs.  I don’t want to have a Resteasy replacement for these APIs.  What I want is something that complements these APIs and makes a specific usecase easier to write.  The use case I want to solve is detaching the processing of an HTTP response from the current thread of execution so that another thread can do the work when ready.  That’s it.  So, there will be no redelivery of a request.

Initially I plan to work with the asynchronous apis within the Tomcat 6.0 NIO transport as this is what is distributed with JBoss 4.2.3.  Next I plan to work with the JBoss Web asynchronous apis, then Jetty 6, then finally with a Servlet 3.0 implementation.

If you have any ideas on this subject, please let me know.  I can work them into the prototype I’m developing.

RESTEasy DZone articles and Beta 8

September 22, 2008

billburke java, JAX-RS, jboss, opensource, REST, Webservices 6 Comments

If you’re wondering why I haven’t done a signficant blog since June, wonder no more.  Instead of creating content for my blog, I’ve written a couple of articles for DZone.com:

Also, today I put out RESTEasy 1.0 Beta 8.  I small tiny bug was found by the community which I rolled up quick into a release.  Follow the downloads link at http://jboss.org/resteasy to check it out.

Caching, Parallelism, and Scalability

September 19, 2008

billburke java, jboss Leave a comment

Manik Surtani has written a great article on Caching, Parallelism, and Scalability.  Check it out.

RESTEasy 1.0-beta-7: quick bug fixes

September 16, 2008

billburke java, JAX-RS, jboss, REST, Webservices 2 Comments

A quick release from our beta-6 to fix a few quick bugs that users found.  See our main project page at:

http://jboss.org/resteasy

to download and browse docs.  Have fun.

RESTEasy 1.0-Beta-6 Released!

September 10, 2008

billburke java, JAX-RS, REST, Webservices 3 Comments

A lot of changes in this release.  The specification is basically done and going final any day now.  The spec lead just needs to finish the TCK and it will go to a vote with the JCP EC and become final.  We are up to date with the specification.  We have a lot more providers in this release.  YAML, JSON, to name a few.  The distribution now contains docbook generated documentation and a bunch of example apps so you can see RESTEasy in action.  I want to especially thank:

  • Ryan McDonnough, for his hard work on writing the JAXB+XML,  JAXB+JSON, and Multipart providers.
  • Olivier Brand, for his Spring/Hibernate/RESTEasy example application
  • Martin Algesten, for all his bug fixing, patching, and the YAML provider

One final thing,  The main project page has moved off of JBoss WIKI to jboss.org:

http://jboss.org/resteasy

Have fun. The next release will be focusing on innovations, specifically:  Asynchronous JAX-RS, JAX-RS Messaging, Asynchronous HTTP, and HTTP Caching.

RESTEasy MOM: An exercise in JAX-RS RESTful WS design

June 16, 2008

billburke java, JAX-RS, REST, Webservices 16 Comments

Edited 10/28/2010: Since I wrote this blog, we’ve released the HornetQ REST Interface.  Check it out and tell me what you think.

A few months ago after my first release of RESTEasy JAX-RS, I realized that I had made a cardinal mistake with our JAX-RS implementation. I had yet to build any real application with it. I had yet to test drive it. In fact, I had actually never written a RESTful application period. So, with that in mind, I decided to build something so I could get a feel how good JAX-RS and REST actually was. But what to build? I thought of redoing Pet Store with AJAX and RESTEasy as the backend, but then I’d actually have to learn AJAX or some AJAX framework. Then I thought, I’m a middleware developer, why not make some piece of middleware RESTful? Thus, RESTEasy MOM was born. This blog entry is about both an exercise in RESTful Web Service design as well as its Java counterpart, JAX-RS.

RESTEasy MOM is a RESTful interface over any generic JMS implementation. I’ve only tested it with JBoss MQ on JBoss AS 4.2, but I’m sure it will work just fine with other application servers and/or JMS implementations with a little tweaking. It is a little different than vanilla JMS in that it deals with sending and receiving of HTTP messages and thus is a little more HTTP aware than regular JMS.

RESTful Queues

My first prototype with RESTEasy MOM was to build a publisher and subscriber interface to a JMS queue. I thought I’d start off simple and get more complicated from there. I’ll describe the interface as both an HTTP request/URI template as well as a JAX-RS interface so you can get two perspectives, a client one, and a server-side implementation perspective.

POST /queue/{queue-name}?persistent=[true/false]
GET /queue/{queue-name}

Publishing to the queue would be a POST HTTP request with an optional persistent flag that would tell the JMS provider whether or not the publisher wants the message stored and forwarded. Queue subscribers would pull the queue via a simple GET and they would get the same content posted by the original POST request. Here’s what the server-side JAX-RS implementation would look like:

@Path("/queues")
public interface QueueManager {
   @POST
   @Path("/{name}")
   public void send(@PathParam("name") String destination,
                          @QueryParam("persistent") @DefaultValue("true") boolean persistent,
                          @Context HttpHeaders headers,
                          InputStream body);

   @GET
   @Path(("/{name}")
   public Response receive(@PathParam("name" String destination);
}

We want the persistent parameter to the send() method to default to true so we use the @DefaultValue parameter annotation. We want access to all of the HttpHeaders sent with the request because we may want to forward certain non-request headers to the receiver of the message. We represent the body of the HTTP request as an InputStream because we’re going to deal with the request generically. Basically all we’re really going to do is store the HTTP request and send it as-is as an HTTP response when a client wants to pull a message from the queue.

What’s cool about RESTful messaging here?

*sarcasm* Yay team! We’ve created a RESTful messaging interface over JMS. *end sarcasm* What’s so interesting about it? What do we get out of making something RESTful?

1. Addressability

Because we are representing our queue as a URI we have a standard way to identify where our queues live on the network. The URI specification is old, well understood, and stable. Believe it or not the JMS specification does not specify where a client is supposed to connect to a remote queue. It is up to the vendor. Many vendors use JNDI to do this, but they are not required to.

2. A uniform, constrained interface

We are constraining ourselves to interact with our queue through the methods provided by the HTTP protocol. If we want to evolve the functionality of our RESTful messaging system, we are constrained to only 4-6 HTTP methods (GET, POST, DELETE, PUT, HEAD, OPTIONS). This forces us architecturally to define new functionality as new resources (different URI paths). Why is this a good thing? Well, think of versioning over time. If new functionality is represented as different URI paths we do not have to worry about breaking existing older clients of our messaging system. Even if we add a new attribute to a specific URI request, things like Query parameters are always optional to a request and we won’t break older clients that are not aware of the interface change.

3. A stable protocol

HTTP is an old and well defined protocol. In the JMS specification there really is no remote protocol defined. It is hidden by the implementation. In the distributed world, distributed messaging specifications are either dead or dieing (DCE, CORBA) or a moving target (WS-*) or out of style (JAX-RPC vs. design by contract). HTTP on the otherhand is stable. I’ll also gander that there are a lot more people that know how to pass HTTP messages around than there are people that know CORBA or WS-*.

Points 1-3 are interesting, but honestly, not really *that* compelling. But, when you start analyzing the side-effects of points 1-3 you really start to see why a RESTful messaging system is a great idea:

4. Interoperability

JMS has no interoperable protocol defined between vendors. So there is really no way a IBM client could talk to a JBoss one. If you had an application that needed to interact remotely with both a JBoss MQ and IBM MQSeries, you’d have to install both libraries on your system. Both systems would even probably have different ways to configure, connect to, and bootstrap themselves. With HTTP it is all well understood and simple. Because JMS is not interoperable, what happens if you want to ditch your ultra-expensive IBM MQSeries implementation and slide in the ultra-reliable, ultra-cheaper, JBoss solution? You have to upgrade both your client and server. Not too bad if you have one client, but what if you have a 1000 clients? Not very fun…

With REST over HTTP all you have to worry about is that your server and receivers understand the data you are sending them, which is really no different than what you have to worry about with sending regular JMS messages.

5. Ubiquity == platform/language independence

Most (all?) credible programming languages have an HTTP client stack. Because we have created a RESTful interface to our messaging system, virtually any client from any language can use it. You just cannot say the same about JMS (Java Messaging Service), or really any WS-* or CORBA equivalent (or even AMQP). Because we are using a standard, ubiquitous, messaging protocol (HTTP), we can have a Java server in which a C++ client is posting XML or JSON messages and a Ruby client receiving and processing them. We could have a RESTFul Messaging Service written in PHP in which Java clients are exchanging Java objects (No reason the HTTP body can’t be a Java object!). IMO, this is the most compelling reason to use REST and HTTP.

6. Familiarity

Software developers are usually a curious animal. They don’t mind learning new technologies, its fun to the. I hope I don’t insult anybody too much with the next statement, but, IT operations aren’t as open to change and new things. Usually, its not because they don’t want to learn new technologies. Its just that they just have one more complication to worry about. For the app developer, putting their project into production, they only have to worry about their application, their world. For ops, they have to worry about the whole organization. So, what does this have to do with REST? Well, take security for instance. In a Java EE, WS-*, or CORBA world, the ops guy is going to have to learn how that particular platform configures security. For REST? Well, securing HTTP requests is a well known and practiced trade. All the ops professional needs is a set of URI’s and HTTP operations the application exposes. Since we’re contrained by the HTTP uniform interface, its also intuitive on what is being secured. GET is read access. PUT/POST update/insert access. DELETE delete access. Most admins know how to define authentication and authorization for URI patterns and HTTP methods. Many times it is as simple as configuring the good, old reliable Apache HTTPD.

What’s cool about JAX-RS?

The easiest way to appreciate JAX-RS is just to write the same application as a raw servlet and compare the two. I think if you do this, you’ll find that JAX-RS is simpler, more elegant, and just plain fun. It doesn’t get in your way and you can focus on writing your application rather than struggling with an API. Details?

1. Easy access to URI parameters

If we were implementing our messaging service as a servlet, we’d have to do a lot of URI parsing. @PathParam makes things so much easier

2. WYSIWYG Development

Another thing I like about JAX-RS, is that, because of the annotations, you can look at a servler class and know exactly how to invoke on it. You know what headers are being used, params, and the base URI.

3. A la carte HTTP

With servlet development, you have to extend a specific class and follow a specific contract. You get access to the entire HTTP request and response, but, you have to write a bit of gluecode to pull that information out. With JAX-RS, you can pull in exactly what you want from the HTTP request and have JAX-RS do any String->to->primitive conversions you need.

4. Pluggable message body mappings

RESTEasy MOM does not take advantage of this feature of JAX-RS because it is dealing with HTTP message generically and is really just a router of messages. It doesn’t really care about the content. But, this will not be true in other, less-generic, business oriented applications. Through the @Provider contract, developers can write plugins that know how to marshal and/or unmarshall a specific MIME type to a specific Java class. JAX-RS itself requires built-in support for JAXB. Other mappings like JSON->Java could easily be plugged in as well.

Evolving our RESTFul Messaging interface

There are few things that need to be tweeked in this simple RESTful interface.

GET is idempotent

Our interface for receiving messages has broken our constrained uniform interface. How you ask? Well, the HTTP GET operation is supposed to idempotent. What that means is no matter how many times you invoke it it is not supposed to change the state of the underlying resource. It is supposed to be repeatable with no side affects. This is not the case here as when we GET from the queue, we are consuming a message, sending it back to the receiving client, and removing the message from the queue. This is changing the state of the resource and using GET in the manner is poor RESTful design.

This may sound like nit-picking, but its not. Consider if the Unix file system allowed users to change the state of a file or device with a read access? It would break file permissions and our system wouldn’t be consistent. Later, in another blog , (this article is already too long) when I talk about caching and Cache Control headers, I’ll show you why this might be important for a RESTful messaging system. But for now, I hope you get what I’m saying.

What HTTP method should we use then?

So, we have to replace GET. Your first insite might be, what about DELETE? We are consuming/deleting a message from the queue when we send it back to the client and DELETE is allowed to send back content as a response. Unfortunately, again, DELETE is supposed to be an idempotent method. It is supposed to be repeatable. The only logic choice is POST because POST allows you to change the state of your resource. Unfortunately, we already have POST assigned to an operation so what should we do?

I don’t like overloaded POST

What some people like to do is overload a POST operation by adding an action query parameter:

POST /queues/{name}?action=[send|receive]

I don’t like this because what you’ve done is provided multiple operations for what is supposed to be a uniform operation. You’ve created a mini-rpc framework. Beyond that, it doesn’t map very well to JAX-RS. You’d have to do some mini dispatching in a specialized generic method. Might as well just write a servlet :).

Just create a new resource

Instead, a better plan is to just create a different resource, a.k.a. just extend the URI path.

POST /queues/{name}/receiving

Now that receiving is its own resource we use POST for getting and consuming the message. We could, in the future, use GET to peek at the message. In much the same way in programming we’d want to separate the sender and receiver into two different interfaces and/or objects, we can do the same with URI resources.

PUT instead of POST for sending

When we send a message via POST, POST does not guarantee any idempotency. This kinda sucks because we could possibly send a duplicate message. For example, lets say we POST a message, the server locally sends it over JMS successfully, but crashes before it can send the acknowledgement. The sending client has no idea whether the message posted to the internal JMS queue or not, so, it might resend the message. So can we solve this problem? We can make sending a message idempotent by writing to a specific URI that is dedicated to that particular message. The client would be responsible for generating a unique identifier for that and PUTing it at a specific URI:

PUT /queues/{name}/messages/{message-id}

Using PUT in this way makes sending the message idempotent. Why? Well, if, on the initial send, there is a failure, the sending client can just retransmit. When the server processes a send, it has the identifier of the message. It can look in its persistent store to see if the message already exists and has been posted, or look in memory. If a message of that id doesn’t exist, just post it, if it does exist then do nothing. Our logic doesn’t even have to be that complicated. Anytime a message comes in, just log a message of that id and overwrite any existing one. Since they should be the same data, no harm done. We would use a PUT instead of POST here because the standard HTTP 1.1 definition of PUT tells the user of our messaging system that sending a message is idempotent and they don’t have to worry about performing resends.

Unfortunately, we cannot implement this with our JAX-RS facade over JMS. While JMS does let you specify a message id, it is really undefined on what happens if you post a message of the same id. We could browse the queue to see if the message is there, but, if the message gets processed before we can’t check to see if it already exists. We would have to extend the functionality of an open source JMS implementation like JBoss Messaging.

Improvements to JAX-RS design

So, after our RESTful design improvements we have the follow JAX-RS design:

@Path("/queues")
public interface QueueManager {
   @POST
   @Path("/{name}")
   public void send(@PathParam("name") String destination,
                          @QueryParam("persistent") @DefaultValue("true") boolean persistent,
                          @Context HttpHeaders headers,
                          InputStream body);

   @POST
   @Path(("/{name}/receiving")
   public Response receive(@PathParam("name" String destination);
}

One of the problems with defining our JAX-RS interface in this way is that we now have two things coupled together when they shouldn’t be. Any implementation of our QueueManager interface would have to both locate a JMS ConnectionFactory and Destination as well as process the request and interact with a Connection and JMS Session. The problem is that, locating the ConnectionFactory and Destination is going to be implementation specific. Each JMS implementation could have their own specific way of finding these items. So, it would be better if we could separate these concerns into two separate classes. What’s cool is that JAX-RS allows us to model such a scenario with Subresources and Subresource Locators.

JAX-RS Subresources and Subresource Locators

A cool feature of JAX-RS is that you can divide the processing of an individual requests between multiple objects. A top-level resource could gather some information from part of the HTTP request and hand back to the JAX-RS runtime an object that can handle the remainder of the request. These are called Subresource Locators and Subresources. What we are going to use them for in our messaging system is to break up finding a JMS destination and processing a sending or receiving of a message into two separate classes/objects/resources.

@Path("/queues")
public class RestfulJBossQueueManager {

   @Path("/{name}")
   public RestfulQueue locate(@PathParam("name") String destination) {
       ConnectionFactory connectionFactory = (ConnectionFactory)jndi.lookup("java:/ConnectionFactory");
       Destination queue = (Destination)jndi.lookup("queue/" + destination);
       return new RestfulQueue(connectionFactory, queue);
   }
}

The sole purpose of the RestfulJBossQueueManager class is to find the target destination through the destination @PathParam parameter. It must not have a HTTP method annotation like @POST and must return an object that can process the rest of the request. The locate() method does just this. It finds a ConnectionFactory and Destination and uses these variables to construct an instance of the subresource class, RestfulQueue.

public class RestfulQueue {
   public RestfulQueue(ConnectionFactory factory, Destination destination) {
     ...
   }

   @POST
   public void send(@QueryParam("persistent") @DefaultValue("true") boolean persistent,
                    @Context HttpHeaders headers,
                    InputStream body) {...}

   @POST
   @Path(("/receiving")
   public Response receive(@PathParam("name") String destination) {...}
}

The JAX-RS runtime dynamically scans the returned RestfulQueue class for annotated methods that could be used to finish processing the HTTP request. This is a simple example of Subresources and their locators, but imagine if your Hibernate object/rational model served the dual purpose of a data and restful interface. Just some thoughts… 🙂

More to come…

I had a lot more to write but I’m already at 3000 words, a little bit long for a blog entry. I plan on writing one or two more entries diving into some more complexities and features you could add to RESTEasy MOM including content negotiation, listeners, tighter reliability, and overall better RESTFul design, applying more RESTFul techniques and architecture to a distributed messaging system. For now, you can check out the RESTEasy MOM project, download the code, read the WIKI and see what I’ve experimented with so far. If you like what you see, post us an email on resteasy-developers@lists.sourceforge.net

Older Entries Newer Entries