Bill the Plumber

Draft 5 of REST-* messaging talked about iterating on the reliable posting protocol.  Recently, I was arguing with Jan Algermissen on a completely unrelated subject.  As a result of this conversation, I ended up re-reading Roy’s post on hypermedia for the 4th time in 2 years. Re-reading Roy’s blog got me thinking a bit about improving the message posting protocol of REST-* Messaging so that it is driven more by in-band information, rather than out-of-band information.

Firstly, I want to remove the post-message-once and post-message link relationships.  Instead, a destination resource would only publish the create-next link.  When a client wants to post a message to a queue or topic, it will use this link to create a new message resource on the server.    The type of this link would be “*/*” meaning it accepts any media type.

The key change to the protocol would be for the client to be aware that responses to creating messages through a create-next link may contain a new create-next links.  The client is encouraged to use these new, on-the-fly, create-next links to post additional messages to the topic or queue.  An important point of this change is that the server is not required to send a create-next link with its responses.  How, and if, the server protects itself from duplicate message posting is up to the server.

So how could the server protect itself from duplicate message posting?  One implementation could be that the server returns a 307 response code, “Temporary Redirect” for the initial POST to the static top-level create-next link published by the destination resource.  This 307 requires the client to re-POST the request to a URL contained in a response Location header as defined by the HTTP 1.1 specification.  The Location header would point to a one-off URL (like the previous protocol defined in Draft 5).    If a network failure happens, then the client re-POSTs to this  URL.  If the messages was previously successfully processed by the server, the server would respond with a 405, Method Not Allowed.  If no network failure happens on the re-POST from the 307 redirection, then the server would just return a success code.  In either response, the server would also return a new create-next link as a Link header within the response.  The client would use this new create-next link to post new messages.  Subsequent posts to these new links would not have to go through the re-direct protocol because they would be newly generated one-off URLs.

I have been reading a bit that some weird or undesirable behavior may be experience with some user agents when using 307 and POST/PUT.  So, I think that if the REST-* Messaging specification leaves it undefined how a server implementation handles the initial response of a duplicate-message protection protocol, we can let it evolve on its own.  The key here is that the client should be encouraged to look within the response for new create-next links even from error responses.  For example, if instead of 307, the initial POST return a 415, Preconditions Failed and that error response contained a create-next link header, the client should use that link to re-post the request.  NOTE!  I think 307 is probably the best implementation, but maybe its best to give flexibility to implementors.

Keep the *-batch links

I still want to have separate links for submitting batches of messages.  Specifically rename post-batch to create-next-batch (and remove post-batch-once).  I want the distinction so that the server knows that it is receiving a collection of messages vs. the server just forwarding a message to message consumers that just happens to be a collection media type.

I’ve made some small changes to REST-* Message Draft 5.  First is to the reliable posting of messages to a message destination.  The second is to the push model default subscription creation method.

New post-message-once protocol

Previously, the post-message-once link used the POE pattern to avoid duplicate message posting.  I asked around and it seems that the POE pattern isn’t used a lot in practice.  I’m glad because it kinda breaks the uniform interface (unsafe GET) and isn’t really consistent with the other protocols I defined.  It is also very inefficient as you have to make two round trips to post each message.  Nathan Winder, on the reststar-messaging list suggested using a one-off link generated with each message post.  Here’s how it looks:

The post-message-once link URL provided by this link is not used to actually create a message, but rather to obtain a new, one-off, URL. An empty POST should be executed on the post-message-once link. The response provides a new “create-next” link which the client can then post their message to. The link is a “one-off” URL. What that means is that is that if the client re-posts the message to the create-next URL it will receive a 405 error response if the message has already successfully been posted to that URL. If the client receives a successful response or a 405 response, there should be a Link header returned containing a new “create-next” link that the client can post new messages to. Continuously providing a “create-next” link allows the client to avoid making two round-trip requests each and every time it wants to post a message to the destination resource. It is up to the server on whether the create-next URL is a permanent URL for the created message. If it is not permanent, the server should return a Content-Location header to the message.

post-message-once example

Change to push model subscription

I also added a minor change to the push model’s subscriber registration protocol.  In the previous version of the spec, the client would post form parameters to a subscribers URL on the server.  The form parameter would define a URL to forward messages to and whether or not to use the POE protocol to post this message.  I changed this to simple require the client to post an Atom Link.  Since links define protocol semantics, the server can look at the link relationship registered to know how to interact with the subscriber when forwarding messages.  So, if the client registers a post-message-once link when it creates its subscription, the server knows how to interact with the link.  This gives the client and server a lot of simple flexibility in describing how messages should be forwarded.  For example:

This example shows the creation of a subscription and the receiving of a message by the subscriber.

Just finished draft 4 of REST-* Messaging.  Please check our our discussion group if you want to talk more about it.  Here’s a list of resources and their corresponding relationships for a high level overview.  See the spec for more details.  It relies heavily on Link headers.  The current draft shows a lot of example HTTP request/responses to give you a good idea of what is going on.

Destination

A queue or a topic resource.

Relationships:

• post-message – link to POST/create a message to a queue or topic.  This can be
• post-message-once – link to POST/create a message to a queue or topic using the POST-Once-Exactlly (POE) pattern
• post-batch – POST/create a batch of messages using Atom or Multipart
• post-batch-once – POST/create a batch using the POE pattern.

Message

Every message posted creates a message resource that can be viewed for adminstration, auditing, monitoring, or usage.

Links Relationships:

• reply-to.  Destination that you can reply to
• reply-to-once.  Destination that you can reply to once using the POST-Only-Once pattern
• reply-to-batch. Destination that you can reply with a batch of messages
• reply-to-batch-once.  Batch using POE.
• next.  If pulling from a topic, messages will have this link.  This is the next message after this one.
• acknowledgement.  If pulling from a queue, this link allows you to change the state of the message to acknowleged or not.
• generator. Destination that the message came from

Topic

Has the same links as Destination with these added:

Link Relationships:

• next.  GET the next incoming message.  Clients should use the Accept-Wait header to set a blocking timeout (see later).
• last.  Last posted message to the topic
• first.  First message posted to topic, or, the first message that is still around.
• subscribers. URL that allows you to view (GET) or create/register (POST) a list of URLs to forward messages to.

Queue

Same as Destination, but these additional link relationships:

Link Relationships:

• poller.  Alias to the next available message in the queue.  Returns a message representation along with a Content-Location header pointing to the real one.  a POST is required to access this because the state of the queue is changed.
• subscribers.  Same as topic push model.