Bill the Plumber

Plumbing the network

  • Recent Posts

  • Categories

  • Archives

REST-* Messaging Draft 3

Posted by billburke on November 6, 2009

After prototyping, I’m back to writing another draft. This is a little bit more formal draft. I created a OSS project at:

http://sf.net/projects/rest-star

A draft of the PDF is at:

http://reststar-messaging.googlegroups.com/web/REST-Star-Messaging-draft-3.pdf

(You’ll have to click through an additional link that is *very* long). This draft only talks about the Message Publishing Protocol. I still have to write up the topic and queue push and pull model.  To discuss the draft please post at the REST-* Messaging Group.  I’m also looking for people to help prototype the specification.  Post to the RESTEasy developers list if you’re interested.

Posted in REST, REST-star | Leave a Comment »

RESTEasy 1.2.GA Released

Posted by billburke on November 4, 2009

After a few months RESTEasy 1.2.GA is finally ready.  This is mostly a cleanup, bug fix, and refactoring release, but here are some features of note:

I’d also like to thank Attila Kiraly for fixing some bugs that cropped up in Multipart and XOP support.  Pascal de Kloe, a new committer, also helped refactor content negotiation to support charset variants.  Solomon Duskis continues to be a seasoned veteran and helps with bugs and features here and there.

What’s next?  I’d like to focus next on getting complete OAuth support in.  I think it will help out our REST-* efforts as we look to secure the services we’re defining there.  RESTEasy is also going to expand beyond a simple JAX-RS implementation.  As JBoss projects like HornetQ, jBPM, Drools, Transactions, and Infinispan obtain RESTful interfaces, I’ll be creating a REST profile under the RESTEasy umbrella.

Useful links:

Posted in JAX-RS, REST, RESTEasy, java | Leave a Comment »

Parsing Link headers with Javascript and Java

Posted by billburke on October 15, 2009

I wrote a Link header parser in Javascript. Thanks Mark Nottingham for sending me a link to his python equivalent.

var linkexp=/<[^>]*>\s*(\s*;\s*[^\(\)<>@,;:"\/\[\]\?={} \t]+=(([^\(\)<>@,;:"\/\[\]\?={} \t]+)|("[^"]*")))*(,|$)/g;
var paramexp=/[^\(\)<>@,;:"\/\[\]\?={} \t]+=(([^\(\)<>@,;:"\/\[\]\?={} \t]+)|("[^"]*"))/g;

function unquote(value)
{
    if (value.charAt(0) == '"' && value.charAt(value.length - 1) == '"') return value.substring(1, value.length - 1);
    return value;
}

function parseLinkHeader(value)
{
   var matches = value.match(linkexp);
   var rels = new Object();
   var titles = new Object();
   for (i = 0; i < matches.length; i++)
   {
      var split = matches[i].split('>');
      var href = split[0].substring(1);
      var ps = split[1];
      var link = new Object();
      link.href = href;
      var s = ps.match(paramexp);
      for (j = 0; j < s.length; j++)
      {
         var p = s[j];
         var paramsplit = p.split('=');
         var name = paramsplit[0];
         link[name] = unquote(paramsplit[1]);
      }

      if (link.rel != undefined)
      {
         rels[link.rel] = link;
      }
      if (link.title != undefined)
      {
         titles[link.title] = link;
      }
   }
   var linkheader = new Object();
   linkheader.rels = rels;
   linkheader.titles = titles;
   return linkheader;
}

I had written a non-regular-expression, hand-coded parser in Java for RESTEasy. Thought it might be faster than regexp. That could probably be ported to Javascript too.

Posted in REST, java | Leave a Comment »

Link headers vs. Custom headers

Posted by billburke on October 14, 2009

There’s a really nice RFC out that defines the concept of a Link header.  Link headers are like Atom links within documents except they are specified within request or response headers.  For example let’s say we wanted to GET a medical image of an MRI.  This image might have additional relationships to a front, back, top, bottom up view.  A response with link headers might look like this:

HTTP/1.1 200, OK
Content-Type: image/jpeg
Link: <http:/.../front.jpeg>; rel=front; type="image/jpeg",<http://.../back.jpeg>; rel=back;type="image/jpeg"

...

The jpeg format cannot embed hyperlinks within it so to obtain relationship information, the Link header is used.  Other than data formats that don’t support hyperlinks, using the Link header can also be very valuable to intermediaries that don’t care or want to know about the content being exchanged.

Link headers vs. Custom headers

I really like the logical concept of Link headers.  The syntax of the header is also solid and fine.  The problem I have with it though is that it is non-trivial to parse.    Please, please, correct me if I’m wrong, but parsing a Link header, according to the current version of the spec, isn’t as easy as doing a few string.splits or simple regular expressions.  This is because ‘;=,’ characters can be embedded within quotes.  The ‘;’ can also be embedded within links and media types.  Not only this, but to get at a specific link I have to parse all the links defined within the header and create a map, before I can define a specific link.

Granted, its not hard to write a parser for this.  It is a micro format.  The problem is that HTTP libraries generally do not have native support for this.  Not a very big deal, but IMO, one of the big selling point of REST is its low footprint and the ability to leverage existing libraries to make HTTP invocations.  I came across this problem when writing a Javascript client.

This made me think about redefining the problem.  Instead, isn’t a Custom HTTP header easier to interact with from an API point-of-view?  Do clients and servers really need the media type information (or really any other metadata) about the link?  Could we instead have a simple custom header?

HTTP/1.1 200, OK
Front-link: http/.../front.jpg
Back-link: http://.../back.jpg
Content-Type: image/jpeg

...

Then its just a matter of hash lookup using existing HTTP library facilities.

The downside of course to all of this is that you lose all the self-description aspects that are provided by Link headers.  Specifically the media type (type attribute) as well as IETF registered links vs. custom link relationships referenced via a URL.  IMO, this self-description metadata within Link headers also helps to mitigate against namespace clashes where different domains might define the same header names.

Header Templates?

What if the Link header spec defined a template for header names?  like Link-{rel}, Link-{rel}-Type, Link-{rel}-Title, etc…  Link-{rel}-{attribute}  So, my response might look like this instead:

Link-Front: http:/.../frong.jpg
Link-Front-Type: image/jpeg
Link-Back: http:/.../back.jpeg
Link-Back-Type: image/jpeg

To find links published by a resource is just a matter of iterating through and doing a “startsWith(‘Link’)” on the header.

Posted in REST | 7 Comments »

Announcing a new distributed computing paradigm: ULSER

Posted by billburke on October 7, 2009

I’m pleased to announce a new distributed effort named ULSER.  ULSER stands for a set of uncompromisable architectural principles, specifically:

  • Uniform constrained interface.
  • Links as the engine of application state
  • Statelessness for scalability
  • Enterprise applicable
  • Resource and representation oriented

Why call it ULSER?  Well, even though these architectural principles are extremely useful to design distributed applications (and middleware), I’ve found that discussing any of these  principles publicly have given me (and others) an ulcer.   So, I thought I’d make these principles my own and let others share the pain.  I’m also going to create a set of standards called ULSER-*.  ULSER-*.org will focus on bringing ULSER principles to middleware technology.  Since middleware generally produces ulcers in engineers I think the name is perfect!  I’ve also created an open source Java project called EasyULSER so that you can let Java give you an ulcer too!

Also, don’t worry, I’ve deliberated with the doctor who discovered ulcers and coined the term “ulcer”.  He didn’t mind me using the term as long as what I’m doing will cause me an ulcer.  The ulcer community didn’t seem to mind either as when I tried to explain to them the principles of ULSER and how I’m using them, it gave them an ulcer too, so they were happy.  So, it looks like we’re ready to go!  ONWARD!

Posted in flame bait, sarcasm, satire, spam | 8 Comments »

Thanks James and Cheyenne

Posted by billburke on September 30, 2009

I just want to publicly thanks James Cobb and Cheyenne Weaver for putting together the branding, skin, look and feel for REST-*.org.  The site is nice to look at and gives a professional look and feel that I could never in my wildest dreams created by myself.  I hope to tap their expertise and artistic ability to perform similar branding efforts for RESTEasy.

Posted in REST-star, jboss | 1 Comment »

RESTEasy 1.2.RC1 Released

Posted by billburke on September 28, 2009

We’re getting ready to release 1.2.GA.  This is a release candidate that fixes a bunch of outstanding bugs and reorganizes the project a little bit as well.  It is strongly suggested you take a look at the migration guide because a few pieces of RESTEasy have been refactored into separate Maven modules.

All links to documentation and downloads are available at the RESTEasy Project Website.

Posted in spam | Leave a Comment »

New REST-* Workflow/BPM Effort

Posted by billburke on September 24, 2009

I’ve put together some preliminary thoughts that have been swimming around in my head for the past few months on workflow and BPM.  Its hosted on our new REST-* work-group on the subject.  While we still have some work to do on defining some of the initial media types and going beyond simple use cases, the design is very link driven.  I’ve asked Tom Baeyerns and the jBPM team to help flush out the most common use cases users deal with.  I also need a REST heavyweight lurking around to make sure we’re not introducing unRESTful designs.  Anyways, Enjoy!

Posted in REST, REST-star | Leave a Comment »

We screwed up with first REST-* submissions

Posted by billburke on September 23, 2009

We really screwed up with the initial drafts we put up for the initial REST-* specifications on a number of levels.  First the specifications were written 1+ years ago by me for the messaging submission and 8+ years ago by  Mark Little.  Both of us, at the time were still trying to come to terms with what REST actually was (we still are) and how you design applications/systems/services around it.  As a result, those initial drafts, while they did follow the uniform interface principle of REST, weren’t very RESTful beyond that.  The thing is, I knew this!  I knew these initial drafts were crap.  I had planned to completely rework them and have draft 2 ready for when we announced.  So, when the press picked up Mark Little’s announcement of REST-* at JBoss World (It was actually only one bullet point on a “future directions” slide), the cat was out of the bag too early and we ended up seriously botching the announcement because we werent’ ready.

While the REST-* name itself was bound to cause friction within the hard-core REST community, when the lurkers on the rest-discuss mail list actually looked at these initial drafts, to them, their greatest fears were realized:  That what we were doing was just a re-hash of the same old architecture vendors have been rehashing over and over again for almost 20 years now (DCE, rehashed to CORBA, rehashed to WS-*) and that we were just repackaging our existing products and calling it REST to be buzzword compliant.  Even a few feared that we were trying to coopt REST itself which was totally incorrect.  We just want to figure out where middleware fits into this new model, specifically the middleware problems that Red Hat is used to solving.

So, we definitely are marketing morons, well specifically me.  I botched the announcement by not being ready.  All I can say is mae culpa.  If you’ve read my blog over the past few days you’ve seen me scurrying to catch up on writing the technical material I had always intended to write.  Hopefully you can see that we’re headed in the right direction in that we’re describing things that are much more RESTful than before.  All we can do is plug on…

Posted in REST, REST-star | 1 Comment »

Credit Cards, Transactions, and REST

Posted by billburke on September 21, 2009

One of the interesting thing about credit cards is that actions performed on your account are not operations, but instead modeled as things that are first related to your account, then that actually change the state of your account.  When you buy something at a store online or in person, you are not actually manipulating your account directly.  A credit card purchase is converted into a thing:  specifically a Debit.  A refund of a purchase is converted into a thing called a Credit.  This credit or debit is logged at Visa or Mastercard.  Once a week (i forget the frequency), a company like Capital One (I used to work for them in the mid-90s) receives a huge magnetic tap roll with all the transactions that are applicable to Capital One’s accounts.  Capital One then pulls these individual credits and debits and matches them with a specific credit card account.  Finally it processes the debits and credits for a account to calculate a final balance.

The key thing to take from this is that credit card transactions are not operations, but actual things.  Another similar construct is an Airline Reservation.  A reservation is very similar to a credit or debit.  It is a representation of the final thing you want to do.  Create a ticket on an airplane.

Where REST and Transactions meet

I think if we constrain ourselves to modeling transactions as things instead that are themselves resources REST and Transactions can co-exist.  So, what transaction processing becomes in RESTful terms is relating one resource to another.  For Credit Cards, transaction processing is the result of relating Debit and Credit resources with Credt Card Account resources.

I think the wrong way to merge REST and Transactions is the way the O’Reilly RESTful Web Services book suggests.  In the book, the authors talk about manipulating an account balance by creating a parallel resource of an account as a subresource of a transaction object:

/transactions/111/account/222

The transaction sub resource is manipulated with representations of the account.  At “commit” time, the “real” account is updated with a new representation of the account.  The funny thing is, as I talked about earlier, the real world doesn’t work in this manner.  This idea that a RESTful Transaction must work in the same way you interact (with JDBC or something) with your database and that uncommitted state modifications to a resource must be viewable just isn’t true and not really feasible, and it is just not RESTful.

Where do Transaction Managers fit in?

A Transaction Manager is a generic service whose sole purpose is to ensure that a distributed system is in a consistent state.  It does this by coordinating actions between various things on the network.  I think if we model actions as things and model transaction processing as the action of relating one or more resource to one or more other resources, Transactions, Transaction Managers, and REST can all fit together.  To illustrate this let’s model a Travel Agent transaction where the Travel Agent is reserving a flight and a room within a hotel.   While there are many different transactional protocols, we’ll model the protocol that RESTafarians deem the most-unRESTful, 2-Phase-Commit.

The resources in our example will be: Reservation, Ticket, Room, Transaction-State, Transaction, and Transaction Manager.  Let’s walk through on how these resources are created.  The end product of these interactions will be a Ticket and  Room.

1. Travel Agent creates a transaction

The Travel Agent interacts withte Transaction Manager resource to create a transaction resource.  For example:

POST /transaction-manager
Content-Type: application/tx+xml

<transaction timeout="1 minute"/>

The Transaction Manager responds with:

HTTP/1.1 200, OK
Content-Type application/tx+xml
Location: /transaction-manager/transactions/11

<transaction timeout="11" state="uncommitted">
   <link rel="participants" href="/transaction-manager/transactions/11/participants"/>
</transaction>

2. The Travel Agent makes an Airline Reservation

POST /airline/reservations
Content-Type: application/airline-rsv+xml

<reservation>
   <date>...</date>
   <flight>...</flight>
   <seat>...</seat>
   <link rel="tranasaction" href="/transaction-manager/transactions/11" type="application/tx+xml"/>
</reservation>

Here the client is creating the reservation.  Notice that the reservation created is related to a transaction.  The server responds with:

HTTP/1.1 200, OK
Location: /airline/reservations/222
Content-Type: application/airline-res+xml

<reservation>
   <date>...</date>
   <flight>...</flight>
   <seat>...</seat>
   <link rel="transaction" href="/transaction-manager/transactions/11" type="application/tx+xml"/>
   <link rel="transaction-state" href="/airline/reservations/22/transaction-state"/ type="application/tx-state+xml"/>
</reservation>

Notice that a new relationship is defined on the Reservation: “transaction-state”.  This is the state of the reservation.  Another thing to note is that this representation returned with the response of the POST would be the same representation returned if you did a GET /airline/reservations/22.

3. Register Transactional State With Transaction

IMO, it doesn’t matter who relates the Reservation’s Transactional State resource with the Transaction resource.  Both the Travel Agent and the Reservation have the links needed to perform the operation.  Whoever does this, the operation might look like this:

POST /transaction-manager/transactions/11/participants
Content-Type: uri-list

http:.../airline/reservations/22/transaction-state

Another way to do this might be to use Link headers and the HTTP LINK operation (don’t quote me on how exactly to do this!):

LINK /transaction-manager/transactions/11/participants
Link: <http:.../airline/reservations/22/transaction-state>; rel="airline-reservation"; type="application/tx-state+xml"

4. Travel Agent Creates Hotel Room Reservation

The Hotel reservation works in the same way as the Airline Reservation, so I don’t want to burden you with reading duplicate interactions :)

5. Client Commits the Transaction

To Commit the transaction, the client simply POSTs to the Transaction resource

POST /transaction-manager/transactions/11
Content-Type:  application/tx+xml

<transaction state="committed"/>

This would block until the Transaction Manager has coordinated all services.  Or, it could return a 202, Accepted and the client could check back later to see if the state changed or not.

6. Transaction Manager Prepares

The particpants of the transaction have been registered.  The TM initiates the “Prepare” phase of the 2pc protocol by trying to change the state of each partcipants Transaction-State resource to “prepared”, i.e.:

PUT /airline/reservations/22/transaction-state
Content-Type: application/tx-state+xml

<transactional-state state="prepared"/>

What’s happening behind the scenes?  IMO, it doesn’t really matter.  Maybe we’re locking up database resources, maybe we’re just checking to see if it is still possible to purchase a ticket for that flight.

7. Commit each participant

Now that all the participants are in prepare mode, the TM will guarantee that each participant commits by logging the vote to its transaction log.  If any body fails, ( a crash or something), including the Tranactaion Manager, the TM can recover and put things in a consistent state, or if, it can’t, notify a human that it must do something.  This is what a transaction maanger was designed for.  Reliability.  So, the TM changes the state of each particpant to COMMIT. i.e.

PUT /airline/reservations/22/transaction-state
Content-Type: application/tx-state+xml

<transactional-state state="committed"/>

8. Retrieve the Ticket and Room

In this example system, the act of committing the reservation will create a Ticket or Room behind the scenes.  So, now, when the Travel Agent does a GET on the Airline or Hotel Reservation, a new link will pop up that points to a Ticket or Hotel Resource:

GET /airline/reservations/22
Content-Type: application/airline-res+xml

<reservation>
...
   <link rel="ticket" href="/tickets/333" type="application/ticket+xml"/>
</reservation>

Conclusion

An interesting thing to note about this example is that when the interaction completes, there is a complete log of who participated within the transaction through the use of link relationships.

Finally, the point of this blog was not to promote the use of 2PC or Transaction Managers, but to show that it may be possible to model transactions RESTfully by turning actions into things and modeling transaction processing by relating one resource to another.  This is how I envision REST-*.org defining the relationship with REST and Transactions.  If you want to discuss this more, ping us on our working group.

Posted in REST, REST-star | 1 Comment »