The poor JAX-RS Request Dispatching Algorithm

5 Comments

As we’re rolling out Resteasy 3.0, we have to pass the JAX-RS TCK.  The good thing about this is that the TCK has grown massively is size and has a lot more test coverage for all old and new features of JAX-RS.  It allowed me to uncover a few bugs I would not have found without the TCK.  An unfortunate downside the TCK also got a lot stricter in some of the weak areas of the JAX-RS specification, particularly the request dispatching algorithm.  I’ll be blunt, the algorithm is poor.  IMO, the old spec leads made a huge mistake in introducing implementation details to the specification and now we have a poor algorithm we are stuck with.  Us vendors cannot innovate and improve it because the TCK has backed us into a corner and the licensing fine print of Java EE makes it really hard for us to ship things that diverge from the spec.  Here are a bunch of problems that used to work in Resteasy, but will no longer work because the TCK tests every fine detail of the JAX-RS matching algorithm.

  •  The @Path annotation at the class level is matched first before matching any resource methods.  Only classes with the best and exact regular expressions are picked.  Then the rest of the request is matched with remaining methods.  So this won’t work anymore with a spec compliant algorithm:
Request: OPTIONS /foo

@Path("/foo")
public class Foo {
   @GET
   public String get() {...}
}
@Path("/{.*}")
public class OptionsDefault {
   @OPTIONS
   public String options() {...}
}

Earlier versions of Resteasy would match OptionsDefault.options().  Now, this method will not match according to the spec rules and you’ll get the default JAX-RS OPTIONS behavior.

  • Locators are never resolved if there are resource methods that match the request.  For example
PUT /foo/sub

@Path("/foo")
public class Foo {
   @GET
   @Path("sub")
   public String get() {...}

   @Path("{id}")
   public Locator locator() { return new Locator(); }
}

public class Locator{
   @PUT
   public void put() {...}
}

You’d think that the request would resolve to Locator.put() but you’d be wrong! Because there is a resource method whose path matches the request, but not the method you’d get a 405 response from the server. What’s interesting if you flip the expressions, a PUT request would work, but a GET request wouldn’t!

PUT /foo/sub

@Path("/foo")
public class Foo {
   @GET
   @Path("{id}")
   public String get() {...}

   @Path("sub")
   public Locator locator() { return new Locator(); }
}
  • It is possible to have poorer matches
GET /fart
Accept: text/plain

  @GET
  @PATH("foo") 
  @Produces("text/plain") 
  public String get1() {} 

  @GET
  @Path("{text}") 
  @Produces("text/plain") 
  public String get2() {} 

  @GET
  @Path("f{text}") 
  @Produces("text/*") 
  public String get3() {}

You would think that GET /fart would match the get3() method because it is more specific path, but you’d be wrong.  Because get3() has a less specific @Produces get2() would match.  This is weird because the spec originally tells you to sort expressions on a best-match basis but then ditches this information to match Accept headers.

Another related note is the default returned media type.Right now the default is dependent on the deployment.  If there is no Produce header, then the returned media type defaults to a union of the Accept header and explicit media types of all available MessageBodyWriters.  There goes your portability!  Instead, implementations should be allowed to specify their own default or even make it configurable.  But, of course we can’t do that!

Granted some of these issues are edge cases, but IMO, some are not.  The specification has 2 pages on english/pseudo-academic algorithm syntax to describe this very complex, but poor algorithm.  Users will get frustrated trying to understand it.  The experts themselves argued for days on interpretation of the specification.  Users will scratch there head wondering why certain classes will match and some won’t and blame the vendor’s implementation.  Resteasy had at least 4 user-reported regression tests that failed as a result of following the specfication matching algorithm religiously.  I know these users will be back complaining that Resteasy 3.0 does not work for them when Resteasy 2.3.x did.

Resteasy 3.0-beta-3 – Latest Spec Updates

1 Comment

Resteasy 3.0-beta-3 has been released.  Follow the links from our main jboss.org page to download and view the documentation.  Here are the highlights:

  • The latest and greatest from the master branch of the JAX-RS 2.0 spec.  Many of the client builder SSL changes I introduced in 3.0-beta-2 have made it into the spec.  Thanks Marek for giving the thumbs up on them.
  • There are a few minor features of JAX-RS 2.0 we don’t have implemented yet.  You’ll get a NotImplementedYetExceptoin if you invoke them.

Next I’ll be focusing on my book, implementing our missing features, refactoring, and general test coverage.

 

Resteasy 3.0-beta-2 Released with New OAuth 2.0 Features

3 Comments

Resteasy 3.0-beta-2 has been released.  Follow the links from our main jboss.org page to download and view the documentation.  Here are the highlights:

  • Added a new ResteasyClientBuilder class to make it easier to create HTTPS/SSL connections on the client side
  • Extensive work on OAuth 2.0 support including tight AS7 integration.

You can find out more about our OAuth 2.0 stuff here, and the distribution comes with an extensive example.  Here’s the overall features of it:

  • Turn an existing servlet-form-auth-based web application into an OAuth 2.0 provider.
  • Provide Distributed Single-Sign-On (SSO) from a central authentication server. Log in once, and you can securely access any browser-based app configured to work in the domain.
  • Provide Distributed Logout. Following one link from any application can log you out of all your distributed applications configured to use SSO.
  • Web apps can interact securely with any remote restful service by forwarding access tokens through the standard Authorization header.
  • Access tokens are digitally signed by the oauth2 framework and can be used to access any service configured to work in the domain. The tokens contain both identity and role mapping information. Because they are digitally signed, there’s no need to overload the central authentication server with each request to verify identity and to determine permissions.

What’s next for Resteasy?  Next release I’ll be focusing on getting it up to date with the latest JAX-RS 2.0 snapshot.  I also have to get started on my O’Reilly book.

Write your own logging abstraction

3 Comments

As software developers we are ingrained to believe that copying and pasting code into multiple places is a bad idea.  As everything in life, you should always question what you’re taught as there are always exception to the rule.  I think logging abstractions are is one of these exceptions.  How many of you have struggled with logging frameworks?  How many times has one project or library used a different framework or version that causes incompatibilities?  How many often do you use a maven exclude for apache commons logging, log4j, or slf4j because each dependency uses a different framework or version of that framework?  Its a pain in the ass and a bleeping mess!

HornetQ was the first project I saw where they said, “To hell with logging abstractions!”.  They wanted to reduce external dependencies and logging was at the top of their list.  They implemented their own logging abstraction that used JUL by default, but that could also delegate to log4j, slf4j, etc.  Generic findClass invocations were done to determine which logging framework was installed in the environment HornetQ was running in.

So, what I’m suggesting is that each project should cut and paste what HornetQ has done and incorporate it into their own projects.  Let each project own their own logging abstraction and reduce their dependencies.  A side effect is they’ve also reduced any potential maven transitive dependency conflicts as well.  We did this very thing in Resteasy last year.  This is one case where copy and pasting is a good thing.

Enjoy.

Java EE wins over Spring

170 Comments

The past 1-2 years since the release of Java EE 6, you’ve seen a lot of articles like this latest on TSS that talk about the niceities of Java EE 6’s component model over Spring and how Spring is now legacy.  Yup legacy.  Who would have thought it?  (other than me of course 😉 ) I remember internal JBoss emails 4-5 years ago arguing whether we should give up on promoting Java EE as a component model (aka on EJB) and just concede to Spring.  Now, 4-5 years later, Java EE 6 has answered the challenge and is a viable, rich, integration technology.  So what happened?

Spring always depended on Java EE

Spring was and has always been a wrapper over core middleware infrastructure: ORM, Transactions, Messaging, HTTP.  It always depended core Java EE specs like JPA, JTA, JMS, and Servlet.  So, since you couldn’t deploy a Spring app without at least one of these core technologies/specifications, Java EE stayed in users minds.  There was always the opportunity that Java EE could get its act together in component model development.  While Rod Johnson always tried to position Spring as a Java EE alternative and JBoss killer, the Spring “platform” was never a true alternative to Java EE and JBoss, and in fact, couldn’t really exist without it.  IMO, this was a huge missed opportunity for the Spring folks.

Being the anti-Romney doesn’t work in the long run

J2EE was a machine with a huge massive install base.  A machine with a massive amount of money invested into it.  Java EE was its own market.  While Rod positioned himself over and over as the alternative to Java EE did he really think that this massive machine wouldn’t respond to the challenge?  While there are a lot of radical technology enthusiasts out there, the core Java constituency is pretty much moderate.  They are slow to adopt and tend to wait to see who is going to win the war over a long time.  Spring could not replace Java EE because technology wise, they were dependent on it.  All Java EE had to do was improve its component API message to the people, outspend Spring, and win over it in the long run.

Annotations were a game changer

The first thing that happened to shake Spring was the introduction of annotations in Java 5.  Annotations were a game changer.  Annotations were the opportunity to introduce mini-DSLs and pluggable keywords into Java.  Java EE 5 grabbed this opportunity with a huge facelift and refactoring of EJB and the introduction of JPA.  Basically, this was a standardization of Hibernate and its integration into EJB.  Complex EJB 2.x XML was replaced by few new Java keywords (well, annotations).  Simplicity ruled the day.  Middleware started to look more and more like a language feature rather than something hacked together via XML.  When annotations came out, I remember the Spring folks writing multiple blogs and forum posts about how evil they were.  IMO, they were just terrified of this new technology as it made much of Spring 2.x obsolete, and, well, much of Spring more complicated than Java EE 5.

CDI closed API hole

Thank you Gavin and the Seam folks.  CDI and Java EE 5 pretty much closed the technology gap.  Not only did they fill the integration holes that Spring exposed, they innovated far beyond what Spring had and created something new.  Beyond core IoC and DI, CDI’s event model was truly innovative and cool.

App Servers got their act together

Application server started to get their act together with regards to boot time.  It started with Glassfish and ended with JBoss 7.  Both of which can boot in a matter of seconds.  The whole Spring complaint that you needed Spring to mock out and test your code because app-servers started so slow was moot.

Arquillian made a mock of mocks

The final game changer was Arquillian.  One huge advantage Spring had was a unit testing story.  They gave you the ability to mock out core services like transactions and allow you to test application code outside of the application server.  This is huge for continuation integration and automated builds as well.  Combined with the fast boot times of JBoss 7 and Glassfish, you no longer have to hope your mocks will work when you actually run it in its real environment.  Arquillian allows you to run your unit tests in a real environment with real transactions, etc.  Personally I always despised mocks because they didn’t test in the environment you were going to run in.  I thought they were pointless and to this day, I refuse to use this testing pattern.

Anyways, in retrospect, I’m glad Rod and company were able to cash out with the VMWare acquisition before Java EE was able to regain its dominance.  SpringSource pushed Java EE to innovate and for that I’m very grateful.  For Java EE, it was either evolve or die.  They evolved, now its time for Spring to die.

 

Exchanging digital signatures with Python and Java

4 Comments

I’ve been testing my Content-Signature framework discussed earlier and wanted to see if I could exchange digital signatures generated and verified from both Python and Java code.   After a bit of research here’s what I’ve found so far on how to do this.

Generate keys with openssl

The first step is to generate a private key and a certificate using the openssl program.  This is a common utility.  Do a search if it is not available on your computer and you’ll find support and instructions to install on various platforms.  It came with my macbook pro (I think maybe with Darwin tools).  You’ll have to generate the keys in both .pem format (for Python) and .der format (for Java).

# generate pems
$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert-private.pem -out mycert.pem

# create private key .der file
$ openssl pkcs8 -topk8 -nocrypt -in mycert-private.pem -out mycert-private.der -outform der

# create certificate .der file
$ openssl x509 -in mycert.pem -out mycert.der -outform der

From this you should have 2 sets of files: mycert-private.pem, mycert-private.der and mycert.pem and mycert.der

Import private key sign in Java

Here’s a nice tool for loading in the .der files created into a Java KeyStore.  I’ve extracted some of the code so that you can see the whole manual, programmatic process of importing a private key and signing a message.

import org.jboss.resteasy.util.Hex;
import java.io.DataInputStream;
import java.io.*;
import java.security.*;
import java.security.cert.*;
import java.security.spec.PKCS8EncodedKeySpec;

public class ExampleSignTest
{
   @Test
   public void testDerFile() throws Exception
   {
      // import private key
      InputStream is = Thread.currentThread().getContextClassLoader().getResourceAsStream("mycert-private.der");
      DataInputStream dis = new DataInputStream(is);
      byte[] derFile = new byte[dis.available()];
      dis.readFully(derFile);
      KeyFactory kf = KeyFactory.getInstance("RSA");
      PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(derFile);
      PrivateKey privateKey = kf.generatePrivate(spec);

      Signature instance = Signature.getInstance("SHA256withRSA");
      instance.initSign(privateKey);
      instance.update("from-java".getBytes());
      byte[] signatureBytes = instance.sign();
      System.out.println("Signature: ");
      System.out.println(Hex.encodeHex(signatureBytes));
   }
}

The code prints out the signature in hex using a simple routine from Resteasy.

Import certificate and verify in Java

Here’s an example of verifying:

@Test
public void testDerFile() throws Exception
{
   CertificateFactory cf = CertificateFactory.getInstance("X.509");
   is = Thread.currentThread().getContextClassLoader().getResourceAsStream("mycert.der");
   Certificate cert = cf.generateCertificate(is);
   PublicKey publicKey = cert.getPublicKey();

   String hexSignature = "4e3014a3a0ff296c07927e846221ee68f70e0b06ed54a1fe974944ea17b836b92279635a7e0bb6b8923df94f4023de95ef07fa76506888897a88ac440eb185b6b117f4c906cba989ffb4e1f81c6677db12e7dc22d51d9369df92165709817792dc3e647dae6b70a0d84c386b0228c2442c9a6a0107381aac8e4cb4c367435d52";
   // loading CertificateChain
   Signature verify = Signature.getInstance("SHA256withRSA");
   verify.initVerify(publicKey);
   verify.update("from-python".getBytes());
   Assert.assertTrue(verify.verify(Hex.decodeHex(pythonHexSignature)));
}

The code has hardcoded a generated signature produced from signing the “from-python” string.

Import private key and sign in Python

The Python code requires the M2Crypto library.  I tried PyCrypto, but I could get it to work.  My code was tested on macbook pro with Python 2.6.1 M2Crypto version   0.21.1.  Also notice that the .pem files are used instead of .der.  I couldn’t figure out if M2Crypto fully supported .der so I just used the .pems.

from M2Crypto import EVP, RSA, X509
import binascii

key = EVP.load_key("mycert-private.pem")
key.reset_context(md='sha256')
key.sign_init()
key.sign_update("from-python")
signature = key.sign_final()
print "Signature:"
print binascii.b2a_hex(signature)

Importing certificate and verifying in Python

Here’s the verification:

rom M2Crypto import EVP, RSA, X509
import binascii

hexSignature = "0a11ab4ebcd2b0803d6e280a1d45b5b5d5d53688949f5a4f2d6436f15df3b10633c79760b9fe3b64eb9d84371c35e8b7d946052dfdd99ebb5cf7f3092762e1a91b261117e6675f2d28afe2ec4\
d90abfe3559a1259d2c66f3dc42ca3bfce7498705833445170bd8c293d60448b6c599abfe2d06882d3fff9ef887379eb7da3fe0"
java_sig = binascii.a2b_hex(hexSignature)

cert = X509.load_cert("mycert.pem")
pubkey = cert.get_pubkey()
pubkey.reset_context(md="sha256")
pubkey.verify_init()
pubkey.verify_update("from-java")
assert pubkey.verify_final(java_sig) == 1

Hope you enjoy.  If you know a better way to set up the certs and key files, let me know.  Using openssl was the best way I could find.

Remember why we don’t have Java 7

22 Comments

This is a bit of a reiteration of my previous blog, but, I wanted to be a bit more clear:

Ask yourself this question…Why do we not have a Java 7 release?  Mainly it is because of Apache (not the developers, but the bureaucrats) filibustering the Java 7 vote in the JCP Executive Committee all because they didn’t want a Field Of Use restriction for Harmony.   They felt entitled to the Java brand just because they are Apache.  For those of you who don’t know the Field Of Use, (IIRC) was that Harmony wouldn’t have been able to be used within a mobile environment.  IMO, I’d much rather have had a Java 7 release than to lift the FOU restriction just to make one Apache open source project  happy.  I’m upset with my company for supporting this fiasco.

Another side point:

The “I’m leaving the JCP because it isn’t working” play that seems to be popular at the moment, is, IMO, a big slap in the face to those of us who have put a lot of time, effort, engineering, and dollars to improve the Java platform, specifically on the EE side of things.  Specifically, the Apache CXF project who have created a top-notch SOAP implementation, as well, of course the Tomcat effort.  For Red Hat, we’ve put huge amount of engineering time into EJB, JPA, JSF, CDI, JAX-RS, and Validation.  There are many other companies, individuals, and open source projects that have made similar contributions.  Those of us who cared enough about the platform (and Sun and Oracle are both in this camp) have improved and evolved Java EE so that it is viable platform into the next decade, despite the best efforts of the “Party of NO” coalition of non-contributors on the EC and on the Java EE JSR.

IMO, if you are unwilling to give up something to obtain the Java brand, if you’re creating competing technologies that you have no intention of bringing back to the JCP to be standardized, if you or your company are not consumers or implementors of JCP specifications, then, you probably should leave the JCP.  In fact, I encourage it, so that the rest of us can have less obstacles in moving the platform forward.

And we care why?

15 Comments

So, Apache leaves JCP.  Surprise surprise.  Their biggest contribution in the past few years has been to filibuster the Java 7 JSR, and is the primary reason why there is no final version of Java 7 (or 8) today.  I’m all for non-contributing members leaving the JCP.  Less noise, and more people, who actually care about the language and EE platform working on improving it.

Improved HornetQ Spring Integration

6 Comments

I’ve been doing some work on HornetQ SVN trunk to improve the embeddable HornetQ experience (you’ll see these improvements in the next HornetQ release).  You can disagree with me, but I thought that embedding HornetQ was a bit verbose, especially for JMS, so first of all, I wrote two wrapper classes to make this easier.  While these classes will be in the next release along with a full docbook explanation, here’s what configuring JMS looks like now:

import org.hornetq.jms.server.embedded.EmbeddedJMS;

...
EmbeddedJMS jms = new EmbeddedJMS();
jms.start();

This class will look for hornetq-configuration.xml, hornetq-jms.xml, and hornetq-users.xml within your classpath.  It also has the option of manually creating configuration objects via pojo instantiation if you desire (or if you want to wire it with Spring for instance).

Simple Spring Integration

Another thing I did was to provide some simple Spring integration with the HornetQ JMS implementation.  I wrote a simple class that extends EmbeddedJMS that will register any configured queues, topics, and ConnectionFactorys direction within the Spring application context so that other beans can reference them.  Here’s an example of bootstrapping HornetQ JMS and referencing a queue within a Spring message listener container.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

   <bean id="EmbeddedJms" 
                class="org.hornetq.integration.spring.SpringJmsBootstrap" 
                init-method="start" destroy-method="stop"/>

   <bean id="listener"
             class="org.hornetq.tests.integration.spring.ExampleListener"/>

   <bean id="listenerContainer"
             class="org.springframework.jms.listener.DefaultMessageListenerContainer">
      <property name="connectionFactory" ref="ConnectionFactory"/>
      <property name="destination" ref="/queue/exampleQueue"/>
      <property name="messageListener" ref="listener"/>
   </bean>

</beans>

Again, this assumes you have configured HornetQ properly within HornetQ specific configuration files.  You can also manually declare HornetQ config objects within Spring and inject them into the EmbeddedJMS bean instance too if you don’t want to use HornetQ config files.

Right now, the code does not also register HornetQ objects within JNDI.  Do you think I should add this capability?  Anyways, I hope you’ll find this HornetQ + Spring integration useful in the next HornetQ release.

Want to rewrite JAXB?

4 Comments

I’ve been pretty frustrated with Sun’s JAXB and JBoss’s JBossXB implementations.  Here’s why:

* JAXB is not very extendable.  For example, I’d love to be able to define custom annotations that trigger callback APIs so that I can read/write custom properties from an XML document.  For example, ever try to do Map support?  I’ve also wanted to convert URLs to/from Link objects in resteasy with hooks into RESTEasy runtime for automatic URL writing.  I just couldn’t find a way to do it within resteasy.

* JAXB (both Sun’s and JBoss’s) is very slow to initialize.  Yeah, they parse very fast, but initialization speed is very slow because it has to do a lot of reflection on the classes that make up the JAXBContext and build a lot of data structures to do quick parsing.  You might be saying, “Big Deal!”.  But what I’ve found with JAXB and JBoss’s impl is that for the meta-models we’re trying to parse in JBoss AS, it can take as much as 1 second to initialize all the models.  That’s 1 second more added to boot time.  1 second to make embedded/unit testing slower.

So, this is why I started the fast-jaxb project under the RESTEasy SVN.  The way it works is that you provide it a set of classes and it generates Java code that uses SAX to parse XML.  The problem is, I haven’t had the time finish it, which is why I’m blogging to ask if anybody is interested in taking it over (or suggesting a good alternative for solving the goals/problems mentioned above).  Fast-JAXB has basically no initialization overhead compared to Sun’s JAXB implimentation.  I haven’t worked on the extension points though, nor do I support all the JAXB annotations.  I have also only worked on parsing XML, not generating XML.  Anybody interested in taking it over and driving it?  Give me a ping here on this blog…

Older Entries Newer Entries