There were some interesting side conversations going on in the comments of my Java EE wins over Spring blog.  In particular, a few people were arguing over the value of Mocks.  I always considered Mocks a bogus pattern.  Only time I ever use them is when I’m initially starting a project and I don’t have full implementations of a particular subsystem and I need to test that something works.  You see to me, your GIT master branch, or your SVN trunk is a sacred place.  I fully believe in the idea of continuous integration.  But to have a pure Trunk, you have to test your code, as close as possible, to the target environment it is supposed to run in.  Mocks go counter to this idea.  And give you false sense of security.

My colleague, Stan Silvert said it best:

When you DO use mocks you are testing in an environment that is biased toward a passing test. An artificial environment will always make invalid assumptions about the behavior and stability of that environment.

What you end up with is a (hard to maintain) mock library that allows your tests to pass and gives you a warm feeling that you’ve done a great job. But you are fooling yourself.

The ONLY reason to use mocks is when test execution in the real environment is unacceptably slow. This used to be the case quite often. But with frameworks like Arquillian and super-fast startup times for app servers this is no longer a frequent concern.

I’ll end with an even more radical assertion. The difference between integration testing and unit testing is purely semantic. Even the smallest, most focused unit test pulls in a lot of the environment (JVM, OS, etc). In-container testing just pulls in more of the target environment and allows you to find more errors sooner.

Test in the real environment and get results that are truly valid. Stop kidding yourself with mocks.

Personally, I’m a very paranoid person when it comes to development.  Earlier in my career I did a lot of demos and live coding in front of an audience.  When you’re in this type of environment you live by the mantra that anything that can go wrong will.  So, when rehearsing, you always mimic as much as possible what your setup will be.  I carried this paranoia to Iona (an Irish company), when I worked on Orbix there.  We had a rule, that if even one test (integration or whatever) failed, you were awarded the Guiness Penalty.  If you broke the build, you had to buy every one on the team a pint of Guiness.  IN this environment, Mocks are a waste of time, both in build CPU and in coding time as you still had to run integration testing before you committed anything (unless of course you just wanted an excuse to have a few beers).

So unless a particular integration test is ungodly slow, don’t use mocks.  Stan again, said it best:

…the days of the 2 hour in-container smoke test are over.
Since mocks no longer provide much of a speed advantage their use is outweighed by the fact that they give you a phony environment, maintenance headaches, and false confidence.

But you do touch on a truth about testing that a lot of people miss. The purpose of testing is to uncover errors. And you will uncover more errors in a real environment than you would in a mock environment.

With projects like Arquillian and fast boot up times like AS7 (1.75 seconds on my laptop), there’s no need to mock anymore.