Project Lombok: Creating Custom Transformations

Project Lombok aims to reduce Java boiler-plate via annotations that perform class transformations at compile time. Project Lombok comes with a decent set of transformations, but you may also want to create your own custom Lombok tranformations. In this blog, I will walk you through the process of extending Project Lombok to do a simple Hello World transformer. What I present here is an approach that worked for me. At the moment, there are scarce few resources out there on this subject. I started by reading Nicolas Frankel's blog and a post in the Project Lombok discussion group but mostly it came down to groking the Project Lombok source code. With that disclaimer, let's dive in. Overview Project Lombok runs as an annotation processor. The annotation processor acts as a dispatcher that delegates to Lombok annotation handlers (this is what we're going to create). Handlers are discovered via a framework called SPI. Lombok annotation handlers declare the specific annotation that they handle. When delegating to a handler, the Lombok annotation processor provides an object representing the Abstract Syntax Tree (AST) of the annotated node (e.g. class, method, field, etc). The Lombok annotation handler is free to modify the AST by injecting new nodes such as methods, fields and expressions. After the annotation processing stage, the compiler will generate byte code from the modified AST. Here's an overview of the compilation process and how Project Lombok fits in: The basic classes you'll need to write are: - Annotation class - Eclipse handler - Javac handler In this example, we will create a very simple Lombok transformation that adds a helloWorld method to any class annotated as @HelloWorld. This is a trivial and useless transformation but serves as a simple starting point. For example, given the following source code:

@HelloWorld
public class MyClass {

}

Our custom handler will transform the class to the following equivalent source:

public class MyClass {
  public void helloWorld() {
    System.out.println("Hello World");
  }
}

Annotation class This is the easy part. We just need to create an annotation called HelloWorld that can be applied to classes.

package lombok;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.SOURCE)
public @interface HelloWorld {}

This simple annotation can only be applied to Types (interface, class, enum). Since we plan on adding a concrete method, this annotation should only be used on classes but @Target doesn't give us that granularity. Our annotation handler will be responsible for generating an error if @HelloWorld is used on an interface. The annotation only needs to be retained in the source because we're only using the annotation to generate a method during compilation. The annotation is not needed at runtime. Handler Overview The handler class will be responsible for creating the AST that represents the helloWorld method and then injecting it into the AST of the Class declaration. A simplified AST for a Class (type) declaration looks something like this: Our handler will be adding a new Method Declaration to the Type Declaration. A Method Declaration is composed of several components. The AST for our Method Declration will have this form: Filling in the above template with our implementation specifics, the AST starts to resemble something that looks more like Java: Writing the Handler class Since AST modifications are compiler specific, we'll need to provide implementations for both Javac and Eclipse. If Intellij supported Lombok, we'd likely have to provide a 3rd implementation. Since NetBeans uses Javac, when we're done we'll be able to compile using commandline javac (ant and maven included), NetBeans and Eclipse. Both the Javac and Eclipse handlers must use "lombok" as the top-level package. In order for our handler to be discovered by the Lombok annotation processor, the Eclipse handler must be annotated as @ProviderFor(EclipseAnnotationHandler.class) and implement the EclipseAnnotationHandler interface. Likewise the Javac handler must be annotated as @ProviderFor(JavacAnnotationHandler.class) and implement the JavacAnnotationHandler interface. Here is the starting point for our Eclipse handler:

package lombok.eclipse.handlers;
import lombok.HelloWorld;
import lombok.core.AnnotationValues;
import lombok.eclipse.EclipseAnnotationHandler;
import lombok.eclipse.EclipseNode;
import org.mangosdk.spi.ProviderFor

@ProviderFor(EclipseAnnotationHandler.class)
public class HandleHelloWorld implements EclipseAnnotationHandler<HelloWorld> {

 @Override
 public boolean handle(AnnotationValues<HelloWorld> annotation, Annotation ast,
   EclipseNode annotationNode) {
  // our logic here
 }
}

Here is the starting point for our Javac handler:

package lombok.javac.handler;

import lombok.HelloWorld;
import lombok.core.AnnotationValues;
import lombok.javac.JavacAnnotationHandler;
import lombok.javac.JavacNode;
import org.mangosdk.spi.ProviderFor

@ProviderFor(JavacAnnotationHandler.class)
@SuppressWarnings("restriction") 
public class HandleHelloWorld implements JavacAnnotationHandler<HelloWorld>{

 public boolean handle(AnnotationValues<HelloWorld> annotation, JCAnnotation ast,
   JavacNode annotationNode) {
          // logic here
        }
}

Handle Logic Our handle method will need to do the following:

1. Mark annotation as processed (Javac only)
2. Create the helloWorld method
3. Inject the helloWorld method into the AST of the annotated class

Our handle method looks very similar for both Eclipse and Javac. Eclipse version:

@Override
 public boolean handle(AnnotationValues<HelloWorld> annotation, Annotation ast,
   EclipseNode annotationNode) {
  EclipseNode typeNode = annotationNode.up();

  MethodDeclaration helloWorldMethod = 
   createHelloWorld(typeNode, annotationNode, annotationNode.get(), ast);
  
  EclipseHandlerUtil.injectMethod(typeNode, helloWorldMethod);
  
  return true;
 }

Javac version:

@Override public boolean handle(AnnotationValues<HelloWorld> annotation, JCAnnotation ast,
   JavacNode annotationNode) {
  JavacHandlerUtil.markAnnotationAsProcessed(annotationNode, HelloWorld.class);
  JavacNode typeNode = annotationNode.up();
  
  JCMethodDecl helloWorldMethod = createHelloWorld(typeNode);
  
  JavacHandlerUtil.injectMethod(typeNode, helloWorldMethod);
  return true;
 }

Lombok provides our handler with the AST of the annotation node. We know that the annotation can only be applied to a Type. To get the AST for the Type (Class), we call annotationNode.up(). The annotation is a child of the Type AST, so by calling up() we get the parent AST which is the Type AST we need to modify. For simplicity, I've omitted the logic to check that the Type is actually a Class. Next we create the node representing createHelloWorld method node. We still need to write this method which we'll look at in the next section. Once we've created the method method, we inject it the AST of our Type. This is accomplished by Lombok utility classes JavacHandlerUtil and EclipseHandlerUtil. We'll also be using these utility classes to help implement the createHelloWorld method. Creating the helloWorld Method Now we get the crux of the problem: creating the helloWorld method. The basic recipe will be the same for both Javac and Eclipse:

• Start with a method node.
• Add the return type, parameters, access level, throw clause, etc to the method node.
• Create an expression statement to represent System.out.println("Hello World")
• Add the expression to the method node.

Implementing this logic is by far the hardest part. We'll need to figure out how to programatically create the various AST objects. To really grok this code, you'll need to look at the Java source for the various classes in the Javac and Eclipse syntax tree packages. You'll see that Eclipse and Javac implementations differ drastically. Here is the Javac implementation

 private JCMethodDecl createHelloWorld(JavacNode type) {
  TreeMaker treeMaker = type.getTreeMaker();
  
  JCModifiers           modifiers          = treeMaker.Modifiers(Modifier.PUBLIC);
  List<JCTypeParameter> methodGenericTypes = List.<JCTypeParameter>nil();
  JCExpression          methodType         = treeMaker.TypeIdent(TypeTags.VOID);
  Name                  methodName         = type.toName("helloWorld");
  List<JCVariableDecl>  methodParameters   = List.<JCVariableDecl>nil();
  List<JCExpression>    methodThrows       = List.<JCExpression>nil();

  JCExpression printlnMethod = 
   JavacHandlerUtil.chainDots(treeMaker, type, "System", "out", "println"); 
  List<JCExpression> printlnArgs = List.<JCExpression>of(treeMaker.Literal("hello world"));
  JCMethodInvocation printlnInvocation = 
   treeMaker.Apply(List.<JCExpression>nil(), printlnMethod, printlnArgs);
  JCBlock methodBody = 
   treeMaker.Block(0, List.<JCStatement>of(treeMaker.Exec(printlnInvocation)));

  JCExpression defaultValue = null;

  return treeMaker.MethodDef(
    modifiers, 
    methodName, 
    methodType,
    methodGenericTypes, 
    methodParameters, 
    methodThrows, 
    methodBody, 
    defaultValue 
   );
 }

With Javac, we need to generate an object for all parts of the method: modifiers, generic types, return type, method name, parameters, throw clause, and method body. Creating these object is via a TreeMaker class that is part of Javac. TreeMaker is a factory class for creating all the different types of nodes. The method body is comprised of various nodes as well: method reference to System.out.println, arguments to println which includes the String literal "hello world". It's all tied together as a method invocation of the println method reference. Finally we use treeMaker to combine all the pieces into a method defintion. Now let's look at the Eclipse implementation:

private MethodDeclaration createHelloWorld(EclipseNode typeNode, EclipseNode errorNode, ASTNode astNode, Annotation source) {
  TypeDeclaration typeDecl = (TypeDeclaration) typeNode.get();

  MethodDeclaration method = new MethodDeclaration(typeDecl.compilationResult);
  Eclipse.setGeneratedBy(method, astNode);
  method.annotations = null;
  method.modifiers = Modifier.PUBLIC;
  method.typeParameters = null;
  method.returnType = new SingleTypeReference(TypeBinding.VOID.simpleName, 0);
  method.selector = "helloWorld".toCharArray();
  method.arguments = null;
  method.binding = null;
  method.thrownExceptions = null;
  method.bits |= ECLIPSE_DO_NOT_TOUCH_FLAG;
  
  NameReference systemOutReference = createNameReference("System.out", source);
  Expression [] printlnArguments = new Expression[] { 
   new StringLiteral("Hello World".toCharArray(), astNode.sourceStart, astNode.sourceEnd, 0)
  };
  
  MessageSend printlnInvocation = new MessageSend();
  printlnInvocation.arguments = printlnArguments;
  printlnInvocation.receiver = systemOutReference;
  printlnInvocation.selector = "println".toCharArray();
  Eclipse.setGeneratedBy(printlnInvocation, source);
  
  method.bodyStart = method.declarationSourceStart = method.sourceStart = astNode.sourceStart;
  method.bodyEnd = method.declarationSourceEnd = method.sourceEnd = astNode.sourceEnd;
  method.statements = new Statement[] { printlnInvocation };
  return method;
 }

With the Eclipse implementation, instead of using a TreeMaker to create the various components, we just create the components via their constructors and attach child nodes to parents via property assignments. Eclipse developers apparently aren't big on encapsulation. They also seem to be fixated on using char arrays instead of Strings. Another thing we need to deal with is telling Eclipse that the HelloWorld annotation was responsible for generating the method. That way if there is an error with our generated method, Eclipse will associate the error with @HelloWorld annotation. The complete source along with a Maven project setup can be found here Going beyond HelloWorld Hopefully by now the architecture of Lombok is a little clearer. Once you've mastered Hello World you're ready to start creating more useful transformations. As you would expect, more interesting behavior requires more complex AST transformations. You'll probably spend a lot of time looking at the source code for the handlers that the Project Lombok maintainers have already created. This is the downside of using private APIs to generate code. You're on your own to figure out how these APIs work. Appendix - Project Setup Before you can build any Lombok transformations, you'll need a way to build your code. Where to put your code The first thing you need to decide is where your code will live. There are 2 options: 1. Fork Project Lombok source 2. Create a new source project Let's look at both options in more detail: Forking Lombok By far the easiest way to get started is just to clone the Project Lombok git repo and place your custom classes alongside the core Lombok classes. This is going to be the fastest and easiest way to get started. You won't have to deal with configuring library dependencies or patching Eclipse. Down the road you can easily move to a standalone project for your custom classes. The downside is that you are forking the project is that you'll have to adopt Lombok's project structure and build system (Ant+Ivy). Using ant, you can generate an Eclipse project which makes things easy. Cloning Project Lombok is simple. You'll need to have git installed. Then run the command:

git clone https://github.com/rzwitserloot/lombok.git

Navigating source project The top-level lombok directory contains a build.xml. To compile and build the lombok jar run: ant To generate an eclipse project run: ant eclipse The lombok jar is generated under the dist subdirectory. Once you've added your custom handlers, you'll need to patch Eclipse with your updated jar by running: java -jar <new lombok jar>: Source code is organized into the following directories: src/core/lombok - Annotations src/core/lombok/eclipse/handlers - Eclipse annotation handlers src/core/lombok/javac/handlers - Javac annotation handlers Creating a stand-alone project Skip this section if you chose to fork Project Lombok. Instead of forking the Project Lombok repo, another more complicated approach is to place your custom Lombok extensions in a separate standalone project. Setting up custom project comes with some pain though. You'll need to handle dependency management and Eclipse patching yourself. At a minimum your project will need to provide the following libraries: - Project Lombok - SPI - Eclipse Core JDT - Sun's tools.jar (this contains javac classes) The example I provide here uses Maven2. This presents a challenge because SPI library does not have a public maven repo (that I could find), nor does Eclipse Core JDT or tools.jar. So these have to be installed manually into your local repo. The easiest way to obtain theses jars is by checking out Project Lombok and doing a build (see above). The build will download dependent libraries under lib/build. The lombok dist jar can be found under the dist/. With Maven, you'll need to install these jars these jars in your local repo. Download the Maven project for the HelloWorld project here Once you've built the jar, you'll need to patch it manually into Eclipse. Copy the jar to the same folder as your eclipse.ini. Edit eclipse.ini, and add your jar to the bootclasspath. Here is an example for adding the hello-lombok.jar from the example project to eclipse.ini.

...
-javaagent:lombok.jar
-Xbootclasspath/a:lombok.jar
-Xbootclasspath/a:hello-lombok.jar
...

Project Lombok - Trick Explained

In my previous blog post, I introduced Project Lombok, a library that can inject code into a class at compile time. When you see it in action, it almost seems magical. I will attempt to explain the trick behind the magic. Java Compilation To understand how Project Lombok works, one must first understand how Java compilation works. OpenJDK provides an excellent overview of the compilation process. To paraphrase, compilation has 3 stages: 1. Parse and Enter 2. Annotation Processing 3. Analyse and Generate In the Parse and Enter phase, the compiler parses source files into an Abstract Syntax Tree (AST). Think of the AST as the DOM-equivalent for Java code. Parsing will only throw errors if the syntax is invalid. Compilation errors such as invalid class or method usage are checked in phase 3. In the Annotation Processing phase, custom annotation processors are invoked. This is considered a pre-compilation phase. Annotation processors can do things like validate classes or generate new resources, including source files. Annotation processors can generate errors that will cause the compilation process to fail. If new source files are generated as a result of annotation processing, then compilation loops back to the Parse and Enter phase and the process is repeated until no new source files are generated. In the last phase, Analyse and Generate, the compiler generates class files (byte code) from the Abstract Syntax Trees generated in phase 1. As part of this process, the AST is analyzed for broken references (e.g. class not found, method not found), valid flow is checked (e.g. no unreachable statements), type erasure is performed, syntactic sugar is desugared (e.g. enhanced for loops become iterator loops) and finally, if everything is successful, class files are written out. Project Lombok and Compilation Project Lombok hooks itself into the compilation process as an annotation processor. But Lombok is not your normal annotation processor. Normally, annotation processors only generate new source files whereas Lombok modifies existing classes. The trick is that Lombok modifies the AST. It turns out that changes made to the AST in the Annotation Processing phase will be visible to the Analyse and Generate phase. Thus, changing the AST will change the generated class file. For example, if a method node is added to the AST, then the class file will contain that new method. By modifying the AST, Lombok can do things like generate new methods (getter, setter, equals, etc) or inject code into an existing method (e.g. cleaning up resources). Trick or Hack? Some people call Lombok's trick a hack, and I'd agree. But don't pass judgement yet. Like any hack, you should examine the risk/reward and alternatives before determining if you are comfortable with it. The "hack" in Lombok is that, strictly speaking, the annotation processing spec doesn't allow you to modify existing classes. The annotation processing API doesn't provide a mechanism for changing the AST of a class. The clever people at Project Lombok got around this through some unpublished APIs of javac. Since Eclipse uses an internal compiler, Lombok also needs access to internal APIs of the Eclipse compiler. If Java officially supported compile-time AST transformations then Lombok wouldn't need to rely on backdoor APIs. This makes Project Lombok vulnerable to future changes in the JDK. There is no guarantee the private APIs won't change in a later JDK and break Project Lombok. If that happens, then you're left hoping that the guys at Lombok will be responsive about patching their library to work with the new JDK. Same thing goes for the new Eclipse compilers. Given how often we get a new version of Java, this may not be that big of an issue. Alternatives in Java There are other alternatives for modifying the behavior of classes. One approach is to use byte-code manipulation at runtime via a library like CGLib or ASM. This is how Hibernate is able to do things like lazily initialize a persistent Collection the first time it is accessed. In general, this can be used to enhance the behavior of existing methods. This trick could possibly be used to implement the @Cleanup behavior in Lombok, so that a resource is closed when it goes out of scope. Runtime byte-code manipulation is no help for generating getters and setters which you intend to reference in source code. Another approach is to use byte-code manipulation on the class files. For example, Kohsuke Kawaguchi of Hudson fame created a library called Bridge Method Injector, that helps perserve binary compatibility when changing a method's return type in a way that is source compatible but not binary compatible. Kohsuke implements this by using ASM to modify the byte-code in a class file after compilation. This trick could be used to mimic the behavior of the Getter/Setter/ToString/EqualsHashCode annotations of Lombok with one caveat: generated methods would only be visible to classes external to your library but not to classes within your library. In other words, projects that depended on classes in your library as a jar would see your getters and setters, but classes within your library would not see these getters and setters at compile time. The trick that makes Lombok special is that the code it generates is weaved in before Analyze and Generate phase of compilation. This allows classes within the same compilation unit to have visibility to the generated methods. It appears another library called Juast may be using a similar trick (modifying the AST) to do things like operator overloading. For some developers, the immediate benefits of Lombok's approach may outweigh the potential risks. Alternatives outside Java If you're willing to switch to Scala, Lombok becomes a moot point. Scala has Case classes that eliminate the getter/setter/toString/hashCode/equals boiler-plate. Scala also has Automatic Resource Management that covers Lombok's @Cleanup behavior. Another option is Groovy if you don't care about static typing. Groovy has similar support for Scala-like Case classes. Groovy also officially supports compile-time, AST transformations. Final thoughts Project Lombok can do tricks that are impossible via other dynamic code generation methods in Java but you should be aware the it uses some back-door APIs to accomplish it.

Project Lombok: Annotation-driven development - Part 1

As the Java language has reached an evolutionary plateau, annotations are emerging as a way to extend the language. Two interesting frameworks that I've been looking at are Project Lombok and the Checker Framework. In this first entry of a series I'm calling "Annotation-driven development", I'll be looking at Project Lombok.

Project Lombok

Project Lombok aims to eliminate boilerplate code. For example most POJO classes are littered with trivial getters/setters like the following:

public class Person {

  private final String firstName; // read-only
  
  private String lastName;

  public Person(String firstName, String lastName) {
    this.firstName = firstName;
    this.lastName = lastName;
  }

  public String getFirstName() { 
    return firstName;
  }

  public String getLastName() { 
    return lastName;
  }
  public void setLastName(String value) {
    this.lastName = value;
  }  

}

With Lombok, you can use annotations to create the equivalent set of methods:

@AllArgsConstructor
public class Person {
  @Getter @Setter private final String firstName;
  @Getter private String lastName;
}

Or even simpler:

@Data
public class Person {
  private final String firstName;
  private String lastName;
}

Actually, the final example generates more than just getters/setters and constructor. You also get a toString and hashCode method, which are often "forgotten" because they are a pain to write correctly (unless you are using Pojomatic). Lombok isn't just about POJO boilerplate. From the Project Lombok features page, here are all the annotations available currently:

• @Getter / @Setter Never write public int getFoo() {return foo;} again.<
• @ToString No need to start a debugger to see your fields: Just let lombok generate a toString for you!
• @EqualsAndHashCode Equality made easy: Generates hashCode and equals implementations from the fields of your object.
• @NoArgsConstructor, @RequiredArgsConstructor and @AllArgsConstructor Constructors made to order: Generates constructors that take no arguments, one argument per final / non-null field, or one argument for every field.
• @Data All together now: A shortcut for @ToString, @EqualsAndHashCode, @Getter on all fields, and @Setter on all non-final fields, and @RequiredArgsConstructor!
• @Cleanup Automatic resource management: Call your close() methods safely with no hassle.
• @Synchronized synchronized done right: Don't expose your locks.
• @SneakyThrows To boldly throw checked exceptions where no one has thrown them before!

Setting up Lombok

To use lombok, you'll need the lombok.jar. This can be obtained from the Project Lombok website. Maven users can simply add the lombok dependency and repository. For example:

<dependencies>
  <dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>0.9.3</version>
  </dependency>
</dependencies>
<repositories>
    <repository>
      <id>projectlombok.org</id>
      <url>http://projectlombok.org/mavenrepo</url>
    </repository>
  </repositories>
</repositories>

This will allow you to use lombok using javac. For the 99.9% of developers who use an IDE, you'll want Lombok IDE support unless you enjoy seeing code that won't compile. You're in luck if you're using Eclipse or NetBeans, as those are the only IDEs supported (sorry IntelliJ users). For Eclipse 3+ and NetBeans 6.8+, simply run java -jar lombok.jar and an install wizard will guide you through adding Lombok support to your chosen IDE(s). The wizard will modify your IDE start script to include Lombok.jar as a Java agent. NetBeans 6.9 users also have the option of using Lombok as a inline annotation processor.

Using Lombok

Once you're got your environment set up to use Lombok, you simply add a Lombok annotation your class, and your class is magically enhanced. Lombok generates code during the compilation phase. Within the IDE, you'll now have access to methods that aren't in your source file. The methods exist in the generated class so they show up in your class outline and are available for code completion. If you ask your IDE to open/jump to the method, it will open the source file but obviously you won't see the code for the method. If you want to view the generated code, you can use JAD to decompile the class or you can use the delombok tool to generate a Lomboked source file from your original source file. You can run delombok manually from the command-line or automatically via the Maven Lombok plugin. Despite being a Maven user, I found it easier to use delombok from the command-line because the maven plugin requires moving all Lomboked files to a non-standard directory. Delombok serves a few useful purposes. You may be curious to see what code Lombok is generating. Or you later decide you want to remove your dependency on Lombok, then you can delombok all your source, and replace the original source with the delomboked source. Running Delombok on the previously mentioned @Data example, generated the following source code:

public class Person {
        private final String firstName;
        private String lastName;

        @java.beans.ConstructorProperties({"firstName"})
        @java.lang.SuppressWarnings("all")
        public Person(final String firstName) {
                this.firstName = firstName;
        }

        @java.lang.SuppressWarnings("all")
        public String getFirstName() {
                return this.firstName;
        }

        @java.lang.SuppressWarnings("all")
        public String getLastName() {
                return this.lastName;
        }

        @java.lang.SuppressWarnings("all")
        public void setLastName(final String lastName) {
                this.lastName = lastName;
        }

        @java.lang.Override
        @java.lang.SuppressWarnings("all")
        public boolean equals(final java.lang.Object o) {
                if (o == this) return true;
                if (o == null) return false;
                if (o.getClass() != this.getClass()) return false;
                final Person other = (Person)o;
                if (this.getFirstName() == null ? other.getFirstName() != null : !this.getFirstName().equals(other.getFirstName())) return false;
                if (this.getLastName() == null ? other.getLastName() != null : !this.getLastName().equals(other.getLastName())) return false;
                return true;
        }

        @java.lang.Override
        @java.lang.SuppressWarnings("all")
        public int hashCode() {
                final int PRIME = 31;
                int result = 1;
                result = result * PRIME + (this.getFirstName() == null ? 0 : this.getFirstName().hashCode());
                result = result * PRIME + (this.getLastName() == null ? 0 : this.getLastName().hashCode());
                return result;
        }

        @java.lang.Override
        @java.lang.SuppressWarnings("all")
        public java.lang.String toString() {
                return "Person(firstName=" + this.getFirstName() + ", lastName=" + this.getLastName() + ")";
        }
}

Like any source code generator, delombok produces code that looks, well, generated. It seems that Lombok lazily adds @java.lang.SuppressWarnings("all") to all methods. The toString/hasCode/equals methods are definitely uglier that what you'd get if you were using Pojomatic.

Extending Lombok

After playing with Lombok, you will likely think of other boilerplate code you'd like to eliminate using the Lombok. The Builder pattern came to my mind and apparently others others as well. There are very few resources that explain how to extend Lombok. You can obviously download the source. Besides that, the best resource I could find was a blog by Nicolas Frankel that describes the basic steps as well as example source. Nicolas states that writing custom Lombok plugins is not for the faint-hearted and I'd agree. You'll quickly discover that you need to know something about annotation processors as well as some rather low-level Javac APIs. Honestly, when you look under the covers of Lombok, things get a bit scary. Some have called Lombok a hack because it relies on internal javac APIs.

Final Thoughts

Lombok is a very interesting use of annotations to extend the Java language. For those who really hate writing getters/setters/etc, Lombok is worth checking out. Although I don't enjoy writing getters/setters/etc, I don't spend a lot of my time writing those types of methods and the IDE can generate much of the boilerplate. It should be noted that Scala has eliminated many of the pain points that Lombok aims to remedy. Right now I'm just using Lombok for prototyping. It definitely speeds up the process of creating POJOs and let's me focus on the more interesting aspects of the prototype. I am not using Lombok for production code. Although I use Eclipse, other developers at Overstock use IntelliJ and the lack of support for IntelliJ is a show-stopper. Even if all IDEs were supported, I'm still not comfortable unleashing Lombok until I've spent some more time with it. I don't think I've found all its warts yet. I'm also concerned about later releases of JDK (or Eclipse) breaking Lombok compatibility. It wouldn't be the end of the world because I can always delombok my source, but that could be a painful process for large projects. That said, it's definitely worth looking Lombok. Even if it never makes it into my Java toolkit, it's a fascinating library to examine and my knowledge of Java has increased because of it.

Passing properties to the Maven release plugin

The Maven release plugin does not propagate system properties to the maven release build. This can cause confusion when you try to set properties for a maven release. For example, if your pom.xml uses a custom property "some-property" that you intend to set for a release, you might be tempted to try this:

mvn release:prepare -Dsome-property=foo

Although this works for regular build, it will not work for a release build. This is because the release plugin forks a new instance for the release build, and properties passed to the parent are not propagated to the forked build.

To solve this problem, you need to propagate the properties yourself by configuring the argument property of the release plugin. For example:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.overstock.example</groupId>
  <artifactId>helloworld</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>helloworld</name>
  <url>http://maven.apache.org</url>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-release-plugin</artifactId>
        <configuration>
          <arguments>-Dsome-property=${some-property}</arguments>  
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

If you need to propagate more than 1 property, the arguments property allows for multiple values separated by spaces.

Custom Hudson Plugins

Where to Begin

So you want to write a Hudson plugin but don't know where to start. Luckily, it's fairly easy to get started by following the tutorial on the official Hudson wiki: http://wiki.hudson-ci.org/display/HUDSON/Plugin+tutorial. If all you need is a simple Hudson builder, the Hudson tutorial should suffice. The purpose of this blog post is to supplement that tutorial with additional hints and explanations to make your journey a bit easier.

After following the Hudson plugin tutorial you should end up with a fully-functioning HelloWorld Hudson plugin that can be run from the command-line by running mvn clean hpi:run. Running clean is optional, but I highly recommend cleaning because I have had many moments where my plugin wasn't working as expected, only to discover after much troubleshooting that I didn't clean. If everything works, you should be able to connect to your custom Hudson instance via localhost:8080.

Testing the HelloWorld Builder plugin

Once you've got Hudson running, you're ready to configure the HelloWorld Builder plugin. Hudson provides 2-levels of configuration, global and job. To access, global configuration navigate to Hudson->Manage Hudson->Configure System. Here you will find a section labeled "Hello World Builder" with a configuration checkbox for French. This controls the default language of the plugin.

Now it's time to configure this plugin. Create a new Hudson job and on the project configuration screen under Build, notice that the Add Build Step menu has an option "Say hello". Adding this build step will invoke the custom HelloWorld Builder which prints a "Hello" message to the console output of a job build.

Under the Covers

Plugin Discovery

Hudson automagically discovered the HelloWorldBuilder class because it is annotated with @Extension and it extends Builder, which is a defined Hudson extenson point. Other defined extensions points can be found here: http://wiki.hudson-ci.org/display/HUDSON/Extension+points

Jelly

After discovering the HelloWorldBuilder, Hudson found the UI resources for this plugin using the package and classname. Hudson uses Jelly for generating HTML.

The plugin ships with 2 jelly files, global.jelly and config.jelly. As the name implies, global.jelly provides the UI elements that are displayed on the global Hudson configuration screen. config.jelly provides the UI elements that are displayed in the job configuration screen.

Explaining jelly is beyond the scope of this document. I found jelly easy enough to learn by example using other Hudson plugins for reference.

Stapler

Hudson uses the Stapler to associate jelly files with plugins. In MVC terms, your plugin class is the "M", Jelly is the "V", Stapler is the "C". To associate Jelly files with a plugin class using Stapler, you create a resource directory under src/main/resources/{package}/{Class}/ and drop the jelly files in there. For example, if the fully-qualified name of your plugin is demo.hudson.HelloWorldBuilder then your jelly files must be located under src/main/resources/demo/hudson/HelloWorldBuilder. If you ever rename or repackage your plugin class, you must also reorganize your resource subdirectories appropriately.

Stapler is used for more than just stapling jelly files to plugins. It's also used to dispatch requests to the plugin classes. Once again, Stapler does this all by convention. If you have a form that submits to /foo/bar/SomePlugin then Stapler will try to invoke the doSubmit method on foo.bar.SomePlugin. Likewise, if a form field needs to be validated, then Stapler will call the method SomePlugin.checkSomeField. I found this to be the most confusing part of plugin development. It's fairly straightforward until it doesn't work, then it's a lot of spellchecking and consulting the Stapler docs to try to figure out what you're doing wrong.

Configuration Persistance

After configuring the plugin, you may be wondering how it gets stored or why it's not being stored. When you fire up the plugin via hpi:run, you'll see a work directory created under the project directly. This directory contains all plugin and job configuration as well as job run history logs. Global plugin configuration is stored under work/{plugin-name}.xml. Per job plugin configuration is stored under work/jobs/{jobname}/{plugin-name}.xml.

More information on configuration persistance can be found here: http://wiki.hudson-ci.org/display/HUDSON/Architecture

Going Beyond HelloWorldBuilder

Once you've mastered the HelloWorldBuilder plugin, you're ready to invent your plugin. The first place to start is by find the right extension point(s) to hook into. The complete list of extension points can be found here: http://wiki.hudson-ci.org/display/HUDSON/Extension+points

Most plugins seem to extend Builder, BuildWrapper and Action. Hopefully by now I've provided you enough information to dissect how an existing plugin works. I found that looking at existing plugins gave me nearly enough insight to accomplish what I wanted. There are a wealth of plugins on Hudson with full-source code. Start here to find a plugin that looks similar to what you are trying to accomplish. Then download the project source from https://hudson.dev.java.net/svn/hudson/trunk/hudson/plugins/ (login credentials are user=guest with empty password).

Overall, I found writing a Hudson plugin fairly easy once I understood the architecture. There isn't a lot of reference material out there beyond the Hudson tutorial. The Javadoc is generally better than most open source projects I've encountered. I definitely had some frustrating moments, especially when working with Stapler. There's a fairly good community of Hudson developers with ample source code to refer to when you get stuck.

Trends in Java

This week I'll be attending No Fluff Just Stuff: Salt Lake City edition. I last attended NFJS 3 years ago. Looking at the list of sessions and topics this year compared to 3 years ago has me thinking about trends in Java.

I'm glad to see there are a number of presentations on core Java. 3 years ago I was surprised that a Java conference featured so few topics about Java. Most surprising was a keynote by Neal Ford where he essentially claimed Java was dead (or at least dying). I don't remember all his arguments, but there were a few interesting that I'll paraphrase. He said that Java was nearly 15 years old which is typically the lifetime of a language; Java has too much ceremony and is too verbose; Java has become too complicated; the legacy of Java is the JVM and its future will be alternative languages on the JVM. I'd agree with the latter point the most.

Java is not dead no matter how hard Neal Ford wishes it was. If Java is dead, then there is a lot of software necrophilia going on. It's still the number 1 language being used for application development. At Overstock, we are heavy into Java and we are still getting a lot of mileage out of it. Senior Java developers are under extremely high demand in the Salt Lake City area.

Alternative languages on the JVM are still gaining momentum though. NFJS this year features a number of sessions on Groovy and Scala. Newcomer Clojure has a small mention. Gone from 3 years ago is JRuby. No mention of Jython either. Can we call JRuby and Jython dead? I hardly hear them talked about anymore.

Despite the strength of Groovy and Scala, I don't think they will kill Java. My prediction is that Java will commit suicide though we're likely years away from that. My guess is that at some point Oracle will realize that evolving Java is too difficult and costly and declare it end of life. The release of Java 7 has convinced me of this. Consider the timeline of previous Java releases:
1.0 (January 23, 1996)
1.1 (February 19, 1997)
1.2 (December 8, 1998)
1.3 (May 8, 2000)
1.4 (February 6, 2002)
5.0 (September 30, 2004)
6.0 (December 11, 2006)
7.0 2011???

Trends like this are hard to correct. It's a vicious cycle when big projects are continually delayed. Morale goes down, short cuts are taken and developers abandon ship. It takes drastic changes to reverse the cycle and so far it doesn't appear that has happened, though Oracle may be playing this one close to the chest. It's good to see Oracle has at least put some developers on the closures feature that was announced last year.

Despite the dreary state of Java 7, the Java community is still going strong. There are still libraries emerging that demonstrate some powerful features in the JDK that have yet to be tapped to their potential. You don't hear much about Java Instrumentation but take a look at what JMockit has done with it and you'll be pretty amazed. Likewise, you don't hear much about Annotation Processing Tool but Project Lombok has done some interesting (and slightly scary) things.

For now, I'll be attending the core Java sessions at NFJS. It's what I use today and what I expect to be using for the forseeable future. I may attend a Scala session because I've been fooling on and off with Scala, mostly because its interesting. Overall, I'm pretty excited about the sessions.

Interesting change to method signature erasure rules in Java 7

I found an interesting "bug" that has been fixed in Java 7 compiler. I say "bug" because some may have considered the old behavior to be a nice feature. Consider the following class:

public class ListPrinter {

  public static String getExample(List<String> list) { 
    return list.get(0); // return first
  }

  public static Integer getExample(List<Integer> list) { 
    return list.get(list.size() - 1);  // return last
  }

  public static void main(String[] args) {
    System.out.println(getExample(Arrays.asList("1", "2", "3")));
    System.out.println(getExample(Arrays.asList(1, 2, 3)));
  }
}

In Java 5 and 6, this compiles fine. We have 2 overloaded methods to get an example element from a list. For Lists of String, the first element is returned. For Lists of Integer, the last element is returned. Running the code prints the expected output: 1 3 Obviously, the compiler did the right thing, so what's the problem? Let's look at what happens after type erasure. Running javap on the above class compiled with JDK 6, yields:

public class ListPrinter extends java.lang.Object{
    public ListPrinter();
    public static java.lang.String getExample(java.util.List);
    public static java.lang.Integer getExample(java.util.List);
    public static void main(java.lang.String[]);
}

That's interesting, we've got 2 methods with the same signature getExample(java.util.List) but different return types. The discussion section for 8.4.8.3 of the Java language spec states that "...methods declared in the same class with the same name must have different erasures." The first part is completely clear, but the last part requires some thought. Does different erasure mean: A) different arguments after erasure B) different arguments and return type after erasure In Java 7, the answer is A. But in Java 5 and 6, the answer was B. By strict interpretation, Java 7 got it right. Method overloading requires changing the number and/or types of method arguments. Method signatures are not allowed to differ only by return type. As proof, let's perform the type erasure on the source code:

import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

public class ListPrinter {

  public static String getExample(List list) { 
    return (String) list.get(0); // return first
  }

  public static Integer getExample(List list) { 
    return (Integer) list.get(list.size() - 1);  // return last
  }

  public static void main(String[] args) {
    System.out.println(getExample(Arrays.asList("1", "2", "3")));
    System.out.println(getExample(Arrays.asList(1, 2, 3)));
  }

}

Compiling this version of the code in JDK 6 generates the following error:

ListPrinter.java:11: getExample(java.util.List) is already defined in ListPrinter

The compiler is telling us that we cannot have more one than method with the signature getExample(java.util.List). Notice that this is the exact signature that JDK 6 compiler generated twice in the original example. The compiler let us cheat. In Java 7, the original example fails to compile with the error:

ListPrinter.java:11: name clash: getExample(List) and getExample(List) have the same erasure

Thank goodness Java finally fixed that bug. But wait...it was kind of cool that JDK 6 let us do that. Is this a bug or a feature? The compiler figured it out and did the right thing so again I ask, "what's the problem"? The problem is erasure. And it's a problem that Java will likely always be stuck with. I guess it's time to start using Scala. But wait, Scala has type erasure too. Now, here's the million dollar question. What does Scala do in this situation? Here's the equivalient code written in Scala:

object ListPrinter extends Application {

  def getExample(list: List[String]):String = list.head

  def getExample(list: List[Int]):Int = list.last

  override def main(args: Array[String])
  {
    println(getExample(List("1", "2", "3")))
    println(getExample(List(1, 2, 3)))
  }

}

Scala inherits the same rules for overloading with type erasure from Java. But which interpretation of this rule does it use? As an incentive to get people to try Scala, I'm going to let the reader answer this question for themselves by compiling the code. The result may surprise you.