How to hide your Java with a DSL using JRuby

Last time I showed some Java code for building a simple tree. Then I showed a DSL for representing that tree. The point was basically to make you want to create such a DSL. This time I am going to show you how to make the DSL actually do something. We are going to write all the code that will allow that DSL to build a tree. If you've never seen it, or don't remember, please look back at the code from the last post. It's important to see the java code we started with and the DSL that we are going to implement. It's kind of like those before and after pictures you're always looking at. Seriously, come back when you are done, and this will make more sense.

It's not odd to start with the DSL. In fact I usually start with a language that would be ideal and then modify it just enough so that it is valid Ruby syntax. It is important to understand the language features and know how much you can get away with as far as syntax. As you begin implementing it, your language will probably change. In the end you hopefully end up with a concise and powerful abstraction.

Ruby has a lot of features that make it ideal for internal DSL's. Literal Arrays and Hashes are simple and very useful here, even though we won't use them today. We are going to focus on blocks and method_missing. Note that all of these concepts are missing from Java.

We are going to decorate our Java TreeNode class with JRuby. That's right, we are going to slap behavior implemented in Ruby directly on our Java class. And we are going to make it do things that can't even be done in Java. But first I want to introduce the IRB.

The IRB is a command line that you can execute Ruby in. With JRuby you get a JIRB, which is the same thing except you can execute Java there too. Sweet, interactive Java! As a Java developer, it is worth learning about JRuby just for this new ability. It's a more efficient way to explore how Java classes work than implementing main methods all over the place. I have also prototyped new behavior on my Java objects very effectively through the JIRB. The following examples are what I typed into the JIRB to test my ideas for the implementation of the DSL. If something doesn't work out, you just keep typing. The JIRB lets you try something and see the results very quickly. I have never felt so close to my Java code.

	require 'java'
	require 'Trees.jar'
	TreeNode = com.claymccoy.trees.TreeNode
	class TreeNode
	def to_s
	puts to_string.gsub("\\r", "\\n")
	end
	end

view raw OpenTreeNode.rb hosted with ❤ by GitHub

This first piece of new behavior we added to TreeNode is a good example. The new line character is different in Java and Ruby so we switch them out in the toString method. We had to require the jar for this class. Then we assign the qualified class name to a constant for convenience. After that adding a new method to our Java class is done as if it were Ruby. We could call toString, but JRuby allows us to call the Java methods with Ruby style so we call to_string instead. Note that is is completely unnecessary for the DSL. I only did it because I already had a toString that wrote out in the DSL format, and I wanted it to look right in the JIRB. The next example is where the meat is.

	class TreeNode
	def method_missing(methodname, *args, &block)
	child = self.children().find { |child| methodname.to_s == child.name }
	if child.nil?
	child = TreeNode.new(methodname.to_s)
	self.add_child(child)
	end
	child.instance_eval(&block) if block_given?
	child
	end
	end

view raw TreeNode_MethodMissing.rb hosted with ❤ by GitHub

This code does most of the work for us. Method_missing is a special method in Ruby. A call to method_missing is what happens if you call a method that doesn't explicitly exist. Java wouldn't even compile if you did something like this. In this case we find or create a TreeNode with the name of the method. This lets us represent a path in our tree like this: shape.ellipse.circle

As opposed to this in Java.

	TreeNode ellipse = null;
	for (TreeNode treeNode : shape.getChildren()) {
	if ("ellipse".equals(treeNode.getName())) {
	ellipse = treeNode;
	break;
	}
	}
	if (null == ellipse) {
	ellipse = new TreeNode("ellipse");
	shape.addChild(ellipse);
	}
	TreeNode circle = null;
	for (TreeNode treeNode : ellipse.getChildren()) {
	if ("circle".equals(treeNode.getName())) {
	circle = treeNode;
	break;
	}
	}
	if (null == circle) {
	circle = new TreeNode("circle");
	ellipse.addChild(circle);
	}

view raw BuildShapePath.java hosted with ❤ by GitHub

So this is 1 line of our DSL rather than 22 lines of Java. Even if your developers don't know Ruby it should be obvious that they are going to have an easier time writing this immediately.

This path stuff is cool, but it's not what I promised. Since our data structure is a tree, we want to express multiple children for each TreeNode. To do this we are going to use blocks. Blocks are a way to pass executable code into a method. A good example of using a block is the call to the find method in the first line of our method_missing way up there. For each child in the Collection returned by getChildren() on this TreeNode we are checking to see if it has the same name as the method name that invoked method_missing.

Blocks will also allow us to describe a tree structure. The way this works is that method_missing gets called recursively. Each time it will find or create the named TreeNode, and then it allows that TreeNode to evaluate the remaining structure. All methods in Ruby can be passed a block. Blocks are placed after a method call and fit within '{...}' or 'do...end.' I chose the brackets because I liked the way it made the language look, but either would actually work. Nesting the blocks allows for a natural tree structure. The block can be accessed in the method with an extra parameter. In this case &block. Don't get too worried about the '&' just yet, it is the concepts that are important. The second to the last line deals with constructing the children. If a block which represents substructure of the tree is given, it is evaluated on our child. The way that we recursively traverse the described children is the most complicated and important part of the whole thing. Now we can build a tree described like this.

	shape {
	ellipse.circle
	polygon {
	petagon
	triangle {
	scalene
	equilateral
	isosceles
	}
	hexagon
	quadrilateral.rectangle.square
	googolgon
	}
	}

view raw Shapes2.rb hosted with ❤ by GitHub

For readability it is nice to leave off the parens on the methods called. Everything that is a word in this DSL is actually a method call that invokes method missing, which creates a TreeNode that is added as a child. There is a bit of a chicken and egg problem with this example though. Once we have a TreeNode we are rolling, but how do we get our hands on the root TreeNode, in this case "shape?" Here is a way to start the whole thing off that is consistent with our DSL.

	Tree.create {
	shape {
	ellipse.circle
	polygon {
	petagon
	triangle {
	scalene
	equilateral
	isosceles
	}
	hexagon
	quadrilateral.rectangle.square
	googolgon
	}
	}
	}

view raw Shapes3.rb hosted with ❤ by GitHub

To make this work we just have to implement Tree to create our first TreeNode, and then hand the rest of the hierarchy off to it.

	class Tree
	def Tree.create(&block)
	Tree.new.instance_eval(&block)
	end
	def method_missing(methodname, *args, &block)
	@root = TreeNode.new(methodname.to_s)
	@root.instance_eval(&block) if block_given?
	@root
	end
	end

view raw Tree.rb hosted with ❤ by GitHub

One of my first Ruby projects was a DSL, and it was not a bad way to learn the language. I have taken these concepts further in production code by describing a complicated meta model with frames and slots, effective dating, and internal rules. The ideas presented here scaled well to the more complicated project. The performance was also acceptable. There are a few problems that I neglected to mention because their solutions are straight forward. One of the more noteworthy is the possibility that you would have to encode and decode the names of TreeNodes into valid method names if they weren't already. This is one of the costs of using an internal DSL. That's fine with me because it is much easier than writing a parser. In the next entry I would like to show how this DSL can be used for more than just building the tree.

Why hide your Java with a DSL using JRuby?

The Java vs. Ruby dialogs are everywhere. I intend to show how you can use the complementary strengths of the Ruby language and the Java platform. This basically amounts to trying to avoid the Java language as much as possible, while leveraging the JVM and libraries. There is enough information around about the merits of Java, so I am going to focus on how Ruby can enable Java with new super powers.

The problem with the Java language is the lack of expressiveness. If you are naive enough to think that Java has no real problem here then do two things. First, keep reading while I give an example. Second, work with a better language for a while. Be warned that once you start down this path you will be permanently changed.

This will work a little like a magic trick. We will start with some Java code. As usual, java will look awkward and feel clumsy. Then we will use JRuby and some concepts that are missing from Java to make all the ugliness disappear. By the end we will have code that people can read or write without knowing a thing about Java or programming at all. Here's the Java code, check it over and make sure it's authentic.

	public class TreeNode {
	private final String name;
	private final Collection<TreeNode> children;
	public TreeNode(String name) {
	this.name = name;
	children = new HashSet<TreeNode>();
	}
	public String getName() {
	return this.name;
	}
	public Collection<TreeNode> getChildren() {
	// don't show your privates!
	return Collections.unmodifiableCollection(children);
	}
	public void addChild(final TreeNode treeNode) {
	children.add(treeNode);
	}
	}

view raw TreeNode.java hosted with ❤ by GitHub

This is simply a node in a tree. Sure it is contrived for this example, that's the point. It knows its name and the nodes below it. We can add children to it. This would be useful for building up a little hierarchy like this.

	final TreeNode shape = new TreeNode("shape");
	final TreeNode ellipse = new TreeNode("ellipse");
	shape.addChild(ellipse);
	ellipse.addChild(new TreeNode("circle"));
	final TreeNode polygon = new TreeNode("polygon");
	shape.addChild(polygon);
	final TreeNode triangle = new TreeNode("triangle");
	polygon.addChild(triangle);
	triangle.addChild(new TreeNode("equilateral"));
	triangle.addChild(new TreeNode("isosceles"));
	triangle.addChild(new TreeNode("scalene"));
	final TreeNode quad = new TreeNode("quadrilateral");
	polygon.addChild(quad);
	final TreeNode rectangle = new TreeNode("rectangle");
	quad.addChild(rectangle);
	rectangle.addChild(new TreeNode("square"));
	polygon.addChild(new TreeNode("petagon"));
	polygon.addChild(new TreeNode("hexagon"));
	polygon.addChild(new TreeNode("googolgon"));

view raw Shapes.java hosted with ❤ by GitHub

Java got the job done, but that's ugly. In fact I bet you skipped right over that code block. That's okay, I'll write it out like this so that you can easily see that it is a hierarchy of shape types.

	shape {
	ellipse {
	circle
	}
	polygon {
	petagon
	triangle {
	scalene
	equilateral
	isosceles
	}
	hexagon
	quadrilateral {
	rectangle {
	square
	}
	}
	googolgon
	}
	}

view raw Shapes.rb hosted with ❤ by GitHub

Now the nested structure of shapes is pretty obvious at a glance. This is actually valid Ruby code. No matter how much more Java than Ruby you have written, our DSL is going to immediately be more readable to you. This is the point where it should start to dawn on you that this isn't about Java vs. Ruby, it's about using the tools available effectively.

A DSL is just an abstraction like a good API. While an API can insulate you from details with simpler concepts, a DSL can insulate you from unintuitive syntax and language features as well. Ruby has the ability to do this way better than Java can.

If you are just concerned with the business of using a hierarchy of things then you don't need to know Ruby or even Java now. The fact that this DSL is internal to Ruby, implemented in JRuby, and delegates to existing Java is just implementation details to the user. That means they don't need to know. Only the author of the DSL needs to know Ruby. Ruby does a lot for us since we don't have to write a parser and can use existing tools like IDE's.

One concern about DSL's is the creation of lots of mini languages and the investment to learn them. But there are mini languages everywhere. Picking up a DSL is similar to learning a new API or XML dialect.

In this case the DSL can potentially do much more than the original Java code. The DSL is focused only on describing the data structure while the original Java is tied to the action of building the structure. Separation of concerns here is useful because it lets you use the description for actions other than building. For example, I've used a similar DSL for making assertions about the structure of a tree and also writing queries against it. This works out especially handy in unit tests where you need a convenient way to express an initial structure and an expected value.

Hopefully I've convinced you that this is genuinely useful stuff. Java is ugly, and sometimes it is just better to hide it. A Ruby DSL can be a nice alternative. Now that the why is out of the way, next time I will focus on the how.