Theory In Practice

The more I use the combination of git-flow and gerrit as our code review process the more I feel I’d never want to use anything different.

As we continue to break up our projects into smaller repositories I find my change sets often cross repository boundaries.

When ever I find myself with a large, epic, cross-repository refactoring I find myself wanting a way to group the disparate reviews under a common “tag” to signify their interdependence - and it turns out Gerrit supports this via “topics”.

Traditionally we setup our repositories to push into gerrit with a for-review remote for the develop branch along the lines of:

[remote "for-review"]
  url = gerrit:dev/$GIT_REPO_NAME
  fetch = +refs/heads/*:refs/remotes/for-review/*
  push = HEAD:refs/for/develop

Which means submitting reviews is a simple matter of running git push for-review. But if I want to submit a review with a topic defined, I need to modify my .git/config’s push reference, or write a long winded command line.

Being the lazy developer that I am, I followed wrote up a simple git-submit-review script to handle it all for me:

#!/bin/sh
url=$(git config --get remote.for-review.url)
push=$(git config --get remote.for-review.push)
branch_name=$(git symbolic-ref -q HEAD)
branch_name=${branch_name##refs/heads/feature/}
branch_name=${branch_name:-HEAD}

git push $url $push/$branch_name

echo "Open browser to: \
  http://gerrit/#/q/status:open+topic:$branch_name,n,z \
  to view topic reviews"

Sticking this on the path lets me run git submit-review and any commits will be pushed to gerrit, along with the current feature branch name as the topic.

Now, for each repository I’m working on, if I use the same feature name, such as TEST-102-some-epic-fix then all of those reviews will be grouped nicely together so I can just send that filtered search to a coworker.

Note: This works for Gerrit 2.5, by looking at documentation changes for the upcoming 2.6 release, the topic is set by appending %topic=TOPICNAME_ to the push reference.

It’s been awhile since I’ve posted anything on the state of HalBuilder and where the project is going, so now that I’d started heading towards a release I thought it was about time I updated the blog about it.

The first thing to note is the Github source repository has moved to a new HalBuilder Organisation and split up into a set of 6-7 separate maven artifacts/git repositories:

• halbuilder-test-resources
• halbuilder-api
• halbuilder-core
• halbuilder-java
• halbuilder-scala
• halbuilder-kotlin
• halbuilder-clojure

halbuilder-test-resources

A reusable set of HAL representations in both XML and JSON format to use in tests. This artifact is bundled using the maven-remote-resources plugin for easy reuse in other projects.

Currently this is the only 2.0.1 series artifact that has been released.

<dependency>
 <groupId>com.theoryinpractise</groupId>
 <artifactId>halbuilder-test-resources</artifactId>
 <version>2.0.1</version>
</dependency> 

halbuilder-api

Next up is the HalBuilder API artefact, under the 1.x releases this used to be the contents of the com.theoryinpractise.halbuilder.spi package.

There are two fundamental changes in the 2.0.1 series of HalBuilder:

1. The base API contains no dependency on the Google Guava library, this means the things no longer return Optional<T> instances ( see the halbuilder-java module for this ).
2. All classes/interfaces/methods with the word Resource in their name have been renamed to Representation to more clearly separate HAL from your actual REST API. HalBuilder is all about the representation of your resources, not the resources themselves.

halbuilder-core

The core HalBuilder library - this is 1.x restructured, and without any outgoing Google Guava dependencies.

halbuilder-java

The halbuilder-java module provides a simple “extension” class (com.theoryinpractise.halbuilder.java.Representations) containing Guava based extensions to the HalBuilder library.

halbuilder-scala

The new halbuilder-scala module contains implicit conversions providing idiomatic Scala extensions to the HalBuilder library. Similar to halbuilder-java only using Scala’s Option and closures.

halbuilder-kotlin ( coming )

The planned halbuilder-kotlin module provides idiomatic Kotlin extension methods to HalBuilder. The intent is to be a sister to module to halbuilder-scala only using Kotlin nullable types and closures.

halbuilder-clojure ( coming )

The planned halbuilder-clojure module provides idiomatic Clojure functions wrapping HalBuilder. Much like halbuilder-kotlin and halbuilder-scala - the intent is to provide support for clojure functions/predicates/maps when working with HalBuilder.

Releasing?

So far I’m pretty happy with the shape of halbuilder-api, halbuilder-core, and halbuilder-java so I hope to start releasing these artifacts over the next week or two unless something crops up, or someone reports some heinous design flaw…

The other day I was updating a rather complex data provider driven TestNG tests and I needed to introduce another element to data being provided but this would entail repeating, or reworking how the initial provider was being generated.

After thinking about the problem for a while, it hit me that I what I wanted was a way of combining multiple dataproviders into a single larger one - a cartesian product of results for want of a better term, and soon enough with the help of the Google Guava library I had what I needed.

Given the following three independant data providers:

@DataProvider
public Object[][] providerNamesAndAges() {
    return new Object[][]{
      {"Peter", 53},
      {"Paul", 25},
      {"Mary", 16}};
}

@DataProvider
public Object[][] providerColours() {
    return new Object[][]{
      {"Red"},
      {"Green"},
      {"Blue"}};
}

@DataProvider
public Object[][] providerCounts() {
    List<Object[]> counts = Lists.newArrayList();
    for (int i = 0; i < 10; i++) {
        counts.add(new Object[] {i});
    }
    return counts.toArray(new Object[][] {});
}

I can now add a fourth provider which is actually used by the test:

@DataProvider
public Iterator<Object[]> providerData() {
    return DataProviderUtil.cartesianProviderFrom(
      providerNamesAndAges(), providerColours(),
      providerCounts());
}

@Test(dataProvider = "providerData")
public void testFest(String name, int age,
      String color, int count) {
    System.out.println(String.format(
      "%s - %d - %s - %d", name, age, color, count));
}

which yields a test run of the cartesian combinations of each:

It’s small things like this that keep be coming back to TestNG for my unit testing in Java land.

For the last few weeks I’ve been reading about Frege, a Haskell based language for the JVM but only in the last few days have I actually started to experiment with actually using it.

My initial experiments were made using the maven-exec-plugin but I knew this wouldn’t last too long so last night I whipped up a quick small maven plugin and pushed it out to maven central.

Amusingly, it was sometime after writing this that I discovered there was a compiler mojo already in the Frege codebase :)

Anyway, I’ve also pushed my simple frege test project to github which makes use of the plugin if anyone wants to play along with me.

<plugin>
  <groupId>com.theoryinpractise.frege</groupId>
  <artifactId>frege-maven-plugin</artifactId>
  <version>1.0.1</version>
</plugin>

Currently one needs to explicitly mention the source and test source files to compile, and the order in which they’re used as it looks like frege compiler doesn’t resolve this for you.

Along with learning frege one experiment I wish to look at is how well it plays with OSGi and module systems - I’m hoping with the static nature of the compiled functions that this should be quite clean, however interacting with OSGi services might be another issue!

Work has begun slowly on an API breaking 2.0 release of my HalBuilder library for the REST Hypertext Application Language mediatype. Along with API breaking changes, updates to support for the new IETF proposed specification is also being tracked:

• Everything is now a “Representation”, one thing I regret not changing prior to the 1.0 release was using Resource as the basis for HalBuilder’s class hierarchy, what this library produces and consumes is NOT a REST resource, but merely it’s representation. So the first change in 2.0 is a wholesale renaming of classes, interfaces, and method names ( if that’s not API breaking what is? ).
• Optional HREF/self-link for resources
• Link/Embedded representations are bound by a single reltype, not a space separated list of reltypes. As part of this, the API standardises on specifying the rel as the first argument to all relevant methods.

When experimenting with HalBuilder in non-java languages such as Clojure and Kotlin I was stuck with the mild irritation caused by exposing Google Gauva’s Optional classes from the API, as an attempt to rectify this HalBuilder will be split into multiple artefacts/repositories as a core library with several language specific modules providing more idiomatic language support were appropriate, i.e. nullable types and closures in Kotlin, Guava Optionals/Functions in Java, Clojure bindings/macros/fns.

The core module, and each supported language module will be kept as separate git repositories under the new HalBuilder Organisation on Github.

As with the existing release, a Google Guava based Java module will be the first supported shadow API and hopefully the first commits on that new repository will start to appear over the next short period as time allows.

At the end of the last post I mentioned how to create a ReadableResource from external content

ReadableResource somethingPostedToUs = rf.newResource(
  new InputStreamReader(request.getInputStream()));

and mentioned we’d look at converting that to static class, but first we’ll look at the generic contract’s system I added to the API. A Contract is a simple SAM Type interface:

public interface Contract {
  boolean isSatisfiedBy(ReadableResource resource);
}

The ReadableResource interface provides an isSatisfiedBy method which simply takes a Contract:

Contract validName = new Contract() {
  Pattern p = Pattern.compile("\\W*");
  public boolean isSatisfiedBy(ReadableResource resource) {
    return (((String) resource.get("name").or("")).matches("[a-zA-Z\\w]+"));
  }
};

The contract above simply asserts that the resource has a name property that matches the given regex, and can be used rather simply:

if (resource.isSatisfiedBy(validName)) {
  System.out.println("Name matches!");
} else {
  System.out.println("Name doesn't match");
}

Getting back to our initial question about static typing however, the HalBuilder library has in its internal API an InterfaceContract which uses reflection to assert that a resource contains a matching property for every declared method in the interface. getXxx style methods are resolved to an xxx property whilst other methods are resolved verbatim.

Currently the InterfaceContract is an internal class that is used via the ifSatisfiedBy method on ReadableResource and the renderClass method on RenderableResource. Given an interface such as:

public static interface Nameable {
  String getName();
}

we call call:

Optional<Integer> length = resource.ifSatisfiedBy(Nameable.class, new Function<Nameable, Integer>() {
  public Integer apply(@Nullable Nameable input) {
    System.out.println(input.getName());
    return input.getName().length();
  }
});

System.out.println("Name is " + length.get() + " characters long.");

The call to ifSatisfiedBy takes an interface class, and a Google Guava function object which accepts a proxy of that interface based on the resource. The result of the call is a Guava Optional of the functions return value (absent when the contract is not safisfied).

Beneath the covers this uses the renderClass method on RenderableResource to return the proxy.

Note to self: renderClass should really be named renderInterface.

The combination of contracts and proxies give us a powerful programming model for working with HAL resource content from a static language/spec point of view, and whilst fairly low level, can be used to offer great flexibility in their usage.

The other day when I posted about the release of the Halbuilder library I stupidly reused big portions of an email I was writing about the library, and included some client usage code using Jetbrain’s new language Kotlin.

Several people (and rightfully so) slapped me for that – so this post is all about how you generate, and consume HAL resources in pure Java.

Creating a basic Resource

The HalBuilder library, as its name suggests takes on a fluent builder approach to its API, but before we get to building we need a ResourceFactory as our entry point.

ResourceFactory rf = new ResourceFactory();

A ResourceFactory contains various defaults for our HAL resources, the only default default being a base URL of http://localhost for all new resources (overridable via a constructor argument).

Now that we have our factory – we’re ready to start building!

Resource r = rf.newResource("/user/talios")
               .withLink("http://twitter.com/talios", "twitter")
               .withProperty("name", "Mark Derricutt");

This gives us a Resource with two links:

1. http://localhost/user/talios with a rel of self
2. http://twitter.com/talios with a rel of twitter

and a name property with the value of Mark Derricutt. This can be converted to an XML representation with:

String xml = r.asRenderableResource().renderContent("application/hal+xml");

which yields:

<resource href="http://localhost/user/talios">
  <link rel="twitter" href="http://twitter.com/talios" />
  <name>Mark Derricutt</name>
</resource>

As well as individually adding properties using withProperty, HalBuilder provides additional withBean, withFields, and withSerializble methods to automatically add all “JavaBean” fields via get/set methods, all public fields as properties, or delegate to a Serializable interface.

Example’s of each usage of these can be found in the unit tests over on Github.

Embedded Sub-Resources

Along with links, the HAL format allows for embedding additional resources inside the current resource. This can easily be done using the withSubresource method:

Resource r2 = rf.newResource("/user/illegalargument")
                .withLink("http://twitter.com/illegalargument", "twitter")
                .withProperty("name", "Illegal Argument");

r.withSubresource("podcast", r2);

which generates the following XML:

<resource href="http://localhost/user/talios">
  <link rel="twitter" href="http://twitter.com/talios" />
  <name>Mark Derricutt</name>
  <resource href="http://localhost/user/illegalargument" rel="podcast self">
    <link rel="twitter" href="http://twitter.com/illegalargument" />
    <name>Illegal Argument</name>
  </resource>
</resource>

or the following JSON:

{
  "_links" : {
    "self" : {
      "href" : "http://localhost/user/talios"
    },
    "twitter" : {
      "href" : "http://twitter.com/talios"
    }
  },
  "name" : "Mark Derricutt",
  "_embedded" : {
    "podcast" : {
      "_links" : {
        "podcast self" : {
          "href" : "http://localhost/user/illegalargument"
        },
        "twitter" : {
          "href" : "http://twitter.com/illegalargument"
        }
      },
      "name" : "Illegal Argument"
    }
  }
}

Reading External Resources

Generating resources is one thing, but what about reading HAL resources from external sources? The ResourceFactory#newResource that we used to create the above resources, can also take a Reader, but rather than a Resource we get an immutable ReadableResource given to us (no withXXX methods).

ReadableResource somethingPostedToUs = rf.newResource(
  new InputStreamReader(request.getInputStream()));

In the next post we’ll look at how you can take that ReadableResource and convert it into a static, type safe Java class…

Last night I pushed out the first public release of my Java based Hypertext Application Language (HAL) parser/generator library to Maven Central:

<dependency>
  <groupId>com.theoryinpractise</groupId>
  <artifactId>halbuilder</artifactId>
  <version>1.0.1</version>
</dependency>

The HalBuilder library provides an API for both generating, and consuming HAL based resource representations. Using the library, we can write the following code (examples are using Jetbrains Kotlin):

val gadgetForumOwner = openResource("http://localhost:4567/")
  .follow("forums")
  .findResourceByIdProperty("gadgets")
  .follow("self")
  .get("owner").get()

The above snippet uses four custom helper/extension methods outside of the base library: openResource, follow, and findResourceByIdProperty which are defined as:

inline fun openResource(href: String) : ReadableResource {
  val stream = httpClient.prepareGet(href)?.execute()
    ?.get()?.getResponseBodyAsStream().sure()
  return factory.newResource(InputStreamReader(stream)).sure();
}

inline fun Link.openResource() : ReadableResource {
  return openResource(this.getHref().sure());
}

inline fun ReadableResource.follow(rel: String) : ReadableResource {
  return this.getLinkByRel(rel)?.get()?.openResource().sure();
}

inline fun ReadableResource.findResourceByIdProperty(id: String) : ReadableResource {
  return this.getResources()?.findFirst(
    { it -> id.equals(it?.get("id")?.get()) }).sure()
}

Our example application REST API is a mock “GeekZone” forum server, when we open the initial resource at http://localhost:4567/ (our only fixed entry URL) we’re returned a resource with a link telling us where the forums resource can be found:

<resource href="http://localhost:4567/">
  <link rel="forums" href="http://localhost:4567/forums" />
</resource>

When we follow the link, a resource containing a set of embedded forum resources is returned:

<resource href="http://localhost:4567/forums">
  <resource href="http://localhost:4567/forums/offtopic" rel="forum self">
    <description>Anything goes baby!</description>
    <id>offtopic</id>
  </resource>
  <resource href="http://localhost:4567/forums/gadgets" rel="forum self">
    <description>Gadgets and Toys</description>
    <id>gadgets</id>
  </resource>
</resource>

The extension method findResourceByIdProperty walks the embedded resources looking for a matching id property. Once we’ve identified the embedded resource we wish to explore further, if the embedded representation’s content doesn’t satisfy our requirements, we can follow the self link to get the full representation:

<resource href="http://localhost:4567/forums/gadgets">
  <link rel="forums" href="http://localhost:4567/forums" />
  <link rel="rename" href="http://localhost:4567/forums/gadgets/rename" />
  <description>Gadgets and Toys</description>
  <id>gadgets</id>
  <name>Gadget Talk</name>
  <owner>example@example.com</owner>
</resource>

and then finally get the owner property.

The uniform nature of the HAL specification – a common structure and processing model of resources, links, and embedded resources means we can remove a large portion of ambiguity which often plagues REST based APIs.

Unfortunately HAL’s still not the saviour of everything REST, there’s still plenty of other questions surrounding your API to be decided.

A followup post will expand on how to use Halbuilder, and how resource ‘requirements’ are satisfied.

It’s been a few months since I’ve managed to find time to do any work on the clojure-maven-plugin but since there’s a few bug fixes and updates sitting on the development branch it’s high time for a new release!

1.3.8 has been released to Maven Central with the following changes:

• Add autodocTargetDirectory property to specify location of autodoc output.
• Add dependency info in README.
• Add mojo for running marginalia.
• Honour skipTests to prevent test execution.
• Switched to using SourceInclusionScanner to find which source files to recompile.
• Added integration test for clojure 1.3-beta1.
• Fixes issue with test-with-junit not failing build if there are test failures.
• clojure:swank now requires swank-clojure 1.3.1

Thanks to Hugo Duncan and Paudi Moriarty for contributions to this rather belated release.