Maven Mysteries

Maven is a dependency management tool for Java / JVM projects which has been around since 2004. Version 2 came out in 2005 and version 3 which was mostly compatible with version 2 was released in 2010. The fact that version 3 is mostly compatible with version 2 can be seen in the directory structure still having folders called .m2 for 'Maven 2'. Maven uses a different paradigm than its predecessor Ant which functioned as an imperative build tool where every build step had to be explicitly declared. Even though Gradle is used instead of Maven in many projects, Maven still has a very large footprint on the JVM ecosystem.

Cycle of Life

Maven introduces the concept of lifecycle phases like compile and test with a lot of sane defaults which it uses to build your application or library. Plugins can be added to your project which will bind to one of these cycles to perform their tasks during that phase of the lifecycle. Maven comes with a set of default plugins, such as the compiler plugin which compiles your project during the compile phase.

Dependencies

To me the most revolutionary feature of Maven is its ability to manage your dependencies. In the 'old days' you would have to manually download a new JAR file and add it to your build folder (maybe even check it into your version control system) to update a dependency. Maven offers the ability to manage dependencies in your POM file. You could add a dependency on Apache Commons Lang like this:

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-lang3</artifactId>
  <version>${commons-lang3.version}</version>
</dependency>

The ${commons-lang3.version} is a placeholder; oftentimes version numbers are specified inside the <properties> section of your POM to be able to easily find and update them in one place.

In order to update this dependency we only have to change its version. Now where does this dependency come from? This brings us to the next revolutionary feature: the central repository.

Out of the box Maven uses both a local repository (saved on disk in folder <home>/.m2/repository) and a central repository. If a dependency needs to be resolved for your project, it first searches if there is a copy in your local repository and if there isn't it downloads it from the central repository and adds it to the local repository for future use. This also means that dependencies themselves don't have to be checked into version control anymore. Every developer could just run mvn compile/package/install and Maven would download the needed dependencies to the local repository.

Transitive dependencies

The dependencies you add to your project can also have dependencies themselves, specified in their respective POM files, these dependencies are called transitive dependencies. And those dependencies can have dependencies, basically it's dependencies all the way down. If you ever managed a Spring Boot application: it is not out of the ordinary to end up with 100+ dependencies this way. Now here is the tricky part, if dependencies are both specified by you and by dependencies you use, which version of those dependencies will Maven pick?

Dependency resolution

This brings us to one of the most awesome and problematic features of Maven: dependency resolution.

Let's say you have the following tree of (transitive) dependencies:

• your application
  • dependency A version 1.0.0
  • dependency B (some version)
    • dependency A version 1.1.0

In this example your application depends on version 1.0.0 of dependency A, but dependency B depends on version 1.1.0 of dependency A, which one will it choose? The answer might surprise you (or not): it chooses version 1.0.0. Why? Maven looks at the 'depth' of the dependencies and it will choose the one which has the shortest path. As dependency A is declared right from within your application, it has a path length of 1, the path going through dependency B has a length of 2 so it is longer and thus version 1.0.0 is used.

Another example:

• your application
  • dependency B
    • dependency A version 1.1.0
  • dependency C
    • dependency D
      • dependency A version 1.0.0

In this example Maven will choose version 1.1.0 as the path to dependency A through dependency B is shorter than the path through dependencies C and D.

You might be inclined to think Maven would understand semantic versioning and always choose version major.minor+1.patch over version major.minor.patch as a version with only a higher minor version number should be backwards compatible, but alas, it does not work this way.

Overriding dependency versions

There is a way to fix dependency versions for your dependencies, whether they are directly included or transitive: dependency management. You can add the following section to your POM file:

<dependencyManagement>
    <dependencies>
      <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-lang3</artifactId>
      <version>${commons-lang3.version}</version>
    </dependency>
  </dependencies>
</dependencyManagement>

This tells Maven that anytime it encounters the dependency commons-lang3 it must choose the specified version. The added bonus of this is that you no longer have to add the version in places where you actually include this dependency directly. You can leave it out and it will still resolve the version you specified.

Scopes

Dependencies do not just have versions, they also have scopes. Scopes tell Maven at which point a dependency is needed, such as when running tests or only during runtime. The default scope is compile which tells Maven that the dependency is always needed: when compiling the sources, running tests and during runtime. A dependency with the test scope is only available when running tests. By specifying the correct scopes for your dependencies, you can prevent unneeded dependencies from being packaged with your application if you were building a fat jar (such as a Spring Boot application).

Advanced dependency resolution

Remember how I said that Maven doesn't understand semantic versioning? Well, that wasn't entirely true. It's possible to specify version ranges in Maven, but also highly dangerous, it uses semantic versioning 1.0.0 so more complex version numbers which are valid according to semantic versioning 2.0.0 won't work.

You could in theory specify a version for a dependency like [1.0,2.0) meaning 'any version greater than or equal to 1.0, but lower than 2.0', but this could still break compatibility as there are no guarantees. Using version ranges might seem like a useful feature, but it turns your build into a box of chocolates: you never know which version you're gonna get. Your build should be predictable and reproducible, it's a red flag to get different artefacts depending on when the build runs.

Enforcing dependencies

Maven has a useful plugin called Maven Enforcer which enforces stuff. You can use it to enforce requirements on your build or the managed dependencies, here's an example:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>  
      <artifactId>maven-enforcer-plugin</artifactId>  
      <version>${maven-enforcer-plugin.version}</version>  
      <executions>  
        <execution>
          <id>enforce-maven</id>  
          <goals>
            <goal>enforce</goal>  
          </goals>
          <configuration>
            <rules>
              <requireMavenVersion>
                <version>3.3.9</version>  
              </requireMavenVersion>
              <requirePluginVersions/>
              <requireReleaseDeps>
                <onlyWhenRelease>true</onlyWhenRelease>  
              </requireReleaseDeps>
              <banDuplicatePomDependencyVersions/>
              <reactorModuleConvergence/>
            </rules>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

The important part is the <rules> section which specifies which of the (built-in) rules you want to activate for this build. I have specified a version for Maven itself which is the minimum required version of Maven this build and the plugins it uses need.

• requirePluginVersions: this forces all plugins to have an explicitly specified version. While not necessary, it does make your build more stable
• requireReleaseDeps (with onlyWhenRelease): this prevents the build from succeeding if there are any SNAPSHOT dependencies if the build itself is not a SNAPSHOT version. Snapshot dependencies are unstable versions which may change over time, so it's best not to depend on them
• banDuplicatePomDependencyVersions: having duplicate versions of direct dependencies can lead to surprises, especially if they have different versions, this rule prevents that
• reactorModuleConvergence: enforces best practice for multi-module builds, such that children inherit the version of their parent

I usually use only those rules in my projects, but for the bold here are some more interesting ones:

• dependencyConvergence: ensures that dependencies which are included through multiple paths all resolve to the same version. Given Maven's dependency resolution mechanism, this would only work in practice if you specify those versions in your <dependencyManagement> , but if you start specifying versions for all of your transitive dependencies, well, good luck to you.
• requireUpperBoundDeps: ensures that dependencies are resolved to the highest version found among them across all paths. Again, due to Maven's mechanisms, this would only really work if you use <dependencyManagement> but it can definitely be a bit more useful than dependencyConvergence.

Mystery solved

So many things can go awry with dependencies it's sometimes a mystery it even works at all! At least now we understand more of how Maven deals with the many nuances of dependency management. As a takeaway I will give you one last tip if you're still having issues, just run mvn dependency:tree -Dverbose (might take a while). This will list all of your (transitive) dependencies in one giant tree and tell you which ones are used and which are omitted so you can see the duplicate dependencies with different versions.