2/17/2014

Simplest Explanation of Ivy Configurations

If you are here then you are probably trying to understand IVY's configuration concept. And quite frankly after getting comfortable with it I still can't understand their docs. They are freaking obtuse. Going on Stackoverflow proves frustrating as well. I'm going to try and explain this in a really straight forward example. One because the information out there isn't good, and two so people can throw stones at my explanation and improve my understanding. Here goes.

What does Ivy do?

Ivy downloads dependencies and put them into directories your Ant script will use when compiling, packaging, etc. The important part of that is Ivy downloads dependencies and organizes them. It's up to your Ant script to use them appropriately.
An ivy-module (ie ivy.xml file) has two main parts:
  • What dependencies do you need?
  • How do you want them organized?
The first part is configured under the <dependencies> element. The 2nd is controlled by the <configurations> element. For example:
This is fairly straightforward. We have three dependencies. If we have an Ant build file configured for Ivy this will download all three of these jar files and put them into the ${ivy.lib.dir}/compile/jar directory. That's great, but we when we go to package our application some of these aren't needed. For one we don't care to ship junit with our application so can we segment that out?
You could do this with filesets and excludes in Ant but that is tedious and error prone. Ivy will do this for you if you know how to ask to ask it. Ivy will put the dependencies in different directories based on if that dependency is needed for testing, compilation, or runtime. This is where configurations start to matter. So let's change what we have such that we divide up our dependencies using configurations. Let's create a 'test' configuration for this purpose.
Ok that was easy right? Well if you run this you'll find two directories under ${ivy.lib.dir}:
  • ${ivy.lib.dir}/compile
  • ${ivy.lib.dir}/test
However, all three dependencies in test, and the other two will be in compile! Doh! That's not what we wanted so what happened?! This comes from the fact that if you don't specify a conf attribute on each dependency it defaults to "*". Well sort of it's a bit more complicated, but you can think of it like match all configs. And because that dependency matches all configs mysql and log4j was copied to both test and compile directories. So let's fix that.
Alright now everything should be as we expect! But it's annoying to have to specify conf="compile" every time we add an dependency. This is where defaults come into play. Remember I said conf attribute defaults to "*" when nothing is specified? Well we can override that by setting the defaultconf on the dependencies tag.
Alright! Now we can just add dependencies and they will always be added to the compile configuration by default! Much easier.

Transitive Dependencies

Now there are some complexities about Ivy that I shielded you from thus far. And it has to do with the decisions Ivy has to make while trying to resolve dependencies. See when you declare you depend on A well A might also depend on B and C. Therefore you depend on not just A, but A, B, and C. B and C are called transitive dependencies. These are hidden from you because using Maven's POM files Maven (and Ivy) can figure those transitive dependencies. And there is where the information I've shielded from you lies in Maven's POM file.
See Maven has a different way to section out dependencies called scopes. And unlike Ivy they are fixed. But when Ivy is downloading these dependencies it needs to know what scopes to use when pulling these transitive dependencies (are we pulling this for testing, runtime, compilation, etc?). That should make your head spin a bit. But this is a real problem because we have to tell Ivy how to map our configurations to Maven scopes so it knows what to pull.
Without mapping our configurations they don't really work well so you have to understand this, but it's not this complicated once it's explained. So let's say we want to pull all of the dependencies JUnit has we'd do the following:
Whoa what the heck is test->default? This looks weird, but what we are saying is our configuration is test and we want to map that to the default scope in Maven. This will have the effect of pulling all junit's transitive dependencies. If we did the following:
That would only pull the dependencies junit directly declares, but not ITS transitive dependencies. You might do test->master if you wanted to compile against just junit, but not actually package it up in your application because it's optional. The user of your library must provide that library if they want to use that integration for example. Servlet API is a good example where you only need it for compilation, but you don't need to shipped with your WAR.
So here is the mystery of the -> operator in Ivy. It maps Ivy configurations onto Maven scopes when resolving dependencies so Ivy knows exactly what to pull down. It's that simple.
Back to our example now because we used defaultconf attribute to specify compile, but we didn't map it to scopes yet. So we can do that by doing the following:
We can go further and simply specify this at the configurations level so that we don't have to specify it every time we change a conf attribute.
Notice we didn't use test->default anymore? That's because we specified that at the configurations level and all our our configs are mapped to default scope in Maven for us.
There is a lot more to configurations that I don't fully understand, but I think this will demystify most things about configurations so you can start to structure your project appropriately using Ivy without trolling Stackoverflow and Ivy docs for vague answers.

23 comments:

mi0x said...

Thanks! This helped me a lot!

Unknown said...

Thank you for writing this. We are using Ivy to manage cross-compiled binaries. This is a great help. Thank you.

Unknown said...

Thank you for writing this. We are using Ivy to manage cross-compiled binaries. This is a great help. Thank you.

Unknown said...

I've been trying to find some clues about ivy dependencies configurations to no avail. Was about to succumb into the self deprecating stupor, until I've found this article. Thanks a mil!

Unknown said...

Thanks indeed for a nice and detailed explanation. I find the official Ivy docs regarding this topic very vague too. Well, after your post I'm going to read it again, maybe I'll finally get it and learn something new, though the most important things I already know, thanks to you.

Rahul said...

Very useful! I am currently working on a sbt application and ivy is driving me nuts!

PC said...

People like you are true leaders .. I dont really care how much is not right in this article .. but at least your intentions are undoubtedly right and from here what you have mentioned in this article anyone can get a head start.

a small feedback - when a newbie is reading here about ivy .. suddenly from somewhere Maven jumps into the scene.. are they even related?

Unknown said...

Wrong notes better than most of the right notes. Thanks

RS said...

Thanks a lot! A very clear, detailed and progressive explanation. Helped a lot in demystifying the conf and -> operator's working. Thanks again!

bhavana said...

This is the first post about ivy confs that didn't leave my head spinning. Thanks!

Kees said...

I had to change the line:

to the following line to get it working:

Kees said...

I had to change the line:

to the following line to get it to work:

Unknown said...

Thanks is a great explanation.

Unknown said...

As weird as it is, yours is the best approach of the many articles (among ivy's own) i've come across to explain ivy. Thanks!

Nirmit Shetty said...

thanks! really simple explanation.

Dave Conrad said...

Thanks! Very helpful.

Unknown said...

Thanks. It was a great help.

Ameya said...

Incredibly simplified !

Unknown said...

Years after your post, but, the pictures aren't visible. :/

Martin d'Anjou said...

Can you please fix the css, I can't see the code snippets unless I view the page as source. Thanks!

Unknown said...

None of the examples are showing, it's just blank space. How frustrating. I'll keep looking elsewhere.

Unknown said...

Still useful ;)

Thanks a lot for your post!

chubbsondubs said...

I have no problem seeing the code sections in Chrome or Firefox. If anyone wants to share what browser is giving them trouble I'll be happy to test it.