Fun with Opam: Advice to my Past Self

Most instructions on how to get started with OCaml packages now advise the user to get started with opam, which is excellent advice. Getting up and running with opam is pretty easy, but I wasn’t sure where to go from there when I wanted to modify other people’s packages and use the modifications in my environment. I wish I’d realized that the documentation for making packages has a lot of applicable advice for that use case, as well as the apparent target (making your own packges from scratch).

Most of the documentation on opam seems to focus on its use as a dependency manager, but it’s also capable of providing many virtual environments via opam switch. Often buried in documentation is the very useful opam switch --alias-of, which one can use to make new virtual environments using a standard OCaml compiler. Most of my current switches are based off the 4.02.1 compiler (just a bit short of up-to-date), so I make new switches for projects with opam switch new_project --alias-of 4.02.1.

I wish, also, that I’d started writing opam files for my own projects and installing them locally via opam pin sooner. The earlier I do this, the less likely I am to publicize a repository with missing or broken dependencies, and it reduces the friction of sharing early-stage code with other people. Combined with the switch-per-project approach, having opam files ready means you can easily have fresh switches with the minimal amount of dependencies installed; you can also install a minimal set of reverse-dependencies (if any!), to limit the amount of time you spend waiting for downstream packages to recompile when you want to test a change. I have great sorrow when I think back on the number of times I waited through a complete recompile of all zillion packages that depend on mirage-types when I really only wanted to test tcpip; a switch that doesn’t have mirage-console-unix and friends installed won’t try to recompile them, and I won’t have to spend any minutes of my only life waiting for that compilation to finish.

Having a lot of switches does lead to some confusion, since executing opam switch results in a filesystem change in the user’s .opam directory but can’t enforce the change in all open terminals. The problem is compounded by the requirement to eval $(opam config env) after running opam switch – if the user forgets to do this, opam switch will report the expected branch, but any commands that try to use the configuration will use the values for the previous switch. After having been caught out by this a few times, I caved and customized my command prompt to report on the current actual switch (along with the current git branch):

function git-current-branch {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1) /'
}
function opam-sw {
  # this is pretty brittle but good enough
  # have to adjust last cut if .opam is not /home/username/.opam or similar depth
  echo $CAML_LD_LIBRARY_PATH|cut -d: -f 1|cut -d'/' -f 5
}
export PS1="[\$(opam-sw)] \$(git-current-branch)$PS1"

It’s embarrassing to admit how many times I was confused by the apparently nonsensical output of make and opam install before I made this change, but at least that’s decreased since I started showing myself this information by default.

Something that still trips me up, though, is the setup.data file created by projects that use oasis to generate build files. setup.data contains cached information on how to build projects, including full paths to compilers and libraries, which isn’t updated if you use opam switch. When I began writing this post, my “solution” for this was compulsively removing setup.data every time there’s a weird-looking problem building a project that uses oasis, but writing I’ve been inspired to do better by making another adjustment:

function setup-data {
  grep -E "^standard_library=" setup.data 2>/dev/null|cut -d'/' -f5 
}
function opam-sw {
  setup=$(setup-data)
  switch=$(echo $CAML_LD_LIBRARY_PATH|cut -d: -f 1|cut -d'/' -f 5)
  if [ -n "$setup" ] && [ "$switch" != "$setup" ]
  then echo "(!!! setup.data $switch)"
  else echo $switch
  fi  
}
export PS1="[\$(opam-sw)] \$(git-current-branch)$PS1"

Now my prompt will warn me when setup.data and my current switch conflict, as they do in the following example where I last built mirage-tcpip in the separate-arp-tcpip switch:

[4.02.1] user@computer:~$ cd mirage-tcpip
[(!!! setup.data 4.02.1)] (separate_arp) user@computer:~/mirage-tcpip$ 

This hasn’t saved me any grief yet, but I’m sure it will soon enough.