So, just a quick overview of where we are, what territory we're in: I like Emacs, I like Erlang, and I like running unit tests on my code. This is my attempt to gather some pieces together to fuse all of this into a seamless experience, so that whenever something goes wrong, I can quickly get to exactly where in the code the error occurred.

This will involve patches both to Emacs and to Erlang/OTP. My hope is that these changes will make it into upstream versions fairly soon, but this is where we are at the moment.

To start with, I run my unit tests from within Emacs, using M-x compile. It takes any shell command—make test or rebar eunit come to mind—and runs it inside a buffer in Emacs. That means that you can use the normal Emacs editing commands to move around, search for things, copy pieces of text, etc. Emacs also attempts to highlight error and warning messages in the output, and turn them into links to the corresponding position of the source code. With a stock Emacs, there are many possibilities left untouched, and that is where our journey begins.

Get stacktraces from eunit failures (pre-R15B02)

First of all, if you're running Erlang/OTP R15B or R15B01, you have a version of eunit that doesn't print stacktraces when an error occurs. That can be a serious time waster, compared to knowing what code was running three or four stack frames down from the error site (not to mention even knowing in what function the error occurred). So make sure that you're running R15B02 or later.

If you're stuck with an earlier version, fortunately there is a patch for that. I've written up instructions on how to apply it to your existing Erlang/OTP installation with minimal effort.

So before this patch, eunit output looks like this:

foo:6: my_test_ (module 'foo')...*failed*
::error:foo

And after the patch, we get this:

foo:6: my_test_ (module 'foo')...*failed*
::error:foo
  in function foo:'-my_test_/0-fun-0-'/0 [foo.erl:6]

Drop compilation error regexp from Emacs Erlang mode

I received great satisfaction the other day when I submitted a patch to Erlang/OTP consisting entirely of line removals. I'm not sure at what point in history the code I removed was needed, but the comments suggest at some point around Emacs 19. My Emacs 24 does compilation error highlighting just as well without it.

Even better, in fact—once the Erlang mode had installed this regexp globally, Emacs lost the ability to distinguish between compilation errors (pink) and compilation warnings (orange). This regexp was installed first in a long list (compilation-error-regexp-alist), thus hiding more sophisticated regexps.

If you have Erlang checked out from Github, you can apply this patch with the following commands (adjusting the git-fetch command if needed):

git fetch
git cherry-pick a87a9699735b0a25f99397fba9576f5756da54d3

Theoretically you could undo the changes that have been done to your Emacs session, but the simplest way is to just restart Emacs.

So far, so good: if you compile your Erlang code from within Emacs with M-x compile, warnings will be orange instead of pink, and M-g M-n (the next-error command) will skip warnings and jump directly to errors.

Jump from failed test cases to code

So let's return to the output we got from eunit:

foo:6: my_test_ (module 'foo')...*failed*

That tells us that the test on line 6 in module foo failed. That's pretty unambiguous, so there's no reason why I as a human should spend any effort on finding that location when the computer can do it for me.

This calls for a regular expression, of course. Here it is:

(setq compilation-error-regexp-alist-alist
      (delq (assq 'erlang-eunit compilation-error-regexp-alist-alist)
            compilation-error-regexp-alist-alist))
(add-to-list
 'compilation-error-regexp-alist-alist
 (cons
  'erlang-eunit
  (list
   "^ *\\(\\([^.:( \t\n]+\\):\\([0-9]+\\)\\):.*\\.\\.\\.\\(?:\\([^*]\\)\\|[*]\\)"
   ;; file
   (list 2 "%s.erl" "src/%s.erl" "test/%s.erl")
   ;; line
   3
   ;; column
   nil
   ;; type - need to match [^*] after the three dots to be info,
   ;; otherwise it's an error
   (cons nil 4)
   ;; highlight
   1
   )))
(add-to-list 'compilation-error-regexp-alist 'erlang-eunit)

The first line makes sure that any earlier attempts are purged before adding the new regexp, to avoid accumulating cruft. I did refine this quite a few times before arriving at this version ☺

I won't bore you with the details of that piece of code (see C-h v compilation-error-regexp-alist if you're interested), but let me just draw your attention to one of the lines:

   (list 2 "%s.erl" "src/%s.erl" "test/%s.erl")

Since the text we're matching is the module name, not the file name, we need to tell Emacs how to make a file name out of it. I added two common subdirectory names to make it do the right thing in most cases.

However, there is a bug in Emacs in Emacs that prevents that from working—if we try this with our example above, Emacs will just ask where the foo file is. So head to that bug report and apply the patch, to continue your journey to instant link bliss.

Done? Great! You're probably itching to try this out, and thus frantically looking for a failing test case. There's a simpler way: just put the example output above in a text file, and hit M-x compilation-minor-mode and try all the links. To edit the text file again, type M-x fundamental-mode.

Jump from stacktrace lines to code

Since R15B, Erlang stacktraces include file names and line numbers (and with the patch above, we got Eunit to display them to us). So wouldn't it be great to use that information to jump directly from the test output to the corresponding point in the code?

The output looks like this, and we might want to jump to any of the line numbers given:

foo: bza_test...*failed*
::error:{badmatch,c}
  in function foo:b/0 [foo.erl:31]
  in call from foo:a/0 [foo.erl:27]
  in call from foo:bza_test/0 [foo.erl:23]

This calls for another regexp:

(setq compilation-error-regexp-alist-alist
      (delq (assq 'erlang-eunit-stacktrace compilation-error-regexp-alist-alist)
            compilation-error-regexp-alist-alist))
(add-to-list
 'compilation-error-regexp-alist-alist
 (cons
  'erlang-eunit-stacktrace
  (list
   "^ *in \\(?:function\\|call from\\) .* \\[\\(\\([^:]+\\):\\([0-9]+\\)\\)\\]$"
   ;; file
   2
   ;; line
   3
   ;; column
   nil
   ;; type
   2
   ;; hyperlink
   1
   )))
(add-to-list 'compilation-error-regexp-alist 'erlang-eunit-stacktrace)

So from an error like the one we saw above, we can now move point to a line in the stacktrace and hit Enter, and Emacs will take us to the right line and file. (You could of course just click on the stacktrace line, if you're into that kind of thing.)

Jump to failing assertions

And while we're at it, why not create links for assertion failures as well? Eunit's assert macros (assert, assertEqual, assertMatch etc) create error messages that look like this:

foo: foo_test (module 'foo')...*failed*
::error:{assertEqual_failed,[{module,foo},
                           {line,6},
                           {expression,"2"},
                           {expected,1},
                           {value,2}]}
  in function foo:'-foo_test/0-fun-0-'/1 [foo.erl:6]

Here, the module name and the line number are on different lines, but that doesn't stop this regexp from working:

(setq compilation-error-regexp-alist-alist
      (delq (assq 'erlang-eunit-assert compilation-error-regexp-alist-alist)
            compilation-error-regexp-alist-alist))
(add-to-list
 'compilation-error-regexp-alist-alist
 (cons
  'erlang-eunit-assert
  (list
   (concat
    "^\\(::error:{assert[A-Za-z]+_failed\\),"
    "[ \n]*\\[{module,\\([^}]+\\)},"
    "[ \n]*{line,\\([0-9]+\\)}")
   ;; file
   (list 2 "%s.erl" "src/%s.erl" "test/%s.erl")
   ;; line
   3
   ;; column
   nil
   ;; type
   2
   ;; hyperlink
   1
   )))
(add-to-list 'compilation-error-regexp-alist 'erlang-eunit-assert)

Again, we have module names, not file names, so the same caveat applies.

Jump from stacktraces embedded in other output

The above snippets work very well as long as you get straightforward errors—something has crashed, and the error gets propagated up to your test function, and further up to Eunit, which formats a nice error report. But you're not always so lucky. You might have an error in a linked process:

foo: bar_test...
=ERROR REPORT==== 1-Aug-2012::19:56:58 ===
Error in process <0.79.0> with exit value: {badarith,[{foo,baz,0,[{file,"foo.erl"},{line,13}]}]}

*skipped*
undefined
*unexpected termination of test process*
::{badarith,[{foo,baz,0,[{file,"foo.erl"},{line,13}]}]}

Or there might be a catch somewhere deep in the code, and an error gets propagated into a comparison or something:

foo: frobozz_test...*failed*
::error:{badmatch,
      {ok,{'EXIT',
          {badarg,
          [{erlang,list_to_integer,[x],[]},
           {foo,frobozz,1,[{file,"foo.erl"},{line,19}]},
           {foo,frobozz_test,0,[{file,"foo.erl"},{line,16}]},
           {eunit_test,'-function_wrapper/2-fun-0-',2,
               [{file,[...]},{line,...}]},
           {eunit_test,run_testfun,1,[{file,...},{...}]},
           {eunit_proc,run_test,1,[{...}|...]},
           {eunit_proc,with_timeout,3,[...]},
           {eunit_proc,handle_test,2,...}]}}}}
  in function foo:frobozz_test/0 [foo.erl:16]

So here you can see that the crash actually occurred on line 19, but the only stacktrace line that our regexps so far can recognise is for line 16. But since the information is there, let's spare ourselves the trouble of moving to that line manually:

(setq compilation-error-regexp-alist-alist
  (delq (assq 'erlang-raw-stacktrace compilation-error-regexp-alist-alist)
    compilation-error-regexp-alist-alist))
(add-to-list
 'compilation-error-regexp-alist-alist
 (cons
  'erlang-raw-stacktrace
  (list
   "{file,\"\\([^\"]+\\)\"},[[:space:]]*{line,\\([0-9]+\\)}"
   ;; file
   1
   ;; line
   2
   ;; column
   nil
   ;; type
   2
   ;; hyperlink
   1
   )))
(add-to-list 'compilation-error-regexp-alist 'erlang-raw-stacktrace)

There it is. Now, every time the compilation output contains {file,"something.erl"} and {line,42}, possibly separated by whitespace, the file name will be turned into a link.

Conclusion

That is all I've been able to come up with for now. I've automated the boring things (finding where my test crashed) so I can spend more time and energy on the fun things (actually fixing the test)—which is what Emacs is all about, of course.

Some elisp snippets for improving highlighting and linking of output from Eunit (the Erlang unit test tool) when run inside Emacs.