Some people like to read their email in Emacs. Of course, Emacs comes with not one but two email clients, the more advanced one being Gnus. Gnus lets you choose from a number of formats to store your email on your local disk. Perhaps you're using offlineimap to fetch your email from an IMAP server, and since offlineimap uses the Maildir format, you might think that using Gnus' nnmaildir backend is exactly what you're looking for.

However, as has been documented, there are some issues with nnmaildir. In particular, nnmaildir and offlineimap disagree about how to mark a message as read. Normally, offlineimap will synchronise that flag between the server and your local mail store, such that a message that you've read in one place eventually gets marked as read in the other, but in this case we're not so lucky; you'd have to mark old messages as read in both places, which pretty much defeats the purpose of offlineimap synchronisation.

At this point, most people give up on nnmaildir and install Dovecot, a small IMAP server, locally, make it serve the Maildir directory over IMAP, and point Gnus' nnimap backend to the local IMAP server. That's a very reasonable thing to do, and probably the most painless way.

Of course, that precludes the chance of becoming immortal by writing code, so I dug into nnmaildir.el. I found that it paid no attention to what the Maildir specification says about message flags, but instead stores flags in a Gnus-specific format inside the Maildir directory. The latter needs to be kept, since not all Gnus marks can be represented as Maildir flags, but the most important ones ("read", "replied to" and "flagged") can be stored as part of the file names of the messages. I came up with a patch and posted it to the bug report for this feature. As of 2012-09-05, it's merged into Gnus' master branch, and the change was included in GNU Emacs 24.3.

So there it is, ready to be tested. There appear to be some glitches, in that it might sometimes not mark a message as read that should be read, but so far it seems it's not in the habit of eating people's email.

Not fast enough

There is more to do about nnmaildir, though; my changes so far have been about making it correct, so the next step is to make it fast. It works fairly well for mailboxes with thousands of messages, but when you get into the tens of thousands, there is a long delay when starting Gnus. (The logical solution to that is to never exit Gnus, but…) I'm getting ahead of myself here, but I've sprinkled debug messages over the code and so far they suggest that most of that time is spent in nnmaildir--grp-add-art. If I'm reading the code correctly, it seems that it's entering every article, one by one, into an ordered list called nlist. Obviously, in the worst case, that has quadratic time complexity. My thoughts at this moment are:

• Does this need to be an ordered list? (Other parts of the code performs operations on subranges of this list, but maybe we could use a tree or something…)
• Do we really need to read all of this into memory? This is data about every single message in the folder; most likely we won't ever use most of it.

That's where I am right now. I'll keep working on this as time permits.

nnmaildir, the Maildir implementation of Gnus, doesn't agree with offlineimap about what a read message looks like. I'm trying to fix that, and other issues with nnmaildir as well.