Some people like to read their email in Emacs. Of course, Emacs comes with not one but two email clients, the more advanced one being Gnus. Gnus lets you choose from a number of formats to store your email on your local disk. Perhaps you're using offlineimap to fetch your email from an IMAP server, and since offlineimap uses the Maildir format, you might think that using Gnus' nnmaildir backend is exactly what you're looking for.
However, as has been documented, there are some issues with nnmaildir. In particular, nnmaildir and offlineimap disagree about how to mark a message as read. Normally, offlineimap will synchronise that flag between the server and your local mail store, such that a message that you've read in one place eventually gets marked as read in the other, but in this case we're not so lucky; you'd have to mark old messages as read in both places, which pretty much defeats the purpose of offlineimap synchronisation.
At this point, most people give up on nnmaildir and install Dovecot, a small IMAP server, locally, make it serve the Maildir directory over IMAP, and point Gnus' nnimap backend to the local IMAP server. That's a very reasonable thing to do, and probably the most painless way.
Of course, that precludes the chance of becoming immortal by writing
code, so I dug into
nnmaildir.el. I found that it paid no attention
to what the Maildir specification says about message
flags, but instead stores flags in a Gnus-specific format inside the
Maildir directory. The latter needs to be kept, since not all Gnus
marks can be represented as Maildir flags, but the most important ones
("read", "replied to" and "flagged") can be stored as part of the file
names of the messages. I came up with a patch and posted
it to the bug report for this feature. As of
2012-09-05, it's merged into Gnus' master branch.
So there it is, ready to be tested. There appear to be some glitches, in that it might sometimes not mark a message as read that should be read, but so far it seems it's not in the habit of eating people's email.
Not fast enough
There is more to do about nnmaildir, though; my changes so far have
been about making it correct, so the next step is to make it fast. It
works fairly well for mailboxes with thousands of messages, but when
you get into the tens of thousands, there is a long delay when
starting Gnus. (The logical solution to that is to never exit Gnus,
but…) I'm getting ahead of myself here, but I've sprinkled debug
messages over the code and so far they suggest that most of that time
is spent in
nnmaildir--grp-add-art. If I'm reading the
code correctly, it seems that it's entering every article, one by one,
into an ordered list called
nlist. Obviously, in the worst case,
that has quadratic time complexity. My thoughts at this moment are:
- Does this need to be an ordered list? (Other parts of the code performs operations on subranges of this list, but maybe we could use a tree or something…)
- Do we really need to read all of this into memory? This is data about every single message in the folder; most likely we won't ever use most of it.
That's where I am right now. I'll keep working on this as time permits.