reading and searching gmane with gnus, fast

Reading mailing lists via Gnus by pointing it to the usenet service news.gmane.io is a well-known trick among emacsers. It has a couple of drawbacks, though: network latency and no search. The two problems have, as almost always with almost any problem in Emacs land, a cure. The names of the game are, in this case, leafnode and notmuch.

I've been using leafnode since i was young to avoid network latency issues when Gnus fetches news from remote usenet servers. Leafnode is a store & forward NNTP proxy that can be used to give a regular newsreader off-line functionality. It works by fetching in the background news articles from a number of configured remote servers (gmane.io in our case), storing them locally and offering a local NNTP server to Gnus (or any other newsreader, for that matter). That way, one configures Gnus to fetch news from localhost, which is fast and will never block, even when one is disconnected from the interwebs. Leafnode's server implements the full protocol, so one can also post to the remote servers.

For our case, leafnode's configuration file is very simple:

## Unread articles will be deleted after this many days
expire = 365

## This is the NNTP server leafnode fetches its news from.
## You need read and post access to it. Mandatory.
server = news.gmane.io

## Fetch only a few articles when we subscribe a new newsgroup. The
## default is to fetch all articles.
initialfetch = 100

With leafnode in place, i've rarely needed to subscribe to a mailing list¹, and all their messages are available with the Gnus interface that we all know and love.

With one caveat: one can search over e-mails, using either IMAP (i like dovecot's lucene indexes) or (even better) notmuch. Can we do the same with those messages we access through leafnode? Well, it turns out that, using notmuch, you can!

First of all, leafnode stores its articles in a format recognised by notmuch's indexer. In my debian installation, the live in the directory /var/spool/news/gmane. On the other hand, my notmuch configuration points to ~/var/mail as the parent directory where my mailboxes are to be found. I just created a symlink in the latter to the former and voila, notmuch is indexing all the messages retrieved by leafnode and i can search over them!²

With the version of Gnus in current emacs master, it's even better. I can tell Gnus that the search engine for the news server is notmuch:

(setq gnus-select-method
      '(nntp "localhost"
             (gnus-search-engine gnus-search-notmuch
                                 (remove-prefix "/home/jao/var/mail/"))))

and perform searches directly in Gnus using the notmuch indexes. Or, if you prefer, you can use directly notmuch.el to find and read those usenet articles: they look just like good old email³ :)