Phil Sung : Emacs resources : VM configuration and workflow

13 Sep 2008

I read my mail in Emacs, using VM. This page describes some of the steps I did to get it to a point where I liked it, and some of the rationale for my setup.

You can download my .vm and/or just keep reading.

Contents

Prerequisites

On Debian/Ubuntu, install the vm package. Then install stunnel4 (for POP over SSL) and gnutls-bin (for SMTP over SSL/STARTTLS). You will also need to configure your Emacs to use stunnel4 (as described below) in order to use POP over SSL; otherwise, you may get the following unhelpful error:

vm-pop-check-connection: POP connection not open: POP over SSL

General overview of workflow

I get mail from two places: school e-mail (mit.edu), and local system mail (reports from cron jobs, etc., but only a couple of messages a day).

School e-mail

VM retrieves mail from a POP server over SSL and files it in a mail folder called inbox.

When I get around to opening VM, it displays inbox. I could read all my mail from there, but I like to read one mailing list at a time in order to (1) reduce the amount of context-switching and (2) allow me to postpone reading mail from high-volume lists until the end of each day.

To manage my mail, I've defined a number of virtual folders, which are subsets of inbox defined by specific rules— if you've used Evolution, you may have seen this concept. A mail folder is backed by a file (e.g. ~/mail/inbox), but a virtual folder is not; it's just a view of some of the messages in a "real" mail folder. For example, I have virtual folders for mail from co-workers (ddmg), general lab announcements (csail), Emacs mailing lists (emacs), etc.

I don't keep anything in my inbox permanently. When I'm done reading a message, it goes to a permanent storage location (just another mail folder). To keep me sane, these mail folders roughly parallel my virtual folders. For example, when I'm done reading a message from the emacs virtual folder, it's moved from inbox into a mail folder called emacs (it then disappears from the emacs virtual folder, which is a subset of inbox). In addition, I use another mail folder named general as a catch-all for saving my remaining mail. The process of moving mail to its permanent home is called saving or archiving.

Your VM configuration can define rules for how to construct virtual folders, as well how to decide to which mail folders messages get archived. I've chosen this particular scheme to make it easy to search for mail later, but this is by no means the only way to do it; like most other parts of Emacs, VM is flexible enough to let you really define your own workflow. You can choose to archive to one big mail folder, to mail folders based on senders or recipients, or by date, or any combination of the above, or not at all; and you can customize numerous other aspects of VM's behavior as well.

System mail

VM retrieves mail from my local spool file, /var/spool/mail/phil and puts it in a mail folder called system.

Eventually, I read each message in the system mail folder. Then I delete them all. They go away, forever.

Typical workflow

I've bound F12 to VM (M-x vm):

(global-set-key [f12] 'vm)

This pops me into my inbox. You can use v to open another mail folder or V V to open a virtual folder. I typically jump straight to a virtual folder I've defined called nonjunk, which is all of my mail minus selected high-volume mailing lists. It's this virtual folder that I usually keep open, so I can get work done without getting distracted by new mail every three minutes. More about this later.

If a VM buffer is already open, pressing g checks for new mail.

For each folder or virtual folder in which I have mail, I use n and p to move between messages and SPC to page through messages. When I'm done with a message, there are a couple of ways to proceed:

Alternatively, I sometimes just scan over the entire folder, and then press A ("auto-archive") to save every message in that folder to its default destination mail folder.

After either saving or auto-archiving, Emacs has copied the message(s) to the destination folder and marked the originals for deletion. To expunge these messages (as well as any spam that I marked for deletion earlier) and get them out of my inbox, I press # # # (which is hard to press on accident :) ). Then I either move to another folder or virtual folder, or press q to stop reading mail.

To compose mail, press m in a VM buffer to open up a new composition buffer. Write your message. Press C-c C-c to send it. To reply to an existing message, press R or F ("followup" i.e. reply to all).

My .vm file

Here's the configuration I use to achieve the workflow described above. In order to just run VM, though, the only really essential stuff is the account setup part near the top.

You could in principle put all this configuration in .emacs, but it's conventional to put VM-related configuration into ~/.vm instead. VM loads that file before it starts.

You can download the whole thing here, or read it with annotations below.

This code is for instructional and inspirational purposes only. Read the VM manual before (and preferably, after) trying any of this stuff out. You should not add any code to your init file unless you understand its implications, even if an upstanding person (such as myself) said it was in his init file.

;; -*- mode: emacs-lisp -*-

Tell Emacs my name and e-mail address:

(setq user-full-name "Phil Sung")
(setq user-mail-address "psung@mydomain.com")

Tell VM which mail folder is the default (the "primary inbox"), and where to put sent mail. Usually one stores all mail folders in the same directory, the vm-folder-directory:

(setq vm-folder-directory "~/mail")
(setq vm-primary-inbox "~/mail/inbox")
(setq mail-archive-file-name "~/mail/sent")

Here's how we tell VM how to retrieve mail and put it in a mail folder. Each part names a mail folder, a specification for where to get mail, and a "crash box", which is important but only for crash recovery purposes. An excellent blog post by Bill Clementson was invaluable in getting me started here.

(setq vm-spool-files
      '(("~/mail/system" "/var/spool/mail/phil" "~/mail/system.crash")
        ("~/mail/inbox" "pop-ssl:pop.gmail.com:995:pass:YOUR_USERNAME:*"
         "~/mail/inbox.crash")))

In the second line above, you should substitute two things: your mail server and your username. You may also need to change the method (pop-ssl) if you are not, in fact, using POP over SSL. Here's a configuration for outgoing mail:

(setq smtpmail-starttls-credentials '(("smtp.gmail.com" 587 nil nil))
      smtpmail-smtp-server "smtp.gmail.com"
      smtpmail-default-smtp-server "smtp.gmail.com"
      send-mail-function 'smtpmail-send-it
      message-send-mail-function 'smtpmail-send-it
      smtpmail-smtp-service 587
      smtpmail-auth-credentials '(("smtp.gmail.com" 587
                                   "YOUR_USERNAME@gmail.com" nil)))

When you send your outgoing mail through smtp.gmail.com, Gmail is nice enough to make it show it up your Gmail Sent Mail folder and also insert those messages into conversations.

On Debian/Ubuntu, make sure you're using stunnel4 for retrieving mail, below:

(setq password-cache-expiry 86400)      ; Time (in sec) to cache SMTP password

(setq vm-stunnel-program "stunnel4")

Now, we'll tell VM about how we want to send and read mail. I like to BCC myself on all outgoing messages so they later get saved alongside messages I received in the same thread.

;; In outgoing mail, write "From" field as "My Name <name@host.net>"
(setq mail-from-style 'angles)

(setq mail-self-blind t)                ; BCC self on all messages.

(setq mail-interactive nil)             ; Background sending mail.

(setq vm-delete-empty-folders nil)

(setq-default vm-summary-show-threads t) ; Show threaded view by default

VM typically displays only the headers of a message in the preview pane until you press SPC. You might want this if people looking over your shoulder is often an issue. I turned this option off to make VM display the entire message immediately:

(setq vm-preview-lines nil)             ; Don't preview, just display msg

Ask VM to delete messages when I've auto-archived or saved them. This is desirable in my workflow because I don't save/archive a message until I'm done with it. Also, when I save a message, make VM automatically jump to the next message.

(setq vm-delete-after-archiving t)
(setq vm-delete-after-saving t)
(setq vm-move-after-deleting t)

Don't let VM create new frames for anything!

(setq vm-frame-per-completion nil)
(setq vm-frame-per-composition nil)
(setq vm-frame-per-edit nil)
(setq vm-frame-per-folder nil)
(setq vm-frame-per-folders-summary nil)
(setq vm-frame-per-help nil)
(setq vm-frame-per-summary nil)

Sometimes, messages come in with charsets which aren't ASCII. You have to tell Emacs for which charsets it's possible to display these messages inline. If you don't, you will be prompted to save the text of the message to a file.

(setq vm-mime-charset-converter-alist
      '(("utf-8" "iso-8859-1" "iconv -c -f utf-8 -t iso-8859-1")))

(and (boundp 'vm-mime-default-face-charsets)
     (progn
       (add-to-list 'vm-mime-default-face-charsets "Windows-1251")
       (add-to-list 'vm-mime-default-face-charsets "Windows-1252")
       (add-to-list 'vm-mime-default-face-charsets "Windows-1257")))

Now, the good stuff: defining virtual folders and targets for saving/auto-archiving. First, we want to make sure that regexp matching for these purposes is done in a case-insensitive manner.

(setq vm-virtual-check-case-fold-search t)
(setq vm-auto-folder-case-fold-search t)

Now we'll define some virtual folders. The first element on each line is the name of the virtual folder. We then tell where that virtual folder's messages should be drawn from, and criteria for matching: usually, a regexp match on some header or a sender or recipient.

(setq vm-virtual-folder-alist
      '(("reuse" (("inbox") (header "reuse@mit\.edu")))
        ("intrepid-changes" (("inbox") (header "intrepid-changes\.lists\.ubuntu\.com")))
        ("launchpad" (("inbox") (header "X-Generated-By: Launchpad")))
        ("ubuntu" (("inbox") (header "ubuntu-.*\.lists\.ubuntu\.com")))
        ("emacs" (("inbox")
                  (header "\\(help-gnu-emacs\\|emacs-devel\\|emacs-pretest-bug\\)\.gnu\.org")))
        ("xemacs" (("inbox") (header "xemacs-beta\.xemacs\.org")))
        ("openmoko" (("inbox")
                     (header "\\(devel\\|support\\|community\\)\.lists\.openmoko\.org")))
        ("ddmg" (("inbox")
                 (author-or-recipient "\\(colleague1\\|colleague2\\)@example\.com")))))

I programmatically construct an additional virtual folder, nonjunk which contains all my mail, except those in a couple of the above virtual folders, corresponding to high-volume mailing lists, which I've chosen to filter out. (I read those excluded virtual folders typically just once or twice a day, and keep nonjunk open the rest of the time.)

(setq pps-unimportant-virtual-folders
      '("launchpad" "intrepid-changes" "ubuntu" "openmoko" "emacs" "xemacs"))
(add-to-list
 'vm-virtual-folder-alist
 `("nonjunk"
   (("inbox")
    (not (or ,@(mapcar (lambda (c)
                         (second (second (assoc c vm-virtual-folder-alist))))
                       pps-unimportant-virtual-folders))))))

Here are some rules for deciding which mail folder to save a particular message to by default. I just wrote rules that mostly parallel my virtual folder rules. It's somewhat unfortunate that the specification language here is not quite the same as that for virtual folders. However, it is more flexible in that you can ask VM to dynamically compute a the name of the mail folder to use (not shown here); this would be useful for, presumably, making mail folders named by year or month, etc.

You probably want to include a catch-all rule at the end for filing mail which doesn't fall into any other category. (I put all of that stuff in its own mail folder, general.) If you don't, when you auto-archive a folder, messages which didn't match any of the rules don't get moved anywhere, which can be kind of annoying.

(setq vm-auto-folder-alist
      '(("X-BeenThere"
         ("reuse@mit\.edu" . "reuse"))
        ("From"
         ("\\(colleague1\\|colleague2\\)@example\.com" . "ddmg"))
        ("List-Id"
         ("intrepid-changes\.lists\.ubuntu\.com" . "intrepid-changes")
         ("ubuntu-.*\.lists\.ubuntu\.com" . "ubuntu")
         ("\\(help-gnu-emacs\\|emacs-devel\\|emacs-pretest-bug\\)\.gnu\.org" . "emacs")
         ("xemacs-beta\.xemacs\.org" . "xemacs")
         ("\\(devel\\|support\\|community\\)\.lists\.openmoko\.org" . "openmoko"))
        ("X-Generated-By"
         ("Launchpad" . "launchpad"))
        ("To"
         (".*" . "general"))))

Additional notes

One neat thing about virtual folder handling in VM is that it's easy to define new virtual folders on an ad-hoc basis so you can deal with multiple messages quickly, e.g. messages with a particular subject. Type V C to make a new virtual folder which selects messages from the folder you're looking at. Type V C ? for help with the possible options.

I also use BBDB to manage my contacts, the setup for which is described here.

Peeves

As a Gmail user, I wish there were more thread-oriented features. However, the ad-hoc virtual folders actually get you most of the way there.