Importing contacts from OSX Addressbook to Emacs BBDB

A major aspect of using Emacs is that it is more than a text editor – it is more of an application platform – some would call it an Operating System – that allows a host of applications to run within the same context and session – and vastly enrich the user experience. This is in fact a primary reason why many Emacs users spend their entire computer interaction via Emacs.

This unique aspect of Emacs is made possible by the availability of Emacs Lisp as a first class scripting/programming interface for the editor – and has made possible application packages such as W3 (a web browser), Gnus (the NNTP and mail reader), Eshell (a lisp based shell), Wanderlust (a very capable email client), and the Insidious Big Brother Database – aka BBDB – which is a contact manager and the subject of this post.

(BTW, the list of application is only a very small fraction of what is actually available on Emacs – see for a ton of other packages).

As EmacsWiki points out, BBDB is:

“A complete address book for nearly every email and news client written for Emacs, and its functionality supports (by default) making entries in the address book automatically when reading email or posts by people, and easy lookup of addresses while composing messages.”

In essence, it is an always available address book that works with other packages such as mail, Gnus and org-mode to provide a easy to access repository of contact information. The repository itself is a plain text lisp file which by default is stored as ~/.bbdb.

However, as with any contact manager, the problems are usually with:

  1. Initial data import, and
  2. Synchronization with other repositories and devices

For my platform (Apple OSX), the first problem above translates to data import from my primary OS level address book – the The second problem is not much of an issue for me since I use as the primary contact list elsewhere and sync my devices to that. For keeping BBDB in sync, I have a cron job to automate the scripts below.

The contacts import to BBDB requires a data conversion of the address from custom format in which stores the contacts to the BBDB lisp format. The steps required are as follows:

  1. Extract the records from
  2. Parse and reformat the relevant fields that BBDB requires, and
  3. Output the reformatted records in the BBDB lisp format

Luckily, for step # 1, an open source command line utility already exists – appropriately called contacts. It is available from and is a svelte 194KB download. This utility allows querying of the OSX Address book and extraction of specific fields and records. The output is a simple CSV text file.

For step # 2 and #3, additional scripting is required. I have written a simple Ruby script called contacts2bbdb.rb to run the contacts utility and output the BBDB repository file:

#!/usr/bin/env ruby -w
# Filename: contacts2bbdb.rb
# Description: Converts addresses from OSX to BBDB format
# Author: Anupam Sengupta (anupamsg ... A-T ... G - M - A - I - L )
# Maintainer: Anupam Sengupta
# (c) 2007-2016 Anupam Sengupta.
# Created: Fri May 25 15:53:22 2007
# Version: 1.0
# Last-Updated:
#           By: Anupam Sengupta
#     Update #: 733
# URL:
# Keywords: `BBDB’, ‘OSX’, ‘contacts’, ‘convert’
# Compatibility: GNU Emacs 21 and above
# Commentary:
#  Converts the addresses and contacts from Apple OSX’s system addressbook
#  format ( to Emacs’ BBDB format.
#  Requires the following additional software:
#  1. Ruby V1.8 and above (
#  2. Big Brother Database (BBDB) package for Emacs (
#  3. The ‘contacts’ program to read Addressbook’s contacts
#     (
# Usage:
#  1. Install Ruby, BBDB and contacts, if not already present
#  2. Backup the OSX Address book (export the addresses as addressbook archive)
#  3. Run this Ruby script to generate the converted records in BBDB format in the STDOUT
#  4. Save the STDOUT output of this script to
#     $ ruby contacts2bbdb.rb >
#  5. Replace your .bbdb file with
# Change log:
# License: GNU GPL V2.
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 51 Franklin Street, Fifth
# Floor, Boston, MA 02110-1301, USA.
Delim = “\t”                    # Default delimiter
ContactsProg = ‘/usr/bin/env contacts’ # The contacts program
ContactsProgParams = ‘-H -l’
# A map between the LDIF field names and the actual field name
ContactFields = {
:fn => ‘firstName’,
:ln => ‘lastName’,
:c  => ‘company’,
:nn => ‘nickName’,
:he => ‘homeEmail’,
:we => ‘workEmail’,
: oe => ‘otherEmail’,
:hp => ‘Home’,
:mp => ‘Mobile’,
:Mp => ‘Main’,
:wp => ‘Work’,
ContactFormatOpts = ContactFields.keys #  Options to pass to the contacts program
ContactsProgFormat =  “‘%” + ContactFormatOpts.inject { |s, f| s.to_s + “#{Delim}%#{f.to_s}” } + “‘“
ContactsFullExec = “#{ContactsProg} #{ContactsProgParams} -f #{ContactsProgFormat}”
output = `#{ContactsFullExec}`.split(/\n/) # Read the output of contacts program
# Start parsing the contacts output
records = do |line|
record =
line.chomp.split(Delim).each_with_index { |f, i| record[ContactFields[ContactFormatOpts[i]]] = f.strip unless f =~ /^\s*$/ }
# Start outputing the details to STDOUT
puts <<END
;; -*-coding: utf-8-emacs;-*-
;;; file-version: 6
for r in records do
r[‘nickName’] = nil           # No need for the nick names.
outs = %w{ firstName lastName nickName company }.inject(“[“) { |s, f| s + (r[f] ? “\”#{r[f]}\” “ : “nil “) }
outs = %w{ Home Main Mobile Work}.inject(outs + “(“) { |s, f| r[f] ? s + “[\”#{f}\” \”#{r[f].strip}\”] “ : s } + “) “
outs = %w{ homeEmail workEmail otherEmail }.inject(outs + “ nil (“) { |s, f| r[f] ? s + “\”#{r[f]}\” “ : s } + “) “
outs += “((creation-date . \”2009-02-08\”) (timestamp . \”2009-02-08\”)) nil]”
puts outs
# End of contacts2bbdb.rb script

A companion shell script to automate the execution of the Ruby program and also backup the previous BBDB file is also provided. Before usage, please edit the BBDB folder and file names:

# Filename:
# Description: Converts addresses from OSX to BBDB format
# Author: Anupam Sengupta (
# Maintainer: Anupam Sengupta
# (c) 2007-2016 Anupam Sengupta.
# Created: Fri May 25 15:53:22 2007
# Version: 1.0
# Last-Updated:
#           By: Anupam Sengupta
#     Update #: 740
# URL:
# Keywords: `BBDB’, ‘OSX’, ‘contacts’, ‘convert’
# Compatibility: GNU Emacs 21 and above
# Commentary:
# Runs the export of OS X Address Book contacts to the Emacs BBDB file.
# Requires:
#   1. The contacts2bbdb.rb Ruby script
#      (
#   2. The Ruby interpreter (Version 1.8 and above)
#      (
#   3. The Big Brother Database (BBDB) Emacs Package
#      (
#   4. The contacts command line program to extract the records
#      (
# Usage:
#   1. Backup the’s addresses
#   2. Install BBDB and ‘contacts’, if not already present
#   3. Edit the BBDBDIR and RUBYPROG variables below
#   4. Open and run this script
#   5. The script will also create a backup of the previous bbdb file
# TODO: Edit these variables to match the file locations
BBDBDIR=$HOME/.emacs.d/data     # Edit this to match location of the bbdb file
BBDBFILE=bbdb                   # Edit this to match your bbdb file name
RUBYPROG=$HOME/bin/contacts2bbdb.rb # Name and loction of the Ruby Script
#######################################  Do not edit beyond this point !
RUBYEXEC=`which ruby`
if [ -x ${RUBYEXEC} ]; then
echo “BBDB Export and conversion completed.”
# End of

Do remember to take a backup of your existing BBDB file as well as a backup of the OSX address-book before running the scripts!


Update [17th Aug 2009]: Emacs-fu has a nice bbdb tutorial at emacs-fu: managing e-mail addresses with bbdb


Replacing with Gmail on OSX

Off late the email client on OSX has been acting up on me. It often stalls or displays the spinning beach ball of death till a forced quit is required. Also, it seems to have display rendering issues when a mail classified incorrectly as junk is moved back to the inbox.

I tend to use the as a local email client for three main purposes:

  1. Offline mode for reading my emails when there is no Internet connection available
  2. Easy desktop search for the mails via Spotlight, and
  3. Easy archival of important mail in my project workspaces (either in Devonthink or plain text export)

The other features (TODO and Notes) are nice, but not really useful for me as I use other tools for these items. The mail itself is served out via gmail accounts and IMAP synchronization.

However, is one of the few (only?) Apple provided applications that I have a love-hate relationship with. I love the fact that it integrates seamlessly with iCal and Address Book, and the other features such as support for multiple signatures, digital signing and encryption via the GPG plugin and the smart quote while responding are quite good. the threading is also functional.

But the incessant crashes and freeze ups are getting to be more than annoying. I have tried the various cures such as re-synching the entire mailbox, rebuilding the mailbox, trashing preferences, changing the cache setting and a ton of other voodoo. These seem to be temporary solutions however, and the problems come back pretty fast. Apparently I am not alone in my suffering though – Apple’s discussion and support forums are full of unhappy users.

So I am now on the hunt for an alternative offline/local client solution. I have already tried Mozilla Thunderbird, and it looks too ugly for my tastes (though an excellent client on the Windows platform). In addition, two more deal-breakers exist right now:

  1. Getting Spotlight to work with Thunderbird requires a third party indexer which seems to be a hack. It looks like an experimental mdimporter plugin does come with version 2.0 but is flacky
  2. Missing integration with the system provided application (I need to sync the addresses with my Blackberry, and is a core part of the synchronization workflow)

Another promising solution seems to be using Gmail directly as the primary email client. This requires a few additional steps to enable a seamless offline operation:

  1. Installation of the Google Gears extension for the browser (no support yet for the latest Safari 4.x version) – this enables a copy of the emails to be stored on the local disk and allows access and usage of the gmail interface when offline – note that the first-time sync takes a long time as last 6 months worth of emails are downloaded to your computer – it is pretty peppy from then on
  2. Installation of the Gmail Notifier application which provides notifications on receipt of new emailgooglenotifiersignin.png
  3. Setting up Gmail as the default mail handler from URLs by using an option on the Gmail notifiergooglenotifierpreferences1.png
  4. Using a Site Specific Browser such as or Mozilla Prism and creation of a Gmail SSB application (remember to set a nice icon)fluid1.png
  5. Setting up the Gmail SSB as the default mail application by setting the “Default Mail Reader” option from’s general preferencesmail-appgeneralpreferences1.png
  6. Synchronizing the addresses between and Gmail using iSyncaddressbookpreferences.png
  7. Using Google Desktop for searching the mails on Gmail

Whew! Quite a bit of setup here to do – and it is still not perfect. Lets look at the pros and cons:


  1. Can use the excellent Gmail UI everywhere, online or offline – on all platforms. Hurray for Web apps!
  2. Great keyboard shortcuts – much better than
  3. Using the SSB allows a smooth integration of the mail experience with rest of the desktop
  4. No more waiting for the local client to download the mails before accessing – it is near instantaneous after the first offline synchronization
  5. No more crashes!


  1. It is a hack right now – definitely not a “download and start using” solution
  2. Google Desktop is duplicating search functionality that already exists via Spotlight
  3. Synchronization of the addresses between Gmail and AddressBook is not reliable
  4. No support for multiple signatures (can get around with a auto-typing solution such as Typinator or TextExpander)
  5. Cannot export the emails (not the contacts) from Gmail to the local computer

I intend to use this setup for the next couple of weeks to get a better feel of the system. Will follow up with a post on the findings. Do let me know what you think.

Tips for using emacsclient using Emacs 23.1 daemon mode

In an earlier post I had listed the server-mode mechanism to allow client/server editing using Emacs. However, this require some setup and also an active Emacs session.

With the latest version of Emacs (23.1), a new mechanism to invoke Emacs in a server mode has been introduced. Basically, emacs can be invoked with the –daemon option from the shell command line:

$ emacs --daemon

This will run emacs in the background as a daemon which will be listening in to emacsclient connections. To actually start a edit session, use emacsclient from the command line as before with the file name as a parameter.

A couple of new options have been added to the 23.1 emacsclient which allow the client to either

  1. Invoke a terminal editing session (via the -t option to emacsclient), or
  2. Create or reuse a GUI frame for the editing session (default behavior)

The advantage here is that Emacs can now truly act as a server (without any visible window or terminal session) and can be added to the logon/start up scripts.

Tips for using emacsclient and server-mode on OSX

Emacs is a great editor, but has one setback – it is slow to startup if you have loads of packages to load. On my G4 iBook, it takes around 45 seconds before Emacs becomes usable (OK – I use lots of packages).

However, Emacs is not meant to be restarted every so often. The canonical usage is to start Emacs once, invoke the server mode (see EmacsClient) and then use the emacsclient from the command line to invoke the editor instance when needed. I.e., the classic client/server model for editing!

Note that by default, this uses local sockets and hence the clients can connect only to the local machine’s server. However, there is an alternate mechanism to use standard TCP/IP sockets, which allows connections to remote Emacs servers. See the emacsclient info page  for details.

(Note: With the latest 23.1 version of Emacs, there is an alternate mechanism to start Emacs in a server mode – will post this in a future article).

In order to enable this feature, server-mode needs to be run within an active Emacs session (V22 and above). This can be achieved by:

M-x server-start

Alternatively, you can add the following line at end of your .emacs file:


Note that a running Emacs session needs to be present before the clients can connect.

The next step is to invoke emacsclient from the command line (or from a script) with the file name to edit.

$ emacsclient <file to edit>

This will cause a new file buffer with the requested file to be opened in the running Emacs session. The emacsclient that invoked the edit will block till the running Emacs returns control via ‘C-x #’ (server-edit)

You can use the -n option to the emacsclient invocation to prevent the blocking.

On my system, I have aliased the ‘emacsclient’ binary to be ‘ec’, which is much simpler to type.

I.e., in my `.bashrc’, I have

alias ec emacsclient

I have also defined $EDITOR to use ‘emacsclient’ when possible, and ‘vi’ otherwise. However, instead of directly assigning to the $EDITOR environment variable, I have a small shell-script to define the default editor I want to use. This script is then assigned to the $EDITOR variable.

The code for the default-editor shell-script (named ‘default-editor’) is:

# Set up a default editor for the shell
# Use Emacs if already open, and vi in other cases (including fallback)

# Note that this will set the default emacsclient, which may not be what you want
# if multiple versions of emacs are available. In that case, you can put the absolute path
# of the emacsclient that needs to be used.
# E.g., the Cocoa 23.1 emacsclient on OSX is actually located at:
# /Applications/
EMACS_EDITOR=`which emacsclient`

if [ -x "$EMACS_EDITOR" ]; then
   $EMACS_EDITOR -a vi "$@"
   vi "$@"                     # Never fails!

And then, in the .bashrc file, I have:

if [ -x $HOME/bin/default_editor ]; then
   export EDITOR=$HOME/bin/default_editor
   export EDITOR=`which vi`

BTW, the code above assumes that you use bash as the primary shell in your terminal. You can however trivially modify the scripts above for your specific shell.

Setting the PATH variable from within Emacs

A Small function to set the PATH variable from within Emacs. You can define the function and the path definitions in your .emacs file:

Very useful for OS X, where the default path is set by plists.

;; Define a function to setup additional path
 (defun my-add-path (path-element)
    "Add the specified path element to the Emacs PATH"
    (interactive "DEnter directory to be added to path: ")
    (if (file-directory-p path-element)
        (setenv "PATH"
                (concat (expand-file-name path-element)
                         (getenv "PATH")))))

and use it as:

;;; Set localized PATH for OS X
 (if (fboundp 'my-add-path)
     (let ((my-paths (list
         (dolist (path-to-add my-paths (getenv "PATH"))
         (my-add-path path-to-add))))

Using M-` from switching Emacs Frames on OSX

The standard Emacs key-binding for moving to the other frame (‘other-frame’) is C-x 5 o. That’s quite a few key-strokes, and also does not match with the standard OS X key-mapping for switching windows, which is Command-`. On OS X, the Command key maps to Meta in Emacs.

However, this is easy to fix. Put the following code-fragment into your .emacs file:

(define-key icicle-mode-map [?\M-`] nil) # Only needed if you use Icicles
 (global-set-key [?\M-`] 'other-frame) # This sets the key binding

Note that by default M-` invokes ‘tmm-menubar’ which can also be accessed via [F10]. Also, you do not need the first line (to undefine the key binding in Icicles) if you do not use that package.

Emulate a three button Mouse in Emacs on a single button Mac trackpad/mouse

Traditionally, mice on Macs have had just one button (including the iBook/Powerbook trackpads) – even through Mac OS (including OS X) can accept 2 or 3 button input without any problems.

Emacs on the other hand maps quite a few functions to the second and third chords of the mice. If you want to emulate a three button mouse in Emacs on OS X, a simple customization option will do the trick.


M-x customize-group and enter mac. Navigate to the “Mac Emulate Three Button Mouse” and select option 2 or option 3. Voila!

Note: This works for Carbon Emacs – the customization group is not available on the latest 23.1 Cocoa Emacs.

Musings on my Inbox Zero and GTD workflow

My time-management revolves around email, Calendar and to-do lists. While tools definitely help, it is more important to understand that we are processing information and acting on them, and it is just managing or tracking the items. A few pointers on what I use for Emails:


The newest computer can merely compound, at speed, the oldest problem in the relations between human beings, and in the end the communicator will be confronted with the old problem, of what to say and how to say it.

– Edward R. Murrow

Most of us are inundated with a lot of emails everyday, and the inbox keeps piling up with mails that need to be read and processed. Many of us keep these emails that have not been followed up with an action yet in our default inboxes – or squirrel them away in a maze of folders and sub-folders. Very soon, you land up with hundreds of emails competing for your precious time – and prioritization becomes difficult. This is compounded by the fact that your email inbox is now stuffed with tons of emails – which need to be re-read again before you can decide what to do with them.

However, there is a technique to handle this mess.

  1. Delete Emails! Frankly, a lot of email is never going to be used again, and has dubious value in terms of information content, or are just time-bound communication whose content is going in expire by a certain period. There is absolutely no need to keep these mails lying around and wasting space and time
  2. Never read an email twice. Scan your inbox top to bottom once per read-cycle (more to come on that) and decide the action that this email requires. Most of the time, the action is either:
    • Read and archive (the FYI kind of stuff)
    • Read and respond immediately (anything which requires less than 2 minutes)
    • Read and extract the actionable TODO (e.g. create a report, call some one)
    • Delegate to someone else (you are not the one who should be performing this action)
    • Wait for something else to happen (someone needs to work on an up-stream work item
    • The trick is to ensure that no email slips through the cracks here. You must decide the action and get out of the email reading cycle and actually do what the email triggers as work.
  3. Read email in read-cycles – Turn off the auto-notification every time an email arrives. Set a schedule (say every 45 minutes or an hour) to get into your email client and scan+process your inbox. This will help avoid a lot of distraction and interruption the email system brings
  4. Don’t create more than 5 email folders – Lets be realistic here. How many times were you actually able to retrieve the email you were looking for from the elaborate folder structure you have? Or, think about the frequency of instances where you used the “All Documents/email” view to retrieve that email.
  5. Creating folder structures is really a placebo that feels good while filing away the email but has no other practical value. I use five folders to manage my emails:
    • Action – Emails for which I need to do something that takes longer than 5 minutes
    • Waiting – Emails where someone else is going to provide inputs before I can complete the actionable task
    • Someday – Emails which are really future statements or actions
    • Read/Review – Long announcements, FYIs etc. which take time to read
    • Archive – All my processed emails go here. No hunting around for the right folder to move the email to
  6. Review the Action mail folders at EOD – At end of the day, I scan the action, waiting, someday and the read-review folders to ensure I have NOT missed out on anything for that day
  7. Store the TODO list of actions OUTSIDE the email – An email is a piece of communication, its action should be stored outside – MUAs are not task trackers. A simple text document (with a timestamp and schedule dates) suffices. Your actions and work should be driven via this file, not your inbox


How we spend our days is, of course, how we spend our lives.
– Annie Dillard

Calendars are meant for tracking your workday – not just meetings and calls. I block some personal time (in chunks of 30 – 60 min.) every day where I do not schedule meetings or calls. Early morning and late evenings are great for this.

You can also use the recurring event feature to setup reminders for yourself (e.g. “I have to send the status report on 12th of every month”).

Scheduling deadlines are also easy (e.g. “I have to complete the coding on this module by next Thursday”).

Note that the TODO file can also help with tracking these entries, as long as you enter a date against the item.


If hard work were such a wonderful thing, surely the rich would have kept it all to themselves.
– Lane Kirkland

Todos are the meat of getting the work done. In essence, todos are the actionable next steps to take something forward. The trick is to ensure that the granularity of tasks and their inter-relationships are managed. In a sense, every TODO is also a mini-project with multiple activities.

As an example, creating a report (of any kind) usually involves:

  1. Ensuring the report meta-data is known (e.g. submission date)
  2. Gathering the data
  3. Analyzing and reformatting the data
  4. Updating the report document
  5. Reviewing the document
  6. Sending out the report, and
  7. Confirming receipt of the report

As you can see, this is a multi-activity TODO, with additional level of granularity possible, along with some scheduling in the calendar. For this example, you can have a Parent TODO for the original task, and child TODOs for the next-level activities in your TODO file. For the maximum impact, assign deadlines and schedules for each of the items (nothing fancy, can be as simple as “This Friday”).The advantage this structuring provides is that

(a) the next step is always known, and
(b) you can be sure of all steps being completed and progress is visible

Another important concept around TODOs is context. To put it simply, some activities can only be done in a certain environment, or using certain tools. They are the ‘meta-tags’ which help you classify what you are doing.

For example, a call needs a Phone, reading email needs a computer or some gizmo that can access the mails; attending an in-person meeting means that you have to be physically in a work place etc.

If you can assign context around your TODOs, you can better utilize those short time-slots that become available throughout the day (e.g. waiting for a meeting to start). To take the earlier example, I would track the context as:

  1. TODO: Create the Report @WORK DEADLINE: This Friday (8/17)
  2. NEXT ACTION Find out the submission date from the boss @ANYWHERE:@CALL (DEADLINE: today)
  3. WAITING Gather the data from my team @EMAIL (DEADLINE: today)
  4. NEXT ACTION Analyse and reformat the data @WORK:@COMPUTER
  5. NEXT ACTION Update the report document @COMPUTER (DEADLINE: tomorrow)
  6. NEXT ACTION Review the document with my boss @WORK (DEADLINE: tomorrow)
  7. NEXT ACTION Send out the report @EMAIL (friday)
  8. NEXT ACTION Confirm receipt of the report @EMAIL or @CALL (friday)

The words prefixed with ‘@’ are the contexts, and help you decide if you can do it right now, or need to wait for the next step. Many of the NEXT ACTIONS can be done parallelly (not in this example though – which is a bit trivial, and too granular). BTW, if you are tracking TODOs at this level, it is better to use a custom tool rather than just plain lists in a spreadsheet. I happen to use the excellent Org-Mode mode for Emacs – though there are quite a few custom applications built for this purpose and more which do just that.

In fact, the task-tracking genre seems to be a a very popular software category on all platforms – with all the bells and whistles you could wish for. Do play around with the tool of your choice for a few days though before locking down on the one that fits your work-cadence. After all, it is one application that will be used the most through out your day.


One man’s religion is another man’s belly laugh.
– Robert A. Heinlein

These are some of the techniques I use on a daily basis to gain control on the constant stream of information and the seemingly chaotic and unplanned work that seems to turn up. It does work for me, though is definitely a custom tailored workflow that may not suit everyone.

Do let me know what you think, and how you bring in sanity to your work day!

Further Reading

  1. – Inspiration for most of the mail related tips and workflow
  2. Getting Things Done – The Art of Stress-free productivityThe book for learning GTD methods
  3. ATPO – A comprehensive list of outliners and to-do trackers for the Mac
  4. Org-Mode – The all-in-one note taking, calendaring and everything-else outliner for Emacs