Converting from TaskPaper to Emacs Org-Mode

Why TaskPaper and Org-Mode?

TaskPaper is a simple and elegant task management software for the OSX platform. It combines the simplicity of a text micro-format to mark the tasks, and the elegance of a Mac UI. It also provides a quick launch time and a nice system-wide quick entry window that is accessible with a single shortcut key.

tasks.png

taskpapercapturewindow.png

I have been a heavy user of Emacs’ Org-Mode for some years now, and love the power and flexibility it offers for tracking not just outlines and tasks, but any text based item, including notes and calendar entries. In fact Org-Mode has become one of the primary software that I use regularly, every day.

So where does the link between TaskPaper and Org-Mode come in? Both are text based, and have their own light-weight formats to define outlines and tasks. The underlying files are plain text with the meaning readable even when not viewed by the appropriate software. Org-Mode is obviously much more feature rich than TaskPaper, which by design keeps things simple.

However, there is one big difference that has led me to looking at integration: Org-Mode is Emacs based and hence takes ages to launch. TaskPaper on the other hand launches in under a second, and also offers a nice quick entry form that is available system-wide via a a global short-cut key.

Emacs aficionados will protest now – after all – Emacs is meant to be launched and never shutdown! This is true, but in my usage patterns, Emacs does get closed once in a while and having to launch it just to make a few quick Org-Mode entries (even with the excellent remember mode) becomes a pain.

My usage has now become more of the following:

  1. Use TaskPaper as an initial capture mechanism (sort of as a pre-Inbox store). I make heavy use of the quick entry window here
  2. Use a script to collect the TaskPaper entries and reformat them into a Org-Mode compatible file
  3. Append the converted entries into my primary Org-Mode Inbox whenever I have Emacs open

This (for me) provides best of both worlds – quick and ubiquitous data capture, and the power and flexibility of Org-Mode.

The Ruby Conversion Script

Without further ado, the script to convert from TaskPaper to Org-mode is:

#!/usr/bin/env ruby
#
# Converts Taskpaper files to Emacs org-mode files.
#
# Author: Anupam Sengupta, 2010
#
# Distributed under the BSD license (<a href="http://www.opensource.org/licenses/bsd-license.php">http://www.opensource.org/licenses/bsd-license.php</a>)
#
# Usage: From the command line, enter the command:
#
#  ./tpaper2org.rb &lt;taskpaperfilename&gt;
#
# The output is on STDOUT, which can be redirected to an Org-mode file.
# Whether the generated org-mode file should use odd-level prefix stars
# See <a href="http://orgmode.org/manual/Clean-view.html">http://orgmode.org/manual/Clean-view.html</a> for details.
ORG_USES_ODD_LEVELS = false

LINE_PATTERN = /^(\t*)          # Leading tabs
               -                # Followed by a dash (the taskpaper task identifier)
               (.*?)            # The task description
               ((@\w+\s*)*)     # The tags, if any
               $/x

all_tags = Hash.new(0)

Shiftlvl = ORG_USES_ODD_LEVELS ? 2 : 1 # Determine the number of stars to use in Org-mode entries

while (line = gets()):
  line.chomp!
   md = LINE_PATTERN.match(line)          # Match and extract each line
  if md then                                                # ................ A Task line
    tags = md[3].split(/ +/).reject {|tag| “@done” == tag } # get the tags, except @done tags
    tags = [‘’, tags, ‘’].flatten unless tags.empty?
    puts ‘*’ * (1 + Shiftlvl * (md[1].length + 1)) + (line =~ /@done/ ? “ DONE” : “ TODO”) + md[2] + tags.join(‘:’)
    tags.each { |tag| all_tags[tag] += 1} if tags # Keep a list of all tags
  elsif line =~/:$/                               # ................ A project line
    print “* “
    puts line.chomp(“:”)
  else                          # ................ Any other line
    puts line
  end
end

# Lets do a summary of the tags used.
puts &lt;&lt;END
# The tags used till now.
#+TAGS:#{all_tags.keys.sort.join(‘ ‘)}
END

Note that Org-mode supports multiple prefix styles with ‘*’. In particular, the odd-levels versus the odd-even levels is interesting and useful. The script has a ‘ORG_USES_ODD_LEVELS’ global variable that can be set to true if this is the desired export format.

In addition, the script also adds the tags used in the TaskPaper file as a ‘#+TAGS’ entry in the exported org-mode file. You can comment this if this is not required.

The TaskPaper format

The TaskPaper format is simple, and the file (which by default ends with the extension ‘.taskpaper’) is essentially a plain text file that can be opened and edited in any text editor.

The format can be summarized as (from the TaskPaper User’s Guide):

A project is a line ending with a colon:

      A Project:

A task is a line starting with a dash followed by a space:

    - My First Task

A Note is any line that is NOT a project or a task (i.e., does not start with a dash or end with a colon):

    Notes for a task

A tag is any word prefixed with the @ symbol. The tag can optionally have a value in parentheses after the tag name:

      - My First Task @atag @another_tag(1)

Outlining is done by indenting the tasks with tabs:

     - My First Task @atag
         - My sub-level task

Usage

Using the script is simple. Assuming that TaskPaper’s file is named tasks.taskpaper, from the OS X terminal, run the following command:

        $ tpaper2org.rb tasks.taskpaper >> tinbox.org

Where tinbox.org is the destination org-mode file.

This shell command can be put into a cron job or invoked from within Emacs to pull in the tasks as required. You may also want to delete the TaskPaper file (or empty its contents) after this is done, to prevent duplicate entries being imported the next time the Ruby Script is run.

Links

taskpaper.el is an Emacs mode for emulating the TaskPaper interface with support for projects and tasks. The tags support seems to be missing.

Discussion on the Org-Mode mailing list about Taskpaper and Org-Mode.

Advertisement

Importing contacts from OSX Addressbook to Emacs BBDB

A major aspect of using Emacs is that it is more than a text editor – it is more of an application platform – some would call it an Operating System – that allows a host of applications to run within the same context and session – and vastly enrich the user experience. This is in fact a primary reason why many Emacs users spend their entire computer interaction via Emacs.

This unique aspect of Emacs is made possible by the availability of Emacs Lisp as a first class scripting/programming interface for the editor – and has made possible application packages such as W3 (a web browser), Gnus (the NNTP and mail reader), Eshell (a lisp based shell), Wanderlust (a very capable email client), and the Insidious Big Brother Database – aka BBDB – which is a contact manager and the subject of this post.

(BTW, the list of application is only a very small fraction of what is actually available on Emacs – see www.emacswiki.org for a ton of other packages).

As EmacsWiki points out, BBDB is:

“A complete address book for nearly every email and news client written for Emacs, and its functionality supports (by default) making entries in the address book automatically when reading email or posts by people, and easy lookup of addresses while composing messages.”

In essence, it is an always available address book that works with other packages such as mail, Gnus and org-mode to provide a easy to access repository of contact information. The repository itself is a plain text lisp file which by default is stored as ~/.bbdb.

However, as with any contact manager, the problems are usually with:

  1. Initial data import, and
  2. Synchronization with other repositories and devices

For my platform (Apple OSX), the first problem above translates to data import from my primary OS level address book – the AddressBook.app. The second problem is not much of an issue for me since I use AddressBook.app as the primary contact list elsewhere and sync my devices to that. For keeping BBDB in sync, I have a cron job to automate the scripts below.

The contacts import to BBDB requires a data conversion of the address from custom format in which AddressBook.app stores the contacts to the BBDB lisp format. The steps required are as follows:

  1. Extract the records from AddressBook.app
  2. Parse and reformat the relevant fields that BBDB requires, and
  3. Output the reformatted records in the BBDB lisp format

Luckily, for step # 1, an open source command line utility already exists – appropriately called contacts. It is available from http://gnufoo.org/contacts/ and is a svelte 194KB download. This utility allows querying of the OSX Address book and extraction of specific fields and records. The output is a simple CSV text file.

For step # 2 and #3, additional scripting is required. I have written a simple Ruby script called contacts2bbdb.rb to run the contacts utility and output the BBDB repository file:

#!/usr/bin/env ruby -w
#
# Filename: contacts2bbdb.rb
# Description: Converts addresses from OSX Addressbook.app to BBDB format
# Author: Anupam Sengupta (anupamsg ... A-T ... G - M - A - I - L )
# Maintainer: Anupam Sengupta
#
# (c) 2007-2016 Anupam Sengupta.
#
# Created: Fri May 25 15:53:22 2007
# Version: 1.0
# Last-Updated:
#           By: Anupam Sengupta
#     Update #: 733
# URL: https://slashusr.wordpress.com
# Keywords: `BBDB’, ‘OSX’, ‘contacts’, ‘convert’
# Compatibility: GNU Emacs 21 and above
#
#--------------------------------------------------------------------
#
# Commentary:
#
#  Converts the addresses and contacts from Apple OSX’s system addressbook
#  format (Addressbook.app) to Emacs’ BBDB format.
#
#  Requires the following additional software:
#
#  1. Ruby V1.8 and above (http://www.ruby-lang.org/)
#  2. Big Brother Database (BBDB) package for Emacs (http://bbdb.sourceforge.net/)
#  3. The ‘contacts’ program to read Addressbook’s contacts
#     (http://gnufoo.org/contacts/)
#
# Usage:
#
#  1. Install Ruby, BBDB and contacts, if not already present
#  2. Backup the OSX Address book (export the addresses as addressbook archive)
#  3. Run this Ruby script to generate the converted records in BBDB format in the STDOUT
#  4. Save the STDOUT output of this script to bbdb.new
#
#     $ ruby contacts2bbdb.rb &gt; bbdb.new
#
#  5. Replace your .bbdb file with bbdb.new
#
#--------------------------------------------------------------------
#
# Change log:
#
#
#--------------------------------------------------------------------
#
# License: GNU GPL V2.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, Inc., 51 Franklin Street, Fifth
# Floor, Boston, MA 02110-1301, USA.
#--------------------------------------------------------------------
#
Delim = “\t”                    # Default delimiter
ContactsProg = ‘/usr/bin/env contacts’ # The contacts program
#
ContactsProgParams = ‘-H -l’
#
# A map between the LDIF field names and the actual field name
ContactFields = {
:fn =&gt; ‘firstName’,
:ln =&gt; ‘lastName’,
:c  =&gt; ‘company’,
:nn =&gt; ‘nickName’,
:he =&gt; ‘homeEmail’,
:we =&gt; ‘workEmail’,
: oe =&gt; ‘otherEmail’,
:hp =&gt; ‘Home’,
:mp =&gt; ‘Mobile’,
:Mp =&gt; ‘Main’,
:wp =&gt; ‘Work’,
}
#
ContactFormatOpts = ContactFields.keys #  Options to pass to the contacts program
ContactsProgFormat =  “‘%” + ContactFormatOpts.inject { |s, f| s.to_s + “#{Delim}%#{f.to_s}” } + “‘“
ContactsFullExec = “#{ContactsProg} #{ContactsProgParams} -f #{ContactsProgFormat}”
#
output = `#{ContactsFullExec}`.split(/\n/) # Read the output of contacts program
#
# Start parsing the contacts output
records = output.map do |line|
record = Hash.new(nil)
line.chomp.split(Delim).each_with_index { |f, i| record[ContactFields[ContactFormatOpts[i]]] = f.strip unless f =~ /^\s*$/ }
record
end
#
# Start outputing the details to STDOUT
puts &lt;&lt;END
;; -*-coding: utf-8-emacs;-*-
;;; file-version: 6
END
#
for r in records do
r[‘nickName’] = nil           # No need for the nick names.
outs = %w{ firstName lastName nickName company }.inject(“[“) { |s, f| s + (r[f] ? “\”#{r[f]}\” “ : “nil “) }
outs = %w{ Home Main Mobile Work}.inject(outs + “(“) { |s, f| r[f] ? s + “[\”#{f}\” \”#{r[f].strip}\”] “ : s } + “) “
outs = %w{ homeEmail workEmail otherEmail }.inject(outs + “ nil (“) { |s, f| r[f] ? s + “\”#{r[f]}\” “ : s } + “) “
outs += “((creation-date . \”2009-02-08\”) (timestamp . \”2009-02-08\”)) nil]”
puts outs
end
# End of contacts2bbdb.rb script

A companion shell script to automate the execution of the Ruby program and also backup the previous BBDB file is also provided. Before usage, please edit the BBDB folder and file names:

#!/bin/sh
#
# Filename: contacts2bbdb.sh
# Description: Converts addresses from OSX Addressbook.app to BBDB format
# Author: Anupam Sengupta (anupamsg@gmail.com)
# Maintainer: Anupam Sengupta
#
# (c) 2007-2016 Anupam Sengupta.
#
# Created: Fri May 25 15:53:22 2007
# Version: 1.0
# Last-Updated:
#           By: Anupam Sengupta
#     Update #: 740
# URL: https://slashusr.wordpress.com
# Keywords: `BBDB’, ‘OSX’, ‘contacts’, ‘convert’
# Compatibility: GNU Emacs 21 and above
#
#--------------------------------------------------------------------
#
# Commentary:
#
# Runs the export of OS X Address Book contacts to the Emacs BBDB file.
#
# Requires:
#   1. The contacts2bbdb.rb Ruby script
#      (https://slashusr.wordpress.com/)
#   2. The Ruby interpreter (Version 1.8 and above)
#      (http://www.ruby-lang.org/)
#   3. The Big Brother Database (BBDB) Emacs Package
#      (http://bbdb.sourceforge.net/)
#   4. The contacts command line program to extract the records
#      (http://gnufoo.org/contacts/)
#
# Usage:
#   1. Backup the Addressbook.app’s addresses
#   2. Install BBDB and ‘contacts’, if not already present
#   3. Edit the BBDBDIR and RUBYPROG variables below
#   4. Open Terminal.app and run this script
#   5. The script will also create a backup of the previous bbdb file
#
#
#
# TODO: Edit these variables to match the file locations
BBDBDIR=$HOME/.emacs.d/data     # Edit this to match location of the bbdb file
BBDBFILE=bbdb                   # Edit this to match your bbdb file name
RUBYPROG=$HOME/bin/contacts2bbdb.rb # Name and loction of the Ruby Script
#
#######################################  Do not edit beyond this point !
#
RUBYEXEC=`which ruby`
#
if [ -x ${RUBYEXEC} ]; then
cp ${BBDBDIR}/$BBDBFILE ${BBDBDIR}/bbdb.bak
${RUBYEXEC} ${RUBYPROG} &gt; $BBDBDIR/$BBDBFILE
fi
echo “BBDB Export and conversion completed.”
#
# End of contacts2bbdb.sh

Do remember to take a backup of your existing BBDB file as well as a backup of the OSX address-book before running the scripts!

Enjoy!

Update [17th Aug 2009]: Emacs-fu has a nice bbdb tutorial at emacs-fu: managing e-mail addresses with bbdb