The Other Emacsen

While Emacs remains my primary text editor, there are times when starting a full Emacs session with tons of packages is simply too slow, especially on a terminal window, and when the task at hand is simply to make a few lines of changes in a configuration file.

Yes, I do know about the handy -q command-line option, which prevents the init.el file from being loaded, thereby ensuring a sub-second initialization of Emacs, or the emacsclient route (which I have enabled, and do use), but sometimes it is just more convenient to have a fast editor that has the Emacs feel, without the bloat. And for a text nerd such as myself, it is also a matter of curiosity to try out other editors once in a while.

Emacs actually has had a rich history of variants and alternate implementations, with XEmacs being one of more well known forks. On the Mac OS X, Aquamacs has remained a good option for a number of years. A more comprehensive list is available on the Wikipedia page for Emacs.

Both XEmacs and Aquamacs are forked from, or derivatives of GNU Emacs, which is probably the most used Emacs these days. However, other light-weight Emacs-like editors still exist, and I have been trying out three of these, vile, zile, and jmacs. On the Mac, these are available via Mac Ports, on Fink, and possibly on Homebrew.

vile is an interesting blend of Emacs and vi, and provides the modal commands from vi, but also has many of the window management features of Emacs (including similar Emacs key-bindings). It is really more of a vi variant, but the window management does make it very handy, though it does not support the text objects and other vi extensions as Vim does.

jmacs (the Joe editor’s Emacs emulation) seems to be the most feature rich, and the syntax highlighting seems to be best of the lot.

Then there is zile, yet another Emacs clone that I am beginning to love for being lightweight, and having the most Emacs-like behavior. For simple text entry, zile seems to feel to be the fastest—though in reality—all three editors start up pretty fast.

In the end, while I still end up with using Emacs/emacsclient for most of my editing (after all, I do keep a Emacs session running most of the time like the true faithful), it is still fun to dabble with these editors, if for nothing else than to marvel at the core Emacs editing experience that these micro-editors can provide, in sub-Megabyte packages (vile = 680K, zile = 251K, joe = 440K on my machine).


What’s New in Emacs 24.4

Mickey Petersen has written up an excellent round-up of the latest features
of the Emacs 24.4 release.

Read the whole article at http://www.masteringemacs.org/articles/2013/12/29/whats-new-in-emacs-24-4/.

He had also written a similar article for Emacs 24.3 (current stable version),
which can be found here.

New Settings that I am using

I have already setup the load-prefer-newer, cycle-spacing, and ns-use-srgb-colorspace options.

The new electric-pair-mode options also look interesting, and I have set these
up (I was using skeleton-pair for the auto-pairing function till now).

The new M-s . (isearch-forward-symbol-at-point) is going to be very useful. It
essentially replicates what C-s C-w does, but having a single key-stroke is a
little easier. A closely related command M-s h . (highlight-symbol-at-point)
also looks to be useful. This is especially handy with the
hi-lock-auto-select-face option.

C-x SPC (rectangle-mark-mode) finally allows replacing CUA for me.

Eshell now has even better support for visual commands; i.e., commands
which expect a terminal. See the options eshell-visual-commands,
eshell-visual-subcommands and eshell-visual-options for details.

New Interesting Modes

The new Web Browser for Emacs (eww) is great! I have already
uninstalledw3m and using eww solely right now.

The dired-hide-details-mode is also nice. It has already replaced
dired-details for me.

superword-mode looks to be useful (especially for programming Ruby, where
underscores are usually used to separate words in an identifier).

The new file notification system would be very useful, except that it does
not (yet) work in OSX (bummer).

Other Default Settings of Interest

desktop-auto-save-timeout is going to be a life-saver. The related option for
desktop-restore-frames finally does what I want.

The new electric-pair settings electric-pair-preserve-balance,
electric-pair-delete-adjacent-pairs and
electric-pair-open-newline-between-pairs do exactly what is expected.

Ruby-mode has some interesting updates, especially for indentation.


Enabling Postfix on OSX as a Relay: Revisited

Postfix on OSX: Revisited

A few years back, I had written a post on enabling the Postfix MTA as a relay server on OSX, which was quite well received. The article was originally written for OS X Lion, though it remained valid for OSX Mountain Lion, and more recently on
OSX Mavericks as well.

However, things have changed on the OS since the last two versions, which warrants a revisit of the instructions, and a refresh of some of the steps. While the old set of instructions still work, there are changes needed that will keep the configuration future-proof.

The key changes since the article was posted (in February 2012) are:

  1. The launchd configuration has changed somewhat, and a few elements of the launchctl plist file are now deprecated. The old configuration in the org.postfix.master.plist needs to be refreshed
  2. The old configuration depended on local mail-drop in the Postfix queues for the MTA daemon to be started. While this is quite correct (this is Unix, after all), there is another common scenario where the daemon is launched when a client connects to the SMTP ports (default port 25) on the
    local machine.

The Updated launchd Configuration

We would like to retain running the daemon on a mail-drop in the queue, but also enable running the MTA when a connection is made to the SMTP port (default 25). The updated org.postfix.master.plist configuration file listed below does this for us.

In addition, the redundant and deprecated plist tags have also been removed.

You can replace the existing copy with this version as a drop-in replacement at /System/Library/LaunchDaemons/ by using the following shell commands in Terminal.app (assuming you have downloaded the new version at ~/Desktop):

$ cd /System/Library/LaunchDaemons/
$ sudo cp org.postfix.master.plist org.postfix.master.plist.old
$ sudo cp ~/Desktop/org.postfix.master.plist .

The actual configuration file:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
        <key>Label</key>
        <string>org.postfix.master</string>
        <key>ProgramArguments</key>
        <array>
            <string>/usr/libexec/postfix/master</string>
            <string>-e</string>
            <string>60</string>
        </array>
        <key>Sockets</key>
        <dict>
            <key>Listeners</key>
            <dict>
                <key>SockServiceName</key>
                <string>smtp</string>
                <key>SockType</key>
                <string>stream</string>
            </dict>
        </dict>
        <key>QueueDirectories</key>
        <array>
            <string>/var/spool/postfix/maildrop</string>
        </array>
        <key>AbandonProcessGroup</key>
        <true/>
    </dict>
    </plist>

Once the file has been copied into the correct location (perhaps by using sudo), we need to reload the configuration into launchd:

$ cd /System/Library/LaunchDaemons/
$ sudo launchctl unload org.postfix.master.plist
$ sudo launchctl load -w org.postfix.master.plist

This should be sufficient to reload the new configuration.

Testing out the configuration

We can test the changes via the shell itself by trying to connect via telnet to the smtp port and trying out a few basic SMTP commands (HELO for checking a response, and QUIT to logout of the SMTP session):

$ cd ~                                  # Lets get back to $HOME directory
$ mail &lt;your_id&gt;                        # Send yourself an email to check mail-drop
$ telnet localhost smtp                 # Now lets telnet into the server
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 localhost.local ESMTP Postfix
HELO
501 Syntax: HELO hostname
QUIT
221 2.0.0 Bye
Connection closed by foreign host.

In the sample SMTP session above, the third line (starting with “220” response) indicates that the Postfix MTA is running, and is ready for accepting mail drops via the SMTP port (as opposed to the mail-drop via the mail command).


Copying the previous line to current position in Emacs

Recently, I came across a scenario where I had to quickly copy the previous line in an Emacs buffer to the current position. The usual method for doing this has been to invoke:

C-p C-a C-k C-y RET C-y

Which basically does the following:

  1. Moves to the previous line (C-p)
  2. Moves to the beginning of the previous line (C-a)
  3. Kills the line (i.e., cuts the line with C-k)
  4. Yanks the line back (i.e., restores the original line with C-y)
  5. Creates a newline (RET), and
  6. Yanks another copy of the line with the final C-y (which is what we wanted in the first place)

While this works, it is certainly not the most optimal mechanism to perform such a simple (albeit infrequent) operation.

In searching the Net, and scanning the inbuilt functions using apropos and the excellent Icicles search, I stumbled upon the:

copy-from-above-command

function, which does exactly what is needed here. This function is available in misc.el, and needs to be enabled by requiring the file to be loaded in .emacs:

(require 'misc)

Also, the there is no default key-binding for the function, and one needs to be setup:

(global-set-key (kbd "H-y") 'copy-from-above-command)

In my case, I set up the Hyper-y key-stroke for the key, which nicely mirrors the yanking keys (C-y and M-y).

This function has another nice twist, which is that it will copy the characters from the previous line only from the current column, which allows partial lines to be copied over.


OS X Mountain Lion: Need to reinstall Xcode command line tools

So I upgraded to OS X Mountain Lion yesterday. The install was pretty smooth (though had a tough time with the redeem code for the upgrade, worked on the third code that Apple sent across).

Most of the software is working fine. However, I found that my existing Xcode installation needed an upgrade (via a Xcode install from the App Store).

The problem is that any new Xcode install seems to nuke the command line development tools (think make, autoconf, etc.). The solution is pretty simple though. Once Xcode has been installed, run Xcode and then open the preferences. There is a section on downloads, which lists installing the command line tools as an option.

downloads-2012-07-27-11-51.png

Voila! Issue resolved.


Enabling postfix for outbound relay via Gmail on OS X Lion (and newer OSX versions)

Update on Oct 25, 2014: Updated For OS X Yosemite.

Update on Dec 21, 2013: I have posted an update to the launchd setup for postfix. You should still read through this post, as most of the setup remains common to both posts.

The background

Mac OSX comes with the postfix MTA, which is a fully featured SMTP server. Under normal circumstances, there is usually no need to enable or configure this software, as most email access is usually done via GUI clients such as the Mail.app – which uses the POP/IMAP and SMTP settings to connect with the email service provider.

However, there are certain circumstances in which having a local SMTP server is very useful, such as:

  1. Allowing the batch logs and output from the cron daemon or other scripts to be sent via Internet email (this is otherwise delivered locally)
  2. Testing email based code; which requires a local sendmail like SMTP server to be present

For such use cases, the postfix server is ideal, as it provides all the features needed (and much more), and is also a nice drop-in replacement for the sendmail program.

While postfix can be used as a full-fledged SMTP server that connects directly to the mail-servers on the Internet, for the use cases above, it is usually better to redirect (i.e., relay) the emails via an authenticated and known server (such as Gmail), as this helps avoid a lot of constraints around open-relays, which are mostly blocked these days to prevent email spam.

Note that configuration of postfix does require dropping down to the command-line, and fiddling with system files. While not complicated, it is definitely not for faint of the heart (though much easier than configuring sendmail).

What you need to know (pre-requisites)

Some of the basic pre-requisites are:

  1. Understanding of the shell prompt and the Terminal.app program
  2. Usage of the sudo program (all the configuration files are owned by root, and hence usage of sudo is essential)
  3. Usage of any command line editor such as vim, Emacs, nano, or any other editor of your choice, that can be invoked with super-user rights (usually via sudo)
  4. A basic understanding of the Apple launchd service manager
  5. The configuration files
  6. A Gmail email ID (actually, any SMTP server credentials will do)

While this article will go step-by-step with the configuration process, knowledge of the above will allow a deeper understanding of the “why” for the changes done.

In the steps below. the $ character before any command represents the shell prompt. Also, I will assume usage of the vim editor in the steps below.

The configuration Files

The configuration files that will be changed are:

 
Name Location Purpose
org.postfix.master.plist /System/Library/LaunchDaemons launchd Configuration for postfix
main.cf /etc/postfix The main postfix configuration
aliases /etc/postfix Local recipient aliases
generic /etc/postfix Sender aliases (for external mail)
passwd /etc/postfix/sasl Relay host authentication

Note that the “/etc/postfix/sasl” directory might not exist, in which case, we will need to create it from the shell prompt:

$ sudo mkdir /etc/postfix/sasl

Step 1: Update the launchd configuration

Update Dec 21, 2013 : While this setup still works, you might want to also see an alternate configuration of postfix’s launchd setup, which I have documented in a follow-up article. The new configuration also allows postfix to be launched when network activity happens on the local SMTP port 25.

The org.postfix.master.plist file located at /System/Library/LaunchDaemons/ is used to start or stop the postfix program on demand, as and when any email is submitted to the mail system for processing. The basic Apple setup is fine, but may need a little tweaking (in my case, the file had a couple of tags which prevented postfix from being started.)

We need to edit the file (as a super user) to match the following content:

$ sudo vim /System/Library/LaunchDaemons/org.postfix.master.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>org.postfix.master</string>
        <key>Program</key>
        <string>/usr/libexec/postfix/master</string>
        <key>ProgramArguments</key>
        <array>
                <string>master</string>
                <string>-e</string>
                <string>60</string>
        </array>
        <key>QueueDirectories</key>
        <array>
                <string>/var/spool/postfix/maildrop</string>
        </array>
        <key>AbandonProcessGroup</key>
        <true/>
        <key>OnDemand</key>
        <true/>
</dict>
</plist>

Step 2: Edit the /etc/postfix/main.cf file

The next step is to edit the main configuration file for postfix. Do make a backup of the current file before editing.

$ cd /etc/postfix
$ sudo cp main.cf main.cf.orig
$ sudo vim main.cf

Note that the main.cf file is a pretty large one, and has a lot of commented out sections, which should be left as is. Please add the following lines at end of the file.

# Set the relayhost to the Gmail Server.  Replace with your SMTP server as needed
relayhost = [smtp.gmail.com]:587
# Postfix 2.2 uses the generic(5) address mapping to replace local fantasy email
# addresses by valid Internet addresses. This mapping happens ONLY when mail
# leaves the machine; not when you send mail between users on the same machine.
smtp_generic_maps = hash:/etc/postfix/generic

# These settings (along with the relayhost setting above) will make
# postfix relay all outbound non-local email via Gmail using an
# authenticated TLS/SASL session.
smtp_tls_loglevel=1
smtp_tls_security_level=encrypt
smtp_sasl_auth_enable=yes
smtp_sasl_password_maps=hash:/etc/postfix/sasl/passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_mechanism_filter = plain

Step 3: Edit the /etc/postfix/aliases file

We need to make a minor edit here, to allow mails sent to the root ID to your local user mailbox.

$ cd /etc/postfix
$ whoami                # This will provide your local user name
$ sudo cp aliases aliases.orig
$ sudo vim aliases
$ sudo newaliases

Find the line in the file which is:

#root:              you

and replace the “you” with the username provided by the whoami command above.  Also, remove the “#” from beginning of the line.

Remember to run the newaliases command (the last command above), or else changes will not take effect!

Step 4: Edit the /etc/postfix/generic file

This file maps the local user address (usually of the form yourid@machine.local) to a valid Internet email address you would like to use when sending mails to the outside world. In our case, it would basically map your Unix user name to the Gmail ID.

$ cd /etc/postfix
$ whoami                # This will provide your local user name
$ hostname              # This will provide your machine name
$ sudo cp generic generic.orig
$ sudo vim generic
$ sudo postmap generic

In the file, add the following lines at the end of the file (replacing the <username> with the output of the whoami command, and <machinename> with output of the hostname command):

# Translate my primary email address to the Gmail address
# This is ONLY for the outbound email, and does not apply to
# local email.
<yourusername>@<machinename>  <your gmail ID, e.g. user@gmail.com>
@<machinename>                <your gmail ID, e.g. user@gmail.com>

Remember to run the last command (postmap) as otherwise the changes will not be picked up!

Step 5: Edit/Create the /etc/postfix/sasl/passwd file

In this step, we store the SMTP authentication (user ID and password) for Gmail, so that postfix can connect as any other SMTP client to Gmail via an authenticated session.

Note that the file may not exist prior to this step, in which case we will create it.

$ sudo mkdir -p /etc/postfix/sasl    # In case the directory does not exist
$ cd /etc/postfix/sasl
$ sudo vim passwd
$ sudo postmap passwd

Create the following file, replacing <gmailusername> with the ID you use for Gmail (with the “@gmail.com” added at the end), and <gmailpassword> with the password you use to login to Gmail.

[smtp.gmail.com]:587    <gmailusername>:<gmailpassword>

Note that if you use two-factor authenication with Google, then the password to use will be a new application specific password generated via Google’s account settings.

Final Step: Test the settings

We are now good to go. Lets test our settings from the terminal:

$ cd /System/Library/LaunchDaemons
$ sudo launchctl load -w org.postfix.master.plist
$ cd ~                             # Just to be safe, move to your home directory
$ mail <your_id>  # Output of the `whoami' command
# Type in a test email and hit Control-D on a new line
$ mail
# Check whether the email has arrived. Hit 'q' on the '?' prompt to quit

$ mail <your gmail ID>       # Lets now try to send an external mail.
# Type in a test email and hit Control-D on a new line

After the second step above, check your Gmail account for the test mail. If it has arrived, then we have a good configuration.

Summary

Setting up the postfix system on OSX is not particularly hard, but does require some steps. Also, this is just the basic setup to get things up and running. Postfix is an industrial strength mail server has a lot of features (and a corresponding number of configurations). Thankfully, the documentation at http://www.postfix.org/documentation.html is pretty good.

For more details on this specific setup, additional documentation is available at http://www.postfix.org/SOHOREADME.html.

[Updated on 19th Feb 2012]: Corrected a typo.  Thanks to jamrok for pointing it out.


Heretical Confessions of an Emacs Addict – Joy of the Vim Text Editor

Context

As an experiment, I have recently started using Vim as my primary text editor. While I have been an Emacs aficionado for a very, very long time – clocking in at almost 16 years – Vim is something that I have always been curious about, and tinkered with from time to time, while ending up going back to Emacs. This time however, I intend to stick around for a while, and try to get a feel of the Zen of Vim.

The Emacs and Vim Icons

The experience with Vim has been rather pleasant so far. While  configuration is definitely a must (just like Emacs), out of the box experience is not bare bones either. In fact, the default settings are rather good, and one can get a lot of mileage from the vanilla setup.

The USP of Vim is that it is first and foremost a text editor, and does not attempt to be THE kitchen sink. In other words, its core focus is to provide the best and most functions to manipulate text; it is not to provide a universal platform where one of the applications happens to be an editor.

The Editing Model

The very first experience the user gets on launching Vim is that it is a modal editor, with distinct modes for entering text, and editing it.

This is in stark contrast to most other editors, which provide the editing functions via other mechanisms, such as menus or key chords (à la Emacs). While this might seem odd at first glance, it is definitely a key component of Vim’s power, and arguably its success as a text editor.

In addition, the second key distinguishing factor of Vim is that its editing is based on a “noun-adjective-count-movement-verb” model of editing. Nearly every editing command is a sequence of one or more of these primitives, where the intent of the edit operation is usually fully mapped to the encoding offered by various combinations of these primitives.

As an example, to move 5 lines down, and then 3 words forward, to delete the next two words, the following “primitive encoding” can be used:

      5j 3w 2de 

While this is a rather trivial example, it illustrates a key principle of using Vim – the editing model is based on MOVEMENT and MANIPULATION, explicitly and succinctly made via the modal key-chords which apply the same basic foundational model at all scales.

To a large extent, this underlying pair is what the user usually is thinking about, when an editing operation is being thought of. The primitives are just the vocabulary to tell Vim to execute the intent.

The movement itself can be further separated into raw movements (i.e., lines and characters), or be semantic oriented (words, sentences, paragraphs, functions, etc.) This can sometimes be further refined using “adjectives” such as “begin”, “end”, “between”, or “surrounded”. In addition, the count represents multiples of the movement unit being specified, and allows velocity of the movement to be increased significantly.

The verb is of course where the “real action” happens. In a general sense, every verb is really a “change” verb, with specializations of “delete”, “replace”, “convert”, etc.

Conveniences

In essence, the core feature set of Vim (and certainly its spiritual ancestor – Vi) can be adequately expressed by the core model described above.

However, a basic feature does not make a competent editor. This requires other user comforts such as multi-file edits, windowing mechanisms (including split windows), language syntax, external tool integration (spell checks, compilers, shell interaction), extensive customization to fit the user’s need, an usable in-built help system, and many more such features that make regular usage comfortable and transparent. And Vim does have all of these in abundance, with parity with Emacs in most of these areas in terms of feature completeness, and power.

Granted, Vim does not yet have an inbuilt News Reader, or an email client, but the core editing functions are very much there, or easily added via plug-ins.

Conclusions

The feeling I get from using Vim is really about efficiency and focus, while Emacs seems to exude more a sense of power. Both can be frustrating at times, often from over-abundance of features than anything else, and hidden functionality that takes years to internalize before returns are gained.

Vim and Emacs are both excellent editors, and share a lot in common. The complexity, feature-richness, and the high learning curve are the ones that stand out the most. Also, the vibrant community around both editors is a shared characteristic.

Editor wars aside, both are very competent and complex pieces of software, and will serve the user well, provided an investment is made in learning the unique interfaces that both exhibit.

As for me, I do not anticipate leaving the Emacs bandwagon any time soon — especially with significant investments now in the Orgmode work-flow that I have built up over the past few years. However, I do see Vim as another pro-tool in my toolbox, which is definitely going to get a lot more love and use in the days ahead.


Follow

Get every new post delivered to your Inbox.

Join 81 other followers