Emacslisten

Emacslisten is a set of packages for controlling Emacs by voice. It provides an interface for defining command languages, as well as a default command language for performing different tasks in Emacs.

History

Emacslisten started development in 2024, when Erik Präntare suffered repetitive strain injury (RSI) from hacking too much. Existing solutions for voice control were fine, but weren't well integrated with Emacs and the Emacs way of doing things. The goal of Emacslisten is to fix this issue, and make voice control feel like a first class citizen in Emacs-land.

The packages

Emacslisten is currently comprised of three packages, each solving a different aspect of controlling Emacs by voice. Each may be used independently, but they work best in tandem.

Phony

Phony provides an interface for defining voice commands. With phony, the following code defines a voice command such that saying "hello emacs" will insert "And hello to you too!" in the buffer:

(phony-defun hello "hello emacs"
  (insert "And hello to you too!"))

Commands can become much more sophisticated than this. See phony for more.

Warning

Phony currently only works well with Talon, a proprietary (but gratis) speech recognition engine. There is a libre dragonfly backend packaged with phony that kind of works for simple languages. You might try it out if you just want to experiment with defining some commands, but for the complex language provided by Emacslisten, Talon is currently required.

Any contributions for getting a libre speech recognition engine up and running are appreciated!

Go here for more information on libre backends and contribution.

Cursorfree

When controlling the computer by voice, the roundtrip time between the user doing something and the computer acting is much longer than when using hands. But voice has the advantage that it is more natural for composing complex commands.

Cursorfree defines generic and composable operations for things like editing text, navigating buffers, and managing windows. With phony installed, cursorfree also provides a command language using these operations.

Example: With the following text, point at |:

Emacslisten provides voice control primitives.
Also a ne|at language.
And some fancy graphics!

By saying "chuck word" you get:

Emacslisten provides voice control primitives.
Also a |language.
And some fancy graphics!

If you instead say "chuck line", you get

Emacslisten provides voice control primitives.
|And some fancy graphics!

Similarly, you can copy the word or line by using "copy word" or "copy line".

The language allows you to mix and match operations to compose much more complex commands in one utterance. The language is designed to be extensible, allowing you to define new (potentially mode-specific) operations.

See cursorfree for more.

Hatty

The longer input latency means we need a faster way to refer to precise locations in the buffer. Hatty provides this by indexing each word with a unique character-shape-color combination. This is best illustrated with an image:

Cursorfree integrates hatty, allowing you to perform operations at locations without needing to move point there first. This allows cursorfree to define operations like "bring A to B" for replacing region B with the text in region A.

Here is a video demo of hatty.

See hatty for more.