Eshell as a main shell

Updated 2017-10-18

This article was initially posted on Reddit. I’ve revised some arguments by taking the community feedback into account.

Since its inception, Eshell has suffered from an arguably low popularity. See what the community says:

Hell, even the official manual gives a fair warning against its use:

I’m not arguing that those arguments are not valid. (To most extent, they are.)

I want to argue that the issue is not so much that Eshell is not a proper Unix shell (in the sense of POSIX sh and derivatives), but rather perhaps we do not need a Unix shell and all its historical limitations.

In other words: the non-Unix-ness of Eshell is more of a step forward than a hindrance. Let’s review the situation.

“State of the art”

Take this typical scenario: I’ve got a long running process which finally outputs pages of relevant information. Only then I realize that it would have been smart saving that somewhere. Now I want to copy it to the clipboard. How do I do that?

By clumsily mouse-selecting pages of terminal output, assuming my terminal can handle it.

Why can’t I (fuzzy-)search the shell output?

Why can’t I browse the different prompts of my shell session?

Why can’t I copy/paste without a mouse?

Back in the days, VT-like terminals were our main mean of communicating with a machine. Decades went by, our desktop computers can now handle gigabytes of buffering and display in 24-bit colors, and yet we still stick to terminal emulators, that is, programs that emulate the restrictions of those ancient machines.

Say I run this command:

$ cat file-list.txt
/path/to/foo
/other/path/to/bar

Now I want to see what’s in foo. How do I do that?

  1. Either I mouse-select the file, hoping I’ll not be off-by-one on one side. Then I paste to the prompt.

  2. Or I type the path myself, with a bit of luck completion will bring me there soon enough.

  3. Or I resort to re-call the command from a subcommand and program some output manipulation to extract the filename and then open it with my editor. This is not nearly universal nor even convenient.

All this is terribly backwards and it is high time we moved on.

Terminals vs. shells

It’s important to understand that shells are not (or should not be) semantically bound to terminal emulator restrictions. Shells are a textual interface to the machine. They just need input, evaluation, execution, output. GTK/Qt/Tk/etc. are absolutely capable of handling such a task. Not mentioning Emacs…

So why do shells de facto need a terminal? One thing: curses-like interfaces. Those libraries need terminal capabilities to render. Do we still really need that?

Enter Eshell

Back to my initial example of the long-running process: want to copy that output somewhere? 3 key-strokes with Eshell. And better: could be even 1 if you want to program it your way. The interface is extensible. (Terminals usually aren’t, URxvt and its Perl scripts are not a proud example of extensibility.)

Eshell breaks up with the “terminal-interface” paradigm: the user interface of the Eshell commandline-interface is Emacs it self. The pager is the Emacs buffer. The input is Emacs keystrokes. The extension language is Elisp. Consequences:

Finally, some Helm niceties:

In terms of extensibility, it’s key to understand that (as of Emacs 26) all prompts are properly identified by Emacs. This opens the door to infinite programmability of the shell interface:

There is more to come

It’s undeniable however that, as of Emacs 25.2, Eshell has numerous rough edges. Quite a few bug reports have been sent on debbugs and it’s getting better.

Eshell’s parser has some bugs here and there. It will most notably fail at parsing some sed expressions. Which is not as dramatic as it sounds since you don’t need sed quite as much when you have Emacs Lisp with all regexps functions you can think of, coupled to fuzzy-matching.

Native completion is limited considering very little effort was ever made. The community can help changing that. In the meantime, I’ve implemented a fallback to Bash/fish completion so that I can keep completing just as much as I could with Bash or fish.

Need that curses-based program? Get a (much more) powerful Emacs alternative:

As opposed to their curses-like counterparts, Emacs alternatives share all those features:

In the meantime, there might still be that annoying curses-based program you cannot replace. Eshell allows you to run those programs by transparently switching to a *term* buffer. See eshell-visual-commands.

Curses-based programs are not the only ones that won’t run well on Eshell: REPL are known for working poorly too. REPLs are poorly designed though: from a Unix-philosophy standpoint, only the E is required for every language, while re-implementing the R, P, and L, i.e. the user interface, for each one of them is simply re-inventing the wheel. Instead of that, Emacs has comint-mode, an REPL-lie user interface that can be plugged to any language. With all the powerful features of Emacs that all those specialized REPLs are lacking. For a start, you can write code in Emacs and execute any piece of code from a single keystroke. It’s configurable and extensible in a consistent manner across all languages.

Similarly, “environment setups”, as commonly used by programming language tools such as Python’s virtualenv or Ruby rvm, expect the user to be running a specific shell. This is broken by design. Let me quote /u/rpdillon as I could not have said better:

To OP’s point, though, yes, these environments do make assumptions that you’ll be using some kind of bash-compatible shell, which tells you a little bit about how sub-optimal the state-of-the-art is. I suppose the only alternative I can think of is to have the language itself manage many versions of itself so that every developer’s environment wouldn’t have to be customized/extended to negotiate the maze.

The Go programming language for instance does not have such a tool as I know of.

A long-standing critique of Eshell is that it does not support input redirection. This is hardly a major hindrance since you can always use a work-around such as

cat file | command

Another common argument against Eshell is that it cannot run the widely popular Tmux. The latter is used for many things, out of which:

All that being said, should Eshell not do the job for you, then maybe give M-x shell a try. It has a similar design: it is a shell emancipated from any terminal; instead of being Lisp based, it runs an existing shell as an interpreter. Bash and zsh are known to work. It has all the advantages an Emacs buffer has over terminal emulators. The main difference then lies in the language: Emacs Lisp vs. Bash and related languages.

An open door to Emacs-Everywhere

Switching to Eshell marked a milestone for me: from then on I dropped all my crufty curses-based programs and switched to much more powerful Emacs alternatives. I now use Emacs everywhere to the point that it even is my window manager. Having a consistent environment well glued together really empowers you.

Ironically, the environment is so powerful that I ended up not using any shell so much anymore, only for a few commands here and there. For which I use Eshell, obviously.

Eshell might not be perfect (yet), a little beta perhaps, but this can only be helped by a growing community: the more people join, the more feedback developers get, the better it will become. Hopefully this pamphlet will help raise your curiosity, if not more esteem for this brilliant project.