moreutils: A collection of useful command-line tools

April 15th, 2007 edited by Tincho

Entry submitted by John Beisley. DPOTD needs your help, please contribute !

Moreutils contains a suite of utilities for command-line users. Written by multiple authors, they are individually quite modest, and perhaps too limited in scope to exist in a package in their own right, but together they form a useful accompaniment to existing command-line tools.

Following are a few examples of the utilities in moreutils, and how they can be used.

Sponge - Soaking file redirects

Many users of the command line have made a mistake such as the following:

$ grep -v someuser /etc/passwd > /etc/passwd

As the shell runs this command it will immediately truncate the destination file, and then run the grep command on it. The end result is an empty file - certainly not what was intended! After learning from their mistake, the user might instead redirect the output to a temporary file, then move the file on top of the original:

$ grep -v someuser /etc/passwd > /etc/passwd.tmp
$ mv /etc/passwd.tmp /etc/passwd

Sponge is a simple, but convenient, command that exists to roll this process up into a single step:

$ grep -v someuser /etc/passwd | sponge /etc/passwd

Sponge ’soaks up’ its standard input, waiting for it to complete before writing to the specified output file. In this way the result is what was intended, and without the need to fiddle with temporary files.

Vipe - Interactively editing pipes

Sometimes it is desirable to put yourself in the middle of a pipe processing chain, where it will be quicker to just edit the pipe content, rather than create complex filters with sed and friends. Maybe the pipe content will be unknown, perhaps a list of files that the user would like to manually filter.

Say that you would like to touch all files in the current directory, but would like to do so selectively, you could do the following:

ls | vipe | xargs -d ‘\n’ touch

In this example a list of files is opened up in your editor (as set by either the EDITOR or VISUAL environment variables), in which you may edit the list of files to be touched at your leisure. When you are satisfied, you can save the file and quit your editor, and vipe will regurgitate the content you edited to its standard-out - touching only the files retained in the editor.

Combine - Boolean/Set operations with text files

Performing set operations with lines in two text files can also be useful. This can be useful in knowing what file names are common to two directories, for example:

Directory foo contains the files:

tom and harry
Directory bar contains the files:

tom and dick

The file lists can be written to two files ready for use with combine with the following bash command:

ls /path/to/foo > foo_list && ls /path/to/bar > bar_list

It can be complicated to find which files are present in both directories, which is where combine really comes into its own:

$ combine foo_list and bar_list

How about files that are in foo, but not in bar?

$ combine foo_list not bar_list

A note of caution, however: while using combine, ‘or’ can be used to find the complete list of files, it has the slightly odd effect of listing files common to both files twice:

$ combine foo_list or bar_list

Obtaining the union of the two lists of file names is therefore best performed with the standard sort command:

$ cat foo_list bar_list | sort -u


This introduction to moreutils has only touched upon some of the commands that it provides. It is worth experimenting with the others, very briefly described here:

Checks if a file consists of valid UTF-8
Reproduces standard input on standard output, with a timestamp prefix - good for annotating non-timestamped log files.
Enables a user to “edit” a directory with their text editor. That is, to conveniently delete and rename files.
Outputs requested information about a specific Linux network interface, without the need to parse ifconfig output.
Acts like tee, but pipes to given commands, instead of standard out and files.
Automatically recognises compressed files passed to a given command, uncompresses them to a temporary file and passes the uncompressed files in place of the original compressed file arguments.
Connects two commands with a pipe, as in command1 | command2, but returns the exit value of the first command, rather than the second.

moreutils has been available in Debian in testing and unstable distributions, and in Ubuntu from Edgy onwards.

Posted in Debian, Ubuntu |

5 Responses

  1. Samuel Bucquet Says:

    I just tried ‘vidir’ and lost 9 files in the directory with the message : “/usr/bin/vidir: unknown item number 01
    /usr/bin/vidir: unknown item number 02
    /usr/bin/vidir: unknown item number 03
    /usr/bin/vidir: unknown item number 04
    /usr/bin/vidir: unknown item number 05
    /usr/bin/vidir: unknown item number 06
    /usr/bin/vidir: unknown item number 07
    /usr/bin/vidir: unknown item number 08
    /usr/bin/vidir: unknown item number 09″
    Is there a way to recover them, I did not suppress the line refering to them during the editing session ?

  2. Anonymous Says:

    Samuel, I think you’re out of luck. Maybe try something like undelete.

    I didn’t quite get the last part of your question, but if you’re wondering how it went wrong:
    Your editor automatically padded the line numbers with zeros, which confused the perl script. And instead of quitting and saying “unable to do anything due to confusion” it just goes ahead and deletes whatever lost it’s line number.

    It’s still a useful program though, once you know about this.

  3. Pádraig Brady Says:

    You can define an alias for isutf8 like:
    alias isutf8=’iconv –from=utf8 –to=utf8 >/dev/null 2>&1′

    For set operations using the standard coreutils, please see:

  4. b4hand Says:

    combine is just syntactic sugar for the join command. See man 1 join, which is already part of GNU coreutils. Note that join only works on input that is already sorted. The join command even lets you work with arbitrary fields as opposed to assuming that you want to match entire lines.

  5. Troff Says:

    Strangest thing (sorry, I’m new here). Tried to apt-get update and install, told me the package didn’t exist. Downloaded it manually from a Debian mirror linked from the Deb search/description page and dpkg -i, worked fine.