Pipes are a very powerful core feature of Linux. By chaining programs together, you can carry out complex tasks with a small set of tools. But piping isn't always the best solution; when it's not needed, it can increase complexity and lead to less efficient options.
Many Linux commands have options that let them do the job of another command. For example, ls can sort its own output, even though the sort command exists. Find out which common pipelines have better alternatives and how you can unlock them.
grep | wc
The pipeline is common, but grep has its own counter
This pattern is very common, partly because it's such a good demonstration of wc. The "word counter" tool , which can also count lines and characters, fits well in many pipelines, because you'll often care more about the number of results than what they actually are:
While wc does a perfect job of counting the number of results, grep can do that itself using the -c or --count option.
The -c option has, apparently, been a feature of grep from its very earliest days, so there's never really been any excuse for this pattern to take hold.
cat file | anything
A useless use of cat that can easily be avoided
Another incredibly common use of a pipe that can be avoided. The cat command is useful because, although it stands for "concatenate," it doesn't have to join anything together at all. By default, it writes a file to standard output, even if it's just a single file on its own.
This means that it's often tempting to reach for cat when you want the contents of a file passed as input to another command:
But a command like wc will read input from each file argument you pass it, so piping from cat isn't necessary:
But what if a command doesn't support file arguments? Then you have to use cat, right? Actually, no: Linux has a built-in file-redirection operator that means cat is unnecessary, even in these cases.
It takes a bit of getting used to, especially when piping cat is in your muscle memory, but learning to redirect effectively will open up a whole bunch of command-line opportunities.
ls | sort
A built-in sort brings new benefits
The sort command is another that's perfect for piping: it's a filter that, as you'd expect, sorts all the input it receives and sends it to standard output:
So, for commands like ls, which can produce output in a near-tabulated form, it can be very useful. In fact, sort's -k property, which specifies the number of a field to sort on, makes it very versatile for this purpose.
Still, there's no actual need to use it, at least not in the case of ls, which supports its own option to sort on file sizes:
The -r option reverses the direction of the sort, to show the largest files at the bottom, which is probably what you want.
Using the sort built into ls has another small, but important, benefit. When you pipe ls output to sort, you lose any color formatting, but the sort built into ls maintains it.
head | tail
Extract a range of lines with a single command
The head and tail commands are great for extracting the first or last n lines of input, respectively. Combined, they're a neat trick to extract a specific range of lines from a file:
It's clever, but you can do the same thing with sed, the stream editor :
Note that both this command and the previous pipeline fetch lines 102-105 from the file, but the sed version requires less math.
grep | head
Keep large result sets manageable and save grep the extra work
Sometimes your grep has one result, sometimes it has hundreds or more. When you've got a lot of grep output to handle, it's natural to reach for the head command once more, to stop after a certain number:
This is unnecessary, though, thanks to grep's -m (--max-count) option, which does the exact same thing:
sort | uniq
A classic pipeline with one less command
Here's one more pipeline that you see so often, you'd just assume it's the best way—maybe the only way—to sort and dedupe lines of data:
But there's another way, without using the awkwardly-named uniq at all. The sort command has uniqueness built in as an option, using -u (--unique):
Always consider the alternatives
Linux is a flexible system, with many small tools that you can combine to achieve different tasks. This gives you a lot of flexibility, so be sure to try out other ways of doing things, understand the differences, and be prepared to learn new approaches.
