Heading in the Right Re-Direction

By Jonathan Terrasi • LinuxInsider • ECT News Network

Need a mobile developer, cybersecurity pro or other IT expert?
On ALL EC you can quickly and conveniently research all available technology solutions before making your next planning or purchasing decisions. Gather key insights and information before selecting your vendors. Visit ALL EC today.

If you've taken the time to get the hang of terminal basics, you're probably at the point where you want to start putting together what you've learned. Sometimes issuing commands one at a time is enough, but there are cases when it can be tedious to enter command after command just to perform a simple task. This is where the extra symbols on your keyboard come in.

For the shell, the terminal's command interpreter, those symbols are not wasted keys -- they're powerful operators that can link information together, split it apart, and much more. One of the simplest and most powerful shell operations is redirection.

3 Streams

To understand the workings of redirection, it's important to know what sources of data your shell can redirect. In Linux there are three "streams" of data. The first is "standard input," numbered by your system as stream 0 (since computers count from 0). It consists of the information or instructions submitted to the shell for evaluation. Most of the time, this comes from the user typing things into the terminal window.
The second, "standard output," is numbered as stream 1. As you would imagine, it is the stream of data that the shell outputs after performing some process, usually to the terminal window underneath the command.
The final stream, "standard error," numbered stream 2, is similar to standard output in that it generally takes the form of data dumped into the terminal window. However, it is conceptually separate from standard output so that the streams can be handled independently if desired. This is helpful when you have a command operating on lots of data in a complicated, error-prone operation, and you don't want the data and errors produced to get dumped into the same file.
As you've probably guessed, redirection involves taking these streams and redirecting them from their usual destination to a different one. This is accomplished using the ">" and "<" characters in various combinations, depending on where you want your data to end up.

Redirecting Standard Output

Let's say you want to create a file that lists today's date and time. Luckily for us, there is a command that returns that information, aptly called "date". Commands normally return the information they process to shell's standard output. To get it into a file, we insert ">" after the command and before the name of the destination file (with a space on either side).
With redirection, whatever file is specified after the ">" is overwritten, so unless you're sure you won't lose anything important, it's best to give a new name, in which case a file with that name will be created. Let's call it "date.txt" (the file extension after the period usually isn't important, but helps us humans with organization). Our command then looks like this:
$ date > date.txt
This isn't terribly useful, but we can build on it by executing another step. Let's say you're trying to monitor how the route your traffic takes over the Internet changes from day to day. The "traceroute" command will tell us every router, including the infrastructural ones in the backbone of the Internet, that our connection goes through from source to destination, the latter being a URL given as an argument.
Since we already have a file with a date in it, it would be practical just to tack on the data from our scan to the end of that file ("date.txt"). To do that, we simply use two ">" characters next to each other (">>"). Our new redirection looks like this:
$ traceroute google.com >> date.txt
Now all we need to do is change the name of the file to something more descriptive, using the "mv" command with its original name as the first argument and the new name as the second, like so:
$ mv date.txt trace1.txt
By using a "<" instead of ">", we can redirect standard input by substituting a file for it.
Let's say you have two files, "list1.txt" and "list2.txt", that each contain an unsorted list. While each list contains items the other does not, there is some overlap. We can find the lines that are in common using the "comm" command, but only if the lists are sorted.
There is a "sort" command, but even though it will return a sorted list to the terminal, it won't permanently sort the list, which puts us back at square one. We could save the sorted version of each list to its own file using ">" and then run "comm", but this approach would require two commands when we could accomplish the same thing with one (and without leftover files).
Instead, we can use the "<" to redirect sorted versions of each file into "comm", which would look like this:
$ comm <(sort list1.txt) <(sort list2.txt)
As with parentheses in math, the shell processes commands in parentheses first and then proceeds with what's left. Here, the two files are sorted and then fed into "comm", which then compares them and presents the results.

Redirecting Standard Error

Finally, we can divert the flow of standard error to do things like create error log files, or aggregate errors and returned data.
For instance, what if you wanted to search your entire system for wireless interface information that is accessible to non-root users? For that, we can employ the powerful "find" command.
Normally, when a non-root user runs "find" system-wide, it dumps standard output and standard error to the terminal, but there is usually more of the latter than former, making it hard to pick out the desired information. We can solve this by simply redirecting standard error to a file using "2>" (since standard error is stream 2), which leaves only standard output returned to the terminal window:
$ find / -name wireless 2> denied.txt
What if you wanted to save the valid results to their own file, without cluttering the error file? Since streams can be redirected independently, we can just add our standard output redirection to the end of our command like so:
$ find / -name wireless 2> denied.txt > found.txt
Notice that the first ">" is numbered while the second isn't. This is because standard output is stream 1 and the ">" redirect assumes stream 1 if no number is given.
Finally, if you wanted all the data from this command -- errors and successful finds -- deposited in the same place, you could redirect both streams to the same place using "&>" as follows:
$ find / -name wireless &> results.txt
This is just a basic outline of how redirection in the shell works, but these building blocks are enough to enable endless possibilities. Like everything else on the terminal, though, the best way to get a taste of what it can do is to try it out for yourself.