export default function BashGuide() {
return (
<>
{/* start pandoc */}
<h1 data-number={1} id="bash-quick-guide">
<span className="header-section-number">1</span> Bash Quick Guide
</h1>
<p>
Bash is an acronym for Bourne Again Shell. It is based on the Bourne
shell and is mostly compatible with its features.
</p>
<p>
Shells are command interpreters. They are applications that provide
users with the ability to give commands to their operating system
interactively, or to execute batches of commands quickly. In no way are
they required for the execution of programs; they are merely a layer
between system function calls and the user.
</p>
<p>
Think of a shell as a way for you to speak to your system. Your system
doesn’t need it for most of its work, but it is an excellent interface
between you and what your system can offer. It allows you to perform
basic math, run basic tests and execute applications. More importantly,
it allows you to combine these operations and connect applications to
each other to perform complex and automated tasks.
</p>
<p>
Bash is not your operating system. It is not your window manager. It is
not your terminal (but it often runs inside your terminal). It does not
control your mouse or keyboard. It does not configure your system,
activate your screen-saver, or open your files. It’s important to
understand that bash is only an interface for you to execute statements
(using bash syntax), either at the interactive bash prompt or via bash
scripts. The things that <em>actually happen</em> are usually caused by
other programs.
</p>
<p>
This guide is based on{" "}
<a href="http://mywiki.wooledge.org/BashGuide">
the bash guide in GreyCat’s wiki
</a>{" "}
and aims to be more concise, while still being accurate. It was produced
specifically for the Bash Workshop by{" "}
<a href="www.thealternative.ch">TheAlternative.ch</a>.
</p>
<p>
It is published under the{" "}
<a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">
CC by-nc-sa 4.0 license
</a>
.
</p>
<h1 data-number={2} id="commands-and-arguments">
<span className="header-section-number">2</span> Commands and Arguments
</h1>
<p>
Bash reads commands from its input, which can be either a file or your
terminal. In general, each line is interpreted as a command followed by
its arguments.
</p>
<pre>
<code>
ls{"\n"}touch file1 file2 file3{"\n"}ls -l{"\n"}rm file1 file2 file3
</code>
</pre>
<p>
The first word is always the command that is executed. All subsequent
words are given to that command as argument.
</p>
<p>
Note that <em>options</em>, such as the <code>-l</code> option in the
example above, are not treated specially by bash. They are arguments
like any other. It is up to the program (<code>ls</code> in the above
case) to treat it as an option.
</p>
<p>
Words are delimited by whitespace (spaces or tabs). It does not matter
how many spaces there are between two words. For example, try
</p>
<pre>
<code>echo Hello{" "}World</code>
</pre>
<p>
The process of splitting a line of text into words is called{" "}
<em>word splitting</em>. It is vital to be aware of it, especially when
you come across expansions later on.
</p>
<h2 data-number="2.1" id="preventing-word-splitting">
<span className="header-section-number">2.1</span> Preventing Word
Splitting
</h2>
<p>
Sometimes, you will want to pass arguments to commands that contain
whitespace. To do so, you can use quotes:
</p>
<pre>
<code>touch "Filename with spaces"</code>
</pre>
<p>
This command creates a single file named <em>Filename with spaces</em>.
The text within double quotes is protected from word splitting and hence
treated as a single word.
</p>
<p>Note that you can also use single quotes:</p>
<pre>
<code>touch 'Another filename with spaces'</code>
</pre>
<p>There is, however, an important difference between the two:</p>
<ul>
<li>
Double quotes prevent <strong>word splitting</strong>
</li>
<li>
Single quotes prevent <strong>word splitting and expansion</strong>
</li>
</ul>
<p>
When you use single quotes, the quoted text will never be changed by
bash. With double quotes, expansion will still happen. This doesn’t make
a difference in the above example, but as soon as you e.g. use
variables, it becomes important.
</p>
<p>
In general, it is considered good practice to use single quotes whenever
possible, and double quotes only when expansion is desired. In that
sense, the last example above can be considered “more correct”.
</p>
<h2 data-number="2.2" id="the-importance-of-spaces">
<span className="header-section-number">2.2</span> The Importance of
Spaces
</h2>
<p>
Bash contains various keywords and built-ins that aren’t immediately
recognizable as commands, such as the new test command:
</p>
<pre>
<code>[[ -f file ]]</code>
</pre>
<p>
The above code tests whether a file named “file” exists in the current
directory. Just like every line of bash code, it consists of a command
followed by its arguments. Here, the command is <code>[[</code>, while
the arguments are <code>-f</code>, <code>file</code> and <code>]]</code>
.
</p>
<p>
Many programmers of other languages would write the above command like
so:
</p>
<pre>
<code>[[-f file]]</code>
</pre>
<p>
This, though, is wrong: Bash will look for a command named{" "}
<code>[[-f</code>, which doesn’t exist, and issue an error message. This
kind of mistake is very common for beginners. It is advisable to always
use spaces after any kind of brackets in bash, even though there are
cases where they are not necessary.
</p>
<h2 data-number="2.3" id="scripts">
<span className="header-section-number">2.3</span> Scripts
</h2>
<p>
You have probably interacted with bash through a terminal before. You
would see a bash prompt, and you would issue one command after another.
</p>
<p>
Bash scripts are basically a sequence of commands stored in a file. They
are read and processed in order, one after the other.
</p>
<p>
Making a script is easy. Begin by making a new file, and put this on the
first line:
</p>
<pre>
<code>#!/bin/bash</code>
</pre>
<p>
This line is called an <em>interpreter directive</em>, or more commonly,
a <em>hashbang</em> or <em>shebang</em>. Your operating system uses it
to determine how this file can be run. In this case, the file is to be
run using <code>bash</code>, which is stored in the <code>/bin/</code>{" "}
directory.
</p>
<p>
After the shebang, you can add any command that you could also use in
your terminal. For example, you could add
</p>
<pre>
<code>echo 'Hello World'</code>
</pre>
<p>and then save the file as “myscript”</p>
<p>You can now run the file from the terminal by typing</p>
<pre>
<code>bash myscript</code>
</pre>
<p>
Here, you explicitly called bash and made it execute the script.{" "}
<code>bash</code> is used as the command, while <code>myscript</code> is
an argument. However, it’s also possible to use <code>myscript</code> as
a command directly.
</p>
<p>To do so, you must first make it executable:</p>
<pre>
<code>chmod +x myscript</code>
</pre>
<p>
Now that you have permission to execute this script directly, you can
type
</p>
<pre>
<code>./myscript</code>
</pre>
<p>to run it.</p>
<p>
The <code>./</code> is required to tell bash that the executable is
located in the current directory, rather than the system directory. We
will come back to this in the chapter on Variables.
</p>
<h1 data-number={3} id="variables-and-parameters">
<span className="header-section-number">3</span> Variables and
Parameters
</h1>
<p>
Variables and parameters can be used to store strings and retrieve them
later. <em>Variables</em> are the ones you create yourself, while{" "}
<em>special parameters</em> are pre-set by bash. <em>Parameters</em>{" "}
actually refers to both, but is often used synonymously to special
parameters.
</p>
<p>
To store a string in a variable, we use the <em>assignment syntax</em>:
</p>
<pre>
<code>varname=vardata</code>
</pre>
<p>
This sets the variable <code>varname</code> to contain the string{" "}
<code>vardata</code>.
</p>
<p>
Note that you cannot use spaces around the <code>=</code> sign. With the
spaces, bash would assume <code>varname</code> to be a command and then
pass <code>=</code> and <code>vardata</code> as arguments.
</p>
<p>
To access the string that is now stored in the variable{" "}
<code>varname</code>, we have to use <em>parameter expansion</em>. This
is the most common kind of expansion: A variable is replaced with its
content.
</p>
<p>If you want to print the variable’s value, you can type</p>
<pre>
<code>echo $varname</code>
</pre>
<p>
The <code>$</code> indicates that you want to use expansion on{" "}
<code>varname</code>, meaning it is replaced by its content. Note that
expansion happens before the command is run. Here’s what happens
step-by-step:
</p>
<ul>
<li>
Bash uses variable expansion, changing <code>echo $varname</code> to{" "}
<code>echo vardata</code>
</li>
<li>
Then, bash runs <code>echo</code> with <code>vardata</code> as its
parameter.
</li>
</ul>
<p>
The most important thing here is that{" "}
<strong>variable expansion happens before wordsplitting</strong>. That
means, if you have defined a variable like this:
</p>
<pre>
<code>myfile='bad song.mp3'</code>
</pre>
<p>and then run the command</p>
<pre>
<code>rm $myfile</code>
</pre>
<p>bash will expand this to</p>
<pre>
<code>rm bad song.mp3</code>
</pre>
<p>
Only now, word splitting occurs, and bash will call <code>rm</code> with
two arguments: <code>bad</code> and <code>song.mp3</code>. If you now
had a file called <code>song.mp3</code> in your current directory, that
one would be deleted instead.
</p>
<p>To prevent this from happening, you can use double quotes:</p>
<pre>
<code>rm "$myfile"</code>
</pre>
<p>This will be expanded to</p>
<pre>
<code>rm "bad song.mp3"</code>
</pre>
<p>
which is what we want. In this case, you have to use double quotes, as
single quotes would prevent expansion from happening altogether.
</p>
<p>
Not quoting variable and parameter expansions is a very common mistake
even among advanced bash programmers. It can cause bugs that are hard to
find and can be very dangerous.{" "}
<strong>Always quote your variable expansions.</strong>
</p>
<p>
You can also use variable expansions inside the variable assignment
itself. Consider this example:
</p>
<pre>
<code>
myvariable='blah'{"\n"}myvariable="$myvariable blah"{"\n"}echo
"$myvariable"
</code>
</pre>
<p>What will the output of this script be?</p>
<p>
First, the variable <code>myvariable</code> will get the value{" "}
<code>blah</code>. Then, <code>myvariable</code> is assigned to again,
which overwrites its former content. The assignment contains a variable
expansion, <code>"$myvariable blah"</code>. This is expanded to{" "}
<code>"blah blah"</code>, and that is going to be the new value of{" "}
<code>myvariable</code>. So the last command is expanded to{" "}
<code>echo "blah blah"</code>, and the output of the script is{" "}
<code>blah blah</code>.
</p>
<h2 data-number="3.1" id="special-parameters">
<span className="header-section-number">3.1</span> Special Parameters
</h2>
<p>
<em>Special parameters</em> are variables that are set by bash itself.
Most of those variables can’t be written to and they contain useful
information.
</p>
<table>
<colgroup>
<col style={{ width: "12%" }} />
<col style={{ width: "12%" }} />
<col style={{ width: "75%" }} />
</colgroup>
<thead>
<tr className="header">
<th>Parameter Name</th>
<th>Usage</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr className="odd">
<td>
<code>0</code>
</td>
<td>
<code>"$0"</code>
</td>
<td>Contains the name of the current script</td>
</tr>
<tr className="even">
<td>
<code>1</code> <code>2</code> <code>3</code> etc.
</td>
<td>
<code>"$1"</code> etc.
</td>
<td>
Contains the arguments that were passed to the current script. The
number indicates the position of that argument (first, second…).
These parameters are also called positional parameters.
</td>
</tr>
<tr className="odd">
<td>
<code>*</code>
</td>
<td>
<code>"$*"</code>
</td>
<td>
Contains all the positional parameters. Double quoted, it expands
to a single word containing them all.
</td>
</tr>
<tr className="even">
<td>
<code>@</code>
</td>
<td>
<code>"$@"</code>
</td>
<td>
Contains all the positional parameters. Double quoted, it expands
to{" "}
<strong>several words, where each word is one parameter</strong>.
This is special syntax, it behaves differently from “normal”
expansion in quotes. It retains the arguments exactly as they were
passed to the script.
</td>
</tr>
<tr className="odd">
<td>
<code>#</code>
</td>
<td>
<code>"$#"</code>
</td>
<td>
Contains the number of parameters that were passed to the script
</td>
</tr>
<tr className="even">
<td>
<code>?</code>
</td>
<td>
<code>"$?"</code>
</td>
<td>Contains the exit code of the last executed command</td>
</tr>
</tbody>
</table>
<h2 data-number="3.2" id="environment-variables">
<span className="header-section-number">3.2</span> Environment Variables
</h2>
<p>
<em>Environment variables</em> are special variables that are already
set when you start bash. In fact, these variables aren’t specific to
bash - they are available in every program that runs on your system and
they can affect your system’s behaviour.
</p>
<p>
You can use the command <code>printenv</code> in your terminal to
display all the environment variables you have currently set. Most of
them contain some system configuration, like the variable{" "}
<code>LANG</code>, which designates your preferred language. Some
variables, like <code>TERM</code>, <code>BROWSER</code> or{" "}
<code>SHELL</code>, designate your preferred default programs (terminal,
web browser and shell, respectively. These may not be set on all
systems).
</p>
<p>
Some of these variables can be useful in your scripts. For example, the
variable <code>RANDOM</code> gives you a different random number every
time you read it.
</p>
<p>
Another important environment variable is <code>PATH</code>. It contains
a bunch of file paths, separated by colons. These paths designate where
your system will look for executables when you type a command. For
example, if you type <code>grep</code> in your terminal, your system
will search for an executable called <code>grep</code> in the
directories designated in your <code>PATH</code> variable. As soon as it
finds one, it will execute that. If it doesn’t find it, you will get a
“command not found” error message.
</p>
<p>
You can modify your environment variables, if you want. The guideline
here is to only mess with those variables of which you know what they
do, otherwise you might break something.
</p>
<p>
The place to modify these variables is your <code>~/.bash_profile</code>{" "}
file. This file contains some bash code that is executed whenever you
log in. For example, you could add the following line:
</p>
<pre>
<code>export BROWSER="firefox"</code>
</pre>
<p>
This would set your default browser to firefox. Note that on some
systems, there are other settings–for example in your Desktop
Environment–which can override these environment variables. You’ll have
to test whether this works.
</p>
<p>
Note the <code>export</code> keyword. This is a bash builtin that takes
a variable definition as its argument and puts it in your{" "}
<em>environment</em>. If you omit this, your new variable will just be
an ordinary variable, rather than an environment variable.
</p>
<h2 data-number="3.3" id="ambiguous-names">
<span className="header-section-number">3.3</span> Ambiguous Names
</h2>
<p>
Say you have a variable called <code>name</code> that is declared as
follows:
</p>
<pre>
<code>name='bat'</code>
</pre>
<p>
Now, you want to use this variable in order to print <em>batman</em>:
</p>
<pre>
<code>echo "$nameman"</code>
</pre>
<p>
If you try this, you will notice that it doesn’t work–that is because
bash will now look for a variable called <code>nameman</code>, which
doesn’t exist. Here’s what you can do instead:
</p>
<pre>
<code>
echo "${"{"}name{"}"}man"
</code>
</pre>
<p>
The curly braces tell bash where the variable name ends. This allows you
to add more characters at the end of a variable’s content.
</p>
<h1 data-number={4} id="globbing">
<span className="header-section-number">4</span> Globbing
</h1>
<p>
<em>Globs</em> are an important bash concept–mostly for their incredible
convenience. They are patterns that can be used to match filenames or
other strings.
</p>
<p>
Globs are composed of normal characters and metacharacters.
Metacharacters are characters that have a special meaning. These are the
metacharacters that can be used in globs:
</p>
<ul>
<li>
<code>*</code>: Matches any string, including the empty string
(i.e. nothing)
</li>
<li>
<code>?</code>: Matches any single character
</li>
<li>
<code>[...]</code>: Matches any one of the characters enclosed in the
brackets
</li>
</ul>
<p>
Bash sees the glob, for example <code>a*</code>. It expands this glob,
by looking in the current directory and matching it against all files
there. Any filenames that match the glob are gathered up and sorted, and
then the list of filenames is used in place of the glob. So if you have
three files <code>a</code>, <code>b</code> and <code>albert</code> in
the current directory, the glob is expanded to <code>a albert</code>.
</p>
<p>
A glob always has to match the entire filename. That means{" "}
<code>a*</code> will match <code>at</code> but not <code>bat</code>.
</p>
<p>
Note that globbing is special in that it happens{" "}
<em>after word splitting</em>. This means you never need to worry about
spaces in filenames when you use globbing, and quoting globs is not
necessary. In fact, quotes will prevent globbing from happening.
</p>
<h1 data-number={5} id="expansion">
<span className="header-section-number">5</span> Expansion
</h1>
<p>
We’ve already seen <em>parameter and variable expansion</em>, but that’s
not the only kind of expansion that happens in bash. In this chapter,
we’ll look at all kinds of expansion that aren’t covered elsewhere.
</p>
<h2 data-number="5.1" id="expansions-and-quotes">
<span className="header-section-number">5.1</span> Expansions and Quotes
</h2>
<p>
You already know that it is important to quote parameter and variable
expansions, but we also told you that quoting globs–which are, in fact,
just another form of expansion–is not necessary. So, which expansions
need to be quoted?
</p>
<p>The rule of thumb here is as follows:</p>
<ul>
<li>
Always quote <strong>paramater expansion</strong>,{" "}
<strong>command substitution</strong> and{" "}
<strong>arithmetic expansion</strong>
</li>
<li>
Never quote <strong>brace expansion</strong>,{" "}
<strong>tilde expansion</strong> and <strong>globs</strong>
</li>
</ul>
<p>
The handy thing here is: All the expansions that require quoting have a{" "}
<code>$</code> in their syntax. Parameter expansion is simply a{" "}
<code>$</code> followed by a parameter name. Command substitution starts
with a <code>$(</code>, and arithmetic expansion starts with{" "}
<code>$((</code>.
</p>
<p>
So, the rule of thumb breaks down to the following:{" "}
<strong>If there’s a dollar, you probably need quotes.</strong>
</p>
<p>
Now, what if you want to use two kinds of expansion in the same line,
but one requires quotes and the other doesn’t? Consider the following
script:
</p>
<pre>
<code>prefix='my picture'{"\n"}rm ~/pictures/$prefix*</code>
</pre>
<p>
Here, we use tilde expansion, parameter expansion and globbing in order
to remove all files that start with <code>my picture</code> in the
folder <code>/home/username/pictures/</code>. But because quotes prevent
tilde expansion and globbing, we cannot quote the entire expression.
This means that the parameter expansion, too, goes unquoted–and this is
fatal, because our variable contains a space. So what should we do?
</p>
<p>
The important thing to realize here is that quoting simply prevents word
splitting, but it does not actually designate something as a single
string. So we can do the following:
</p>
<pre>
<code>prefix='my picture'{"\n"}rm ~/pictures/"$prefix"*</code>
</pre>
<p>
Only the parameter expansion is quoted, so it is protected from word
splitting. But that does not automatically separate it from the rest of
the string. Note that there are no spaces between <code>"$prefix"</code>{" "}
and <code>~/pictures/</code>. Since word splitting only happens when
there are spaces, the entire thing will not be split. Here’s what
happens, in order:
</p>
<p>First, tilde expansion occurs:</p>
<pre>
<code>rm /home/username/pictures/"$prefix"/*</code>
</pre>
<p>Next, parameter expansion:</p>
<pre>
<code>rm /home/username/pictures/"my picture"*</code>
</pre>
<p>
At this point, word splitting happens. But since the only space in our
argument is in quotes, the argument remains intact.
</p>
<p>And last, globbing:</p>
<pre>
<code>
rm /home/username/pictures/"my picture"001.jpg
/home/username/pictures/"my picture"002.jpg
</code>
</pre>
<p>
Now, there’s one last step that happens which we didn’t mention before.
It’s called <em>quote removal</em>. All the quotes that were needed to
prevent word splitting are now ignored, which means that the arguments
that are finally given to <code>rm</code> are:
</p>
<ul>
<li>
<code>/home/username/pictures/my picture001.jpg</code>
</li>
<li>
<code>/home/username/pictures/my picture002.jpg</code>
</li>
</ul>
<p>and this is exactly what we wanted.</p>
<p>
So, remember: Quotes don’t need to be at the beginning or end of an
argument, and if you use several kinds of expansion together, you can
add quotes in the middle as required.
</p>
<h2 data-number="5.2" id="expansion-order">
<span className="header-section-number">5.2</span> Expansion Order
</h2>
<p>
All the kinds of expansion happen in a certain order. The order is as
follows:
</p>
<ul>
<li>Brace expansion</li>
<li>Tilde expansion</li>
<li>Parameter and variable expansion</li>
<li>Command substitution</li>
<li>Arithmetic expansion</li>
<li>Word splitting</li>
<li>Globbing</li>
</ul>
<h2 data-number="5.3" id="brace-expansion">
<span className="header-section-number">5.3</span> Brace Expansion
</h2>
<p>
<em>Brace expansions</em> are often used in conjunction with globs, but
they also have other uses. They always expand to all possible
permutations of their contents. Here’s an example:
</p>
<pre>
<code>
$ echo th{"{"}e,a{"}"}n{"\n"}then than{"\n"}$ echo {"{"}1..9{"}"}
{"\n"}1 2 3 4 5 6 7 8 9{"\n"}$ echo {"{"}0,1{"}"}
{"{"}0..9{"}"}
{"\n"}00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19
</code>
</pre>
<p>
Brace expansions are replaced by a list of words. They are often used in
conjunction with globs to match specific files but not others. For
example, if you want to delete pictures from your pictures folder with
filenames IMG020.jpg through IMG039.jpg, you could use the following
pattern:
</p>
<pre>
<code>
rm IMG0{"{"}2,3{"}"}*.jpg
</code>
</pre>
<p>Note that we don’t use quotes here. Quotes prevent brace expansion.</p>
<p>
Brace expansion happens before globbing, so in the above example, the
braces are expanded to
</p>
<pre>
<code>rm IMG02*.jpg IMG03*.jpg</code>
</pre>
<p>
We end up with two glob patterns, the first matches IMG020.jpg through
IMG029.jpg, and the second matches IMG030.jpg through IMG039.jpg.
</p>
<h2 data-number="5.4" id="tilde-expansion">
<span className="header-section-number">5.4</span> Tilde Expansion
</h2>
<p>
You have probably already seen and used the tilde in the terminal. It is
a shortcut to your home directory:
</p>
<pre>
<code>cd ~/files</code>
</pre>
<p>
This will be expanded to <code>cd /home/username/files</code>. Note that
tilde expansion only happens outside of quotes, so the following won’t
work:
</p>
<pre>
<code>cd "~/files"</code>
</pre>
<h2 data-number="5.5" id="parameter-and-variable-expansion">
<span className="header-section-number">5.5</span> Parameter and
Variable Expansion
</h2>
<p>
<em>Parameter and variable expansion</em> is explained in the chapter on
Variables and Parameters.
</p>
<p>
An important sidenote here is that Parameter expansion and Variable
expansion often refer to the same thing. The official name as per the
bash manual is <em>Parameter expansion</em>, but{" "}
<em>Variable expansion</em> is often used instead as it is less
misleading.
</p>
<h2 data-number="5.6" id="command-substitution">
<span className="header-section-number">5.6</span> Command Substitution
</h2>
<p>
<em>Command substitution</em> is a way of using a command’s output
inside your script. For example, let’s say you want to find out where
your script is executed, and then move a file to that location.
</p>
<pre>
<code>echo 'I am at'{"\n"}pwd</code>
</pre>
<p>
This script will simply print the directory it is called from. But you
want to move a file to that directory, which means you need to use it as
an argument to the <code>mv</code> command. For that, you need command
substitution:
</p>
<pre>
<code>mv ~/myfile "$( pwd )"</code>
</pre>
<p>
The <code>$(</code> introduces a command substitution. It works
similarly to variable expansion, but instead of putting a variable’s
content, it puts a command’s output in that place. In this example,{" "}
<code>pwd</code> is run, and the output is (for example){" "}
<code>/home/username/scripts/</code>. Then, the entire command is
expanded to
</p>
<pre>
<code>mv ~/myfile "/home/username/scripts/"</code>
</pre>
<p>
Note that with this kind of expansion, quotes are important. The path
returned by <code>pwd</code> might have contained a space, in which case
we need the argument to be properly quoted.
</p>
<h2 data-number="5.7" id="arithmetic-expansion">
<span className="header-section-number">5.7</span> Arithmetic Expansion
</h2>
<p>
<em>Arithmetic expansion</em> is explained in the chapter on arithmetic
evaluation.
</p>
<h2 data-number="5.8" id="globbing-1">
<span className="header-section-number">5.8</span> Globbing
</h2>
<p>
<em>Globbing</em> is so important, it has a chapter of its own.
</p>
<h1 data-number={6} id="tests-and-conditionals">
<span className="header-section-number">6</span> Tests and Conditionals
</h1>
<p>
Every command you run in your terminal or shell script has a return
value. That value is a number, and by convention, a return value of 0
means <em>success</em>, while any other number indicates an error.
</p>
<p>
You usually don’t see the return value after running a command. What you
do see is the command’s <em>output</em>. If you want to know a command’s
return value, you can read it by issuing
</p>
<pre>
<code>echo "$?"</code>
</pre>
<p>immediately after running that command.</p>
<p>
While you often don’t need to know a command’s return value, it can be
useful to construct conditionals and thereby achieve advanced logic.
</p>
<h2 data-number="6.1" id="control-operators-and">
<span className="header-section-number">6.1</span> Control Operators
(&& and ||)
</h2>
<p>
Control operators can be used to make a command’s execution depend on
another command’s success. This concept is called{" "}
<em>conditional execution</em>:
</p>
<pre>
<code>mkdir folder && cd folder</code>
</pre>
<p>
In the above example, the <code>&&</code> operator is used to
connect two commands <code>mkdir folder</code> and{" "}
<code>cd folder</code>. Using this connection, bash will first execute{" "}
<code>mkdir folder</code>, and then execute <code>cd folder</code>{" "}
<em>only if the first command was successful</em>.
</p>
<pre>
<code>mkdir folder || echo 'Error: could not create folder'</code>
</pre>
<p>
In this example, the <code>||</code> operator is used to connect the
commands. Here, the second command is executed{" "}
<em>only if the first command failed</em>.
</p>
<p>
It is good practice to make your own scripts return an error
(i.e. something other than 0) whenever something goes wrong. To do
that, you can use this construct:
</p>
<pre>
<code>mkdir folder || exit 1</code>
</pre>
<p>
The <code>exit</code> command immediately stops the execution of your
script and makes it return the number you specified as an argument. In
this example, your script will attempt to create a folder. If that goes
wrong, it will immediately stop and return 1.
</p>
<p>
Control operators can be written more legibly by spreading them across
multiple lines:
</p>
<pre>
<code>
mkdir folder \{"\n"}
{" "}|| exit 1
</code>
</pre>
<p>
The backslash at the end of the first line makes bash ignore that line
break and treat both lines as a single command plus arguments.
</p>
<h2 data-number="6.2" id="grouping-commands">
<span className="header-section-number">6.2</span> Grouping Commands
</h2>
<p>
Now, what if you want to execute multiple commands if the first one
fails? Going from the example above, when <code>mkdir folder</code>{" "}
fails, you might want to print an error message <em>and</em> return 1.
</p>
<p>
This can be done by enclosing these two commands in single curly braces:
</p>
<pre>
<code>
mkdir folder || {"{"} echo 'Could not create folder'; exit 1; {"}"}
</code>
</pre>
<p>
The two commands in curly braces are treated as an unit, and if the
first command fails, both will be executed.
</p>
<p>
Note that you need to include spaces between the curly braces and the
commands. If there were no spaces, bash would look for a command named{" "}
<code>{"{"}echo</code> and fail to find it.
</p>
<p>
There’s a semicolon separating the two commands. The semicolon has the
same function as a line break: it makes bash consider both parts as
individual commands-plus-arguments.
</p>
<p>The example above could be rewritten as follows:</p>
<pre>
<code>
mkdir folder || {"{"}
{"\n"}
{" "}echo 'Could not create folder'{"\n"}
{" "}exit 1{"\n"}
{"}"}
</code>
</pre>
<p>
Here, no semicolon is required, because there is a line break between
the two statements. Line breaks and semicolons can be used
interchangeably.
</p>
<h2 data-number="6.3" id="conditional-blocks-if-and">
<span className="header-section-number">6.3</span> Conditional Blocks
(if and [[)
</h2>
<p>
<code>if</code> is a shell keyword. It first executes a command and
looks at its return value. If it was 0 (success), it executes one list
of commands, and if it was something else (failure), it executes
another.
</p>
<p>It looks like this:</p>
<pre>
<code>
if true{"\n"}then {"\n"}
{" "}echo 'It was true'{"\n"}else {"\n"}
{" "}echo 'It was false'{"\n"}fi
</code>
</pre>
<p>
<code>true</code> is a bash builtin command that does nothing and
returns 0. <code>if</code> will run that command, see that it returned
0, and then execute the commands between the <code>then</code> and the{" "}
<code>else</code>.
</p>
<p>
In many programming languages, operators such as <code>></code>,{" "}
<code><</code> or <code>==</code> exist and can be used to compare
values. In bash, operators don’t exist. But since comparing values is so
common, there’s a special command that can do it:
</p>
<pre>
<code>[[ a = b ]]</code>
</pre>
<p>
<code>[[</code> is a command that takes as its arguments a comparison.
The last argument has to be a <code>]]</code>. It is made to look like a
comparison in double brackets, but it is, in fact, a command like any
other. It is also called the <em>new test command</em>. (The{" "}
<em>old test command</em>, often simply called <em>test</em>, exists as
well, so be careful not to confuse them).
</p>
<p>
For that reason, the spaces are absolutely needed. You cannot write
this:
</p>
<pre>
<code>[[a=b]]</code>
</pre>
<p>
This will make bash look for a command called <code>[[a=b]]</code>,
which doesn’t exist.
</p>
<p>
<code>[[</code> does not only support comparing strings. For example,{" "}
<code>[[ -f file ]]</code> will test whether a file named “file” exists.
Here’s a list of the most common tests you can use with <code>[[</code>:
</p>
<ul>
<li>
<code>-e FILE:</code> True if file exists.
</li>
<li>
<code>-f FILE:</code> True if file is a regular file.
</li>
<li>
<code>-d FILE:</code> True if file is a directory.
</li>
<li>
String operators: * <code>-z STRING:</code> True if the string is
empty (its length is zero). * <code>-n STRING:</code> True if the
string is not empty (its length is not zero). *{" "}
<code>STRING = STRING</code>: True if the string matches the glob
pattern (if you quote the glob pattern, the strings have to match
exactly). * <code>STRING != STRING</code>: True if the string does not
match the glob pattern (if you quote the glob pattern, the strings
just have to be different). * <code>STRING < STRING</code>: True if
the first string sorts before the second. *{" "}
<code>STRING > STRING</code>: True if the first string sorts after
the second.
</li>
<li>
<code>EXPR -a EXPR</code>: True if both expressions are true (logical
AND).
</li>
<li>
<code>EXPR -o EXPR</code>: True if either expression is true (logical
OR).
</li>
<li>
<code>! EXPR</code>: Inverts the result of the expression (logical
NOT).
</li>
<li>
<code>EXPR && EXPR</code>: Much like the ‘-a’ operator of
test, but does not evaluate the second expression if the first already
turns out to be false.
</li>
<li>
<code>EXPR || EXPR</code>: Much like the ‘-o’ operator of test, but
does not evaluate the second expression if the first already turns out
to be true.
</li>
<li>
Numeric operators:
<ul>
<li>
<code>INT -eq INT</code>: True if both integers are equal.
</li>
<li>
<code>INT -ne INT</code>: True if the integers are not equal.
</li>
<li>
<code>INT -lt INT</code>: True if the first integer is less than
the second.
</li>
<li>
<code>INT -gt INT</code>: True if the first integer is greater
than the second.
</li>
<li>
<code>INT -le INT</code>: True if the first integer is less than
or equal to the second.
</li>
<li>
<code>INT -ge INT</code>: True if the first integer is greater
than or equal to the second.
</li>
</ul>
</li>
</ul>
<p>You might occasionally come across something like this:</p>
<pre>
<code>[ a = b ]</code>
</pre>
<p>
Here, we use single brackets instead of double brackets. This is, in
fact, an entirely different command, the <code>[</code> command or{" "}
<em>old test command</em>. It has the same purpose–comparing things–but
the <code>[[</code> command is newer, has more features, and is easier
to use. We strongly recommend using <code>[[</code> over <code>[</code>.
</p>
<h2 data-number="6.4" id="conditional-loops-while-until-and-for">
<span className="header-section-number">6.4</span> Conditional Loops
(while, until and for)
</h2>
<p>
Loops can be used to repeat a list of commands multiple times. In bash,
there are <code>while</code> loops and <code>for</code> loops.
</p>
<p>While loops look like this:</p>
<pre>
<code>
while true{"\n"}do{"\n"}
{" "}echo 'Infinite loop'{"\n"}done
</code>
</pre>
<p>
The <code>while</code> keyword will execute the <code>true</code>{" "}
command, and if that returns 0, it executes all commands between the{" "}
<code>do</code> and <code>done</code>. After that, it starts over, until
the <code>true</code> command returns 1 (which it never does, which is
why this loop will run indefinitely).
</p>
<p>
The above example might not be immediately useful, but you could also do
something like this:
</p>
<pre>
<code>
while ping -c 1 -W 1 www.google.com{"\n"}do{"\n"}
{" "}echo 'Google still works!'{"\n"}done
</code>
</pre>
<p>
There’s also a variation of the <code>while</code> loop, called{" "}
<code>until</code>. It works similarly, except it only runs its command
list when the first command <em>fails</em>:
</p>
<pre>
<code>
until ping -c 1 -W 1 www.google.com{"\n"}do{"\n"}
{" "}echo 'Google isn'\''t working!'{"\n"}done
</code>
</pre>
<p>
<code>for</code> loops can be used to iterate over a list of strings:
</p>
<pre>
<code>
for var in 1 2 3{"\n"}do{"\n"}
{" "}echo "$var"{"\n"}done
</code>
</pre>
<p>
After the <code>for</code>, you specify a variable name. After the{" "}
<code>in</code>, you list all the strings you want to iterate over.
</p>
<p>
The loop works by setting the variable you specified to all the values
from the list in turn, and then executing the command list for each of
them.
</p>
<p>
This is especially useful in combination with globs or brace expansions:
</p>
<pre>
<code>
echo 'This is a list of all my files starting with f:'{"\n"}for var in
f*
{"\n"}do{"\n"}
{" "}echo "$var"{"\n"}done{"\n"}
{"\n"}echo 'And now I will count from 1 to 100:'{"\n"}for var in {"{"}
1..100{"}"}
{"\n"}do{"\n"}
{" "}echo "$var"{"\n"}done
</code>
</pre>
<h2 data-number="6.5" id="choices-case-and-select">
<span className="header-section-number">6.5</span> Choices (case and
select)
</h2>
<p>
Sometimes, you want your script to behave differently depending on the
content of a variable. This could be implemented by taking a different
branch of an if statement for each state:
</p>
<pre>
<code>
if [[ "$LANG" = 'en' ]]{"\n"}then{"\n"}
{" "}echo 'Hello!'{"\n"}elif [[ "$LANG" = 'de' ]]{"\n"}then{"\n"}
{" "}echo 'Guten Tag!'{"\n"}elif [[ "$LANG" = 'it' ]]{"\n"}then
{"\n"}
{" "}echo 'Ciao!'{"\n"}else{"\n"}
{" "}echo 'I do not speak your language.'{"\n"}fi
</code>
</pre>
<p>
This is quite cumbersome to write. At the same time, constructs like
this are very common. For that reason, bash provides a keyword to
simplify it:
</p>
<pre>
<code>
case "$LANG" in{"\n"}
{" "}en){"\n"}
{" "}echo 'Hello!'{"\n"}
{" "};;{"\n"}
{" "}de) {"\n"}
{" "}echo 'Guten Tag!'{"\n"}
{" "};;{"\n"}
{" "}it){"\n"}
{" "}echo 'Ciao!'{"\n"}
{" "};;{"\n"}
{" "}*){"\n"}
{" "}echo 'I do not speak your language.'{"\n"}
{" "};;{"\n"}esac
</code>
</pre>
<p>
Each choice of the case statement consists of a string or glob pattern,
a <code>)</code>, a list of commands that is to be executed if the
string matches the pattern, and two semicolons to denote the end of a
list of commands.
</p>
<p>
The string after the keyword <code>case</code> is matched against each
glob pattern in order. The list of commands after the first match is
executed. After that, execution continues after the <code>esac</code>.
</p>
<p>
Since the string is matched against glob patterns, we can use{" "}
<code>*</code> in the end to catch anything that didn’t match before.
</p>
<p>
Another construct of choice is the <code>select</code> construct. It
looks and works similarly to a loop, but it also presents the user with
a predefined choice. You are encouraged to try running this example
yourself:
</p>
<pre>
<code>
echo 'Which one of these does not belong in the group?'{"\n"}select
choice in Apples Pears Crisps Lemons Kiwis{"\n"}do{"\n"}
{" "}if [[ "$choice" = Crisps ]]{"\n"}
{" "}then{"\n"}
{" "}echo 'Correct! Crisps are not fruit.'{"\n"}
{" "}break{"\n"}
{" "}fi{"\n"}
{" "}echo 'Wrong answer. Try again.'{"\n"}done
</code>
</pre>
<p>
The syntax of the <code>select</code> construct is very similar to{" "}
<code>for</code> loops. The difference is that instead of setting the
variable (<code>choice</code> in this example) to each value in turn,
the <code>select</code> construct lets the user choose which value is
used next. This also means that a <code>select</code> construct can run
indefinitely, because the user can keep selecting new choices. To avoid
being trapped in it, we have to explicitly use <code>break</code>.{" "}
<code>break</code> is a builtin command that makes bash jump out of the
current <code>do</code> block. Execution will continue after the{" "}
<code>done</code>. <code>break</code> also works in <code>for</code> and{" "}
<code>while</code> loops.
</p>
<p>
As you can see in the example above, we used an <code>if</code> command
inside a <code>select</code> command. All of these conditional
constructs (<code>if</code>, <code>for</code>, <code>while</code>,{" "}
<code>case</code> and <code>select</code>) can be nested indefinitely.
</p>
<h1 data-number={7} id="input-and-output">
<span className="header-section-number">7</span> Input and Output
</h1>
<p>
Input and output in bash is very flexible and, consequentially, complex.
We will only look at the most widely used components.
</p>
<h2 data-number="7.1" id="command-line-arguments">
<span className="header-section-number">7.1</span> Command-line
Arguments
</h2>
<p>
For many bash scripts, the first input we care about are the arguments
given to it via the command line. As we saw in the chapter on
Parameters, these arguments are contained in some{" "}
<em>special parameters</em>. These are called{" "}
<em>positional parameters</em>. The first parameter is referred to with{" "}
<code>$1</code>, the second with <code>$2</code>, and so on. After
number 9, you have to enclose the numbers in curly braces:{" "}
<code>
${"{"}10{"}"}
</code>
,{" "}
<code>
${"{"}11{"}"}
</code>{" "}
and so on.
</p>
<p>
In addition to referring to them one at a time, you may also refer to
the entire set of positional parameters with the <code>"$@"</code>{" "}
substitution. The double quotes here are{" "}
<strong>extremely important</strong>. If you don’t use the double
quotes, each one of the positional parameters will undergo word
splitting and globbing. You don’t want that. By using the quotes, you
tell Bash that you want to preserve each parameter as a separate word.
</p>
<p>
There are even more ways to deal with parameters. For example, it is
very common for commands to accept <em>options</em>, which are single
letters starting with a <code>-</code>. For example, <code>ls -l</code>{" "}
calls the <code>ls</code> program with the <code>-l</code> option, which
makes it output more information. Usually, multiple options can be
combined, as in <code>ls -la</code>, which is equivalent to{" "}
<code>ls -l -a</code>.
</p>
<p>
You might want to create your own scripts that accept some options. Bash
has the so called <code>getopts</code> builtin command to parse passed
options.
</p>
<pre>
<code>
while getopts 'hlf:' opt{"\n"}do {"\n"}
{" "}case "$opt" in{"\n"}
{" "}h|\?){"\n"}
{" "}echo 'available options: -h -l -f [filename]'{"\n"}
{" "};;{"\n"}
{" "}f){"\n"}
{" "}file="$OPTARG"{"\n"}
{" "};;{"\n"}
{" "}l){"\n"}
{" "}list=true{"\n"}
{" "};;{"\n"}
{" "}esac{"\n"}done{"\n"}
{"\n"}shift "$(( OPTIND - 1 ))"
</code>
</pre>
<p>
As you can see, we use <code>getopts</code> within a while loop.{" "}
<code>getopts</code> will return 0 as long as there are more options
remaining and something else if there are no more options. That makes it
perfectly suitable for a loop.
</p>
<p>
<code>getopts</code> takes two arguments, the <em>optstring</em> and the{" "}
<em>variable name</em>. The optstring contains all the letters that are
valid options. In our example, these are <code>h</code>, <code>l</code>{" "}
and <code>f</code>.
</p>
<p>
The <code>f</code> is followed by a colon. This indicates that the f
option requires an argument. The script could for example be called like
so:
</p>
<pre>
<code>myscript -f file.txt</code>
</pre>
<p>
<code>getopts</code> will set the variable that you specified as its
second argument to the letter of the option it found first. If the
option required an argument, the variable <code>OPTARG</code> is set to
whatever the argument was. In the example above, a case statement is
used to react to the different options.
</p>
<p>
There’s also a special option <code>?</code>. Whenever{" "}
<code>getopts</code> finds an option that is not present in the
optstring, it sets the shell variable (<code>opt</code> in the example)
to <code>?</code>. In the case statement above, that triggers the help
message.
</p>
<p>
After all options are parsed, the remaining arguments are “moved” such
that they are now in <code>$1</code>, <code>$2</code>… even though
previously, these positional parameters were occupied by the options.
</p>
<p>
Also note the line <code>shift "$(( OPTIND - 1 ))"</code> at the end.
The <code>shift</code> builtin can be used to discard command-line
arguments. Its argument is a number and designates how many arguments we
want to discard.
</p>
<p>
This is needed because we don’t know how many options the user will pass
to our script. If there are more positional parameters after all the
options, we have no way of knowing at which number they start.
Fortunately, <code>getopts</code> also sets the shell variable{" "}
<code>OPTIND</code>, in which it stores the index of the option it’s
going to parse next.
</p>
<p>
So after parsing all the option, we just discard the first{" "}
<code>OPTIND - 1</code> options, and the remaining arguments now start
from <code>$1</code> onwards.
</p>
<h2 data-number="7.2" id="file-descriptors">
<span className="header-section-number">7.2</span> File Descriptors
</h2>
<p>
<em>File descriptors</em> are the way programs refer to files, or other
things that work like files (such as pipes, devices, or terminals). You
can think of them as pointers that point to data locations. Through
these pointers, programs can write to or read from these locations.
</p>
<p>By default, every program has three file descriptors:</p>
<ul>
<li>Standard Input (stdin): File Descriptor 0</li>
<li>Standard Output (stdout): File Descriptor 1</li>
<li>Standard Error (stderr): File Descriptor 2</li>
</ul>
<p>
When you run a script in the terminal, then stdin contains everything
you type in that terminal. stdout and stderr both point to the terminal,
and everything that is written to these two is displayed as text in the
terminal. stdout is where programs send their normal information, and
stderr is where they send their error messages.
</p>
<p>
Let’s make these definitions a little more concrete. Consider this
example:
</p>
<pre>
<code>
echo 'What is your name?'{"\n"}read name{"\n"}echo "Good day, $name.
Would you like some tea?"
</code>
</pre>
<p>
You already know <code>echo</code>. It simply prints its argument to{" "}
<em>stdout</em>. Since stdout is connected to your terminal, you will
see that message there.
</p>
<p>
<code>read</code> is a command that reads one line of text from{" "}
<em>stdin</em> and stores it in a variable, which we specified to be{" "}
<code>name</code>. Because stdin is connected to what you type in your
terminal, it will let you type a line of text, and as soon as you press
enter, that line will be stored in the variable.
</p>
<p>So what about stderr? Consider this example:</p>
<pre>
<code>ehco 'Hello!'</code>
</pre>
<p>
The command <code>ehco</code> does not exist. If you run this, bash will
print an error message to <em>stderr</em>. Because stderr is connected
to your terminal, you will see that message there.
</p>
<h2 data-number="7.3" id="redirection">
<span className="header-section-number">7.3</span> Redirection
</h2>
<p>
<em>Redirection</em> is the most basic form of input/output manipulation
in bash. It is used to change the source or destination of{" "}
<em>File descriptors</em>, i.e. connect them to something other
than your terminal. For example, you can send a command’s output to a
file instead.
</p>
<pre>
<code>
echo 'It was a dark and stormy night. Too dark to write.' > story
</code>
</pre>
<p>
The <code>></code> operator begins an <em>output redirection</em>. It
redirects the <em>stdout</em> file descriptor of the command to the
left, and connects it to a file called “story”. That means if you run
this, you will not see the output of <code>echo</code>—after all, stdout
no longer points to your terminal.
</p>
<p>
Note that <code>></code> will just open the file you specify without
checking whether it already exists first. If the file already exists,
its contents will be overwritten and you will lose whatever was stored
in there before. <strong>Be careful.</strong>
</p>
<p>
If you don’t want to overwrite the existing content of a file, but
rather append your output to the end of that file, you can use{" "}
<code>>></code> instead of <code>></code>.
</p>
<p>
Now, let’s look at <em>input redirection</em>. For that, we first
introduce a command named <code>cat</code>. <code>cat</code> is often
used to display the contents of a file, like so:
</p>
<pre>
<code>cat myfile</code>
</pre>
<p>
If <code>cat</code> is called without an argument, however, it will
simply read from <em>stdin</em> and print that directly to{" "}
<em>stdout</em>.
</p>
<p>
Try the following: Run <code>cat</code>, without arguments, in your
terminal. Then type some characters and hit enter. Can you figure out
what is happening?
</p>
<p>
<em>Input redirection</em> uses the <code><</code> operator. It works
as follows:
</p>
<pre>
<code>cat < story</code>
</pre>
<p>
The <code><</code> operator will take a command’s <em>stdin</em> file
descriptor and point it to a file, “story” in this example. This means{" "}
<code>cat</code> now ignores your terminal and reads from “story”
instead. Note that this has the same effect as typing{" "}
<code>cat story</code>.
</p>
<p>
If you want to redirect <em>stderr</em> instead of <em>stdout</em>, you
can do as follows:
</p>
<pre>
<code>ehco 'Hello!' 2> errors</code>
</pre>
<p>
If you run this, you won’t see any error message, even though the
command <code>ehco</code> doesn’t exist. That’s because <em>stderr</em>{" "}
is no longer connected to your terminal, but to a file called “errors”
instead.
</p>
<p>
Now that you know about redirection, there is one subtlety that you have
to be aware of: You can’t have two file descriptors point to the same
file.
</p>
<p>
If you wanted to log a command’s complete output–stdout <em>and</em>{" "}
stderr–you might be tempted to do something like this:
</p>
<pre>
<code>mycommand > logfile 2> logfile</code>
</pre>
<p>
However, this is a <strong>bad</strong> idea. The two file descriptors
will now both point to the same file <em>independently</em>, which
causes them to constantly overwrite each other’s text.
</p>
<p>
If you still want to point both stdout and stderr to the same file, you
can do it like this:
</p>
<pre>
<code>mycommand > logfile 2>&1</code>
</pre>
<p>
Here, we use the <code>>&</code> syntax to duplicate file
descriptor 1. In this scenario, we no longer have two file descriptors
pointing to one file. Instead, we have only one file descriptor that
acts as both stdout and stderr at the same time.
</p>
<p>
To help remember the syntax, you can think of <code>&1</code> as
“where 1 is”, and of the <code>2></code> as “point 2 to”. The whole
thing, <code>2>&1</code>, then becomes “point 2 to wherever 1
is”. This also makes it clear that <code>> logfile</code> has to come{" "}
<em>before</em> <code>2>&1</code>: First you point 1 to
“logfile”, and only then you can point 2 to where 1 is.
</p>
<p>
There’s also a quick way to completely get rid of a command’s output
using redirections. Consider this:
</p>
<pre>
<code>mycommand > /dev/null 2>&1</code>
</pre>
<p>
<code>/dev/null</code> is a special file in your file system that you
can write to, but the things you write to it are not stored. So,{" "}
<code>mycommand</code>’s output is written somewhere where it’s not
stored or processed in any way. It is discarded completely.
</p>
<p>
You could also leave out the <code>2>&1</code>. Then, you’d still
see error messages, but discard the normal output.
</p>
<h2 data-number="7.4" id="pipes">
<span className="header-section-number">7.4</span> Pipes
</h2>
<p>
Now that you know how to manipulate <em>file descriptors</em> to direct
output to files, it’s time to learn another type of I/O redirection.
</p>
<p>
The <code>|</code> operator can be used to connect one command’s{" "}
<em>stdout</em> to another command’s <em>stdin</em>. Have a look at
this:
</p>
<pre>
<code>echo 'This is a beautiful day!' | sed 's/beauti/wonder'</code>
</pre>
<p>
The <code>sed</code> command (“sed” is short for “stream editor”) is a
utility that can be used to manipulate text “on the fly”. It reads text
from stdin, edits it according to some commands, and then prints the
result to stdout. It is very powerful. Here, we use it to replace
“beauti” with “wonder”.
</p>
<p>
First, the <code>echo</code> command writes some text to it’s stdout.
The <code>|</code> operator connected <code>echo</code>’s stout to{" "}
<code>sed</code>’s stdin, so everything <code>echo</code> sends there is
immediately picked up by <code>sed</code>. <code>sed</code> will then
edit the text and print the result to its own stdout. <code>sed</code>’s
stdout is still connected to your terminal, so this is what you see.
</p>
<h1 data-number={8} id="compound-commands">
<span className="header-section-number">8</span> Compound Commands
</h1>
<p>
<em>Compound commands</em> is a catch-all phrase covering several
different concepts. We’ve already seen <code>if</code>, <code>for</code>
, <code>while</code>, <code>case</code>, <code>select</code> and the{" "}
<code>[[</code> keyword, which all fall into this category. Now we’ll
look at a few more.
</p>
<h2 data-number="8.1" id="subshells">
<span className="header-section-number">8.1</span> Subshells
</h2>
<p>
<em>Subshells</em> can be used to encapsulate a command’s effect. If a
command has undesired side effects, you can execute it in a subshell.
Once the subshell command ends, all side effects will be gone.
</p>
<p>
To execute a command (or several commands) in a subshell, enclose them
in parenthesis:
</p>
<pre>
<code>
( {"\n"}
{" "}cd /tmp {"\n"}
{" "}pwd{"\n"}){"\n"}pwd
</code>
</pre>
<p>
The <code>cd</code> and the first <code>pwd</code> commands are executed
in a subshell. All side effects in that subshell won’t affect the second{" "}
<code>pwd</code> command. Changing the current directory is such a side
effect–even though we use <code>cd</code> to go to the <code>/tmp</code>{" "}
folder, we jump back to our original folder as soon as the subshell
ends.
</p>
<h2 data-number="8.2" id="command-grouping">
<span className="header-section-number">8.2</span> Command Grouping
</h2>
<p>
You can group several commands together by enclosing them in curly
braces. This makes bash consider them as a unit with regard to pipes,
redirections and control flow:
</p>
<pre>
<code>
{"{"}
{"\n"}
{" "}echo 'Logfile of my backup'{"\n"}
{" "}rsync -av . /backup{"\n"}
{" "}echo "Backup finished with exit code $#"{"\n"}
{"}"} > backup.log 2>&1
</code>
</pre>
<p>
This redirects stdout and stderr of <em>all three commands</em> to a
file called backup.log. Note that while this looks similar to subshells,
it is not the same. Side effects that happen within the curly braces
will still be present outside of them.
</p>
<h2 data-number="8.3" id="arithmetic-evaluation">
<span className="header-section-number">8.3</span> Arithmetic Evaluation
</h2>
<p>
So far, we’ve only been manipulating strings in bash. Sometimes, though,
it is also necessary to manipulate numbers. This is done through
arithmetic evaluation.
</p>
<p>
Say you want to add the numbers 5 and 4. You might do something like
this:
</p>
<pre>
<code>a=5+4</code>
</pre>
<p>
However, this will result in the variable <code>a</code> containing the
string <code>5+4</code>, rather than the number <code>9</code>. Instead,
you should do this:
</p>
<pre>
<code>(( a=5+4 ))</code>
</pre>
<p>
The double parenthesis indicate that something arithmetic is happening.
In fact, <code>((</code> is a bash keyword, much like <code>[[</code>.
</p>
<p>
<code>((</code> can also be used to do arithmetic comparison:
</p>
<pre>
<code>
if (( 5 > 9 )){"\n"}then{"\n"}
{" "}echo '5 is greater than 9'{"\n"}else {"\n"}
{" "}echo '5 is not greater than 9'{"\n"}fi
</code>
</pre>
<p>
It is important not to confuse <code>((</code> and <code>[[</code>.{" "}
<code>[[</code> is for comparing strings (among other things), while{" "}
<code>((</code> is only for comparing numbers.
</p>
<p>
There’s also <em>arithmetic substitution</em>, which works similarly to{" "}
<em>command substitution</em>:
</p>
<pre>
<code>echo "There are $(( 60 * 60 * 24 )) seconds in a day."</code>
</pre>
<h1 data-number={9} id="functions">
<span className="header-section-number">9</span> Functions
</h1>
<p>
Inside a bash script, functions are very handy. They are lists of
commands–much like a normal script–except they don’t reside in their own
file. They do however take arguments, just like scripts.
</p>
<p>Functions can be defined like this:</p>
<pre>
<code>
sum() {"{"}
{"\n"}
{" "}echo "$1 + $2 = $(( $1 + $2 ))"{"\n"}
{"}"}
</code>
</pre>
<p>
If you put this in a script file and run it, absolutely nothing will
happen. The function <code>sum</code> has been defined, but it is never
used.
</p>
<p>
You can use your function like any other command, but you have to define
it <em>before</em> you use it:
</p>
<pre>
<code>
sum() {"{"}
{"\n"}
{" "}echo "$1 + $2 = $(( $1 + $2 ))"{"\n"}
{"}"}
{"\n"}sum 1 2{"\n"}sum 3 9{"\n"}sum 6283 3141
</code>
</pre>
<p>
As you can see, you can use the function <code>sum</code> multiple
times, but you only need to define it once. This is useful in larger
scripts, where a certain task has to be performed multiple times.
Whenever you catch yourself writing the same or very similar code twice
in the same script, you should consider using a function.
</p>
<h1 data-number={10} id="useful-commands">
<span className="header-section-number">10</span> Useful Commands
</h1>
<p>
This chapter provides an overview of useful commands that you can use in
your scripts. It is nowhere near complete, and serves only to provide a
brief overview. If you want to know more about a specific command, you
should read its manpage.
</p>
<p>
<strong>grep</strong>
</p>
<p>
<code>grep</code> can be used to search for a string within a file, or
within the output of a command.
</p>
<pre>
<code>
# searches logfile.txt for lines containing the word error{"\n"}grep
'error' logfile.txt{" "}
{"\n"}
{"\n"}# searches the directory 'folder' for files {"\n"}# containing
the word 'analysis'{"\n"}grep 'analysis' folder/{" "}
{"\n"}
{"\n"}# searches the output of 'xrandr' for lines that say
'connected'. {"\n"}# only matches whole words, so 'disconnected' will
not match.{"\n"}
xrandr | grep -w 'connected'{" "}
</code>
</pre>
<p>
<code>grep</code> returns 0 if it finds something, and returns an error
if it doesn’t. This makes it useful for conditionals.
</p>
<p>
<strong>sed</strong>
</p>
<p>
<code>sed</code> can be used to edit text “on the fly”. It uses its own
scripting language to describe modifications to be made to the text,
which makes it extremely powerful. Here, we provide examples for the
most common usages of sed:
</p>
<pre>
<code>
# replaces the first occurrence of 'find' in every line by 'replace'
{"\n"}
sed 's/find/replace' inputfile {"\n"}
{"\n"}# replaces every occurrence of 'find' in every line by 'replace'
{"\n"}sed 's/find/replace/g' inputfile {"\n"}
{"\n"}# deletes the first occurrence of 'find' in every line{"\n"}sed
's/find//' inputfile {"\n"}
{"\n"}# deletes every occurrence of 'find' in every line{"\n"}sed
's/find//g' inputfile {"\n"}
{"\n"}# displays only the 12th line{"\n"}sed '12q;d' inputfile{" "}
</code>
</pre>
<p>
<code>sed</code> is often used in combination with pipes to format the
output or get rid of unwanted characters.
</p>
<p>
<strong>curl and wget</strong>
</p>
<p>
<code>curl</code> and <code>wget</code> are two commands that can be
used to access websites or other content from the web. The difference is
that <code>wget</code> will simply download the content to a file, while{" "}
<code>curl</code> will output it to the console.
</p>
<pre>
<code>
curl http://www.thealternative.ch{"\n"}wget
http://files.project21.ch/LinuxDays-Public/16FS-install-guide.pdf
</code>
</pre>
<p>
<strong>xrandr</strong>
</p>
<p>
<code>xrandr</code> can be used to manipulate your video outputs,
i.e. enabling and disabling monitors or setting their screen
resolution and orientation.
</p>
<pre>
<code>
# list all available outputs and their status info{"\n"}xrandr{" "}
{"\n"}
{"\n"}# enables output HDMI-1{"\n"}xrandr --output HDMI-1 --auto{" "}
{"\n"}
{"\n"}# puts output HDMI-1 to the left of output LVDS-1{"\n"}xrandr
--output HDMI-1 --left-of LVDS-1 {"\n"}
{"\n"}# disables output LVDS-1{"\n"}xrandr --output LVDS-1 --off{" "}
</code>
</pre>
<p>
<strong>ImageMagick (convert)</strong>
</p>
<p>
The <code>convert</code> command makes it possible to do image
processing from the commandline.
</p>
<pre>
<code>
# scale fullHDWallpaper.jpg to specified resolution{"\n"}convert
fullHDWallpaper.jpg -scale 3200x1800 evenBiggerWallpaper.jpg {"\n"}
{"\n"}# "automagically" adjusts the gamma level of somePicture.jpg
{"\n"}
convert somePicture.jpg -auto-gamma someOtherPicture.jpg {"\n"}
{"\n"}# transform image to black and white{"\n"}convert
colorfulPicture.jpg -monochrome blackWhitePicture.jpg{" "}
</code>
</pre>
<p>
It is extremely powerful and has lots of options. A good resource is the{" "}
<a href="http://www.imagemagick.org">official website</a>. It also
provides examples for most options.
</p>
<p>
<strong>notify-send</strong>
</p>
<p>
<code>notify-send</code> can be used to display a desktop notification
with some custom text:
</p>
<pre>
<code>
notify-send 'Battery warning' 'Your battery level is below 10%'
</code>
</pre>
<p>
The first argument is the notification’s title, the second is its
description.
</p>
<p>
<code>notify-send</code> requires a <em>notification daemon</em> to be
running, else it won’t work. Most desktop environments come with a
notification daemon set up and running. If you can’t see your
notifications, it might be that you don’t have such a daemon.
</p>
<p>
<strong>find</strong>
</p>
<p>
<code>find</code> can be used to find files in a directory structure.
</p>
<pre>
<code>
# finds all files in the current directory and all subdirectories that
end
{"\n"}# in .png{"\n"}find -name '*.png' {"\n"}
{"\n"}# finds all files ending in .tmp and removes them. {"{"}
{"}"} is replaced by the{"\n"}# file's name when executing the
command.
{"\n"}# Note that we don't use globbing here, but instead pass the *
to find.{"\n"}# find will then interpret the * as a wildcard.{"\n"}
find -name '*.tmp' -exec rm '{"{"}
{"}"}'{"\n"}
{"\n"}# finds all files in the directory 'files' and prints their size
and path{"\n"}find 'files/' -printf '%s %p\n'{" "}
</code>
</pre>
<p>
<code>find</code> has many options that allow you to perform arbitrary
actions on the files it found or pretty-print the output.
</p>
<p>
<strong>sort</strong>
</p>
<p>
<code>sort</code> sorts lines of text files, or lines it reads from{" "}
<em>stdin</em>.
</p>
<pre>
<code>
sort listOfNames.txt # sorts all lines in listOfNames.txt
alphabetically
</code>
</pre>
<p>
<strong>head and tail</strong>
</p>
<p>
<code>head</code> and <code>tail</code> can be used to show the
beginning or the end of a long stream of text, respectively.
</p>
<pre>
<code>
# display the last few lines of the dmesg log{"\n"}dmesg | tail {"\n"}
{"\n"}# display only the first few lines of a very long text file
{"\n"}
head verylongtext.txt{" "}
</code>
</pre>
<p>
<strong>jshon</strong>
</p>
<p>
<code>jshon</code> is a very simple command-line json parser. It can
read data in json format and return specific values.
</p>
<p>
Its most important options are <code>-e</code> and <code>-a</code>.{" "}
<code>-e</code> extracts the value of a given key from a json array or
object:
</p>
<pre>
<code>
# Find the object with key "title" from a json object stored in the
file "json"{"\n"}jshon -e 'title' < 'json'
</code>
</pre>
<p>
The <code>-a</code> option maps all remaining options across the
currently selected element. It has to be combined with other options,
for example the <code>-e</code> option.
</p>
<pre>
<code>
# Find the names of all elements stored in the json object in file
"json"
{"\n"}jshon -a -e 'name' < 'json'
</code>
</pre>
<p>
<strong>shuf</strong>
</p>
<p>
<code>shuf</code> randomly permutates the lines of its input,
effectively <em>shuffling</em> them.
</p>
<pre>
<code>
# Shuffle the lines of file "playlist"{"\n"}shuf 'playlist'{"\n"}
{"\n"}# Get a random line from file "quotes"{"\n"}shuf -n 1 'quotes'
</code>
</pre>
<p>
<strong>tee</strong>
</p>
<p>
<code>tee</code> takes input from <em>stdin</em> and writes it to a
file:
</p>
<pre>
<code>sudo zypper up | tee 'updatelog'</code>
</pre>
<p>Note that this is equivalent to</p>
<pre>
<code>sudo zypper up > 'updatelog'</code>
</pre>
<p>
<code>tee</code> is useful when you want to write to a file that
requires root access. Then you can do the following:
</p>
<pre>
<code>echo 'some configuration' | sudo tee '/etc/systemconfig'</code>
</pre>
<p>
A normal redirection wouldn’t work in this case, as that would open the
file as a normal user instead of root.{" "}
<strong>Please be careful when modifying system files.</strong>
</p>
<p>
<strong>sleep</strong>
</p>
<p>
<code>sleep</code> pauses the execution of your script for a number of
seconds. It basically takes a number as an argument, then does nothing
for that number of seconds, and then returns. It can be useful if you
want to make your script do something in a loop, but want to throttle
the speed a little.
</p>
<pre>
<code>
# prints "Get back to work" once per second. {"\n"}# Note that 'true'
is a command that always returns 0.{"\n"}while true{"\n"}do{"\n"}
{" "}echo 'Get back to work!'{"\n"}
{" "}sleep 1{"\n"}done
</code>
</pre>
<p>
<strong>mplayer or mpv</strong>
</p>
<p>
<code>mplayer</code> and <code>mpv</code> are media players that can be
used from the console. <code>mpv</code> is based on <code>mplayer</code>{" "}
and a lot more modern, but they can do essentially the same.
</p>
<pre>
<code>
# play file "music.mp3"{"\n"}mplayer 'music.mp3'{"\n"}mpv 'music.mp3'
{"\n"}
{"\n"}# play file "alarm.mp3" in an infinite loop{"\n"}mplayer -loop 0
'alarm.mp3'{"\n"}mplayer --loop=inf 'alarm.mp3'{"\n"}
{"\n"}# play a youtube video{"\n"}# this only works with mpv and
requires youtube-dl to be installed{"\n"}mpv
'https://www.youtube.com/watch?v=lAIGb1lfpBw'
</code>
</pre>
<p>
<strong>xdotool</strong>
</p>
<p>
<code>xdotool</code> can simulate keypresses. With this command, you can
write macros that perform actions for you. It allows you to write
scripts that interact with GUI programs.
</p>
<pre>
<code>
# press ctrl+s (in order to save a document){"\n"}xdotool key
'Control+s'
{"\n"}
{"\n"}# type "hello"{"\n"}xdotool type 'hello'
</code>
</pre>
{/* end pandoc */}
</>
);
}