Skip to content
Snippets Groups Projects
bashguide.html 65.5 KiB
Newer Older
Alexander Schoch's avatar
Alexander Schoch committed
<!-- start pandoc -->

Alexander Schoch's avatar
Alexander Schoch committed
<h1 data-number="1" id="bash-quick-guide">
  <span class="header-section-number">1</span> Bash Quick Guide
</h1>
<p>
  Bash is an acronym for Bourne Again Shell. It is based on the Bourne shell and
  is mostly compatible with its features.
</p>
<p>
  Shells are command interpreters. They are applications that provide users with
  the ability to give commands to their operating system interactively, or to
  execute batches of commands quickly. In no way are they required for the
  execution of programs; they are merely a layer between system function calls
  and the user.
</p>
<p>
  Think of a shell as a way for you to speak to your system. Your system doesn’t
  need it for most of its work, but it is an excellent interface between you and
  what your system can offer. It allows you to perform basic math, run basic
  tests and execute applications. More importantly, it allows you to combine
  these operations and connect applications to each other to perform complex and
  automated tasks.
</p>
<p>
  Bash is not your operating system. It is not your window manager. It is not
  your terminal (but it often runs inside your terminal). It does not control
  your mouse or keyboard. It does not configure your system, activate your
  screen-saver, or open your files. It’s important to understand that bash is
  only an interface for you to execute statements (using bash syntax), either at
  the interactive bash prompt or via bash scripts. The things that
  <em>actually happen</em> are usually caused by other programs.
</p>
<p>
  This guide is based on
  <a href="http://mywiki.wooledge.org/BashGuide"
    >the bash guide in GreyCat’s wiki</a
  >
  and aims to be more concise, while still being accurate. It was produced
  specifically for the Bash Workshop by
  <a href="www.thealternative.ch">TheAlternative.ch</a>.
</p>
<p>
  It is published under the
  <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/"
    >CC by-nc-sa 4.0 license</a
  >.
</p>
<h1 data-number="2" id="commands-and-arguments">
  <span class="header-section-number">2</span> Commands and Arguments
</h1>
<p>
  Bash reads commands from its input, which can be either a file or your
  terminal. In general, each line is interpreted as a command followed by its
  arguments.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>ls
touch file1 file2 file3
ls -l
rm file1 file2 file3</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The first word is always the command that is executed. All subsequent words
  are given to that command as argument.
</p>
<p>
  Note that <em>options</em>, such as the <code>-l</code> option in the example
  above, are not treated specially by bash. They are arguments like any other.
  It is up to the program (<code>ls</code> in the above case) to treat it as an
  option.
</p>
<p>
  Words are delimited by whitespace (spaces or tabs). It does not matter how
  many spaces there are between two words. For example, try
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo Hello            World</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The process of splitting a line of text into words is called
  <em>word splitting</em>. It is vital to be aware of it, especially when you
  come across expansions later on.
</p>
<h2 data-number="2.1" id="preventing-word-splitting">
  <span class="header-section-number">2.1</span> Preventing Word Splitting
</h2>
<p>
  Sometimes, you will want to pass arguments to commands that contain
  whitespace. To do so, you can use quotes:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>touch &quot;Filename with spaces&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This command creates a single file named <em>Filename with spaces</em>. The
  text within double quotes is protected from word splitting and hence treated
  as a single word.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>Note that you can also use single quotes:</p>
<pre><code>touch &#39;Another filename with spaces&#39;</code></pre>
<p>There is, however, an important difference between the two:</p>
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li>Double quotes prevent <strong>word splitting</strong></li>
  <li>Single quotes prevent <strong>word splitting and expansion</strong></li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  When you use single quotes, the quoted text will never be changed by bash.
  With double quotes, expansion will still happen. This doesn’t make a
  difference in the above example, but as soon as you e.g. use variables, it
  becomes important.
</p>
<p>
  In general, it is considered good practice to use single quotes whenever
  possible, and double quotes only when expansion is desired. In that sense, the
  last example above can be considered “more correct”.
</p>
<h2 data-number="2.2" id="the-importance-of-spaces">
  <span class="header-section-number">2.2</span> The Importance of Spaces
</h2>
<p>
  Bash contains various keywords and built-ins that aren’t immediately
  recognizable as commands, such as the new test command:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>[[ -f file ]]</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The above code tests whether a file named “file” exists in the current
  directory. Just like every line of bash code, it consists of a command
  followed by its arguments. Here, the command is <code>[[</code>, while the
  arguments are <code>-f</code>, <code>file</code> and <code>]]</code>.
</p>
<p>
  Many programmers of other languages would write the above command like so:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>[[-f file]]</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This, though, is wrong: Bash will look for a command named <code>[[-f</code>,
  which doesn’t exist, and issue an error message. This kind of mistake is very
  common for beginners. It is advisable to always use spaces after any kind of
  brackets in bash, even though there are cases where they are not necessary.
</p>
<h2 data-number="2.3" id="scripts">
  <span class="header-section-number">2.3</span> Scripts
</h2>
<p>
  You have probably interacted with bash through a terminal before. You would
  see a bash prompt, and you would issue one command after another.
</p>
<p>
  Bash scripts are basically a sequence of commands stored in a file. They are
  read and processed in order, one after the other.
</p>
<p>
  Making a script is easy. Begin by making a new file, and put this on the first
  line:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>#!/bin/bash</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This line is called an <em>interpreter directive</em>, or more commonly, a
  <em>hashbang</em> or <em>shebang</em>. Your operating system uses it to
  determine how this file can be run. In this case, the file is to be run using
  <code>bash</code>, which is stored in the <code>/bin/</code> directory.
</p>
<p>
  After the shebang, you can add any command that you could also use in your
  terminal. For example, you could add
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo &#39;Hello World&#39;</code></pre>
<p>and then save the file as “myscript”</p>
<p>You can now run the file from the terminal by typing</p>
<pre><code>bash myscript</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Here, you explicitly called bash and made it execute the script.
  <code>bash</code> is used as the command, while <code>myscript</code> is an
  argument. However, it’s also possible to use <code>myscript</code> as a
  command directly.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>To do so, you must first make it executable:</p>
<pre><code>chmod +x myscript</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Now that you have permission to execute this script directly, you can type
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>./myscript</code></pre>
<p>to run it.</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The <code>./</code> is required to tell bash that the executable is located in
  the current directory, rather than the system directory. We will come back to
  this in the chapter on Variables.
</p>
<h1 data-number="3" id="variables-and-parameters">
  <span class="header-section-number">3</span> Variables and Parameters
</h1>
<p>
  Variables and parameters can be used to store strings and retrieve them later.
  <em>Variables</em> are the ones you create yourself, while
  <em>special parameters</em> are pre-set by bash. <em>Parameters</em> actually
  refers to both, but is often used synonymously to special parameters.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>To store a string in a variable, we use the <em>assignment syntax</em>:</p>
<pre><code>varname=vardata</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This sets the variable <code>varname</code> to contain the string
  <code>vardata</code>.
</p>
<p>
  Note that you cannot use spaces around the <code>=</code> sign. With the
  spaces, bash would assume <code>varname</code> to be a command and then pass
  <code>=</code> and <code>vardata</code> as arguments.
</p>
<p>
  To access the string that is now stored in the variable <code>varname</code>,
  we have to use <em>parameter expansion</em>. This is the most common kind of
  expansion: A variable is replaced with its content.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>If you want to print the variable’s value, you can type</p>
<pre><code>echo $varname</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The <code>$</code> indicates that you want to use expansion on
  <code>varname</code>, meaning it is replaced by its content. Note that
  expansion happens before the command is run. Here’s what happens step-by-step:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li>
    Bash uses variable expansion, changing <code>echo $varname</code> to
    <code>echo vardata</code>
  </li>
  <li>
    Then, bash runs <code>echo</code> with <code>vardata</code> as its
    parameter.
  </li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The most important thing here is that
  <strong>variable expansion happens before wordsplitting</strong>. That means,
  if you have defined a variable like this:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>myfile=&#39;bad song.mp3&#39;</code></pre>
<p>and then run the command</p>
<pre><code>rm $myfile</code></pre>
<p>bash will expand this to</p>
<pre><code>rm bad song.mp3</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Only now, word splitting occurs, and bash will call <code>rm</code> with two
  arguments: <code>bad</code> and <code>song.mp3</code>. If you now had a file
  called <code>song.mp3</code> in your current directory, that one would be
  deleted instead.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>To prevent this from happening, you can use double quotes:</p>
<pre><code>rm &quot;$myfile&quot;</code></pre>
<p>This will be expanded to</p>
<pre><code>rm &quot;bad song.mp3&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  which is what we want. In this case, you have to use double quotes, as single
  quotes would prevent expansion from happening altogether.
</p>
<p>
  Not quoting variable and parameter expansions is a very common mistake even
  among advanced bash programmers. It can cause bugs that are hard to find and
  can be very dangerous. <strong>Always quote your variable expansions.</strong>
</p>
<p>
  You can also use variable expansions inside the variable assignment itself.
  Consider this example:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>myvariable=&#39;blah&#39;
myvariable=&quot;$myvariable blah&quot;
echo &quot;$myvariable&quot;</code></pre>
<p>What will the output of this script be?</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  First, the variable <code>myvariable</code> will get the value
  <code>blah</code>. Then, <code>myvariable</code> is assigned to again, which
  overwrites its former content. The assignment contains a variable expansion,
  <code>"$myvariable blah"</code>. This is expanded to <code>"blah blah"</code>,
  and that is going to be the new value of <code>myvariable</code>. So the last
  command is expanded to <code>echo "blah blah"</code>, and the output of the
  script is <code>blah blah</code>.
</p>
<h2 data-number="3.1" id="special-parameters">
  <span class="header-section-number">3.1</span> Special Parameters
</h2>
<p>
  <em>Special parameters</em> are variables that are set by bash itself. Most of
  those variables can’t be written to and they contain useful information.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<table>
Alexander Schoch's avatar
Alexander Schoch committed
  <colgroup>
    <col style="width: 12%" />
    <col style="width: 12%" />
    <col style="width: 75%" />
  </colgroup>
  <thead>
    <tr class="header">
      <th>Parameter Name</th>
      <th>Usage</th>
      <th>Description</th>
    </tr>
  </thead>
  <tbody>
    <tr class="odd">
      <td><code>0</code></td>
      <td><code>"$0"</code></td>
      <td>Contains the name of the current script</td>
    </tr>
    <tr class="even">
      <td><code>1</code> <code>2</code> <code>3</code> etc.</td>
      <td><code>"$1"</code> etc.</td>
      <td>
        Contains the arguments that were passed to the current script. The
        number indicates the position of that argument (first, second…). These
        parameters are also called positional parameters.
      </td>
    </tr>
    <tr class="odd">
      <td><code>*</code></td>
      <td><code>"$*"</code></td>
      <td>
        Contains all the positional parameters. Double quoted, it expands to a
        single word containing them all.
      </td>
    </tr>
    <tr class="even">
      <td><code>@</code></td>
      <td><code>"$@"</code></td>
      <td>
        Contains all the positional parameters. Double quoted, it expands to
        <strong>several words, where each word is one parameter</strong>. This
        is special syntax, it behaves differently from “normal” expansion in
        quotes. It retains the arguments exactly as they were passed to the
        script.
      </td>
    </tr>
    <tr class="odd">
      <td><code>#</code></td>
      <td><code>"$#"</code></td>
      <td>Contains the number of parameters that were passed to the script</td>
    </tr>
    <tr class="even">
      <td><code>?</code></td>
      <td><code>"$?"</code></td>
      <td>Contains the exit code of the last executed command</td>
    </tr>
  </tbody>
Alexander Schoch's avatar
Alexander Schoch committed
</table>
Alexander Schoch's avatar
Alexander Schoch committed
<h2 data-number="3.2" id="environment-variables">
  <span class="header-section-number">3.2</span> Environment Variables
</h2>
<p>
  <em>Environment variables</em> are special variables that are already set when
  you start bash. In fact, these variables aren’t specific to bash - they are
  available in every program that runs on your system and they can affect your
  system’s behaviour.
</p>
<p>
  You can use the command <code>printenv</code> in your terminal to display all
  the environment variables you have currently set. Most of them contain some
  system configuration, like the variable <code>LANG</code>, which designates
  your preferred language. Some variables, like <code>TERM</code>,
  <code>BROWSER</code> or <code>SHELL</code>, designate your preferred default
  programs (terminal, web browser and shell, respectively. These may not be set
  on all systems).
</p>
<p>
  Some of these variables can be useful in your scripts. For example, the
  variable <code>RANDOM</code> gives you a different random number every time
  you read it.
</p>
<p>
  Another important environment variable is <code>PATH</code>. It contains a
  bunch of file paths, separated by colons. These paths designate where your
  system will look for executables when you type a command. For example, if you
  type <code>grep</code> in your terminal, your system will search for an
  executable called <code>grep</code> in the directories designated in your
  <code>PATH</code> variable. As soon as it finds one, it will execute that. If
  it doesn’t find it, you will get a “command not found” error message.
</p>
<p>
  You can modify your environment variables, if you want. The guideline here is
  to only mess with those variables of which you know what they do, otherwise
  you might break something.
</p>
<p>
  The place to modify these variables is your <code>~/.bash_profile</code> file.
  This file contains some bash code that is executed whenever you log in. For
  example, you could add the following line:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>export BROWSER=&quot;firefox&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This would set your default browser to firefox. Note that on some systems,
  there are other settings–for example in your Desktop Environment–which can
  override these environment variables. You’ll have to test whether this works.
</p>
<p>
  Note the <code>export</code> keyword. This is a bash builtin that takes a
  variable definition as its argument and puts it in your <em>environment</em>.
  If you omit this, your new variable will just be an ordinary variable, rather
  than an environment variable.
</p>
<h2 data-number="3.3" id="ambiguous-names">
  <span class="header-section-number">3.3</span> Ambiguous Names
</h2>
<p>
  Say you have a variable called <code>name</code> that is declared as follows:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>name=&#39;bat&#39;</code></pre>
<p>Now, you want to use this variable in order to print <em>batman</em>:</p>
<pre><code>echo &quot;$nameman&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  If you try this, you will notice that it doesn’t work–that is because bash
  will now look for a variable called <code>nameman</code>, which doesn’t exist.
  Here’s what you can do instead:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo &quot;${name}man&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The curly braces tell bash where the variable name ends. This allows you to
  add more characters at the end of a variable’s content.
</p>
<h1 data-number="4" id="globbing">
  <span class="header-section-number">4</span> Globbing
</h1>
<p>
  <em>Globs</em> are an important bash concept–mostly for their incredible
  convenience. They are patterns that can be used to match filenames or other
  strings.
</p>
<p>
  Globs are composed of normal characters and metacharacters. Metacharacters are
  characters that have a special meaning. These are the metacharacters that can
  be used in globs:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li>
    <code>*</code>: Matches any string, including the empty string
    (i.e. nothing)
  </li>
  <li><code>?</code>: Matches any single character</li>
  <li>
    <code>[...]</code>: Matches any one of the characters enclosed in the
    brackets
  </li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Bash sees the glob, for example <code>a*</code>. It expands this glob, by
  looking in the current directory and matching it against all files there. Any
  filenames that match the glob are gathered up and sorted, and then the list of
  filenames is used in place of the glob. So if you have three files
  <code>a</code>, <code>b</code> and <code>albert</code> in the current
  directory, the glob is expanded to <code>a albert</code>.
</p>
<p>
  A glob always has to match the entire filename. That means
  <code>a*</code> will match <code>at</code> but not <code>bat</code>.
</p>
<p>
  Note that globbing is special in that it happens
  <em>after word splitting</em>. This means you never need to worry about spaces
  in filenames when you use globbing, and quoting globs is not necessary. In
  fact, quotes will prevent globbing from happening.
</p>
<h1 data-number="5" id="expansion">
  <span class="header-section-number">5</span> Expansion
</h1>
<p>
  We’ve already seen <em>parameter and variable expansion</em>, but that’s not
  the only kind of expansion that happens in bash. In this chapter, we’ll look
  at all kinds of expansion that aren’t covered elsewhere.
</p>
<h2 data-number="5.1" id="expansions-and-quotes">
  <span class="header-section-number">5.1</span> Expansions and Quotes
</h2>
<p>
  You already know that it is important to quote parameter and variable
  expansions, but we also told you that quoting globs–which are, in fact, just
  another form of expansion–is not necessary. So, which expansions need to be
  quoted?
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>The rule of thumb here is as follows:</p>
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li>
    Always quote <strong>paramater expansion</strong>,
    <strong>command substitution</strong> and
    <strong>arithmetic expansion</strong>
  </li>
  <li>
    Never quote <strong>brace expansion</strong>,
    <strong>tilde expansion</strong> and <strong>globs</strong>
  </li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The handy thing here is: All the expansions that require quoting have a
  <code>$</code> in their syntax. Parameter expansion is simply a
  <code>$</code> followed by a parameter name. Command substitution starts with
  a <code>$(</code>, and arithmetic expansion starts with <code>$((</code>.
</p>
<p>
  So, the rule of thumb breaks down to the following:
  <strong>If there’s a dollar, you probably need quotes.</strong>
</p>
<p>
  Now, what if you want to use two kinds of expansion in the same line, but one
  requires quotes and the other doesn’t? Consider the following script:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>prefix=&#39;my picture&#39;
rm ~/pictures/$prefix*</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Here, we use tilde expansion, parameter expansion and globbing in order to
  remove all files that start with <code>my picture</code> in the folder
  <code>/home/username/pictures/</code>. But because quotes prevent tilde
  expansion and globbing, we cannot quote the entire expression. This means that
  the parameter expansion, too, goes unquoted–and this is fatal, because our
  variable contains a space. So what should we do?
</p>
<p>
  The important thing to realize here is that quoting simply prevents word
  splitting, but it does not actually designate something as a single string. So
  we can do the following:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>prefix=&#39;my picture&#39;
rm ~/pictures/&quot;$prefix&quot;*</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Only the parameter expansion is quoted, so it is protected from word
  splitting. But that does not automatically separate it from the rest of the
  string. Note that there are no spaces between <code>"$prefix"</code> and
  <code>~/pictures/</code>. Since word splitting only happens when there are
  spaces, the entire thing will not be split. Here’s what happens, in order:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>First, tilde expansion occurs:</p>
<pre><code>rm /home/username/pictures/&quot;$prefix&quot;/*</code></pre>
<p>Next, parameter expansion:</p>
<pre><code>rm /home/username/pictures/&quot;my picture&quot;*</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  At this point, word splitting happens. But since the only space in our
  argument is in quotes, the argument remains intact.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>And last, globbing:</p>
<pre><code>rm /home/username/pictures/&quot;my picture&quot;001.jpg /home/username/pictures/&quot;my picture&quot;002.jpg</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Now, there’s one last step that happens which we didn’t mention before. It’s
  called <em>quote removal</em>. All the quotes that were needed to prevent word
  splitting are now ignored, which means that the arguments that are finally
  given to <code>rm</code> are:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li><code>/home/username/pictures/my picture001.jpg</code></li>
  <li><code>/home/username/pictures/my picture002.jpg</code></li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
<p>and this is exactly what we wanted.</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  So, remember: Quotes don’t need to be at the beginning or end of an argument,
  and if you use several kinds of expansion together, you can add quotes in the
  middle as required.
</p>
<h2 data-number="5.2" id="expansion-order">
  <span class="header-section-number">5.2</span> Expansion Order
</h2>
<p>
  All the kinds of expansion happen in a certain order. The order is as follows:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li>Brace expansion</li>
  <li>Tilde expansion</li>
  <li>Parameter and variable expansion</li>
  <li>Command substitution</li>
  <li>Arithmetic expansion</li>
  <li>Word splitting</li>
  <li>Globbing</li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
Alexander Schoch's avatar
Alexander Schoch committed
<h2 data-number="5.3" id="brace-expansion">
  <span class="header-section-number">5.3</span> Brace Expansion
</h2>
<p>
  <em>Brace expansions</em> are often used in conjunction with globs, but they
  also have other uses. They always expand to all possible permutations of their
  contents. Here’s an example:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>$ echo th{e,a}n
then than
$ echo {1..9}
1 2 3 4 5 6 7 8 9
$ echo {0,1}{0..9}
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Brace expansions are replaced by a list of words. They are often used in
  conjunction with globs to match specific files but not others. For example, if
  you want to delete pictures from your pictures folder with filenames
  IMG020.jpg through IMG039.jpg, you could use the following pattern:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>rm IMG0{2,3}*.jpg</code></pre>
<p>Note that we don’t use quotes here. Quotes prevent brace expansion.</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Brace expansion happens before globbing, so in the above example, the braces
  are expanded to
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>rm IMG02*.jpg IMG03*.jpg</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  We end up with two glob patterns, the first matches IMG020.jpg through
  IMG029.jpg, and the second matches IMG030.jpg through IMG039.jpg.
</p>
<h2 data-number="5.4" id="tilde-expansion">
  <span class="header-section-number">5.4</span> Tilde Expansion
</h2>
<p>
  You have probably already seen and used the tilde in the terminal. It is a
  shortcut to your home directory:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>cd ~/files</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This will be expanded to <code>cd /home/username/files</code>. Note that tilde
  expansion only happens outside of quotes, so the following won’t work:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>cd &quot;~/files&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<h2 data-number="5.5" id="parameter-and-variable-expansion">
  <span class="header-section-number">5.5</span> Parameter and Variable
  Expansion
</h2>
<p>
  <em>Parameter and variable expansion</em> is explained in the chapter on
  Variables and Parameters.
</p>
<p>
  An important sidenote here is that Parameter expansion and Variable expansion
  often refer to the same thing. The official name as per the bash manual is
  <em>Parameter expansion</em>, but <em>Variable expansion</em> is often used
  instead as it is less misleading.
</p>
<h2 data-number="5.6" id="command-substitution">
  <span class="header-section-number">5.6</span> Command Substitution
</h2>
<p>
  <em>Command substitution</em> is a way of using a command’s output inside your
  script. For example, let’s say you want to find out where your script is
  executed, and then move a file to that location.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo &#39;I am at&#39;
pwd</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This script will simply print the directory it is called from. But you want to
  move a file to that directory, which means you need to use it as an argument
  to the <code>mv</code> command. For that, you need command substitution:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mv ~/myfile &quot;$( pwd )&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The <code>$(</code> introduces a command substitution. It works similarly to
  variable expansion, but instead of putting a variable’s content, it puts a
  command’s output in that place. In this example, <code>pwd</code> is run, and
  the output is (for example) <code>/home/username/scripts/</code>. Then, the
  entire command is expanded to
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mv ~/myfile &quot;/home/username/scripts/&quot;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Note that with this kind of expansion, quotes are important. The path returned
  by <code>pwd</code> might have contained a space, in which case we need the
  argument to be properly quoted.
</p>
<h2 data-number="5.7" id="arithmetic-expansion">
  <span class="header-section-number">5.7</span> Arithmetic Expansion
</h2>
<p>
  <em>Arithmetic expansion</em> is explained in the chapter on arithmetic
  evaluation.
</p>
<h2 data-number="5.8" id="globbing-1">
  <span class="header-section-number">5.8</span> Globbing
</h2>
Alexander Schoch's avatar
Alexander Schoch committed
<p><em>Globbing</em> is so important, it has a chapter of its own.</p>
Alexander Schoch's avatar
Alexander Schoch committed
<h1 data-number="6" id="tests-and-conditionals">
  <span class="header-section-number">6</span> Tests and Conditionals
</h1>
<p>
  Every command you run in your terminal or shell script has a return value.
  That value is a number, and by convention, a return value of 0 means
  <em>success</em>, while any other number indicates an error.
</p>
<p>
  You usually don’t see the return value after running a command. What you do
  see is the command’s <em>output</em>. If you want to know a command’s return
  value, you can read it by issuing
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo &quot;$?&quot;</code></pre>
<p>immediately after running that command.</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  While you often don’t need to know a command’s return value, it can be useful
  to construct conditionals and thereby achieve advanced logic.
</p>
<h2 data-number="6.1" id="control-operators-and">
  <span class="header-section-number">6.1</span> Control Operators (&amp;&amp;
  and ||)
</h2>
<p>
  Control operators can be used to make a command’s execution depend on another
  command’s success. This concept is called <em>conditional execution</em>:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mkdir folder &amp;&amp; cd folder</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  In the above example, the <code>&amp;&amp;</code> operator is used to connect
  two commands <code>mkdir folder</code> and <code>cd folder</code>. Using this
  connection, bash will first execute <code>mkdir folder</code>, and then
  execute <code>cd folder</code>
  <em>only if the first command was successful</em>.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mkdir folder || echo &#39;Error: could not create folder&#39;</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  In this example, the <code>||</code> operator is used to connect the commands.
  Here, the second command is executed
  <em>only if the first command failed</em>.
</p>
<p>
  It is good practice to make your own scripts return an error (i.e. something
  other than 0) whenever something goes wrong. To do that, you can use this
  construct:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mkdir folder || exit 1</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The <code>exit</code> command immediately stops the execution of your script
  and makes it return the number you specified as an argument. In this example,
  your script will attempt to create a folder. If that goes wrong, it will
  immediately stop and return 1.
</p>
<p>
  Control operators can be written more legibly by spreading them across
  multiple lines:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>mkdir folder \
    || exit 1</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The backslash at the end of the first line makes bash ignore that line break
  and treat both lines as a single command plus arguments.
</p>
<h2 data-number="6.2" id="grouping-commands">
  <span class="header-section-number">6.2</span> Grouping Commands
</h2>
<p>
  Now, what if you want to execute multiple commands if the first one fails?
  Going from the example above, when <code>mkdir folder</code> fails, you might
  want to print an error message <em>and</em> return 1.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>This can be done by enclosing these two commands in single curly braces:</p>
<pre><code>mkdir folder || { echo &#39;Could not create folder&#39;; exit 1; }</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The two commands in curly braces are treated as an unit, and if the first
  command fails, both will be executed.
</p>
<p>
  Note that you need to include spaces between the curly braces and the
  commands. If there were no spaces, bash would look for a command named
  <code>{echo</code> and fail to find it.
</p>
<p>
  There’s a semicolon separating the two commands. The semicolon has the same
  function as a line break: it makes bash consider both parts as individual
  commands-plus-arguments.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>The example above could be rewritten as follows:</p>
<pre><code>mkdir folder || {
    echo &#39;Could not create folder&#39;
    exit 1
}</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Here, no semicolon is required, because there is a line break between the two
  statements. Line breaks and semicolons can be used interchangeably.
</p>
<h2 data-number="6.3" id="conditional-blocks-if-and">
  <span class="header-section-number">6.3</span> Conditional Blocks (if and [[)
</h2>
<p>
  <code>if</code> is a shell keyword. It first executes a command and looks at
  its return value. If it was 0 (success), it executes one list of commands, and
  if it was something else (failure), it executes another.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>It looks like this:</p>
<pre><code>if true
then 
    echo &#39;It was true&#39;
else 
    echo &#39;It was false&#39;
fi</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  <code>true</code> is a bash builtin command that does nothing and returns 0.
  <code>if</code> will run that command, see that it returned 0, and then
  execute the commands between the <code>then</code> and the <code>else</code>.
</p>
<p>
  In many programming languages, operators such as <code>&gt;</code>,
  <code>&lt;</code> or <code>==</code> exist and can be used to compare values.
  In bash, operators don’t exist. But since comparing values is so common,
  there’s a special command that can do it:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>[[ a = b ]]</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  <code>[[</code> is a command that takes as its arguments a comparison. The
  last argument has to be a <code>]]</code>. It is made to look like a
  comparison in double brackets, but it is, in fact, a command like any other.
  It is also called the <em>new test command</em>. (The
  <em>old test command</em>, often simply called <em>test</em>, exists as well,
  so be careful not to confuse them).
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>For that reason, the spaces are absolutely needed. You cannot write this:</p>
<pre><code>[[a=b]]</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This will make bash look for a command called <code>[[a=b]]</code>, which
  doesn’t exist.
</p>
<p>
  <code>[[</code> does not only support comparing strings. For example,
  <code>[[ -f file ]]</code> will test whether a file named “file” exists.
  Here’s a list of the most common tests you can use with <code>[[</code>:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<ul>
Alexander Schoch's avatar
Alexander Schoch committed
  <li><code>-e FILE:</code> True if file exists.</li>
  <li><code>-f FILE:</code> True if file is a regular file.</li>
  <li><code>-d FILE:</code> True if file is a directory.</li>
  <li>
    String operators: * <code>-z STRING:</code> True if the string is empty (its
    length is zero). * <code>-n STRING:</code> True if the string is not empty
    (its length is not zero). * <code>STRING = STRING</code>: True if the string
    matches the glob pattern (if you quote the glob pattern, the strings have to
    match exactly). * <code>STRING != STRING</code>: True if the string does not
    match the glob pattern (if you quote the glob pattern, the strings just have
    to be different). * <code>STRING &lt; STRING</code>: True if the first
    string sorts before the second. * <code>STRING &gt; STRING</code>: True if
    the first string sorts after the second.
  </li>
  <li>
    <code>EXPR -a EXPR</code>: True if both expressions are true (logical AND).
  </li>
  <li>
    <code>EXPR -o EXPR</code>: True if either expression is true (logical OR).
  </li>
  <li>
    <code>! EXPR</code>: Inverts the result of the expression (logical NOT).
  </li>
  <li>
    <code>EXPR &amp;&amp; EXPR</code>: Much like the ‘-a’ operator of test, but
    does not evaluate the second expression if the first already turns out to be
    false.
  </li>
  <li>
    <code>EXPR || EXPR</code>: Much like the ‘-o’ operator of test, but does not
    evaluate the second expression if the first already turns out to be true.
  </li>
  <li>
    Numeric operators:
    <ul>
      <li><code>INT -eq INT</code>: True if both integers are equal.</li>
      <li><code>INT -ne INT</code>: True if the integers are not equal.</li>
      <li>
        <code>INT -lt INT</code>: True if the first integer is less than the
        second.
      </li>
      <li>
        <code>INT -gt INT</code>: True if the first integer is greater than the
        second.
      </li>
      <li>
        <code>INT -le INT</code>: True if the first integer is less than or
        equal to the second.
      </li>
      <li>
        <code>INT -ge INT</code>: True if the first integer is greater than or
        equal to the second.
      </li>
    </ul>
  </li>
Alexander Schoch's avatar
Alexander Schoch committed
</ul>
<p>You might occasionally come across something like this:</p>
<pre><code>[ a = b ]</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Here, we use single brackets instead of double brackets. This is, in fact, an
  entirely different command, the <code>[</code> command or
  <em>old test command</em>. It has the same purpose–comparing things–but the
  <code>[[</code> command is newer, has more features, and is easier to use. We
  strongly recommend using <code>[[</code> over <code>[</code>.
</p>
<h2 data-number="6.4" id="conditional-loops-while-until-and-for">
  <span class="header-section-number">6.4</span> Conditional Loops (while, until
  and for)
</h2>
<p>
  Loops can be used to repeat a list of commands multiple times. In bash, there
  are <code>while</code> loops and <code>for</code> loops.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>While loops look like this:</p>
<pre><code>while true
do
    echo &#39;Infinite loop&#39;
done</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The <code>while</code> keyword will execute the <code>true</code> command, and
  if that returns 0, it executes all commands between the <code>do</code> and
  <code>done</code>. After that, it starts over, until the
  <code>true</code> command returns 1 (which it never does, which is why this
  loop will run indefinitely).
</p>
<p>
  The above example might not be immediately useful, but you could also do
  something like this:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>while ping -c 1 -W 1 www.google.com
do
    echo &#39;Google still works!&#39;
done</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  There’s also a variation of the <code>while</code> loop, called
  <code>until</code>. It works similarly, except it only runs its command list
  when the first command <em>fails</em>:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>until ping -c 1 -W 1 www.google.com
do
    echo &#39;Google isn&#39;\&#39;&#39;t working!&#39;
done</code></pre>
<p><code>for</code> loops can be used to iterate over a list of strings:</p>
<pre><code>for var in 1 2 3
do
    echo &quot;$var&quot;
done</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  After the <code>for</code>, you specify a variable name. After the
  <code>in</code>, you list all the strings you want to iterate over.
</p>
<p>
  The loop works by setting the variable you specified to all the values from
  the list in turn, and then executing the command list for each of them.
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<p>This is especially useful in combination with globs or brace expansions:</p>
<pre><code>echo &#39;This is a list of all my files starting with f:&#39;
for var in f*
do
    echo &quot;$var&quot;
done

echo &#39;And now I will count from 1 to 100:&#39;
for var in {1..100}
do
    echo &quot;$var&quot;
done</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<h2 data-number="6.5" id="choices-case-and-select">
  <span class="header-section-number">6.5</span> Choices (case and select)
</h2>
<p>
  Sometimes, you want your script to behave differently depending on the content
  of a variable. This could be implemented by taking a different branch of an if
  statement for each state:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>if [[ &quot;$LANG&quot; = &#39;en&#39; ]]
then
    echo &#39;Hello!&#39;
elif [[ &quot;$LANG&quot; = &#39;de&#39; ]]
then
    echo &#39;Guten Tag!&#39;
elif [[ &quot;$LANG&quot; = &#39;it&#39; ]]
then
    echo &#39;Ciao!&#39;
else
    echo &#39;I do not speak your language.&#39;
fi</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  This is quite cumbersome to write. At the same time, constructs like this are
  very common. For that reason, bash provides a keyword to simplify it:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>case &quot;$LANG&quot; in
    en)
        echo &#39;Hello!&#39;
        ;;
    de) 
        echo &#39;Guten Tag!&#39;
        ;;
    it)
        echo &#39;Ciao!&#39;
        ;;
    *)
        echo &#39;I do not speak your language.&#39;
        ;;
esac</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  Each choice of the case statement consists of a string or glob pattern, a
  <code>)</code>, a list of commands that is to be executed if the string
  matches the pattern, and two semicolons to denote the end of a list of
  commands.
</p>
<p>
  The string after the keyword <code>case</code> is matched against each glob
  pattern in order. The list of commands after the first match is executed.
  After that, execution continues after the <code>esac</code>.
</p>
<p>
  Since the string is matched against glob patterns, we can use
  <code>*</code> in the end to catch anything that didn’t match before.
</p>
<p>
  Another construct of choice is the <code>select</code> construct. It looks and
  works similarly to a loop, but it also presents the user with a predefined
  choice. You are encouraged to try running this example yourself:
</p>
Alexander Schoch's avatar
Alexander Schoch committed
<pre><code>echo &#39;Which one of these does not belong in the group?&#39;
select choice in Apples Pears Crisps Lemons Kiwis
do
    if [[ &quot;$choice&quot; = Crisps ]]
    then
        echo &#39;Correct! Crisps are not fruit.&#39;
        break
    fi
    echo &#39;Wrong answer. Try again.&#39;
done</code></pre>
Alexander Schoch's avatar
Alexander Schoch committed
<p>
  The syntax of the <code>select</code> construct is very similar to
  <code>for</code> loops. The difference is that instead of setting the variable
  (<code>choice</code> in this example) to each value in turn, the