Skip to content
Snippets Groups Projects
Commit 2654b110 authored by Aline Abler's avatar Aline Abler
Browse files

Last year's stuff

parent 86618960
No related branches found
No related tags found
No related merge requests found
Showing
with 2988 additions and 0 deletions
*.toc
*.log
*.aux
*.nav
*.snm
*.out
*.pdf
*.synctex.gz
*.vrb
*~
*.swp
guide/bashguide.html
- Reformulate PATH exercise for increased security and decreased chance of bricking system
- Meditate over the brightness script
#!/bin/bash
echo 'Guide'
(
cd guide
./build.sh | sed 's/^/- /'
)
echo 'Presentation'
(
cd pres
./build.sh | sed 's/^/- /'
)
echo 'Exercise sheet'
(
cd exercisesheet
./build.sh | sed 's/^/- /'
)
#!/bin/bash
echo "Building exercises pdf..."
pandoc sheet.md --listings --toc -V links-as-notes -H listings-setup.tex -o sheet.pdf \
&& echo "Build successful"
echo "Building solutions pdf..."
pandoc sol.md --listings -H listings-setup.tex -o sol.pdf \
&& echo "Build successful"
\usepackage{xcolor}
\usepackage[margin=2cm]{geometry}
%various gray colors
\definecolor{slg}{gray}{0.25}
\definecolor{lg}{gray}{0.55}
\definecolor{vlg}{gray}{0.73}
\definecolor{tlg}{gray}{0.9}
%TheAlt colors
\definecolor{ldorange}{HTML}{F18A20}
\colorlet{ldbright}{ldorange!70!white} % tinted version of orange
\definecolor{ldblue}{HTML}{254471}
%%Theme colors
%\definecolor{thgreen}{HTML}{A1A92C}
%\definecolor{thmauve}{HTML}{97284D}
%Theme colors
\definecolor{thgreen}{HTML}{539727}
\definecolor{thmauve}{HTML}{572272}
\lstdefinestyle{custombash}{
belowcaptionskip=1\baselineskip,
captionpos=,
breaklines=true,
frame=L,
xleftmargin=\parindent,
language=bash,
morestring=[b]',
morekeywords=[2]{sudo,zypper,notify-send,feh,youtube-dl,sort,tee,head,tail,shuf,mpv,find,convert,xrandr,curl,wget,grep,xdotool,rm,cp,mv,touch,bash,chmod,mkdir,rsync,mplayer,mpv,xdotool,jshon},
showstringspaces=false,
basicstyle=\scriptsize\ttfamily,
rulecolor=\color{tlg},
backgroundcolor=\color{tlg},
fillcolor=\color{tlg},
rulesepcolor=\color{tlg},
commentstyle=\itshape\color{thmauve!60!black},
keywordstyle=\bfseries\color{thgreen},
identifierstyle=\color{ldblue},
stringstyle=\color{thmauve},
}
\lstset{
basicstyle=\ttfamily,
backgroundcolor=\color{black!10},
showspaces=false,
showstringspaces=false,
showtabs=false,
tabsize=2,
captionpos=b,
breaklines=true,
breakatwhitespace=true,
breakautoindent=true,
linewidth=\textwidth,
style=custombash
}
# About this course
This document provides complementary exercises that you can solve as you go
through the bash guide. For each exercise, the sections of the course that are
required to solve it are listed. Some also rely on external tutorials.
You don’t need to solve all the exercises. You can pick the ones you find
interesting. Some exercises, though, are labelled *recommended*. These either
provide you with useful ’building blocks’ for future exercises and scripts of
your own, or test some basic knowledge that you will need.
We recommend you start with the easy exercises, and once you feel confident,
continue with the more challenging ones. If you run into problems, you can ask
our staff members for help.
## Recommended chapters
Every exercise has a list of "requirements" that designate chapters which you
will need to solve that specific exercise. However, there are some chapters
that we recommend you read first, before you start any exercise, as they
contain important knowledge that will prevent you from making simple "beginner
mistakes".
Please read at least
* 2 Commands and Arguments
* 3 Variables and Parameters
* 5.1 Expansion and Quotes
* 5.2 Expansion Order
before commencing with the exercises.
## A note about solutions
We have master solutions for every exercise, and the requirements and tips are
designed to guide you towards that master solution. However, our master
solutions are certainly not the only way of solving an exercise, and might not
necessarily be the best way. If you think you're onto something, but the tips
don't seem to make sense for your solution, you're encouraged to disregard them
and solve the task your own way.
# Exercises
## Your first script (recommended)
Write a simple script that prints “Hello, World” to the console when called.
Most of these steps are required for every script you write and won’t be
mentioned again in future exercises.
**Requirements:**
* 2.3 Scripts
**Tips:**
* Create a new file named `hello`
* The first line in this file should be a shebang that declares it as a bash
script.
* Then, your script should invoke `echo` and print `Hello, World`.
* Save your script.
* Make your script file executable by invoking `chmod a+x hello` from your
console.
* Now, try your script by invoking `./hello` from your console.
## Count to 100 (recommended)
Write a script that prints the numbers from 1 to 100 to your terminal.
**Requirements**
* 3 Variables and Parameters
* 5.3 Brace Expansion
* 6.4 Conditional Loops
## Set up a script directory for your user (recommended)
Create a directory in which you can put your bash scripts. The directory should
be set up such that you can invoke all the scripts therein simply by typing
their name in a console, rather than the full path.
**Requirements:**
* 3 Variables and Parameters
* 3.2 Environment Variables
**Tips:**
* Create a directory for your scripts. For example, you could use `~/scripts`.
* You will modify the environment variable `$PATH`. Append the path of your
scripts directory to the end of `$PATH`, separated from the rest by a colon.
* You can append to a variable by simply using expansion. For example
`variable="$variable:newtext"` will append `:newtext` to `variable`.
* **Messing with `PATH` can be dangerous!** Please test your modifications in a
terminal before making them permanent.
* In order to make your new `PATH` permanent, you can edit your profile file.
It should be stored in `~/.profile`. If it is not there, you could use
`~/.bash_profile` or `~/.bashrc`.
* The new `PATH` will only become active when you log out and back in, or open
a new terminal (depending on which file you used).
## Parse some options (recommended)
Write a script that parses some options and prints out which options have been
set and which not. While this is not immediately useful, you can reuse it for
many other scripts.
**Requirements:**
* 6.4 Conditional Loops
* 6.5 Choices
* 7.1 Command-line Arguments
**Tips:**
* You can use `getopts` to simplify option parsing.
* `getopts` takes two arguments: an option string and a name.
* The option string contains all the characters that are valid options. If
a character is followed by a colon (:), that option is expected to have
an argument.
* The name is simply the name of a shell variable. `getopts` will set that
variable to the option it found.
* `getopts` needs to be called repeatedly. It will produce a new option with
every call, until there are none left, in which case it returns an error.
That makes it ideal to be used in a `while` loop.
* In the loop body, you need to find out which option was set and print it. You
can use a case block for this.
* Include at least one option that takes an argument. Your script should also
print the argument if that option was set.
* An option’s argument is automatically stored in the shell variable `$OPTARG`.
* If the user provides an invalid option, the variable specified by `getopts`
name argument is set to a question mark.
* Your script should provide a help option, `-h`, which prints all available
options.
* Your script should print an error message if the user provides an invalid
option.
## Output all your arguments (recommended)
Write a script that outputs all the arguments given to it--similar to
`echo`--but makes it clear where one argument ends and the next begins by
enclosing them in angle brackets.
This script actually has some uses--you can find out how exactly your arguments
were affected during word splitting, which is useful for debugging.
**Requirements:**
* 2.1 Preventing Word Splitting
* 3 Variables and Parameters
* 3.1 Special Parameters
* 6.4 Conditional Loops
* 7.1 Command-line Arguments
**Tips:**
* Make sure your arguments are always preserved exactly as they were given.
* You can use a loop to process each argument in turn.
* Note that angle brackets (`<` and `>`) have a special meaning in bash (namely
redirection). That means you have to properly quote them.
## Find big files
Write a script that takes a directory as an argument and prints the five
biggest files found in that directory. You may print the 5 biggest files found
recursively in that directory.
**Requirements:**
* 3.1 Special Parameters
* 6.1 Control Operators
* 7.4 Pipes
**Tips:**
* Your script should take exactly one argument (the directory). If it gets more
or fewer arguments, make it print an error.
* Test whether the given argument is a directory. If not, print an error.
* Have a look at `find`. Make it print a file’s size.
* Find a way to sort the output of `find`. Your script should output the five
largest files found using `find`.
## Self-reproducing script
Write a script that replicates itself, i.e. copies itself to a new file called
backup.sh.
**Requirements:**
* 3.1 Special Parameters
* 7.3 Redirection
**Tips:**
* Do not hardcode the script's file name. Use a special parameter instead, so
that your script still works when moved to another directory.
* This exercise can be done in multiple ways. Try the following:
* Find and use a command that does exactly what you want.
* Use `cat` and a redirection.
## Make your bash prompt fancy
Your bash prompt is what you see in a terminal when you enter commands. It can
be modified by setting the variable `PS1`. Try setting `PS1` to something
different from your terminal.
In this exercise, you will customize your bash prompt. There are some things
you need to know first, though:
* Within your command prompt, you can use several *escape sequences* that will
be replaced by some interesting information:
| Sequence | Decoding |
|----------|----------|
|`\d` | the date in "Weekday Month Date" format (e.g., "Tue May 26") |
|`\e` | an ASCII escape character (033) |
|`\h` | the hostname up to the first '.' |
|`\H` | the hostname |
|`\j` | the number of jobs currently managed by the shell |
|`\n` | a newline |
|`\s` | the name of the shell |
|`\t` | the current time in 24-hour HH:MM:SS format |
|`\T` | the current time in 12-hour HH:MM:SS format |
|`\@` | the current time in 12-hour am/pm format |
|`\u` | the username of the current user |
|`\v` | the version of bash (e.g., 2.00) |
|`\w` | the current working directory |
|`\W` | the basename of the current working directory |
|`\!` | the history number of this command |
|`\#` | the command number of this command |
|`\$` | if you're logged in as root, this prints a #, otherwise a $ |
|`\\` | a backslash |
* You can use the following codes to make your prompt colorful:
| Code | Color |
|----------------|-------|
| `\[\e[0;30m\]` | black |
| `\[\e[0;34m\]` | blue |
| `\[\e[0;32m\]` | green |
| `\[\e[0;36m\]` | cyan |
| `\[\e[0;31m\]` | red |
| `\[\e[0;35m\]` | purple |
| `\[\e[0;33m\]` | brown |
| `\[\e[0;37m\]` | gray |
| `\[\e[1;30m\]` | dark gray |
| `\[\e[1;34m\]` | light blue |
| `\[\e[1;32m\]` | light green |
| `\[\e[1;36m\]` | light cyan |
| `\[\e[1;31m\]` | light red |
| `\[\e[1;35m\]` | light purple |
| `\[\e[1;33m\]` | yellow |
| `\[\e[1;37m\]` | white |
| `\[\e[0m\]` | back to normal |
Now that you know this, write a script that adjusts your bash prompt to your
liking.
**Requirements:**
* 3 Variables and Parameters
**Tips:**
* This endeavour becomes a lot easier if you define variables for the color
codes you need. Be aware that you need to quote these codes properly.
* Set your `PS1` to something you like. Use the variables you defined to change
the color.
* At the end of your prompt, you should use the code for *back to normal*. If
you don't do that, the output of your commands will be colored as well.
* Your prompt will change when you run the script from a terminal. If you want
to make the changes permanent, you can execute your script from the
`~/.bashrc` file--this file is executed whenever you open a terminal.
## Maze generator
Write a script that randomly outputs `/` or `\` in a loop in order to generate
an ASCII maze.
**Requirements**
* 3.2 Environment Variables
* 6.1 Control Operators
* 6.4 Conditional Loops
* 8.3 Arithmetic Evaluation
**Tips:**
* First, find a way to generate random numbers.
* Based on that random number, print a `\` or `/`.
* You want all your `\`s and `/`s to be on the same line. Look at `echo`'s `-n`
option to achieve that.
* It's a good idea to slow down your script a bit. Check out the `sleep`
command.
## Simple backup script
Write a script that backs up all files in your home directory that have been
modified in the last 24 hours and packs them in a tarball (which is a
compressed `.tar.gz` file).
**Requirements:**
* 7.4 Pipes
**Tips:**
* A great utility for finding files with certain properties is the `find`
command.
* By default, `find` also lists directories. You want to make sure it only
lists files.
* Pipes can be used to redirect a command's stdout to another command's stdin.
However, sometimes, you may need to use one command's stdout as *arguments*
to another command. To achieve that, you can use `xargs`.
## Screen brightness control
Write a script that takes a numeric argument and adjusts your screen brightness
accordingly.
**Requirements:**
* 3 Variables and Parameters
* 7.1 Commands and Arguments
**Tips:**
* Have a look at `/sys/class/backlight`. Find a file inside that directory or
one of its subdirectories that lets you control the brightness.
* Be sure not to set your brightness to 0 while experimenting, as that usually
makes your screen turn pitch black.
* You need `sudo` to write to that file. Because of that, you can't use
redirection. Have a look at `tee` to circumvent this problem.
Extend your script to allow absolute and relative changes (for example
’brightness 40’ or ’brightness +10’) and to prevent turning off your screen
entirely by setting the brightness too low.
## Automatic update script
Write a script that installs all available updates, then shuts down. If the
update process failed, the script should still shut down your laptop, but write
the error to a log file.
**Requirements:**
* 6.1 Control Operators
* 6.3 Conditional Blocks
* 7.3 Redirection
**Tips:**
* The easiest way is to create the log file either way, but remove it when the
updates were successful.
* If there already exists a logfile, you should move it first so it doesn't get
overwritten. Have a look at `mv`'s `-b` option for safe moving.
* It is possible use a control operator after a redirection.
## Disable/Enable external monitor output
Write a script that changes your monitor settings. Depending on what you need,
there are many variations for this: you might want a script to simply turn your
external monitor off (even when it's still plugged in), or you can go for more
advanced actions like turning off your internal monitor and only displaying on
your external monitor.
**Requirements:**
* 6.3 Conditional Blocks
**Tips:**
* Have a look at `xrandr`, a command to manage your monitors. Find out how you
can use it to turn outputs on or off.
* `xrandr` can also modify your monitor's orientation, resolution, or position.
* Create a script which uses xrandr to set up your monitors the way you like
it.
* If you have several monitor settings that you use often, create a script for
each of them.
* If you only have two settings and want your script to *toggle* between them,
you can use `xrandr` in combination with `grep` to detect whether a specific
monitor is on or not, and then act accordingly.
* Another possibility is to use xrandr to find out whether a specific monitor
is connected or not, and then configure your setup according to that.
## Display a random headline from Reddit
Write a script that pulls a random post title from a specific subreddit’s front
page and displays it. For example, you could use
https://www.reddit.com/r/Showerthoughts.
**Requirements:**
* 3 Variables and Parameters
* 5.6 Command Substitution
* 7.4 Pipes
**Tips:**
* Reddit has a json API. Just add ’.json’ to the subreddit’s URL. You might
want to use `curl` to look at the json page.
* You can use `wget` to get a local copy of the json page and test it out
locally on the file as input.
* You might need to change the user agent for `curl`. Have a look at the man
page, and google for suitable user agents.
* Find a way to parse the json output in order to get a list of post titles.
* Pick a random post title by generating a random number and picking the
corresponding title.
* You can use `sed` to format the output.
## Wallpaper change every 10 minutes
Write a script that changes your wallpaper every 10 minutes.
**Requirements:**
* 4 Globbing
* 6.4 Conditional Loops
**Tips:**
* Find a way to set your wallpaper from command line.
* You could either use a list of files, or pick a random file from a specific
directory.
* The `sleep` executable could come in handy.
* Make your script wait, then set the next wallpaper, then repeat.
## Your own TODO program
Write a script that maintains a todo list for you. You should be able to
display, remove and add todo items.
**Requirements:**
* 7.1 Command-line Arguments
* 7.2 File Descriptors
* 7.3 Redirection
* 9 Functions
**Tips:**
* Think about how you want to store your todo items. You could use plain text,
or something fancy like json.
* Add one functionality at a time. First make a script that adds todo items,
then extend it to display and finally delete them.
* Find a tool that is suitable for your storage format. (For example, if you
use plain text, text parsers such as `sed` or `grep` are suitable.)
* Optionally, extend your script so you can add priorities or deadlines to your
items.
## Reminder script
Write a script that takes a number of minutes and a string as parameter. After
that many minutes have passed, the script should display the string on your
screen to remind you.
**Requirements:**
* 3.1 Special Parameters
* 7.1 Command-line Arguments
* 7.4 Pipes
**Tips:**
* Have a look at `notify-send`, a command that can display pop-ups on your
screen.
* Have a look at `at`, a command that can be used to postpone execution of a
script to a given time.
* Find out how to combine these two in order to create a reminder script.
## Youtube downloader
Write a script that takes a string as input, searchs for that on youtube, and
then downloads the audio of the first match.
**Requirements:**
* 3 Variables and Parameters
* 7.4 Pipes
**Tips:**
* You can use `curl` to find the search results. Have a look at `curl`’s
`-data-urlencode` option.
* Browse to youtube and search for something. Look at the url of the results
page and find out how to get that page using `curl` and its `-data-urlencode`
option.
* Filter out all youtube video links from the results page. These links start
with *’watch?v=’* followed by an 11-character video ID.
* You can use `grep` for filtering. Have a look at it’s `-o` and `-E` options.
* Have a look at `youtube-dl` and find out how to use it.
* Make your script download the first video you found using `youtube-dl`.
## Web radio
Write a script that, when executed, starts playing your favorite webradio
station.
**Requirements:**
* 6.3 Conditional Blocks
**Tips:**
* You will need a webradio station whose stream is accessible via an IP
address. You can use the [Xatworld radio
search](https://www.xatworld.com/radio-search/) to find such a station.
* Once you have the IP address, find out how to play it from the console.
* Extend your script to take an argument which is the name of a station, and
make it play the station that you provided as argument. The script would have
a predefined list of stations it can play.
## Image resizer
Write a script that creates resized copies of all the pictures in the current
folder.
**Requirements:**
* 4 Globbing
* 6.4 Conditional Loops
**Tips:**
* We recommend downloading some pictures from the web and putting them in a
folder to experiment with (to prevent you from deleting your own pictures).
* Have a look at ImageMagick’s `convert` command. Find out how to use it and
resize a single image.
* Make sure your `convert` command only resizes images that are larger than the
target size.
* `convert` can only operate on one image at a time. Your script will have to
call it repeatedly.
* The resized copies should be stored in a separate, newly created subfolder.
* Find a way to iterate over all the files in a folder and call `convert` for
each.
* Don’t worry about files that are not images---`convert` will ignore them
anyway.
## Assignment-fetcher
Write a script that downloads all pdf files from a given website (for example,
to get all your course assignments at once).
This exercise is a bit more involved, and is best solved using regular
expressions (regex). Regex are not covered in this course, but you can find
great online resources, like this [Quick Start
Tutorial](http://www.regular-expressions.info/quickstart.html). Regex can be
used in combination with `grep`, for example.
**Requirements:**
* 3 Variables and Parameters
* 5.6 Command Substitution
* 6.5 Choices
* 7.1 Command-line Arguments
* 7.4 Pipes
**Tips:**
* Find a way to isolate all URLs that end in ’.pdf’ from a website. You could
use `grep` or `sed`.
* If you want to use `grep`, have a look at its `-o` and `-e` options.
* Depending on your course website, the URLs might be absolute (start with
http) or relative (start with a / or a directory). Your script should be able
to handle both.
* When you get relative URLs, you will have to prepend the base URL. Google for
HTML relative URLs for more information.
* You can get the base url from the full url using `grep` with regular expressions.
* Process all the URLs such that you end up with absolute URLs only.
* Optionally, filter the URLs for a certain keyword that you can give your
script as an argument. That way, you can prevent your script from downloading
unrelated PDFs.
* Download all the files from the URL list to the current directory.
* Course slides at ETH are sometimes password protected. Have a look at wget’s
`-user` and `-password` options. Unfortunately, these don’t work with the
official nethz login, so if your course website uses that, you can’t use a
bash script.
File added
# Master solutions
## Your first script
#!/bin/bash
echo 'Hello, World!'
## Count to 100
Make sure the parameter expansion is quoted properly. It may not be necessary here, but it is good practice.
#!/bin/bash
for i in {1..100}
do
echo "$i"
done
Here, we use brace expansion, so we can't use quotes.
#!/bin/bash
echo {1..100}
## Set up a script directory for your user
The following line should be added to either `~/.profile` or `~/.bashrc`. You have to log out and back in to make it work.
export PATH="$PATH:/home/user/scripts"
You can test whether it worked by running
echo "$PATH"
in your terminal.
## Parse some options
#!/bin/bash
while getopts 'h?abc:' opt; do
case "$opt" in
h|\?)
echo 'Available options: -h, -a, -b, -c ARGUMENT'
exit 0
;;
a)
echo 'Option a selected'
;;
b)
echo 'Option b selected'
;;
c)
echo "Option f selected with argument $OPTARG"
;;
esac
done
shift $((OPTIND-1))
The shift line at the end is important! If you have more arguments (that aren't options), you can't access them otherwise.
## Output all your arguments
#!/bin/bash
for i in "$@"
do
echo "$i"
done
Make sure all quotes are put correctly. Especially the expression `"$@"` MUST be quoted.
## Find big files
#!/bin/bash
usage() {
echo "usage: $0 directory"
exit 1
}
# We want exactly one argument
if [[ "$#" -ne 1 ]]; then
usage
fi
dir="$1"
# Test whether argument is a directory
if ! [[ -d "$dir" ]]
then
usage
fi
# Find files in dir, sort by size and print largest 5
find "$dir" -printf '%s %p\n' | sort -nr | head -n 5
The use of a function for `usage` is not required, but it's good practice.
## Self-reproducing script
#!/bin/bash
cp "$0" backup.sh
This second example also works, but is bad practice: it is discouraged to use `cat` to get the contents of a file in a bash script. There are almost always better ways to do that.
#!/bin/bash
cat "$0" > backup.sh
## Make your bash prompt fancy
Not much to say here, all solutions will be individual
## Maze generator
`RANDOM` is an environment variable that takes on a different numeric value every time you read it.
#!/bin/bash
while true
do
(( $RANDOM % 2 )) \
&& echo -n '/' \
|| echo -n '\' \
sleep 0.07
done
## Simple backup script
The `find` option `-type f` makes sure only files are listed (and not directories). The `-ctime -1` option makes sure only files not older than 1 day (24 hours) are listed.
#!/bin/bash
find . -type f -ctime -1 | xargs tar -czf 'backup.tar.gz'
## Screen brightness control
#!/bin/bash
(( $# == 0 )) && {
echo "No argument given; exiting"
exit 1;
}
# -z tests whether string is empty (zero length). We search for + or - in the string.
if [[ -z $( echo $1 | grep -Eo "\+|-" ) ]]
then
newbrightness="$1"
else
brightness="$(cat /sys/class/backlight/intel_backlight/brightness)"
newbrightness="$(( $brightness $1 ))" # becomes for example 10 +1
fi
# prevent screen from going all black. The threshold value might vary,
# since the scale of brightness values is different on every system
if (( $newbrightness < 10 ))
then
newbrightness=10
fi
# use tee instead of redirections because we need sudo
echo "$newbrightness" | sudo tee /sys/class/backlight/intel_backlight/brightness > /dev/null
The file path might of course vary. If you don't find a brightness file in `/sys/class/backlight`, chances are this script is not possible on your system.
## Automatic update script
This script is designed to be used with `sudo`, i.e. `sudo updatescript`. If you use it like that, you don't need `sudo` for zypper inside the script.
#!/bin/bash
# First, move old files out of the way safely. Discard error messages.
mv -b /home/user/updatelog /home/user/updatelog-old > /dev/null 2>&1
{ zypper up > /home/user/updatelog } \
&& rm /home/user/updatelog
poweroff
Of course, the zypper command is for openSUSE, and other distros require other commands.
## Disable/Enable external monitor output
These are some examples. It depends on your setup what you need.
#!/bin/bash
xrandr --output HDMI-0 --auto --left-of LVDS-0
<!-- -->
#!/bin/bash
xrandr --output HDMI-0 --auto
xrandr --output LVDS-0 --off
<!-- -->
#!/bin/bash
# detect a monitor
if xrandr | grep 'HDMI-0 connected' > /dev/null
then
xrandr --output HDMI-0 --right-of LVDS-0
fi
## Display a random headline from Reddit
This exercise requires you to parse strings to display them nicely. That can be quite tedious, and the commands used for it are often cryptic (cf. `sed`). Don't be scared.
#!/bin/bash
curl -A 'Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)' -s https://www.reddit.com/r/showerthoughts.json \
| jshon -e data -e children -a -e data -e title \
| shuf -n 1 \
| sed 's/\\//g' \
| sed 's/^"//' \
| sed 's/"$//'
The three `sed`s are merely for pretty printing. The first removes backslashes, the second removes the first quote and the last removes the last quote.
In a `sed` expression, `^` matches the beginning of a line and `$` matches the end. Also, backslashes (among other characters) have to be *escaped* using yet another backslash.
## Wallpaper change every 10 minutes
You can run this script in the background, like this: `./wallpaperscript &`
#!/bin/bash
while true
do
feh --bg-fill --randomize ~/pictures/wallpaper/*
sleep $(( 60 * 10 ))
done
## Your own TODO program
#! /bin/bash
# use a subshell to contain the directory change
(
# we work in .config, this isn't mandatory. ~/.config is where many
# programs store user configuration.
mkdir -p ~/.config/todo
cd ~/.config/todo
touch todolist # touch creates a file if it doesn't exist yet
while getopts 'i:d:h' opt; do
case "$opt" in
h|\?)
echo 'Valid flags: -i -d -h'
;;
i)
echo "$OPTARG" >> todolist
;;
d)
# sed '3d' deletes the 3rd line from a file.
sed -e "${OPTARG}d" todolist > /tmp/todolist
cp /tmp/todolist todolist
;;
esac
done
# the -n option adds line numbers
cat -n todolist
)
## Reminder script
#!/bin/bash
mins="$1"
shift 1
echo notify-send "'$@'" | at now + "$mins" min &>/dev/null
echo "Reminder: $@ set for $mins mins from now."
## Youtube downloader
#!/bin/bash
query="$@"
# get youtube search results page
# search for links that contain 'watch\?v=' followed by 11 characters
# take the first one
ytid=$( \
curl -s --data-urlencode "search_query=$query" https://www.youtube.com/results \
| grep -oE 'watch\?v=.{11}' \
| head -n 1 \
)
echo "$ytid"
youtube-dl -f bestaudio -o "%(title)s_%(id)s.%(ext)s" "https://www.youtube.com/$ytid"
## Web radio
#!/bin/bash
# tidy up first
killall mpv
# The & is not required but useful. Cache size is increased for lag free streaming
mpv http://213.251.190.165:9000 -cache 1024 &
## Image resizer
#!/bin/bash
mkdir -p resized
for i in *
do
# The > indicates that ImageMagick should only shrink larger images
convert "$i" -resize '100x100>' resized/"$i"
done
## Assignment-fetcher
#!/bin/bash
while getopts 'h?lf:u:p:' opt; do
case "$opt" in
h|\?)
echo "usage: $0 [-l] [-f FILTER] [-u USER] [-p PASSWORD] URL"
exit 0
;;
f)
filter="$OPTARG"
;;
u)
user="$OPTARG"
;;
p)
pass="$OPTARG"
;;
l)
list=true
esac
done
shift $((OPTIND-1))
url=$1
# Bare url up to the first slash.
# The grep first finds all characters up to the first dot, and then from there on finds
# all characters that are NOT slashes up to the first slash.
# The sed is used to escape all slashes within the url. / becomes \/
baseurl=$( \
echo $url \
| grep -oe '.*\.[^/]*/' \
| sed -e 's/\//\\\//g' \
)
# URL without file name (up to the last slash).
# The grep just finds everything up to a slash. By default it uses greedy matching,
# which means that it finds the longest fitting sequence.
# The sed is used to escape all slashes within the url.
longurl=$( \
echo $url \
| grep -oe '.*/' \
| sed -e 's/\//\\\//g' \
)
# The last command used in the command chain below depends on an option.
# if -l was passed, we just echo the found pdfs. Otherwise download them.
if [ $list == true ]
then
# the -n option is for xargs
lastcmd="-n 1 echo"
else
lastcmd="wget --user $user --password $pass"
fi
# First, find strings that end in .pdf
# Then, prepend the base url to all strings that are relative urls
# Then, apply the filter
# Then, if an url starts with // (relative to root), prepend the longurl
# last, download or display the urls
curl -s $url \
| grep -oe '[^"]*\.pdf' \
| sed -e "s/^\//$baseurl\//"
| grep -e "/[^/]*$filter[^/]*\.pdf" \
| awk "{ if ( \$0 !~ /\/\// ) { print \"$longurl\" \$0 } else {print} }" 2>/dev/null \
| xargs $lastcmd
File added
#!/bin/bash
echo "Building pdf..."
pandoc guide.md --listings --toc -N -V links-as-notes -H listings-setup.tex -o guide.pdf \
&& echo "Build successful"
echo "Building html..."
pandoc --toc -N --template template.html --highlight-style tango guide.md | sed 's/<code>/<code class="prettyprint">/g' > bashguide.html \
&& echo "Build successful"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="generator" content="pandoc" />
<title></title>
<style type="text/css">code{white-space: pre;}</style>
</head>
<body>
<div id="TOC">
<ul>
<li><a href="#bash-quick-guide">Bash Quick Guide</a></li>
<li><a href="#commands-and-arguments">Commands and Arguments</a><ul>
<li><a href="#preventing-word-splitting">Preventing word splitting</a></li>
<li><a href="#the-importance-of-spaces">The importance of spaces</a></li>
<li><a href="#scripts">Scripts</a></li>
</ul></li>
<li><a href="#variables-and-parameters">Variables and Parameters</a><ul>
<li><a href="#special-parameters">Special parameters</a></li>
<li><a href="#environment-variables">Environment variables</a></li>
</ul></li>
<li><a href="#globbing">Globbing</a></li>
<li><a href="#expansion">Expansion</a><ul>
<li><a href="#expansions-and-quotes">Expansions and quotes</a></li>
<li><a href="#expansion-order">Expansion order</a></li>
<li><a href="#brace-expansion">Brace expansion</a></li>
<li><a href="#tilde-expansion">Tilde expansion</a></li>
<li><a href="#parameter-and-variable-expansion">Parameter and variable expansion</a></li>
<li><a href="#command-substitution">Command substitution</a></li>
<li><a href="#arithmetic-expansion">Arithmetic expansion</a></li>
<li><a href="#globbing-1">Globbing</a></li>
</ul></li>
<li><a href="#tests-and-conditionals">Tests and conditionals</a><ul>
<li><a href="#control-operators-and">Control operators (&amp;&amp; and ||)</a></li>
<li><a href="#grouping-commands">Grouping commands</a></li>
<li><a href="#conditional-blocks-if-and">Conditional blocks (if and [[)</a></li>
<li><a href="#conditional-loops-while-until-and-for">Conditional loops (while, until and for)</a></li>
<li><a href="#choices-case-and-select">Choices (case and select)</a></li>
</ul></li>
<li><a href="#input-and-output">Input and Output</a><ul>
<li><a href="#command-line-arguments">Command-line arguments</a></li>
<li><a href="#file-descriptors">File descriptors</a></li>
<li><a href="#redirection">Redirection</a></li>
<li><a href="#pipes">Pipes</a></li>
</ul></li>
<li><a href="#compound-commands">Compound commands</a><ul>
<li><a href="#subshells">Subshells</a></li>
<li><a href="#command-grouping">Command grouping</a></li>
<li><a href="#arithmetic-evaluation">Arithmetic evaluation</a></li>
</ul></li>
<li><a href="#functions">Functions</a></li>
<li><a href="#useful-commands">Useful commands</a><ul>
<li><a href="#grep">grep</a></li>
<li><a href="#sed">sed</a></li>
<li><a href="#curl-and-wget">curl and wget</a></li>
<li><a href="#xrandr">xrandr</a></li>
<li><a href="#imagemagick-convert">ImageMagick (convert)</a></li>
<li><a href="#notify-send">notify-send</a></li>
<li><a href="#find">find</a></li>
<li><a href="#sort">sort</a></li>
<li><a href="#head-and-tail">head and tail</a></li>
<li><a href="#jshon">jshon</a></li>
<li><a href="#shuf">shuf</a></li>
<li><a href="#sleep">sleep</a></li>
<li><a href="#mplayer-or-mpv">mplayer or mpv</a></li>
<li><a href="#xdotool">xdotool</a></li>
</ul></li>
</ul>
</div>
<h1 id="bash-quick-guide">Bash Quick Guide</h1>
<p>Bash is an acronym for Bourne Again Shell. It is based on the Bourne shell and is mostly compatible with its features.</p>
<p>Shells are command interpreters. They are applications that provide users with the ability to give commands to their operating system interactively, or to execute batches of commands quickly. In no way are they required for the execution of programs; they are merely a layer between system function calls and the user.</p>
<p>Think of a shell as a way for you to speak to your system. Your system doesn't need it for most of its work, but it is an excellent interface between you and what your system can offer. It allows you to perform basic math, run basic tests and execute applications. More importantly, it allows you to combine these operations and connect applications to each other to perform complex and automated tasks.</p>
<p>Bash is not your operating system. It is not your window manager. It is not your terminal (but it often runs inside your terminal). It does not control your mouse or keyboard. It does not configure your system, activate your screen-saver, or open your files. It's important to understand that bash is only an interface for you to execute statements (using bash syntax), either at the interactive bash prompt or via bash scripts. The things that <em>actually happen</em> are usually caused by other programs.</p>
<p>This guide is based on <a href="http://mywiki.wooledge.org/BashGuide">the bash guide in GreyCat's wiki</a> and aims to be more concise, while still being accurate. It was produced specifically for the Bash Workshop by <a href="www.thealternative.ch">TheAlternative.ch</a>.</p>
<p>It is published under the <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">CC by-nc-sa 4.0 license</a>.</p>
<h1 id="commands-and-arguments">Commands and Arguments</h1>
<p>Bash reads commands from its input, which can be either a file or your terminal. In general, each line is interpreted as a command followed by its arguments.</p>
<pre><code>ls
touch file1 file2 file3
ls -l
rm file1 file2 file3</code></pre>
<p>The first word is always the command that is executed. All subsequent words are given to that command as argument.</p>
<p>Note that <em>options</em>, such as the <code>-l</code> option in the example above, are not treated specially by bash. They are arguments like any other. It is up to the program (<code>ls</code> in the above case) to treat it as an option.</p>
<p>Words are delimited by whitespace (spaces or tabs). It does not matter how many spaces there are between two words. For example, try</p>
<pre><code>echo Hello World</code></pre>
<p>The process of splitting a line of text into words is called <em>word splitting</em>. It is vital to be aware of it, especially when you come across expansions later on.</p>
<h2 id="preventing-word-splitting">Preventing word splitting</h2>
<p>Sometimes, you will want to pass arguments to commands that contain whitespace. To do so, you can use quotes:</p>
<pre><code>touch &quot;Filename with spaces&quot;</code></pre>
<p>This command creates a single file named <em>Filename with spaces</em>. The text within double quotes is protected from word splitting and hence treated as a single word.</p>
<p>Note that you can also use single quotes:</p>
<pre><code>touch &#39;Another filename with spaces&#39;</code></pre>
<p>There is, however, an important difference between the two:</p>
<ul>
<li>Double quotes prevent <strong>word splitting</strong></li>
<li>Single quotes prevent <strong>word splitting and expansion</strong></li>
</ul>
<p>When you use single quotes, the quoted text will never be changed by bash. With double quotes, expansion will still happen. This doesn't make a difference in the above example, but as soon as you use e.g. variables, it becomes important.</p>
<p>In general, it is considered good practice to use single quotes whenever possible, and double quotes only when expansion is desired. In that sense, the last example above can be considered &quot;more correct&quot;.</p>
<h2 id="the-importance-of-spaces">The importance of spaces</h2>
<p>Bash contains various keywords and built-ins that aren't immediately recognizable as commands, such as the new test command:</p>
<pre><code>[[ -f file ]]</code></pre>
<p>The above code tests whether a file named &quot;file&quot; exists in the current directory. Just like every line of bash code, it consists of a command followed by its arguments. Here, the command is <code>[[</code>, while the arguments are <code>-f</code>, <code>file</code> and <code>]]</code>.</p>
<p>Many programmers of other languages would write the above command like so:</p>
<pre><code>[[-f file]]</code></pre>
<p>This, though, is wrong: Bash will look for a command named <code>[[-f</code>, which doesn't exist, and issue an error message. This kind of mistake is very common for beginners. It is advisable to always use spaces after any kind of brackets in bash, even though there are cases where they are not necessary.</p>
<h2 id="scripts">Scripts</h2>
<p>You have probably interacted with bash through a terminal before. You would see a bash prompt, and you would issue one command after another.</p>
<p>Bash scripts are basically a sequence of commands stored in a file. They are read and processed in order, one after the other.</p>
<p>Making a script is easy. Begin by making a new file, and put this on the first line:</p>
<pre><code>#!/bin/bash</code></pre>
<p>This line is called an <em>interpreter directive</em>, or more commonly, a <em>hashbang</em> or <em>shebang</em>. Your operating system uses it to determine how this file can be run. In this case, the file is to be run using <code>bash</code>, which is stored in the <code>/bin/</code> directory.</p>
<p>After the shebang, you can add any command that you could also use in your terminal. For example, you could add</p>
<pre><code>echo &#39;Hello World&#39;</code></pre>
<p>and then save the file as &quot;myscript&quot;</p>
<p>You can now run the file from the terminal by typing</p>
<pre><code>bash myscript</code></pre>
<p>Here, you explicitly called bash and made it execute the script. <code>bash</code> is used as the command, while <code>myscript</code> is an argument. However, it's also possible to use <code>myscript</code> as a command directly.</p>
<p>To do so, you must first make it executable:</p>
<pre><code>chmod +x myscript</code></pre>
<p>Now that you have permission to execute this script directly, you can type</p>
<pre><code>./myscript</code></pre>
<p>to run it.</p>
<p>The <code>./</code> is required to tell bash that the executable is located in the current directory, rather than the system directory. We will come back to this in the chapter on Variables.</p>
<h1 id="variables-and-parameters">Variables and Parameters</h1>
<p>Variables and parameters can be used to store strings and retrieve them later. <em>Variables</em> are the ones you create yourself, while <em>special parameters</em> are pre-set by bash. <em>Parameters</em> actually refers to both, but is often used synonymously to special parameters.</p>
<p>To store a string in a variable, we use the <em>assignment syntax</em>:</p>
<pre><code>varname=vardata</code></pre>
<p>This sets the variable <code>varname</code> to contain the string <code>vardata</code>.</p>
<p>Note that you cannot use spaces around the <code>=</code> sign. With the spaces, bash would assume <code>varname</code> to be a command and then pass <code>=</code> and <code>vardata</code> as arguments.</p>
<p>To access the string that is now stored in the variable <code>varname</code>, we have to use <em>parameter expansion</em>. This is the most common kind of expansion: A variable is replaced with its content.</p>
<p>If you want to print the variable's value, you can type</p>
<pre><code>echo $varname</code></pre>
<p>The <code>$</code> indicates that you want to use expansion on <code>varname</code>, meaning it is replaced by its content. Note that expansion happens before the command is run. Here's what happens step-by-step:</p>
<ul>
<li>Bash uses variable expansion, changing <code>echo $varname</code> to <code>echo vardata</code></li>
<li>Then, bash runs <code>echo</code> with <code>vardata</code> as its parameter.</li>
</ul>
<p>The most important thing here is that <strong>variable expansion happens before wordsplitting</strong>. That means, if you have defined a variable like this:</p>
<pre><code>myfile=&#39;bad song.mp3&#39;</code></pre>
<p>and then run the command</p>
<pre><code>rm $myfile</code></pre>
<p>bash will expand this to</p>
<pre><code>rm bad song.mp3</code></pre>
<p>Only now, word splitting occurs, and bash will call <code>rm</code> with two arguments: <code>bad</code> and <code>song.mp3</code>. If you now had a file called <code>song.mp3</code> in your current directory, that one would be deleted instead.</p>
<p>To prevent this from happening, you can use double quotes:</p>
<pre><code>rm &quot;$myfile&quot;</code></pre>
<p>This will be expanded to</p>
<pre><code>rm &quot;bad song.mp3&quot;</code></pre>
<p>which is what we want. In this case, you have to use double quotes, as single quotes would prevent expansion from happening altogether.</p>
<p>Not quoting variable and parameter expansions is a very common mistake even among advanced bash programmers. It can cause bugs that are hard to find and can be very dangerous. <strong>Always quote your variable expansions.</strong></p>
<p>You can also use variable expansions inside the variable assignment itself. Consider this example:</p>
<pre><code>myvariable=&#39;blah&#39;
myvariable=&quot;$myvariable blah&quot;
echo &quot;$myvariable&quot;</code></pre>
<p>What will the output of this script be?</p>
<p>First, the variable <code>myvariable</code> will get the value <code>blah</code>. Then, <code>myvariable</code> is assigned to again, which overwrites its former content. The assignment contains a variable expansion, <code>&quot;$myvariable blah&quot;</code>. This is expanded to <code>&quot;blah blah&quot;</code>, and that is going to be the new value of <code>myvariable</code>. So the last command is expanded to <code>echo &quot;blah blah&quot;</code>, and the output of the script is <code>blah blah</code>.</p>
<h2 id="special-parameters">Special parameters</h2>
<p><em>Special parameters</em> are variables that are set by bash itself. Most of those variables can't be written to and they contain useful information.</p>
<table>
<colgroup>
<col width="12%" />
<col width="12%" />
<col width="74%" />
</colgroup>
<thead>
<tr class="header">
<th>Parameter Name</th>
<th>Usage</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><code>0</code></td>
<td><code>&quot;$0&quot;</code></td>
<td>Contains the name of the current script</td>
</tr>
<tr class="even">
<td><code>1</code> <code>2</code> <code>3</code> etc.</td>
<td><code>&quot;$1&quot;</code> etc.</td>
<td>Contains the arguments that were passed to the current script. The number indicates the position of that argument (first, second...). These parameters are also called positional parameters.</td>
</tr>
<tr class="odd">
<td><code>*</code></td>
<td><code>&quot;$*&quot;</code></td>
<td>Contains all the positional parameters. Double quoted, it expands to a single word containing them all.</td>
</tr>
<tr class="even">
<td><code>@</code></td>
<td><code>&quot;$@&quot;</code></td>
<td>Contains all the positional parameters. Double quoted, it expands to <strong>several words, where each word is one parameter</strong>. This is special syntax, it behaves differently from &quot;normal&quot; expansion in quotes. It retains the arguments exactly as they were passed to the script.</td>
</tr>
<tr class="odd">
<td><code>#</code></td>
<td><code>&quot;$#&quot;</code></td>
<td>Contains the number of parameters that were passed to the script</td>
</tr>
<tr class="even">
<td><code>?</code></td>
<td><code>&quot;$?&quot;</code></td>
<td>Contains the exit code of the last executed command</td>
</tr>
</tbody>
</table>
<h2 id="environment-variables">Environment variables</h2>
<p><em>Environment variables</em> are special variables that are already set when you start bash. In fact, these variables aren't specific to bash - they are available in every program that runs on your system and they can affect your system's behaviour.</p>
<p>You can use the command <code>printenv</code> in your terminal to display all the environment variables you have currently set. Most of them contain some system configuration, like the variable <code>LANG</code>, which designates your preferred language. Some variables, like <code>TERM</code>, <code>BROWSER</code> or <code>SHELL</code>, designate your preferred default programs (terminal, web browser and shell, respectively. These may not be set on all systems).</p>
<p>Another important environment variable is <code>PATH</code>. It contains a bunch of file paths, separated by colons. These paths is where your system will look for executables when you type a command. For example, if you type <code>grep</code> in your terminal, your system will search for an executable called <code>grep</code> in the directories designated in your <code>PATH</code> variable. As soon as it finds one, it will execute that. If it doesn't find it, you will get a &quot;command not found&quot; error message.</p>
<p>You can modify your environment variables, if you want. The guideline here is to only mess with those variables of which you know what they do, otherwise you might break something.</p>
<p>The place to modify these variables is your <code>~/.bash_profile</code> file. This file contains some bash code that is executed whenever you log in. For example, you could add the following line:</p>
<pre><code>export BROWSER=&quot;firefox&quot;</code></pre>
<p>This would set your default browser to firefox (note that on some systems, there are other settings--for example in your Desktop Environment--which can override these environment variables. You'll have to test whether this works).</p>
<p>Note the <code>export</code> keyword. This is a bash builtin that takes a variable definition as its argument and puts it in your <em>environment</em>. If you omit this, your new variable will just be an ordinary variable, rather than an environment variable.</p>
<h1 id="globbing">Globbing</h1>
<p><em>Globs</em> are an important bash concept--mostly for their incredible convenience. They are patterns that can be used to match filenames or other strings.</p>
<p>Globs are composed of normal characters and metacharacters. Metacharacters are characters that have a special meaning. These are the metacharacters that can be used in globs:</p>
<ul>
<li><code>*</code>: Matches any string, including the empty string (i.e. nothing)</li>
<li><code>?</code>: Matches any single character</li>
<li><code>[...]</code>: Matches any one of the characters enclosed in the brackets</li>
</ul>
<p>Bash sees the glob, for example <code>a*</code>. It expands this glob, by looking in the current directory and matching it against all files there. Any filenames that match the glob are gathered up and sorted, and then the list of filenames is used in place of the glob. So if you have three files <code>a</code>, <code>b</code> and <code>albert</code> in the current directory, the glob is expanded to <code>a albert</code>.</p>
<p>A glob always has to match the entire filename. That means <code>a*</code> will match <code>at</code> but not <code>bat</code>.</p>
<p>Note that globbing is special in that it happens <em>after word splitting</em>. This means you never need to worry about spaces in filenames when you use globbing, and quoting globs is not necessary. In fact, quotes will prevent globbing from happening.</p>
<h1 id="expansion">Expansion</h1>
<p>We've already seen <em>parameter and variable expansion</em>, but that's not the only kind of expansion that happens in bash. In this chapter, we'll look at all kinds of expansion that aren't covered elsewhere.</p>
<h2 id="expansions-and-quotes">Expansions and quotes</h2>
<p>You already know that it is important to quote parameter and variable expansions, but we also told you that quoting globs--which are, in fact, just another form of expansion--is not necessary. So, which expansions need to be quoted?</p>
<p>The rule of thumb here is as follows:</p>
<ul>
<li>Always quote <strong>paramater expansion</strong>, <strong>command substitution</strong> and <strong>arithmetic expansion</strong></li>
<li>Never quote <strong>brace expansion</strong>, <strong>tilde expansion</strong> and <strong>globs</strong></li>
</ul>
<p>The handy thing here is: All the expansions that require quoting have a <code>$</code> in their syntax. Parameter expansion is simply a <code>$</code> followed by a parameter name. Command substitution starts with a <code>$(</code>, and arithmetic expansion starts with <code>$((</code>.</p>
<p>So, the rule of thumb breaks down to the following: <strong>If there's a dollar, you probably need quotes.</strong></p>
<p>Now, what if you want to use two kinds of expansion in the same line, but one requires quotes and the other doesn't? Consider the following script:</p>
<pre><code>prefix=&#39;my picture&#39;
rm ~/pictures/$prefix*</code></pre>
<p>Here, we use tilde expansion, parameter expansion and globbing in order to remove all files that start with <code>my picture</code> in the folder <code>/home/username/pictures/</code>. But because quotes prevent tilde expansion and globbing, we cannot quote the entire expression. This means that the parameter expansion, too, goes unquoted--and this is fatal, because our variable contains a space. So what should we do?</p>
<p>The important thing to realize here is that quoting simply prevents word splitting, but it does not actually designate something as a single string. So we can do the following:</p>
<pre><code>prefix=&#39;my picture&#39;
rm ~/pictures/&quot;$prefix&quot;*</code></pre>
<p>Only the parameter expansion is quoted, so it is protected from word splitting. But that does not automatically separate it from the rest of the string. Note that there are no spaces between <code>&quot;$prefix&quot;</code> and <code>~/pictures/</code>. Since word splitting only happens when there are spaces, the entire thing will not be split. Here's what happens, in order:</p>
<p>First, tilde expansion occurs:</p>
<pre><code>rm /home/username/pictures/&quot;$prefix&quot;/*</code></pre>
<p>Next, parameter expansion:</p>
<pre><code>rm /home/username/pictures/&quot;my picture&quot;*</code></pre>
<p>At this point, word splitting happens. But since the only space in our argument is in quotes, the argument remains intact.</p>
<p>And last, globbing:</p>
<pre><code>rm /home/username/pictures/&quot;my picture&quot;001.jpg /home/username/pictures/&quot;my picture&quot;002.jpg</code></pre>
<p>Now, there's one last step that happens which we didn't mention before. It's called <em>quote removal</em>. All the quotes that were needed to prevent word splitting are now ignored, which means that the arguments that are finally given to <code>rm</code> are:</p>
<ul>
<li><code>/home/username/pictures/my picture001.jpg</code></li>
<li><code>/home/username/pictures/my picture002.jpg</code></li>
</ul>
<p>and this is exactly what we wanted.</p>
<p>So, remember: Quotes don't need to be at the beginning or end of an argument, and if you use several kinds of expansion together, you can add quotes in the middle as required.</p>
<h2 id="expansion-order">Expansion order</h2>
<p>All the kinds of expansion happen in a certain order. The order is as follows:</p>
<ul>
<li>Brace expansion</li>
<li>Tilde expansion</li>
<li>Parameter and variable expansion</li>
<li>Command substitution</li>
<li>Arithmetic expansion</li>
<li>Word splitting</li>
<li>Globbing</li>
</ul>
<h2 id="brace-expansion">Brace expansion</h2>
<p><em>Brace expansions</em> are often used in conjunction with globs, but they also have other uses. They always expand to all possible permutations of their contents. Here's an example:</p>
<pre><code>$ echo th{e,a}n
then than
$ echo {1..9}
1 2 3 4 5 6 7 8 9
$ echo {0,1}{0..9}
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19</code></pre>
<p>Brace expansions are replaced by a list of words. They are often used in conjunction with globs to match specific files but not others. For example, if you want to delete pictures from your pictures folder with filenames IMG020.jpg through IMG039.jpg, you could use the following pattern:</p>
<pre><code>rm IMG0{2,3}*.jpg</code></pre>
<p>Note that we don't use quotes here. Quotes prevent brace expansion.</p>
<p>Brace expansion happens before globbing, so in the above example, the braces are expanded to</p>
<pre><code>rm IMG02*.jpg IMG03*.jpg</code></pre>
<p>We end up with two glob patterns, the first matches IMG020.jpg through IMG029.jpg, and the second matches IMG030.jpg through IMG039.jpg.</p>
<h2 id="tilde-expansion">Tilde expansion</h2>
<p>You have probably already seen and used the tilde in the terminal. It is a shortcut to your home directory:</p>
<pre><code>cd ~/files</code></pre>
<p>This will be expanded to <code>cd /home/username/files</code>. Note that tilde expansion only happens outside of quotes, so the following won't work:</p>
<pre><code>cd &quot;~/files&quot;</code></pre>
<h2 id="parameter-and-variable-expansion">Parameter and variable expansion</h2>
<p><em>Parameter and variable expansion</em> is explained in the chapter on Variables and Parameters.</p>
<p>An important sidenote here is that Parameter expansion and Variable expansion often refer to the same thing. The official name as per the bash manual is <em>Parameter expansion</em>, but <em>Variable expansion</em> is often used instead as it is less misleading.</p>
<h2 id="command-substitution">Command substitution</h2>
<p><em>Command substitution</em> is a way of using a command's output inside your script. For example, let's say you want to find out where your script is executed, and then move a file to that location.</p>
<pre><code>echo &#39;I am at&#39;
pwd</code></pre>
<p>This script will simply print the directory it is called from. But you want to move a file to that directory, which means you need to use it as an argument to the <code>mv</code> command. For that, you need command substitution:</p>
<pre><code>mv ~/myfile &quot;$( pwd )&quot;</code></pre>
<p>The <code>$(</code> introduces a command substitution. It works similarly to variable expansion, but instead of putting a variable's content, it puts a command's output in that place. In this example, <code>pwd</code> is run, and the output is (for example) <code>/home/username/scripts/</code>. Then, the entire command is expanded to</p>
<pre><code>mv ~/myfile &quot;/home/username/scripts/&quot;</code></pre>
<p>Note that with this kind of expansion, quotes are important. The path returned by <code>pwd</code> might have contained a space, in which case we need the argument to be properly quoted.</p>
<h2 id="arithmetic-expansion">Arithmetic expansion</h2>
<p><em>Arithmetic expansion</em> is explained in the chapter on arithmetic evaluation.</p>
<h2 id="globbing-1">Globbing</h2>
<p><em>Globbing</em> is so important, it has a chapter of its own.</p>
<h1 id="tests-and-conditionals">Tests and conditionals</h1>
<p>Every command you run in your terminal or shell script has a return value. That value is a number, and by convention, a return value of 0 means <em>success</em>, while any other number indicates an error.</p>
<p>You usually don't see the return value after running a command. What you do see is the command's <em>output</em>. If you want to know a command's return value, you can read it by issuing</p>
<pre><code>echo &quot;$?&quot;</code></pre>
<p>immediately after running that command.</p>
<p>While you often don't need to know a command's return value, it can be useful to construct conditionals and thereby achieve advanced logic.</p>
<h2 id="control-operators-and">Control operators (&amp;&amp; and ||)</h2>
<p>Control operators can be used to make a command's execution depend on another command's success. This concept is called <em>conditional execution</em>:</p>
<pre><code>mkdir folder &amp;&amp; cd folder</code></pre>
<p>In the above example, the <code>&amp;&amp;</code> operator is used to connect two commands <code>mkdir folder</code> and <code>cd folder</code>. Using this connection, bash will first execute <code>mkdir folder</code>, and then execute <code>cd folder</code> <em>only if the first command was successful</em>.</p>
<pre><code>mkdir folder || echo &#39;Error: could not create folder&#39;</code></pre>
<p>In this example, the <code>||</code> operator is used to connect the commands. Here, the second command is executed <em>only if the first command failed</em>.</p>
<p>It is good practice to make your own scripts return an error (i.e. something other than 0) whenever something goes wrong. To do that, you can use this construct:</p>
<pre><code>mkdir folder || exit 1</code></pre>
<p>The <code>exit</code> command immediately stops the execution of your script and makes it return the number you specified as an argument. In this example, your script will attempt to create a folder. If that goes wrong, it will immediately stop and return 1.</p>
<p>Control operators can be written more legibly by spreading them across multiple lines:</p>
<pre><code>mkdir folder \
|| exit 1</code></pre>
<p>The backslash at the end of the first line makes bash ignore that line break and treat both lines as a single command plus arguments.</p>
<h2 id="grouping-commands">Grouping commands</h2>
<p>Now, what if you want to execute multiple commands if the first one fails? Going from the example above, when <code>mkdir folder</code> fails, you might want to print an error message <em>and</em> return 1.</p>
<p>This can be done by enclosing these two commands in single curly braces:</p>
<pre><code>mkdir folder || { echo &#39;Could not create folder&#39;; exit 1 }</code></pre>
<p>The two commands in curly braces are treated as an unit, and if the first command fails, both will be executed.</p>
<p>Note that you need to include spaces between the curly braces and the commands. If there were no spaces, bash would look for a command named <code>{echo</code> and fail to find it.</p>
<p>There's a semicolon separating the two commands. The semicolon has the same function as a line break: it makes bash consider both parts as individual commands-plus-arguments.</p>
<p>The example above could be rewritten as follows:</p>
<pre><code>mkdir folder || {
echo &#39;Could not create folder&#39;
exit 1
}</code></pre>
<p>Here, no semicolon is required, because there is a line break between the two statements. Line breaks and semicolons can be used interchangeably.</p>
<h2 id="conditional-blocks-if-and">Conditional blocks (if and [[)</h2>
<p><code>if</code> is a shell keyword. It first executes a command and looks at its return value. If it was 0 (success), it executes one list of commands, and if it was something else (failure), it executes another.</p>
<p>It looks like this:</p>
<pre><code>if true
then
echo &#39;It was true&#39;
else
echo &#39;It was false&#39;
fi</code></pre>
<p><code>true</code> is a bash builtin command that does nothing and returns 0. <code>if</code> will run that command, see that it returned 0, and then execute the commands between the <code>then</code> and the <code>else</code>.</p>
<p>In many programming languages, operators such as <code>&gt;</code>, <code>&lt;</code> or <code>==</code> exist and can be used to compare values. In bash, operators don't exist. But since comparing values is so common, there's a special command that can do it:</p>
<pre><code>[[ a = b ]]</code></pre>
<p><code>[[</code> is a command that takes as its arguments a comparison. The last argument has to be a <code>]]</code>. It is made to look like a comparison in double brackets, but it is, in fact, a command like any other. It is also called the <em>new test command</em>. (The <em>old test command</em>, often simply called <em>test</em>, exists as well, so be careful not to confuse them).</p>
<p>For that reason, the spaces are absolutely needed. You cannot write this:</p>
<pre><code>[[a=b]]</code></pre>
<p>This will make bash look for a command called <code>[[a=b]]</code>, which doesn't exist.</p>
<p><code>[[</code> does not only support comparing strings. For example, <code>[[ -f file ]]</code> will test whether a file named <em>file</em> exists. Here's a list of the most common tests you can use with <code>[[</code>:</p>
<ul>
<li><code>-e FILE:</code> True if file exists.</li>
<li><code>-f FILE:</code> True if file is a regular file.</li>
<li><code>-d FILE:</code> True if file is a directory.</li>
<li><code>-z STRING:</code> True if the string is empty (it's length is zero).</li>
<li><code>-n STRING:</code> True if the string is not empty (it's length is not zero).</li>
<li>String operators:
<ul>
<li><code>STRING = STRING</code>: True if the string matches the glob pattern (if you quote the glob pattern, the strings have to match exactly).</li>
<li><code>STRING != STRING</code>: True if the string does not match the glob pattern (if you quote the glob pattern, the strings just have to be different).</li>
<li><code>STRING &lt; STRING</code>: True if the first string sorts before the second.</li>
<li><code>STRING &gt; STRING</code>: True if the first string sorts after the second.</li>
</ul></li>
<li><code>EXPR -a EXPR</code>: True if both expressions are true (logical AND).</li>
<li><code>EXPR -o EXPR</code>: True if either expression is true (logical OR).</li>
<li><code>! EXPR</code>: Inverts the result of the expression (logical NOT).</li>
<li>Numeric operators:
<ul>
<li><code>INT -eq INT</code>: True if both integers are identical.</li>
<li><code>INT -ne INT</code>: True if the integers are not identical.</li>
<li><code>INT -lt INT</code>: True if the first integer is less than the second.</li>
<li><code>INT -gt INT</code>: True if the first integer is greater than the second.</li>
<li><code>INT -le INT</code>: True if the first integer is less than or equal to the second.</li>
<li><code>INT -ge INT</code>: True if the first integer is greater than or equal to the second.</li>
</ul></li>
<li><code>EXPR &amp;&amp; EXPR</code>: Much like the '-a' operator of test, but does not evaluate the second expression if the first already turns out to be false.</li>
<li><code>EXPR || EXPR</code>: Much like the '-o' operator of test, but does not evaluate the second expression if the first already turns out to be true.</li>
</ul>
<p>You might occasionally come across something like this:</p>
<pre><code>[ a = b ]</code></pre>
<p>Here, we use single brackets instead of double brackets. This is, in fact, an entirely different command, the <code>[</code> command or <em>old test command</em>. It has the same purpose--comparing things--but the <code>[[</code> command is newer, has more features, and is easier to use. We strongly recommend using <code>[[</code> over <code>[</code>.</p>
<h2 id="conditional-loops-while-until-and-for">Conditional loops (while, until and for)</h2>
<p>Loops can be used to repeat a list of commands multiple times. In bash, there are <code>while</code> loops and <code>for</code> loops.</p>
<p>While loops look like this:</p>
<pre><code>while true
do
echo &#39;Infinite loop&#39;
done</code></pre>
<p>The <code>while</code> keyword will execute the <code>true</code> command, and if that returns 0, it executes all commands between the <code>do</code> and <code>done</code>. After that, it starts over, until the <code>true</code> command returns 1 (which it never does, which is why this loop will run indefinitely).</p>
<p>The above example might not be immediately useful, but you could also do something like this:</p>
<pre><code>while ping -c 1 -W 1 www.google.com
do
echo &#39;Google still works!&#39;
done</code></pre>
<p>There's also a variation of the <code>while</code> loop, called <code>until</code>. It works similarly, except it only runs its command list when the first command <em>fails</em>:</p>
<pre><code>until ping -c 1 -W 1 www.google.com
do
echo &#39;Google isn&#39;t working!&#39;
done</code></pre>
<p><code>for</code> loops can be used to iterate over a list of strings:</p>
<pre><code>for var in 1 2 3
do
echo &quot;$var&quot;
done</code></pre>
<p>After the <code>for</code>, you specify a variable name. After the <code>in</code>, you list all the strings you want to iterate over.</p>
<p>The loop works by setting the variable you specified to all the values from the list in turn, and then executing the command list for each of them.</p>
<p>This is especially useful in combination with globs or brace expansions:</p>
<pre><code>echo &#39;This is a list of all my files starting with f:&#39;
for var in f*
do
echo &quot;$var&quot;
done
echo &#39;And now I&#39;m counting from 1 to 100:&#39;
for var in {1..100}
do
echo &quot;$var&quot;
done</code></pre>
<h2 id="choices-case-and-select">Choices (case and select)</h2>
<p>Sometimes, you want your script to behave differently depending on the content of a variable. This could be implemented by taking a different branch of an if statement for each state:</p>
<pre><code>if [[ &quot;$LANG&quot; = &#39;en&#39; ]]
then
echo &#39;Hello!&#39;
elif [[ &quot;$LANG&quot; = &#39;de&#39; ]]
then
echo &#39;Guten Tag!&#39;
elif [[ &quot;$LANG&quot; = &#39;it&#39; ]]
then
echo &#39;Ciao!&#39;
else
echo &#39;I do not speak your language.&#39;
fi</code></pre>
<p>This is quite cumbersome to write. At the same time, constructs like this are very common. For that reason, bash provides a keyword to simplify it:</p>
<pre><code>case &quot;$LANG&quot; in
en)
echo &#39;Hello!&#39;
;;
de)
echo &#39;Guten Tag!&#39;
;;
it)
echo &#39;Ciao!&#39;
;;
*)
echo &#39;I do not speak your language.&#39;
;;
esac</code></pre>
<p>Each choice of the case statement consists of a string or glob pattern, a <code>)</code>, a list of commands that is to be executed if the string matches the pattern, and two semicolons to denote the end of a list of commands.</p>
<p>The string after the keyword <code>case</code> is matched against each glob pattern in order. The list of commands after the first match is executed. After that, execution continues after the <code>esac</code>.</p>
<p>Since the string is matched against glob patterns, we can use <code>*</code> in the end to catch anything that didn't match before.</p>
<p>Another construct of choice is the <code>select</code> construct. It looks and works similarly to a loop, but it also presents the user with a predefined choice. You are encouraged to try running this example yourself:</p>
<pre><code>echo &#39;Which one of these does not belong in the group?&#39;
select choice in Apples Pears Crisps Lemons Kiwis
do
if [[ &quot;$choice&quot; = Crisps ]]
then
echo &#39;Correct! Crisps are not fruit.&#39;
break
fi
echo &#39;Wrong answer. Try again.&#39;
done</code></pre>
<p>The syntax of the <code>select</code> construct is very similar to <code>for</code> loops. The difference is that instead of setting the variable (<code>choice</code> in this example) to each value in turn, the <code>select</code> construct lets the user choose which value is used next. This also means that a <code>select</code> construct can run indefinitely, because the user can keep selecting new choices. To avoid being trapped in it, we have to explicitly use <code>break</code>. <code>break</code> is a builtin command that makes bash jump out of the current <code>do</code> block. Execution will continue after the <code>done</code>. <code>break</code> also works in <code>for</code> and <code>while</code> loops.</p>
<p>As you can see in the example above, we used an <code>if</code> command inside a <code>select</code> command. All of these conditional constructs (<code>if</code>, <code>for</code>, <code>while</code>, <code>case</code> and <code>select</code>) can be nested indefinitely.</p>
<h1 id="input-and-output">Input and Output</h1>
<p>Input and output in bash is very flexible and, consequentially, complex. We will only look at the most widely used components.</p>
<h2 id="command-line-arguments">Command-line arguments</h2>
<p>For many bash scripts, the first input we care about are the arguments given to it via the command line. As we saw in the chapter on Parameters, these arguments are contained in some <em>special parameters</em>. These are called <em>positional parameters</em>. The first parameter is referred to with <code>$1</code>, the second with <code>$2</code>, and so on. After number 9, you have to enclose the numbers in curly braces: <code>${10}</code>, <code>${11}</code> and so on.</p>
<p>In addition to referring to them one at a time, you may also refer to the entire set of positional parameters with the <code>&quot;$@&quot;</code> substitution. The double quotes here are <strong>extremely important</strong>. If you don't use the double quotes, each one of the positional parameters will undergo word splitting and globbing. You don't want that. By using the quotes, you tell Bash that you want to preserve each parameter as a separate word.</p>
<p>There are even more ways to deal with parameters. For example, it is very common for commands to accept <em>options</em>, which are single letters starting with a <code>-</code>. For example, <code>ls -l</code> calls the <code>ls</code> program with the <code>-l</code> option, which makes it output more information. Usually, multiple options can be combined, as in <code>ls -la</code>, which is equivalent to <code>ls -l -a</code>.</p>
<p>You might want to create your own scripts that accept some options. Since parsing options is so common, there's a bash builtin command to facilitate it. It is called <code>getopts</code>.</p>
<pre><code>while getopts &#39;hlf:&#39; opt
do
case &quot;$opt&quot; in
h|\?)
echo &#39;available options: -h -l -f [filename]&#39;
;;
f)
file=&quot;$OPTARG&quot;
;;
l)
list=true
;;
esac
done
shift &quot;$(( OPTIND - 1 ))&quot;</code></pre>
<p>As you can see, we use <code>getopts</code> within a while loop. <code>getopts</code> will return 0 as long as there are more options remaining and something else if there are no more options. That makes it perfectly suitable for a loop.</p>
<p><code>getopts</code> takes two arguments, the <em>optstring</em> and the <em>variable name</em>. The optstring contains all the letters that are valid options. In our example, these are <code>h</code>, <code>l</code> and <code>f</code>.</p>
<p>The <code>f</code> is followed by a colon. This indicates that the f option requires an argument. The script could for example be called like so:</p>
<pre><code>myscript -f file.txt</code></pre>
<p><code>getopts</code> will set the variable that you specified as its second argument to the letter of the option it found first. If the option required an argument, the variable <code>OPTARG</code> is set to whatever the argument was. In the example above, a case statement is used to react to the different options.</p>
<p>There's also a special option <code>?</code>. Whenever <code>getopts</code> finds an option that is not present in the optstring, it sets the shell variable (<code>opt</code> in the example) to <code>?</code>. In the case statement above, that triggers the help message.</p>
<p>After all options are parsed, the remaining arguments are &quot;moved&quot; such that they are now in <code>$1</code>, <code>$2</code>... even though previously, these positional parameters were occupied by the options.</p>
<p>Also note the line <code>shift &quot;$(( OPTIND - 1 ))&quot;</code> at the end. The <code>shift</code> builtin can be used to discard command-line arguments. Its argument is a number and designates how many arguments we want to discard.</p>
<p>This is needed because we don't know how many options the user will pass to our script. If there are more positional parameters after all the options, we have no way of knowing at which number they start. Fortunately, <code>getopts</code> also sets the shell variable <code>OPTIND</code>, in which it stores the index of the option it's going to parse next.</p>
<p>So after parsing all the option, we just discard the first <code>OPTIND - 1</code> options, and the remaining arguments now start from <code>$1</code> onwards.</p>
<h2 id="file-descriptors">File descriptors</h2>
<p><em>File descriptors</em> are the way programs refer to files, or other things that work like files (such as pipes, devices, or terminals). You can think of them as pointers that point to data locations. Through these pointers, programs can write to or read from these locations.</p>
<p>By default, every program has three File Descriptors:</p>
<ul>
<li>Standard Input (stdin): File Descriptor 0</li>
<li>Standard Output (stdout): File Descriptor 1</li>
<li>Standard Error (stderr): File Descriptor 2</li>
</ul>
<p>When you run a script in the terminal, then stdin contains everything you type in that terminal. stdout and stderr both point to the terminal, and everything that is written to these two is displayed as text in the terminal. stdout is where programs send their normal information, and stderr is where they send their error messages.</p>
<p>Let's make these definitions a little more concrete. Consider this example:</p>
<pre><code>echo &#39;What is your name?&#39;
read name
echo &quot;Good day, $name. Would you like some tea?&quot;</code></pre>
<p>You already know <code>echo</code>. It simply prints its argument to <em>stdout</em>. Since stdout is connected to your terminal, you will see that message there.</p>
<p><code>read</code> is a command that reads one line of text from <em>stdin</em> and stores it in a variable, which we specified to be <code>name</code>. Because stdin is connected to what you type in your terminal, it will let you type a line of text, and as soon as you press enter, that line will be stored in the variable.</p>
<p>So what about stderr? Consider this example:</p>
<pre><code>ehco &#39;Hello!&#39;</code></pre>
<p>The command <code>ehco</code> does not exist. If you run this, bash will print an error message to <em>stderr</em>. Because stderr is connected to your terminal, you will see that message there.</p>
<h2 id="redirection">Redirection</h2>
<p><em>Redirection</em> is the most basic form of input/output manipulation in bash. It is used to change the source or destination of <em>File descriptors</em>, i.e. connect them to something other than your terminal. For example, you can send a command's output to a file instead.</p>
<pre><code>echo &#39;It was a dark and stormy night. Too dark to write.&#39; &gt; story</code></pre>
<p>The <code>&gt;</code> operator begins an <em>output redirection</em>. It redirects the <em>stdout</em> file descriptor of the command to the left, and connects it to a file called &quot;story&quot;. That means if you run this, you will not see the output of <code>echo</code> - after all, stdout no longer points to your terminal.</p>
<p>Note that <code>&gt;</code> will just open the file you specify without checking whether it already exists first. If the file already exists, its contents will be overwritten and you will lose whatever was stored in there before. <strong>Be careful.</strong></p>
<p>If you don't want to overwrite the existing content of a file, but rather append your output to the end of that file, you can use <code>&gt;&gt;</code> instead of <code>&gt;</code>.</p>
<p>Now, let's look at <em>input redirection</em>. For that, we first introduce a command named <code>cat</code>. <code>cat</code> is often used to display the contents of a file, like so:</p>
<pre><code>cat myfile</code></pre>
<p>If <code>cat</code> is called without an argument, however, it will simply read from <em>stdin</em> and print that directly to <em>stdout</em>.</p>
<p>Try the following: Run <code>cat</code> -- without arguments -- in your terminal. Then type some characters and hit enter. Can you figure out what is happening?</p>
<p><em>Input redirection</em> uses the <code>&lt;</code> operator. It works as follows:</p>
<pre><code>cat &lt; story</code></pre>
<p>The <code>&lt;</code> operator will take a command's <em>stdin</em> file descriptor and point it to a file, &quot;story&quot; in this example. This means <code>cat</code> now ignores your terminal and reads from &quot;story&quot; instead. Note that this has the same effect as typing <code>cat story</code>.</p>
<p>If you want to redirect <em>stderr</em> instead of <em>stdout</em>, you can do as follows:</p>
<pre><code>ehco &#39;Hello!&#39; 2&gt; errors</code></pre>
<p>If you run this, you won't see any error message, even though the command <code>ehco</code> doesn't exist. That's because <em>stderr</em> is no longer connected to your terminal, but to a file called &quot;errors&quot; instead.</p>
<p>Now that you know about redirection, there is one subtlety that you have to be aware of: You can't have two file descriptors point to the same file.</p>
<p>If you wanted to log a command's complete output--stdout <em>and</em> stderr--you might be tempted to do something like this:</p>
<pre><code>mycommand &gt; logfile 2&gt; logfile</code></pre>
<p>However, this is a <strong>bad</strong> idea. The two file descriptors will now both point to the same file <em>independently</em>, which causes them to constantly overwrite each other's text.</p>
<p>If you still want to point both stdout and stderr to the same file, you can do it like this:</p>
<pre><code>mycommand &gt; logfile 2&gt;&amp;1</code></pre>
<p>Here, we use the <code>&gt;&amp;</code> syntax to duplicate file descriptor 1. In this scenario, we no longer have two file descriptors pointing to one file. Instead, we have only one file descriptor that acts as both stdout and stderr at the same time.</p>
<p>To help remember the syntax, you can think of <code>&amp;1</code> as &quot;where 1 is&quot;, and of the <code>2&gt;</code> as &quot;point 2 to&quot;. The whole thing, <code>2&gt;&amp;1</code>, then becomes &quot;point 2 to wherever 1 is&quot;. This also makes it clear that <code>&gt; logfile</code> has to come <em>before</em> <code>2&gt;&amp;1</code>: First you point 1 to &quot;logfile&quot;, and only then you can point 2 to where 1 is.</p>
<h2 id="pipes">Pipes</h2>
<p>Now that you know how to manipulate <em>file descriptors</em> to direct output to files, it's time to learn another type of I/O redirection.</p>
<p>The <code>|</code> operator can be used to connect one command's <em>stdout</em> to another command's <em>stdin</em>. Have a look at this:</p>
<pre><code>echo &#39;This is a beautiful day!&#39; | sed &#39;s/beauti/wonder&#39;</code></pre>
<p>The <code>sed</code> command (&quot;sed&quot; is short for &quot;stream editor&quot;) is a utility that can be used to manipulate text &quot;on the fly&quot;. It reads text from stdin, edits it according to some commands, and then prints the result to stdout. It is very powerful. Here, we use it to replace &quot;beauti&quot; with &quot;wonder&quot;.</p>
<p>First, the <code>echo</code> command writes some text to it's stdout. The <code>|</code> operator connected <code>echo</code>'s stout to <code>sed</code>'s stdin, so everything <code>echo</code> sends there is immediately picked up by <code>sed</code>. <code>sed</code> will then edit the text and print the result to its own stdout. <code>sed</code>'s stdout is still connected to your terminal, so this is what you see.</p>
<h1 id="compound-commands">Compound commands</h1>
<p><em>Compound commands</em> is a catch-all phrase covering several different concepts. We've already seen <code>if</code>, <code>for</code>, <code>while</code>, <code>case</code>, <code>select</code> and the <code>[[</code> keyword, which all fall into this category. Now we'll look at a few more.</p>
<h2 id="subshells">Subshells</h2>
<p><em>Subshells</em> can be used to encapsulate a command's effect. If a command has undesired side effects, you can execute it in a subshell. Once the subshell command ends, all side effects will be gone.</p>
<p>To execute a command (or several commands) in a subshell, enclose them in parenthesis:</p>
<pre><code>(
cd /tmp
pwd
)
pwd</code></pre>
<p>The <code>cd</code> and the first <code>pwd</code> commands are executed in a subshell. All side effects in that subshell won't affect the second <code>pwd</code> command. Changing the current directory is such a side effect--even though we use <code>cd</code> to go to the <code>/tmp</code> folder, we jump back to our original folder as soon as the subshell ends.</p>
<h2 id="command-grouping">Command grouping</h2>
<p>You can group several commands together by enclosing them in curly braces. This makes bash consider them as unit with regard to pipes, redirections and control flow:</p>
<pre><code>{
echo &#39;Logfile of my backup&#39;
rsync -av . /backup
echo &quot;Backup finished with exit code $#&quot;
} &gt; backup.log 2&gt;&amp;1</code></pre>
<p>This redirects stdout and stderr of <em>all three commands</em> to a file called backup.log. Note that while this looks similar to subshells, it is not the same. Side effects that happen within the curly braces will still be present outside of them.</p>
<h2 id="arithmetic-evaluation">Arithmetic evaluation</h2>
<p>So far, we've only been manipulating strings in bash. Sometimes, though, it is also necessary to manipulate numbers. This is done through arithmetic evaluation.</p>
<p>Say you want to add the numbers 5 and 4. You might do something like this:</p>
<pre><code>a=5+4</code></pre>
<p>However, this will result in the variable <code>a</code> containing the string <code>5+4</code>, rather than the number <code>9</code>. Instead, you should do this:</p>
<pre><code>(( a=5+4 ))</code></pre>
<p>The double parenthesis indicate that something arithmetic is happening. In fact, <code>((</code> is a bash keyword, much like <code>[[</code>.</p>
<p><code>((</code> can also be used to do arithmetic comparison:</p>
<pre><code>if (( 5 &gt; 9 ))
then
echo &#39;5 is greater than 9&#39;
else
echo &#39;5 is not greater than 9&#39;
fi</code></pre>
<p>It is important not to confuse <code>((</code> and <code>[[</code>. <code>[[</code> is for comparing strings (among other things), while <code>((</code> is only for comparing numbers.</p>
<p>There's also <em>arithmetic substitution</em>, which works similarly to <em>command substitution</em>:</p>
<pre><code>echo &quot;There are $(( 60 * 60 * 24 )) seconds in a day.&quot;</code></pre>
<h1 id="functions">Functions</h1>
<p>Inside a bash script, functions are very handy. They are lists of commands--much like a normal script--except they don't reside in their own file. They do however take arguments, just like scripts.</p>
<p>Functions can be defined like this:</p>
<pre><code>sum() {
echo &quot;$1 + $2 = $(( $1 + $2 ))&quot;
}</code></pre>
<p>If you put this in a script file and run it, absolutely nothing will happen. The function <code>sum</code> has been defined, but it is never used.</p>
<p>You can use your function like any other command, but you have to define it <em>before</em> you use it:</p>
<pre><code>sum() {
echo &quot;$1 + $2 = $(( $1 + $2 ))&quot;
}
sum 1 2
sum 3 9
sum 6283 3141</code></pre>
<p>As you can see, you can use the function <code>sum</code> multiple times, but you only need to define it once. This is useful in larger scripts, where a certain task has to be performed multiple times. Whenever you catch yourself writing the same or very similar code twice in the same script, you should consider using a function.</p>
<h1 id="useful-commands">Useful commands</h1>
<p>This chapter provides an overview of useful commands that you can use in your scripts. It is nowhere near complete, and serves only to provide a brief overview. If you want to know more about a specific command, you should read its manpage.</p>
<h2 id="grep">grep</h2>
<p><code>grep</code> can be used to search for a string within a file, or within the output of a command.</p>
<pre><code>grep &#39;error&#39; logfile.txt # searches logfile.txt for lines containing the word error
grep &#39;analysis&#39; folder/ # searches the directory &#39;folder&#39; for files containing the word &#39;analysis&#39;
xrandr | grep -w &#39;connected&#39; # searches the output of &#39;xrandr&#39; for lines that say &#39;connected&#39;. only matches whole words, so &#39;disconnected&#39; will not match.</code></pre>
<p><code>grep</code> returns 0 if it finds something, and returns an error if it doesn't. This makes it useful for conditionals.</p>
<h2 id="sed">sed</h2>
<p><code>sed</code> can be used to edit text &quot;on the fly&quot;. It uses its own scripting language to describe modifications to be made to the text, which makes it extremely powerful. Here, we provide examples for the most common usages of sed:</p>
<pre><code>sed &#39;s/find/replace&#39; inputfile # replaces the first occurrence of &#39;find&#39; in every line by &#39;replace&#39;
sed &#39;s/find/replace/g&#39; inputfile # replaces every occurrence of &#39;find&#39; in every line by &#39;replace&#39;
sed &#39;s/find//&#39; inputfile # deletes the first occurrence of &#39;find&#39; in every line
sed &#39;s/find//g&#39; inputfile # deletes every occurrence of &#39;find&#39; in every line
sed &#39;12q;d&#39; inputfile # displays only the 12th line</code></pre>
<p><code>sed</code> is often used in combination with pipes to format the output or get rid of unwanted characters.</p>
<h2 id="curl-and-wget">curl and wget</h2>
<p><code>curl</code> and <code>wget</code> are two commands that can be used to access websites or other content from the web. The difference is that <code>wget</code> will simply download the content to a file, while <code>curl</code> will output it to the console.</p>
<pre><code>curl http://www.thealternative.ch
wget http://files.project21.ch/LinuxDays-Public/16FS-install-guide.pdf</code></pre>
<h2 id="xrandr">xrandr</h2>
<p><code>xrandr</code> can be used to manipulate your video outputs, i.e. enabling and disabling monitors or setting their screen resolution and orientation.</p>
<pre><code>xrandr # list all available outputs and their status info
xrandr --output HDMI-1 --auto # enables output HDMI-1
xrandr --output HDMI-1 --left-of LVDS-1 # puts output HDMI-1 to the left of output LVDS-1
xrandr --output LVDS-1 --off # disables output LVDS-1</code></pre>
<h2 id="imagemagick-convert">ImageMagick (convert)</h2>
<p>The <code>convert</code> command makes it possible to do image processing from the commandline.</p>
<pre><code>convert fullHDWallpaper.jpg -scale 3200x1800 evenBiggerWallpaper.jpg # scale fullHDWallpaper.jpg to specified resolution
convert somePicture.jpg -auto-gamma someOtherPicture.jpg # &quot;automagically&quot; adjusts the gamma level of somePicture.jpg
convert colorfulPicture.jpg -monochrome blackWhitePicture.jpg # transform image to black and white</code></pre>
<p>It is extremely powerful and has lots of options. A good resource is the official website, &lt;www.imagemagick.org&gt;. It also provides examples for most options.</p>
<h2 id="notify-send">notify-send</h2>
<p><code>notify-send</code> can be used to display a desktop notification with some custom text:</p>
<pre><code>notify-send &#39;Battery warning&#39; &#39;Your battery level is below 10%&#39;</code></pre>
<p>The first argument is the notification's title, the second is its description.</p>
<p><code>notify-send</code> requires a <em>notification daemon</em> to be running, else it won't work. Most desktop environments come with a notification daemon set up and running. If you can't see your notifications, it might be that you don't have such a daemon.</p>
<h2 id="find">find</h2>
<p><code>find</code> can be used to find files in a directory structure.</p>
<pre><code>find -name &#39;*.png&#39; # finds all files in the current directory and all subdirectories that end in .png
find -name &#39;*.tmp* -exec rm &#39;{}&#39; \; # finds all files ending in .tmp and removes them. {} is replaced by the file&#39;s name when executing the command.
find &#39;files/&#39; -printf &#39;%s %p\n&#39; # finds all files in the directory &#39;files&#39; and prints their size and path</code></pre>
<p><code>find</code> has many options that allow you to perform arbitrary actions on the files it found or pretty-print the output.</p>
<h2 id="sort">sort</h2>
<p><code>sort</code> sorts lines of text files, or lines it reads from <em>stdin</em>.</p>
<pre><code>sort listOfNames.txt # sorts all lines in listOfNames.txt alphabedically</code></pre>
<h2 id="head-and-tail">head and tail</h2>
<p><code>head</code> and <code>tail</code> can be used to show the beginning or the end of a long stream of text, respectively.</p>
<pre><code>dmesg | tail # display the last few lines of the dmesg log
head verylongtext.txt # display only the first few lines of a very long text file</code></pre>
<h2 id="jshon">jshon</h2>
<p><code>jshon</code> is a very simple command-line json parser. It can read data in json format and read out specific values.</p>
<h2 id="shuf">shuf</h2>
<h2 id="sleep">sleep</h2>
<h2 id="mplayer-or-mpv">mplayer or mpv</h2>
<h2 id="xdotool">xdotool</h2>
</body>
</html>
# Bash Quick Guide
Bash is an acronym for Bourne Again Shell. It is based on the Bourne shell and is mostly compatible with its features.
Shells are command interpreters. They are applications that provide users with the ability to give commands to their operating system interactively, or to execute batches of commands quickly. In no way are they required for the execution of programs; they are merely a layer between system function calls and the user.
Think of a shell as a way for you to speak to your system. Your system doesn't need it for most of its work, but it is an excellent interface between you and what your system can offer. It allows you to perform basic math, run basic tests and execute applications. More importantly, it allows you to combine these operations and connect applications to each other to perform complex and automated tasks.
Bash is not your operating system. It is not your window manager. It is not your terminal (but it often runs inside your terminal). It does not control your mouse or keyboard. It does not configure your system, activate your screen-saver, or open your files. It's important to understand that bash is only an interface for you to execute statements (using bash syntax), either at the interactive bash prompt or via bash scripts. The things that *actually happen* are usually caused by other programs.
This guide is based on [the bash guide in GreyCat's wiki](http://mywiki.wooledge.org/BashGuide) and aims to be more concise, while still being accurate. It was produced specifically for the Bash Workshop by [TheAlternative.ch](www.thealternative.ch).
It is published under the [CC by-nc-sa 4.0 license](http://creativecommons.org/licenses/by-nc-sa/4.0/).
# Commands and Arguments
Bash reads commands from its input, which can be either a file or your terminal.
In general, each line is interpreted as a command followed by its arguments.
ls
touch file1 file2 file3
ls -l
rm file1 file2 file3
The first word is always the command that is executed. All
subsequent words are given to that command as argument.
Note that *options*, such as the `-l` option in the example above, are not treated specially by bash. They are arguments like any other. It is up to the program (`ls` in the above case) to treat it as an option.
Words are delimited by whitespace (spaces or tabs). It does not matter how many spaces there are between two words. For example, try
echo Hello World
The process of splitting a line of text into words is called *word splitting*. It is vital to be aware of it, especially when you come across expansions later on.
## Preventing Word Splitting
Sometimes, you will want to pass arguments to commands that contain whitespace. To do so, you can use quotes:
touch "Filename with spaces"
This command creates a single file named *Filename with spaces*. The text within double quotes is protected from word splitting and hence treated as a single word.
Note that you can also use single quotes:
touch 'Another filename with spaces'
There is, however, an important difference between the two:
* Double quotes prevent **word splitting**
* Single quotes prevent **word splitting and expansion**
When you use single quotes, the quoted text will never be changed by bash. With double quotes, expansion will still happen. This doesn't make a difference in the above example, but as soon as you e.g. use variables, it becomes important.
In general, it is considered good practice to use single quotes whenever possible, and double quotes only when expansion is desired. In that sense, the last example above can be considered "more correct".
## The Importance of Spaces
Bash contains various keywords and built-ins that aren't immediately recognizable as commands, such as the new test command:
[[ -f file ]]
The above code tests whether a file named "file" exists in the current directory. Just like every line of bash code, it consists of a command followed by its arguments. Here, the command is `[[`, while the arguments are `-f`, `file` and `]]`.
Many programmers of other languages would write the above command like so:
[[-f file]]
This, though, is wrong: Bash will look for a command named `[[-f`, which doesn't exist, and issue an error message. This kind of mistake is very common for beginners. It is advisable to always use spaces after any kind of brackets in bash, even though there are cases where they are not necessary.
## Scripts
You have probably interacted with bash through a terminal before. You would see a bash prompt, and you would issue one command after another.
Bash scripts are basically a sequence of commands stored in a file. They are read and processed in order, one after the other.
Making a script is easy. Begin by making a new file, and put this on the first line:
#!/bin/bash
This line is called an *interpreter directive*, or more commonly, a *hashbang* or *shebang*. Your operating system uses it to determine how this file can be run. In this case, the file is to be run using `bash`, which is stored in the `/bin/` directory.
After the shebang, you can add any command that you could also use in your terminal. For example, you could add
echo 'Hello World'
and then save the file as "myscript"
You can now run the file from the terminal by typing
bash myscript
Here, you explicitly called bash and made it execute the script. `bash` is used as the command, while `myscript` is an argument. However, it's also possible to use `myscript` as a command directly.
To do so, you must first make it executable:
chmod +x myscript
Now that you have permission to execute this script directly, you can type
./myscript
to run it.
The `./` is required to tell bash that the executable is located in the current directory, rather than the system directory. We will come back to this in the chapter on Variables.
# Variables and Parameters
Variables and parameters can be used to store strings and retrieve them later. *Variables* are the ones you create yourself, while *special parameters* are pre-set by bash. *Parameters* actually refers to both, but is often used synonymously to special parameters.
To store a string in a variable, we use the *assignment syntax*:
varname=vardata
This sets the variable `varname` to contain the string `vardata`.
Note that you cannot use spaces around the `=` sign. With the spaces, bash would assume `varname` to be a command and then pass `=` and `vardata` as arguments.
To access the string that is now stored in the variable `varname`, we have to use *parameter expansion*. This is the most common kind of expansion: A variable is replaced with its content.
If you want to print the variable's value, you can type
echo $varname
The `$` indicates that you want to use expansion on `varname`, meaning it is replaced by its content. Note that expansion happens before the command is run. Here's what happens step-by-step:
* Bash uses variable expansion, changing `echo $varname` to `echo vardata`
* Then, bash runs `echo` with `vardata` as its parameter.
The most important thing here is that **variable expansion happens before wordsplitting**. That means, if you have defined a variable like this:
myfile='bad song.mp3'
and then run the command
rm $myfile
bash will expand this to
rm bad song.mp3
Only now, word splitting occurs, and bash will call `rm` with two arguments: `bad` and `song.mp3`. If you now had a file called `song.mp3` in your current directory, that one would be deleted instead.
To prevent this from happening, you can use double quotes:
rm "$myfile"
This will be expanded to
rm "bad song.mp3"
which is what we want. In this case, you have to use double quotes, as single quotes would prevent expansion from happening altogether.
Not quoting variable and parameter expansions is a very common mistake even among advanced bash programmers. It can cause bugs that are hard to find and can be very dangerous. **Always quote your variable expansions.**
You can also use variable expansions inside the variable assignment itself. Consider this example:
myvariable='blah'
myvariable="$myvariable blah"
echo "$myvariable"
What will the output of this script be?
First, the variable `myvariable` will get the value `blah`. Then, `myvariable` is assigned to again, which overwrites its former content. The assignment contains a variable expansion, `"$myvariable blah"`. This is expanded to `"blah blah"`, and that is going to be the new value of `myvariable`. So the last command is expanded to `echo "blah blah"`, and the output of the script is `blah blah`.
## Special Parameters
*Special parameters* are variables that are set by bash itself. Most of those variables can't be written to and they contain useful information.
Parameter Name | Usage | Description
-----------|-----------|---------------------------------------------------------------------
`0` | `"$0"` | Contains the name of the current script
`1` `2` `3` etc. | `"$1"` etc. | Contains the arguments that were passed to the current script. The number indicates the position of that argument (first, second...). These parameters are also called positional parameters.
`*` | `"$*"` | Contains all the positional parameters. Double quoted, it expands to a single word containing them all.
`@` | `"$@"` | Contains all the positional parameters. Double quoted, it expands to **several words, where each word is one parameter**. This is special syntax, it behaves differently from "normal" expansion in quotes. It retains the arguments exactly as they were passed to the script.
`#` | `"$#"` | Contains the number of parameters that were passed to the script
`?` | `"$?"` | Contains the exit code of the last executed command
## Environment Variables
*Environment variables* are special variables that are already set when you start bash. In fact, these variables aren't specific to bash - they are available in every program that runs on your system and they can affect your system's behaviour.
You can use the command `printenv` in your terminal to display all the environment variables you have currently set. Most of them contain some system configuration, like the variable `LANG`, which designates your preferred language. Some variables, like `TERM`, `BROWSER` or `SHELL`, designate your preferred default programs (terminal, web browser and shell, respectively. These may not be set on all systems).
Some of these variables can be useful in your scripts. For example, the variable `RANDOM` gives you a different random number every time you read it.
Another important environment variable is `PATH`. It contains a bunch of file paths, separated by colons. These paths designate where your system will look for executables when you type a command. For example, if you type `grep` in your terminal, your system will search for an executable called `grep` in the directories designated in your `PATH` variable. As soon as it finds one, it will execute that. If it doesn't find it, you will get a "command not found" error message.
You can modify your environment variables, if you want. The guideline here is to only mess with those variables of which you know what they do, otherwise you might break something.
The place to modify these variables is your `~/.bash_profile` file. This file contains some bash code that is executed whenever you log in. For example, you could add the following line:
export BROWSER="firefox"
This would set your default browser to firefox. Note that on some systems, there are other settings--for example in your Desktop Environment--which can override these environment variables. You'll have to test whether this works.
Note the `export` keyword. This is a bash builtin that takes a variable definition as its argument and puts it in your *environment*. If you omit this, your new variable will just be an ordinary variable, rather than an environment variable.
## Ambiguous Names
Say you have a variable called `name` that is declared as follows:
name='bat'
Now, you want to use this variable in order to print *batman*:
echo "$nameman"
If you try this, you will notice that it doesn't work--that is because bash will now look for a variable called `nameman`, which doesn't exist. Here's what you can do instead:
echo "${name}man"
The curly braces tell bash where the variable name ends. This allows you to add more characters at the end of a variable's content.
# Globbing
*Globs* are an important bash concept--mostly for their incredible convenience. They are patterns that can be used to match filenames or other strings.
Globs are composed of normal characters and metacharacters. Metacharacters are characters that have a special meaning. These are the metacharacters that can be used in globs:
* `*`: Matches any string, including the empty string (i.e. nothing)
* `?`: Matches any single character
* `[...]`: Matches any one of the characters enclosed in the brackets
Bash sees the glob, for example `a*`. It expands this glob, by looking in the current directory and matching it against all files there. Any filenames that match the glob are gathered up and sorted, and then the list of filenames is used in place of the glob. So if you have three files `a`, `b` and `albert` in the current directory, the glob is expanded to `a albert`.
A glob always has to match the entire filename. That means `a*` will match `at` but not `bat`.
Note that globbing is special in that it happens *after word splitting*. This means you never need to worry about spaces in filenames when you use globbing, and quoting globs is not necessary. In fact, quotes will prevent globbing from happening.
# Expansion
We've already seen *parameter and variable expansion*, but that's not the only kind of expansion that happens in bash. In this chapter, we'll look at all kinds of expansion that aren't covered elsewhere.
## Expansions and Quotes
You already know that it is important to quote parameter and variable expansions, but we also told you that quoting globs--which are, in fact, just another form of expansion--is not necessary. So, which expansions need to be quoted?
The rule of thumb here is as follows:
* Always quote **paramater expansion**, **command substitution** and **arithmetic expansion**
* Never quote **brace expansion**, **tilde expansion** and **globs**
The handy thing here is: All the expansions that require quoting have a `$` in their syntax. Parameter expansion is simply a `$` followed by a parameter name. Command substitution starts with a `$(`, and arithmetic expansion starts with `$((`.
So, the rule of thumb breaks down to the following: **If there's a dollar, you probably need quotes.**
Now, what if you want to use two kinds of expansion in the same line, but one requires quotes and the other doesn't? Consider the following script:
prefix='my picture'
rm ~/pictures/$prefix*
Here, we use tilde expansion, parameter expansion and globbing in order to remove all files that start with `my picture` in the folder `/home/username/pictures/`. But because quotes prevent tilde expansion and globbing, we cannot quote the entire expression. This means that the parameter expansion, too, goes unquoted--and this is fatal, because our variable contains a space. So what should we do?
The important thing to realize here is that quoting simply prevents word splitting, but it does not actually designate something as a single string. So we can do the following:
prefix='my picture'
rm ~/pictures/"$prefix"*
Only the parameter expansion is quoted, so it is protected from word splitting. But that does not automatically separate it from the rest of the string. Note that there are no spaces between `"$prefix"` and `~/pictures/`. Since word splitting only happens when there are spaces, the entire thing will not be split. Here's what happens, in order:
First, tilde expansion occurs:
rm /home/username/pictures/"$prefix"/*
Next, parameter expansion:
rm /home/username/pictures/"my picture"*
At this point, word splitting happens. But since the only space in our argument is in quotes, the argument remains intact.
And last, globbing:
rm /home/username/pictures/"my picture"001.jpg /home/username/pictures/"my picture"002.jpg
Now, there's one last step that happens which we didn't mention before. It's called *quote removal*. All the quotes that were needed to prevent word splitting are now ignored, which means that the arguments that are finally given to `rm` are:
* `/home/username/pictures/my picture001.jpg`
* `/home/username/pictures/my picture002.jpg`
and this is exactly what we wanted.
So, remember: Quotes don't need to be at the beginning or end of an argument, and if you use several kinds of expansion together, you can add quotes in the middle as required.
## Expansion Order
All the kinds of expansion happen in a certain order. The order is as follows:
* Brace expansion
* Tilde expansion
* Parameter and variable expansion
* Command substitution
* Arithmetic expansion
* Word splitting
* Globbing
## Brace Expansion
*Brace expansions* are often used in conjunction with globs, but they also have other uses. They always expand to all possible permutations of their contents. Here's an example:
$ echo th{e,a}n
then than
$ echo {1..9}
1 2 3 4 5 6 7 8 9
$ echo {0,1}{0..9}
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19
Brace expansions are replaced by a list of words. They are often used in conjunction with globs to match specific files but not others. For example, if you want to delete pictures from your pictures folder with filenames IMG020.jpg through IMG039.jpg, you could use the following pattern:
rm IMG0{2,3}*.jpg
Note that we don't use quotes here. Quotes prevent brace expansion.
Brace expansion happens before globbing, so in the above example, the braces are expanded to
rm IMG02*.jpg IMG03*.jpg
We end up with two glob patterns, the first matches IMG020.jpg through IMG029.jpg, and the second matches IMG030.jpg through IMG039.jpg.
## Tilde Expansion
You have probably already seen and used the tilde in the terminal. It is a shortcut to your home directory:
cd ~/files
This will be expanded to `cd /home/username/files`. Note that tilde expansion only happens outside of quotes, so the following won't work:
cd "~/files"
## Parameter and Variable Expansion
*Parameter and variable expansion* is explained in the chapter on Variables and Parameters.
An important sidenote here is that Parameter expansion and Variable expansion often refer to the same thing. The official name as per the bash manual is *Parameter expansion*, but *Variable expansion* is often used instead as it is less misleading.
## Command Substitution
*Command substitution* is a way of using a command's output inside your script. For example, let's say you want to find out where your script is executed, and then move a file to that location.
echo 'I am at'
pwd
This script will simply print the directory it is called from. But you want to move a file to that directory, which means you need to use it as an argument to the `mv` command. For that, you need command substitution:
mv ~/myfile "$( pwd )"
The `$(` introduces a command substitution. It works similarly to variable expansion, but instead of putting a variable's content, it puts a command's output in that place. In this example, `pwd` is run, and the output is (for example) `/home/username/scripts/`. Then, the entire command is expanded to
mv ~/myfile "/home/username/scripts/"
Note that with this kind of expansion, quotes are important. The path returned by `pwd` might have contained a space, in which case we need the argument to be properly quoted.
## Arithmetic Expansion
*Arithmetic expansion* is explained in the chapter on arithmetic evaluation.
## Globbing
*Globbing* is so important, it has a chapter of its own.
# Tests and Conditionals
Every command you run in your terminal or shell script has a return value. That value is a number, and by convention, a return value of 0 means *success*, while any other number indicates an error.
You usually don't see the return value after running a command. What you do see is the command's *output*. If you want to know a command's return value, you can read it by issuing
echo "$?"
immediately after running that command.
While you often don't need to know a command's return value, it can be useful to construct conditionals and thereby achieve advanced logic.
## Control Operators (&& and ||)
Control operators can be used to make a command's execution depend on another command's success. This concept is called *conditional execution*:
mkdir folder && cd folder
In the above example, the `&&` operator is used to connect two commands `mkdir folder` and `cd folder`. Using this connection, bash will first execute `mkdir folder`, and then execute `cd folder` *only if the first command was successful*.
mkdir folder || echo 'Error: could not create folder'
In this example, the `||` operator is used to connect the commands. Here, the second command is executed *only if the first command failed*.
It is good practice to make your own scripts return an error (i.e. something other than 0) whenever something goes wrong. To do that, you can use this construct:
mkdir folder || exit 1
The `exit` command immediately stops the execution of your script and makes it return the number you specified as an argument. In this example, your script will attempt to create a folder. If that goes wrong, it will immediately stop and return 1.
Control operators can be written more legibly by spreading them across multiple lines:
mkdir folder \
|| exit 1
The backslash at the end of the first line makes bash ignore that line break and treat both lines as a single command plus arguments.
## Grouping Commands
Now, what if you want to execute multiple commands if the first one fails? Going from the example above, when `mkdir folder` fails, you might want to print an error message *and* return 1.
This can be done by enclosing these two commands in single curly braces:
mkdir folder || { echo 'Could not create folder'; exit 1 }
The two commands in curly braces are treated as an unit, and if the first command fails, both will be executed.
Note that you need to include spaces between the curly braces and the commands. If there were no spaces, bash would look for a command named `{echo` and fail to find it.
There's a semicolon separating the two commands. The semicolon has the same function as a line break: it makes bash consider both parts as individual commands-plus-arguments.
The example above could be rewritten as follows:
mkdir folder || {
echo 'Could not create folder'
exit 1
}
Here, no semicolon is required, because there is a line break between the two statements. Line breaks and semicolons can be used interchangeably.
## Conditional Blocks (if and [[)
`if` is a shell keyword. It first executes a command and looks at its return value. If it was 0 (success), it executes one list of commands, and if it was something else (failure), it executes another.
It looks like this:
if true
then
echo 'It was true'
else
echo 'It was false'
fi
`true` is a bash builtin command that does nothing and returns 0. `if` will run that command, see that it returned 0, and then execute the commands between the `then` and the `else`.
In many programming languages, operators such as `>`, `<` or `==` exist and can be used to compare values. In bash, operators don't exist. But since comparing values is so common, there's a special command that can do it:
[[ a = b ]]
`[[` is a command that takes as its arguments a comparison. The last argument has to be a `]]`. It is made to look like a comparison in double brackets, but it is, in fact, a command like any other. It is also called the *new test command*. (The *old test command*, often simply called *test*, exists as well, so be careful not to confuse them).
For that reason, the spaces are absolutely needed. You cannot write this:
[[a=b]]
This will make bash look for a command called `[[a=b]]`, which doesn't exist.
`[[` does not only support comparing strings. For example, `[[ -f file ]]` will test whether a file named "file" exists. Here's a list of the most common tests you can use with `[[`:
* `-e FILE:` True if file exists.
* `-f FILE:` True if file is a regular file.
* `-d FILE:` True if file is a directory.
* String operators:
* `-z STRING:` True if the string is empty (its length is zero).
* `-n STRING:` True if the string is not empty (its length is not zero).
* `STRING = STRING`: True if the string matches the glob pattern (if you quote the glob pattern, the strings have to match exactly).
* `STRING != STRING`: True if the string does not match the glob pattern (if you quote the glob pattern, the strings just have to be different).
* `STRING < STRING`: True if the first string sorts before the second.
* `STRING > STRING`: True if the first string sorts after the second.
* `EXPR -a EXPR`: True if both expressions are true (logical AND).
* `EXPR -o EXPR`: True if either expression is true (logical OR).
* `! EXPR`: Inverts the result of the expression (logical NOT).
* `EXPR && EXPR`: Much like the '-a' operator of test, but does not evaluate the second expression if the first already turns out to be false.
* `EXPR || EXPR`: Much like the '-o' operator of test, but does not evaluate the second expression if the first already turns out to be true.
* Numeric operators:
* `INT -eq INT`: True if both integers are equal.
* `INT -ne INT`: True if the integers are not equal.
* `INT -lt INT`: True if the first integer is less than the second.
* `INT -gt INT`: True if the first integer is greater than the second.
* `INT -le INT`: True if the first integer is less than or equal to the second.
* `INT -ge INT`: True if the first integer is greater than or equal to the second.
You might occasionally come across something like this:
[ a = b ]
Here, we use single brackets instead of double brackets. This is, in fact, an entirely different command, the `[` command or *old test command*. It has the same purpose--comparing things--but the `[[` command is newer, has more features, and is easier to use. We strongly recommend using `[[` over `[`.
## Conditional Loops (while, until and for)
Loops can be used to repeat a list of commands multiple times. In bash, there are `while` loops and `for` loops.
While loops look like this:
while true
do
echo 'Infinite loop'
done
The `while` keyword will execute the `true` command, and if that returns 0, it executes all commands between the `do` and `done`. After that, it starts over, until the `true` command returns 1 (which it never does, which is why this loop will run indefinitely).
The above example might not be immediately useful, but you could also do something like this:
while ping -c 1 -W 1 www.google.com
do
echo 'Google still works!'
done
There's also a variation of the `while` loop, called `until`. It works similarly, except it only runs its command list when the first command *fails*:
until ping -c 1 -W 1 www.google.com
do
echo 'Google isn'\''t working!'
done
`for` loops can be used to iterate over a list of strings:
for var in 1 2 3
do
echo "$var"
done
After the `for`, you specify a variable name. After the `in`, you list all the strings you want to iterate over.
The loop works by setting the variable you specified to all the values from the list in turn, and then executing the command list for each of them.
This is especially useful in combination with globs or brace expansions:
echo 'This is a list of all my files starting with f:'
for var in f*
do
echo "$var"
done
echo 'And now I will count from 1 to 100:'
for var in {1..100}
do
echo "$var"
done
## Choices (case and select)
Sometimes, you want your script to behave differently depending on the content of a variable. This could be implemented by taking a different branch of an if statement for each state:
if [[ "$LANG" = 'en' ]]
then
echo 'Hello!'
elif [[ "$LANG" = 'de' ]]
then
echo 'Guten Tag!'
elif [[ "$LANG" = 'it' ]]
then
echo 'Ciao!'
else
echo 'I do not speak your language.'
fi
This is quite cumbersome to write. At the same time, constructs like this are very common. For that reason, bash provides a keyword to simplify it:
case "$LANG" in
en)
echo 'Hello!'
;;
de)
echo 'Guten Tag!'
;;
it)
echo 'Ciao!'
;;
*)
echo 'I do not speak your language.'
;;
esac
Each choice of the case statement consists of a string or glob pattern, a `)`, a list of commands that is to be executed if the string matches the pattern, and two semicolons to denote the end of a list of commands.
The string after the keyword `case` is matched against each glob pattern in order. The list of commands after the first match is executed. After that, execution continues after the `esac`.
Since the string is matched against glob patterns, we can use `*` in the end to catch anything that didn't match before.
Another construct of choice is the `select` construct. It looks and works similarly to a loop, but it also presents the user with a predefined choice. You are encouraged to try running this example yourself:
echo 'Which one of these does not belong in the group?'
select choice in Apples Pears Crisps Lemons Kiwis
do
if [[ "$choice" = Crisps ]]
then
echo 'Correct! Crisps are not fruit.'
break
fi
echo 'Wrong answer. Try again.'
done
The syntax of the `select` construct is very similar to `for` loops. The difference is that instead of setting the variable (`choice` in this example) to each value in turn, the `select` construct lets the user choose which value is used next.
This also means that a `select` construct can run indefinitely, because the user can keep selecting new choices. To avoid being trapped in it, we have to explicitly use `break`. `break` is a builtin command that makes bash jump out of the current `do` block. Execution will continue after the `done`. `break` also works in `for` and `while` loops.
As you can see in the example above, we used an `if` command inside a `select` command. All of these conditional constructs (`if`, `for`, `while`, `case` and `select`) can be nested indefinitely.
# Input and Output
Input and output in bash is very flexible and, consequentially, complex. We will only look at the most widely used components.
## Command-line Arguments
For many bash scripts, the first input we care about are the arguments given to it via the command line. As we saw in the chapter on Parameters, these arguments are contained in some *special parameters*. These are called *positional parameters*. The first parameter is referred to with `$1`, the second with `$2`, and so on. After number 9, you have to enclose the numbers in curly braces: `${10}`, `${11}` and so on.
In addition to referring to them one at a time, you may also refer to the entire set of positional parameters with the `"$@"` substitution. The double quotes here are **extremely important**. If you don't use the double quotes, each one of the positional parameters will undergo word splitting and globbing. You don't want that. By using the quotes, you tell Bash that you want to preserve each parameter as a separate word.
There are even more ways to deal with parameters. For example, it is very common for commands to accept *options*, which are single letters starting with a `-`. For example, `ls -l` calls the `ls` program with the `-l` option, which makes it output more information. Usually, multiple options can be combined, as in `ls -la`, which is equivalent to `ls -l -a`.
You might want to create your own scripts that accept some options. Bash has the so called `getopts` builtin command to parse passed options.
while getopts 'hlf:' opt
do
case "$opt" in
h|\?)
echo 'available options: -h -l -f [filename]'
;;
f)
file="$OPTARG"
;;
l)
list=true
;;
esac
done
shift "$(( OPTIND - 1 ))"
As you can see, we use `getopts` within a while loop. `getopts` will return 0 as long as there are more options remaining and something else if there are no more options. That makes it perfectly suitable for a loop.
`getopts` takes two arguments, the *optstring* and the *variable name*. The optstring contains all the letters that are valid options. In our example, these are `h`, `l` and `f`.
The `f` is followed by a colon. This indicates that the f option requires an argument. The script could for example be called like so:
myscript -f file.txt
`getopts` will set the variable that you specified as its second argument to the letter of the option it found first. If the option required an argument, the variable `OPTARG` is set to whatever the argument was. In the example above, a case statement is used to react to the different options.
There's also a special option `?`. Whenever `getopts` finds an option that is not present in the optstring, it sets the shell variable (`opt` in the example) to `?`. In the case statement above, that triggers the help message.
After all options are parsed, the remaining arguments are "moved" such that they are now in `$1`, `$2`... even though previously, these positional parameters were occupied by the options.
Also note the line `shift "$(( OPTIND - 1 ))"` at the end. The `shift` builtin can be used to discard command-line arguments. Its argument is a number and designates how many arguments we want to discard.
This is needed because we don't know how many options the user will pass to our script. If there are more positional parameters after all the options, we have no way of knowing at which number they start. Fortunately, `getopts` also sets the shell variable `OPTIND`, in which it stores the index of the option it's going to parse next.
So after parsing all the option, we just discard the first `OPTIND - 1` options, and the remaining arguments now start from `$1` onwards.
## File Descriptors
*File descriptors* are the way programs refer to files, or other things that work like files (such as pipes, devices, or terminals). You can think of them as pointers that point to data locations. Through these pointers, programs can write to or read from these locations.
By default, every program has three file descriptors:
* Standard Input (stdin): File Descriptor 0
* Standard Output (stdout): File Descriptor 1
* Standard Error (stderr): File Descriptor 2
When you run a script in the terminal, then stdin contains everything you type in that terminal. stdout and stderr both point to the terminal, and everything that is written to these two is displayed as text in the terminal. stdout is where programs send their normal information, and stderr is where they send their error messages.
Let's make these definitions a little more concrete. Consider this example:
echo 'What is your name?'
read name
echo "Good day, $name. Would you like some tea?"
You already know `echo`. It simply prints its argument to *stdout*. Since stdout is connected to your terminal, you will see that message there.
`read` is a command that reads one line of text from *stdin* and stores it in a variable, which we specified to be `name`. Because stdin is connected to what you type in your terminal, it will let you type a line of text, and as soon as you press enter, that line will be stored in the variable.
So what about stderr? Consider this example:
ehco 'Hello!'
The command `ehco` does not exist. If you run this, bash will print an error message to *stderr*. Because stderr is connected to your terminal, you will see that message there.
## Redirection
*Redirection* is the most basic form of input/output manipulation in bash. It is used to change the source or destination of *File descriptors*, i.e. connect them to something other than your terminal. For example, you can send a command's output to a file instead.
echo 'It was a dark and stormy night. Too dark to write.' > story
The `>` operator begins an *output redirection*. It redirects the *stdout* file descriptor of the command to the left, and connects it to a file called "story". That means if you run this, you will not see the output of `echo`---after all, stdout no longer points to your terminal.
Note that `>` will just open the file you specify without checking whether it already exists first. If the file already exists, its contents will be overwritten and you will lose whatever was stored in there before. **Be careful.**
If you don't want to overwrite the existing content of a file, but rather append your output to the end of that file, you can use `>>` instead of `>`.
Now, let's look at *input redirection*. For that, we first introduce a command named `cat`. `cat` is often used to display the contents of a file, like so:
cat myfile
If `cat` is called without an argument, however, it will simply read from *stdin* and print that directly to *stdout*.
Try the following: Run `cat`, without arguments, in your terminal. Then type some characters and hit enter. Can you figure out what is happening?
*Input redirection* uses the `<` operator. It works as follows:
cat < story
The `<` operator will take a command's *stdin* file descriptor and point it to a file, "story" in this example. This means `cat` now ignores your terminal and reads from "story" instead. Note that this has the same effect as typing `cat story`.
If you want to redirect *stderr* instead of *stdout*, you can do as follows:
ehco 'Hello!' 2> errors
If you run this, you won't see any error message, even though the command `ehco` doesn't exist. That's because *stderr* is no longer connected to your terminal, but to a file called "errors" instead.
Now that you know about redirection, there is one subtlety that you have to be aware of: You can't have two file descriptors point to the same file.
If you wanted to log a command's complete output--stdout *and* stderr--you might be tempted to do something like this:
mycommand > logfile 2> logfile
However, this is a **bad** idea. The two file descriptors will now both point to the same file *independently*, which causes them to constantly overwrite each other's text.
If you still want to point both stdout and stderr to the same file, you can do it like this:
mycommand > logfile 2>&1
Here, we use the `>&` syntax to duplicate file descriptor 1. In this scenario, we no longer have two file descriptors pointing to one file. Instead, we have only one file descriptor that acts as both stdout and stderr at the same time.
To help remember the syntax, you can think of `&1` as "where 1 is", and of the `2>` as "point 2 to". The whole thing, `2>&1`, then becomes "point 2 to wherever 1 is". This also makes it clear that `> logfile` has to come *before* `2>&1`: First you point 1 to "logfile", and only then you can point 2 to where 1 is.
There's also a quick way to completely get rid of a command's output using redirections. Consider this:
mycommand > /dev/null 2>&1
`/dev/null` is a special file in your file system that you can write to, but the things you write to it are not stored. So, `mycommand`'s output is written somewhere where it's not stored or processed in any way. It is discarded completely.
You could also leave out the `2>&1`. Then, you'd still see error messages, but discard the normal output.
## Pipes
Now that you know how to manipulate *file descriptors* to direct output to files, it's time to learn another type of I/O redirection.
The `|` operator can be used to connect one command's *stdout* to another command's *stdin*. Have a look at this:
echo 'This is a beautiful day!' | sed 's/beauti/wonder'
The `sed` command ("sed" is short for "stream editor") is a utility that can be used to manipulate text "on the fly". It reads text from stdin, edits it according to some commands, and then prints the result to stdout. It is very powerful. Here, we use it to replace "beauti" with "wonder".
First, the `echo` command writes some text to it's stdout. The `|` operator connected `echo`'s stout to `sed`'s stdin, so everything `echo` sends there is immediately picked up by `sed`. `sed` will then edit the text and print the result to its own stdout. `sed`'s stdout is still connected to your terminal, so this is what you see.
# Compound Commands
*Compound commands* is a catch-all phrase covering several different concepts. We've already seen `if`, `for`, `while`, `case`, `select` and the `[[` keyword, which all fall into this category. Now we'll look at a few more.
## Subshells
*Subshells* can be used to encapsulate a command's effect. If a command has undesired side effects, you can execute it in a subshell. Once the subshell command ends, all side effects will be gone.
To execute a command (or several commands) in a subshell, enclose them in parenthesis:
(
cd /tmp
pwd
)
pwd
The `cd` and the first `pwd` commands are executed in a subshell. All side effects in that subshell won't affect the second `pwd` command. Changing the current directory is such a side effect--even though we use `cd` to go to the `/tmp` folder, we jump back to our original folder as soon as the subshell ends.
## Command Grouping
You can group several commands together by enclosing them in curly braces. This makes bash consider them as a unit with regard to pipes, redirections and control flow:
{
echo 'Logfile of my backup'
rsync -av . /backup
echo "Backup finished with exit code $#"
} > backup.log 2>&1
This redirects stdout and stderr of *all three commands* to a file called backup.log. Note that while this looks similar to subshells, it is not the same. Side effects that happen within the curly braces will still be present outside of them.
## Arithmetic Evaluation
So far, we've only been manipulating strings in bash. Sometimes, though, it is also necessary to manipulate numbers. This is done through arithmetic evaluation.
Say you want to add the numbers 5 and 4. You might do something like this:
a=5+4
However, this will result in the variable `a` containing the string `5+4`, rather than the number `9`. Instead, you should do this:
(( a=5+4 ))
The double parenthesis indicate that something arithmetic is happening. In fact, `((` is a bash keyword, much like `[[`.
`((` can also be used to do arithmetic comparison:
if (( 5 > 9 ))
then
echo '5 is greater than 9'
else
echo '5 is not greater than 9'
fi
It is important not to confuse `((` and `[[`. `[[` is for comparing strings (among other things), while `((` is only for comparing numbers.
There's also *arithmetic substitution*, which works similarly to *command substitution*:
echo "There are $(( 60 * 60 * 24 )) seconds in a day."
# Functions
Inside a bash script, functions are very handy. They are lists of commands--much like a normal script--except they don't reside in their own file. They do however take arguments, just like scripts.
Functions can be defined like this:
sum() {
echo "$1 + $2 = $(( $1 + $2 ))"
}
If you put this in a script file and run it, absolutely nothing will happen. The function `sum` has been defined, but it is never used.
You can use your function like any other command, but you have to define it *before* you use it:
sum() {
echo "$1 + $2 = $(( $1 + $2 ))"
}
sum 1 2
sum 3 9
sum 6283 3141
As you can see, you can use the function `sum` multiple times, but you only need to define it once. This is useful in larger scripts, where a certain task has to be performed multiple times. Whenever you catch yourself writing the same or very similar code twice in the same script, you should consider using a function.
# Useful Commands
This chapter provides an overview of useful commands that you can use in your scripts. It is nowhere near complete, and serves only to provide a brief overview. If you want to know more about a specific command, you should read its manpage.
**grep**
`grep` can be used to search for a string within a file, or within the output of a command.
# searches logfile.txt for lines containing the word error
grep 'error' logfile.txt
# searches the directory 'folder' for files
# containing the word 'analysis'
grep 'analysis' folder/
# searches the output of 'xrandr' for lines that say 'connected'.
# only matches whole words, so 'disconnected' will not match.
xrandr | grep -w 'connected'
`grep` returns 0 if it finds something, and returns an error if it doesn't. This makes it useful for conditionals.
**sed**
`sed` can be used to edit text "on the fly". It uses its own scripting language to describe modifications to be made to the text, which makes it extremely powerful. Here, we provide examples for the most common usages of sed:
# replaces the first occurrence of 'find' in every line by 'replace'
sed 's/find/replace' inputfile
# replaces every occurrence of 'find' in every line by 'replace'
sed 's/find/replace/g' inputfile
# deletes the first occurrence of 'find' in every line
sed 's/find//' inputfile
# deletes every occurrence of 'find' in every line
sed 's/find//g' inputfile
# displays only the 12th line
sed '12q;d' inputfile
`sed` is often used in combination with pipes to format the output or get rid of unwanted characters.
**curl and wget**
`curl` and `wget` are two commands that can be used to access websites or other content from the web. The difference is that `wget` will simply download the content to a file, while `curl` will output it to the console.
curl http://www.thealternative.ch
wget http://files.project21.ch/LinuxDays-Public/16FS-install-guide.pdf
**xrandr**
`xrandr` can be used to manipulate your video outputs, i.e. enabling and disabling monitors or setting their screen resolution and orientation.
# list all available outputs and their status info
xrandr
# enables output HDMI-1
xrandr --output HDMI-1 --auto
# puts output HDMI-1 to the left of output LVDS-1
xrandr --output HDMI-1 --left-of LVDS-1
# disables output LVDS-1
xrandr --output LVDS-1 --off
**ImageMagick (convert)**
The `convert` command makes it possible to do image processing from the commandline.
# scale fullHDWallpaper.jpg to specified resolution
convert fullHDWallpaper.jpg -scale 3200x1800 evenBiggerWallpaper.jpg
# "automagically" adjusts the gamma level of somePicture.jpg
convert somePicture.jpg -auto-gamma someOtherPicture.jpg
# transform image to black and white
convert colorfulPicture.jpg -monochrome blackWhitePicture.jpg
It is extremely powerful and has lots of options. A good resource is the [official website](http://www.imagemagick.org). It also provides examples for most options.
**notify-send**
`notify-send` can be used to display a desktop notification with some custom text:
notify-send 'Battery warning' 'Your battery level is below 10%'
The first argument is the notification's title, the second is its description.
`notify-send` requires a *notification daemon* to be running, else it won't
work. Most desktop environments come with a notification daemon set up and
running. If you can't see your notifications, it might be that you don't have
such a daemon.
**find**
`find` can be used to find files in a directory structure.
# finds all files in the current directory and all subdirectories that end
# in .png
find -name '*.png'
# finds all files ending in .tmp and removes them. {} is replaced by the
# file's name when executing the command.
# Note that we don't use globbing here, but instead pass the * to find.
# find will then interpret the * as a wildcard.
find -name '*.tmp' -exec rm '{}'
# finds all files in the directory 'files' and prints their size and path
find 'files/' -printf '%s %p\n'
`find` has many options that allow you to perform arbitrary actions on the files it found or pretty-print the output.
**sort**
`sort` sorts lines of text files, or lines it reads from *stdin*.
sort listOfNames.txt # sorts all lines in listOfNames.txt alphabetically
**head and tail**
`head` and `tail` can be used to show the beginning or the end of a long stream of text, respectively.
# display the last few lines of the dmesg log
dmesg | tail
# display only the first few lines of a very long text file
head verylongtext.txt
**jshon**
`jshon` is a very simple command-line json parser. It can read data in json format and return specific values.
Its most important options are `-e` and `-a`. `-e` extracts the value of a given key from a json array or object:
# Find the object with key "title" from a json object stored in the file "json"
jshon -e 'title' < 'json'
The `-a` option maps all remaining options across the currently selected element. It has to be combined with other options, for example the `-e` option.
# Find the names of all elements stored in the json object in file "json"
jshon -a -e 'name' < 'json'
**shuf**
`shuf` randomly permutates the lines of its input, effectively *shuffling* them.
# Shuffle the lines of file "playlist"
shuf 'playlist'
# Get a random line from file "quotes"
shuf -n 1 'quotes'
**tee**
`tee` takes input from *stdin* and writes it to a file:
sudo zypper up | tee 'updatelog'
Note that this is equivalent to
sudo zypper up > 'updatelog'
`tee` is useful when you want to write to a file that requires root access. Then you can do the following:
echo 'some configuration' | sudo tee '/etc/systemconfig'
A normal redirection wouldn't work in this case, as that would open the file as a normal user instead of root.
**Please be careful when modifying system files.**
**sleep**
`sleep` pauses the execution of your script for a number of seconds. It basically takes a number as an argument, then does nothing for that number of seconds, and then returns. It can be useful if you want to make your script do something in a loop, but want to throttle the speed a little.
# prints "Get back to work" once per second.
# Note that 'true' is a command that always returns 0.
while true
do
echo 'Get back to work!'
sleep 1
done
**mplayer or mpv**
`mplayer` and `mpv` are media players that can be used from the console. `mpv` is based on `mplayer` and a lot more modern, but they can do essentially the same.
# play file "music.mp3"
mplayer 'music.mp3'
mpv 'music.mp3'
# play file "alarm.mp3" in an infinite loop
mplayer -loop 0 'alarm.mp3'
mplayer --loop=inf 'alarm.mp3'
# play a youtube video
# this only works with mpv and requires youtube-dl to be installed
mpv 'https://www.youtube.com/watch?v=lAIGb1lfpBw'
**xdotool**
`xdotool` can simulate keypresses. With this command, you can write macros that perform actions for you. It allows you to write scripts that interact with GUI programs.
# press ctrl+s (in order to save a document)
xdotool key 'Control+s'
# type "hello"
xdotool type 'hello'
No preview for this file type
\usepackage{xcolor}
\usepackage[margin=2cm]{geometry}
%various gray colors
\definecolor{slg}{gray}{0.25}
\definecolor{lg}{gray}{0.55}
\definecolor{vlg}{gray}{0.73}
\definecolor{tlg}{gray}{0.9}
%TheAlt colors
\definecolor{ldorange}{HTML}{F18A20}
\colorlet{ldbright}{ldorange!70!white} % tinted version of orange
\definecolor{ldblue}{HTML}{254471}
%%Theme colors
%\definecolor{thgreen}{HTML}{A1A92C}
%\definecolor{thmauve}{HTML}{97284D}
%Theme colors
\definecolor{thgreen}{HTML}{539727}
\definecolor{thmauve}{HTML}{572272}
\lstdefinestyle{custombash}{
belowcaptionskip=1\baselineskip,
captionpos=,
breaklines=true,
frame=L,
xleftmargin=\parindent,
language=bash,
morestring=[b]',
morekeywords=[2]{sudo,zypper,notify-send,feh,youtube-dl,sort,tee,head,tail,shuf,mpv,find,convert,xrandr,curl,wget,grep,xdotool,rm,cp,mv,touch,bash,chmod,mkdir,rsync,mplayer,mpv,xdotool,jshon},
showstringspaces=false,
basicstyle=\scriptsize\ttfamily,
rulecolor=\color{tlg},
backgroundcolor=\color{tlg},
fillcolor=\color{tlg},
rulesepcolor=\color{tlg},
commentstyle=\itshape\color{thmauve!60!black},
keywordstyle=\bfseries\color{thgreen},
identifierstyle=\color{ldblue},
stringstyle=\color{thmauve},
}
\lstset{
basicstyle=\ttfamily,
backgroundcolor=\color{black!10},
showspaces=false,
showstringspaces=false,
showtabs=false,
tabsize=2,
captionpos=b,
breaklines=true,
breakatwhitespace=true,
breakautoindent=true,
linewidth=\textwidth,
style=custombash
}
<div id="bashnav" class="left">
<div id="bashnav-scroll">
<div id="bashnav-wrap">
$toc$
</div>
</div>
</div>
<div id="bashguide">
<?prettify?>
$body$
</div>
#!/bin/bash
echo "Building pdf..."
pandoc -t beamer --template template.tex --listings pres.md -o pres-part.pdf --pdf-engine pdflatex \
&& pandoc -t beamer --template template.tex --listings firstslide.md -o firstslide.pdf --pdf-engine pdflatex \
&& pdfunite firstslide.pdf pres-part.pdf pres.pdf \
&& rm firstslide.pdf pres-part.pdf \
&& echo "Build successful"
# Welcome to the Bash Workshop!
* If you prefer to work on your own, already know programming or are confident in your abilities, please **sit in the back**.
<!-- -->
* If you prefer guided exercises, are completely new to programming, or want to have your hands held, please **sit in the front**.
\usepackage[T1]{fontenc} %pipes don't display properly without this
\usepackage[utf8]{inputenc}
\usepackage{listings}
\usepackage{color}
\usepackage{datapie}
\usepackage{multicol}
\usepackage{siunitx} %pretty measurement unit rendering
\usepackage{hyperref} %enable hyperlink for urls
\usepackage{caption} % needed to tweak caption size
\usefonttheme[onlymath]{serif}
\setcounter{MaxMatrixCols}{20}
\DeclareSIUnit\pixel{px}
\usecolortheme[RGB={37,68,113}]{structure}
\usetheme{Dresden}
\newenvironment{figurehere}
{\def\@captype{figure}}
{}
\makeatother
%commands to exclude sections from miniframes
\makeatletter
\let\beamer@writeslidentry@miniframeson=\beamer@writeslidentry
\def\beamer@writeslidentry@miniframesoff{%
\expandafter\beamer@ifempty\expandafter{\beamer@framestartpage}{}% does not happen normally
{%else
% removed \addtocontents commands
\clearpage\beamer@notesactions%
}
}
\newcommand*{\miniframeson}{\let\beamer@writeslidentry=\beamer@writeslidentry@miniframeson}
\newcommand*{\miniframesoff}{\let\beamer@writeslidentry=\beamer@writeslidentry@miniframesoff}
\beamer@compresstrue
\makeatother
%various gray colors
\definecolor{slg}{gray}{0.25}
\definecolor{lg}{gray}{0.55}
\definecolor{vlg}{gray}{0.73}
\definecolor{tlg}{gray}{0.9}
%TheAlt colors
\definecolor{ldorange}{HTML}{F18A20}
\colorlet{ldbright}{ldorange!70!white} % tinted version of orange, used in miniframes
\definecolor{ldblue}{HTML}{254471}
%reduce caption font size:
\captionsetup{font={scriptsize,color=lg}}
%do not prepend numbering/lettering to figures/subfigures
\captionsetup{labelformat=empty} %do not prepend letters to figure captions
%Apply TheAlt colors to theme
% section titles in top navigation bar
\setbeamercolor{section in head/foot}{parent=palette tertiary,fg=ldorange}
\setbeamertemplate{section in head/foot shaded}{\color{ldbright}\usebeamertemplate{section in head/foot}}
% miniframes (little navigation circles)
\setbeamercolor*{mini frame}{fg=ldorange,bg=ldbright}
\setbeamertemplate{mini frame in other section}[default][0]
\setbeamertemplate{mini frame in other subsection}[default][0]
% others
\setbeamercolor{author in head/foot}{fg=white}
\setbeamercolor{subsection in head/foot}{fg=white}
\setbeamercolor{caption name}{fg=vlg}
\setbeamercolor{caption}{fg=vlg}
\setbeamercolor{frametitle}{fg=ldblue}
\setbeamertemplate{caption}{\raggedright\insertcaption\par}
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{bibliography item}[text]
\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}
\lstdefinestyle{custombash}{
belowcaptionskip=1\baselineskip,
captionpos=,
breaklines=true,
frame=L,
xleftmargin=\parindent,
language=bash,
showstringspaces=false,
basicstyle=\scriptsize\ttfamily,
rulecolor=\color{tlg},
backgroundcolor=\color{tlg},
fillcolor=\color{tlg},
rulesepcolor=\color{tlg},
commentstyle=\itshape\color{purple!60!black},
keywordstyle=\bfseries\color{ldorange!80!black},
%keywordstyle=\bfseries\color{green!40!black},
identifierstyle=\color{blue},
stringstyle=\color{orange},
}
\lstset{language=Bash,style=custombash,caption={Descriptive Caption Text},label=DescriptiveLabel}
\title{Bash Workshop}
\author{Aline Abler}
\institute{\includegraphics[width=0.35\textwidth]{img/logo_blue.pdf}}
\renewcommand{\emph}[1]{\textcolor{ldorange}{#1}}
\newcommand{\soft}[1]{\textcolor{lg}{#1}}
\newcommand{\textt}[1]{\textcolor{blue}{\texttt{#1}}}
\newcommand{\bigtext}[1]{\centering\Huge \textbf{\textcolor{ldorange}{#1}}}
%shortcut to insert small logo in footline
\def\logo{%
\resizebox{!}{3ex}{\includegraphics{img/logo_white.pdf}}
}
% Define a custom footline that includes our logo
\setbeamertemplate{footline}
{%
\begin{beamercolorbox}[wd=\paperwidth,ht=2.5ex,dp=1.125ex,%
leftskip=.3cm,rightskip=.3cm plus1fil]{title in head/foot}
\usebeamerfont{title in head/foot}%
\insertshorttitle\hfill\insertframenumber
\end{beamercolorbox}
\begin{beamercolorbox}[wd=\paperwidth,ht=3.5ex,dp=1.625ex,%
leftskip=.3cm,rightskip=.3cm plus1fil]{author in head/foot}
\usebeamerfont{author in head/foot}
\raisebox{0.5ex}{\insertshortauthor}\hfill\raisebox{-0.5ex}{\logo}
\end{beamercolorbox}
}
File added
File added
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment