Skip to content

GitLab

C
courses
Commit e31ebba0 authored by Alexander Schoch's avatar Alexander Schoch
Browse files
Options

Merge branch 'master' of gitlab.ethz.ch:thealternative/courses 

merge
parents a9f88eac b7cd1042
Hide whitespace changes
Inline Side-by-side
Showing with 1057 additions and 5 deletions
.gitignore 0 → 100644
View file @ e31ebba0
# LaTeX traces
*.log
*.toc
*.aux
*.out
*-blx.bib
*run.xml
*.fdb_latexmk
*.fls
*.synctex.gz
*.snm
*.nav
# Bibtex traces
*.bak
*.blg
*.bbl
# PythonTeX traces and binaries
pythontex-*/
*.pytxcode
# Gedit
*~
.style/header.tex
View file @ e31ebba0
\documentclass[aspectratio=169]{beamer}
\usepackage[T1]{fontenc} %pipes don't display properly without this
\usepackage[T1]{fontenc} % the pipe character doesn't display properly without this
\usepackage[utf8]{inputenc}
\usepackage{listings}
\usepackage{color}
\usepackage{datapie}
\usepackage{minted} % COMPILE WITH `pdflatex -shell-escape`!
\usepackage{multicol}
\usepackage{upgreek}
\usepackage{siunitx} %pretty measurement unit rendering
\usepackage{hyperref} %enable hyperlink for urls
\usepackage{siunitx} % pretty measurement unit rendering
\usepackage{hyperref} % enable hyperlink for URLs
\usepackage{lmodern}
\usepackage{setspace}
\usepackage[os=win]{menukeys} % by default keys are given for MacOS
\usefonttheme[onlymath]{serif}
\setcounter{MaxMatrixCols}{20}
......@@ -26,9 +28,9 @@
%Define generic colors
\definecolor{slg}{gray}{0.25}
\definecolor{lg}{gray}{0.55}
\definecolor{lg}{gray}{0.53}
\definecolor{vlg}{gray}{0.73}
\definecolor{tlg}{gray}{0.9}
\definecolor{tlg}{gray}{0.91}
%Define data colors
\definecolor{mygreen}{rgb}{0,0.6,0}
......
git_gitlab_course/boilerplate-single.html 0 → 100644
View file @ e31ebba0
This source diff could not be displayed because it is too large. You can view the blob instead.
git_gitlab_course/outline.txt 0 → 100644
View file @ e31ebba0
git /gitlab tutorial planned for UZH zentrale informatik
target group:
bachelor informatik
master/phd students
both mostly want to use git with code or latex
targets:
version control (backup, recover old state, make changes visible)
publish (how, which license)
contribute (work together on same project)
structure:
presentation and then practical examples for each topic
for each topic add link/resource to more advanced explanations
motivation 10'/5':
demonstrate how git can recover files and make visible who changed what
explain service provider benefits (backup, collaboration)
ask what interest of participants are to focus advanced topics
git intro 5'/15':
install git
create first local repository
do first commit
show it worked with git status / git log
remote intro 10'/30':
create repository online
do steps of git global config on welcome screen
do steps of create new repository on welcome screen
edit Readme to learn about markdown
do add license on gitlab webpage
git GUI client 5'/15':
install github desktop
open repo & do commit
sync with push/pull
view changes of previous commits
advanced topics sorted by preference (each around 20'):
create branch & merge request
create issue & mention it in commit
fork & merge back into origin
add different remotes
use SSH key
resources considered:
compicampus course gives a good overview, but not suitable structured
git-scm.com has many advanced resources
git boot on git-scm.com has many advanced resources
cheatsheet github
"git for ages 4 and up" (well made youtube tutorial)
personal experience
\ No newline at end of file
git_gitlab_course/remark-latest.min.js 0 → 100644
View file @ e31ebba0
This source diff could not be displayed because it is too large. You can view the blob instead.
git_gitlab_course/template.html 0 → 100644
View file @ e31ebba0
<html>
<body>
<h1>Hallo Welt</h1>
<h2>Hallo Welt</h2>
</body>
</html>
\ No newline at end of file
scientific_software_management/bib.bib 0 → 100644
View file @ e31ebba0
@conference{repsep,
author = {Horea-Ioan Ioanas and Markus Rudin},
title = {Reproducible Self-Publishing for {P}ython-Based Research},
year = 2018,
month = aug,
publisher = {EuroSciPy},
url = {https://figshare.com/articles/Reproducible_Self-Publishing_for_Python-Based_Research/7247339},
doi = {10.6084/m9.figshare.7247339.v1},
}
@article{Ioanas2017,
author = {Horea-Ioan Ioanas and Bechara Saab and Markus Rudin},
title = {Gentoo Linux for Neuroscience - a replicable, flexible, scalable, rolling-release environment that provides direct access to development software},
volume = {3},
number = {},
year = {2017},
doi = {10.3897/rio.3.e12095},
publisher = {Pensoft Publishers},
abstract = {},
issn = {},
pages = {e12095},
URL = {https://doi.org/10.3897/rio.3.e12095},
eprint = {https://doi.org/10.3897/rio.3.e12095},
journal = {Research Ideas and Outcomes}
}
scientific_software_management/compile.sh 0 → 100755
View file @ e31ebba0
#!/usr/bin/env bash
pdflatex -shell-escape slides.tex
bibtex slides
pdflatex -shell-escape slides.tex
pdflatex -shell-escape slides.tex
scientific_software_management/get_samri_data.sh 0 → 100755
View file @ e31ebba0
#!/usr/bin/env bash
wget https://raw.githubusercontent.com/IBT-FMI/SAMRI/master/setup.py -O samri/setup.py
scientific_software_management/reset_system.sh 0 → 100644
View file @ e31ebba0
# reset the system
rm -rf /var/db/repos/chymeric/
emerge -C samri
eix-sync
# reset the overlay repo
cd overlay/location/with/Chymera/account
git reset --hard commitwhichwasbeforethepull
git push -f origin master
scientific_software_management/samri/metadata.xml 0 → 100644
View file @ e31ebba0
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">
<pkgmetadata>
<maintainer type="person">
<email>chr@chymera.eu</email>
<name>Horea Christian</name>
</maintainer>
<longdescription lang="en">
SAMRI (Small Animal Magnetic Resonance Imaging) — pronounced "Sam-rye" — provides
fMRI preprocessing, metadata parsing, and data analysis functions and workflows.
SAMRI integrates functionalities from a number of other packages to create
higher-level tools. The resulting interfaces aim to maximize reproducibility,
simplify batch processing, and minimize the number of function calls required to
generate figures and statistical summaries from the raw data.
</longdescription>
<upstream>
<remote-id type="github">IBT-FMI/SAMRI</remote-id>
</upstream>
</pkgmetadata>
scientific_software_management/samri/samri-0.4.ebuild 0 → 100644
View file @ e31ebba0
# Copyright 1999-2019 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
EAPI=7
PYTHON_COMPAT=( python{3_5,3_6} )
inherit distutils-r1
DESCRIPTION="Small Animal Magnetic Resonance Imaging"
HOMEPAGE="https://github.com/IBT-FMI/SAMRI"
SRC_URI="https://github.com/IBT-FMI/SAMRI/archive/${PV}.tar.gz -> ${P}.tar.gz"
LICENSE="GPL-3"
SLOT="0"
IUSE="test"
KEYWORDS="~amd64 ~x86"
DEPEND="
test? (
dev-python/pytest[${PYTHON_USEDEP}]
sci-biology/samri_bidsdata
sci-biology/samri_bindata
)
"
RDEPEND="
dev-python/argh[${PYTHON_USEDEP}]
dev-python/joblib[${PYTHON_USEDEP}]
>=dev-python/matplotlib-2.0.2[${PYTHON_USEDEP}]
>=dev-python/numpy-1.13.3[${PYTHON_USEDEP}]
dev-python/pandas[${PYTHON_USEDEP}]
dev-python/seaborn[${PYTHON_USEDEP}]
dev-python/statsmodels[${PYTHON_USEDEP}]
media-gfx/blender
>=sci-biology/fsl-5.0.9
sci-biology/bru2nii
sci-biology/mouse-brain-atlases
sci-libs/nibabel[${PYTHON_USEDEP}]
>=sci-libs/nipy-0.4.1[${PYTHON_USEDEP}]
>=sci-libs/nipype-0.14.0_pre20170830[${PYTHON_USEDEP}]
<=sci-libs/pybids-0.6.5[${PYTHON_USEDEP}]
sci-libs/scikits_image[${PYTHON_USEDEP}]
sci-libs/scipy[${PYTHON_USEDEP}]
sci-biology/ants
sci-biology/afni
sci-biology/nilearn[${PYTHON_USEDEP}]
"
S="${WORKDIR}/SAMRI-${PV}"
scientific_software_management/samri/setup.py 0 → 100644
View file @ e31ebba0
from setuptools import setup, find_packages
packages = find_packages(exclude=('samri.tests*', 'samri.*.tests*'))
setup(
name="SAMRI",
version="9999",
description = "Small animal magnetic resonance imaging via Python.",
author = "Horea Christian",
author_email = "chr@chymera.eu",
url = "https://github.com/IBT-FMI/SAMRI",
keywords = ["fMRI", "pipelines", "data analysis", "bruker"],
classifiers = [],
install_requires = [],
provides = ["samri"],
packages = packages,
include_package_data=True,
extras_require = {
},
entry_points = {'console_scripts' : \
['SAMRI = samri.cli:main']
},
)
scientific_software_management/slides.tex 0 → 100644
View file @ e31ebba0
\input{../.style/header}
\title{Software Management for Open Science}
\author{Horea Christian}
\institute{SSC TheAlternative | ETHZ and UZH}
\begin{document}
\begin{frame}
\titlepage
\end{frame}
\begin{frame}{These Slides}
Type one link, click all others:
\begin{itemize}
\item Download \textcolor{lg}{\href{https://thealternative.ch/ssm/slides.pdf}{\texttt{thealternative.ch/ssm/slides.pdf}}}
\end{itemize}
\end{frame}
\section{Requirements}
\subsection{... for the demo session}
\begin{frame}{SSH}
Linux and MacOS:
\begin{itemize}
\item Check that you can run:
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|ssh YOURUSER@130.60.24.66|
\end{itemize}
Windows:
\begin{itemize}
\item Download and launch “Git for Windows” from \textcolor{lg}{\href{https://git-for-windows.github.io}{\texttt{git-for-windows.github.io}}}.
\item Check that you can run:
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|ssh YOURUSER@130.60.24.66|
\end{itemize}
\end{frame}
\begin{frame}{Command Line Text Editor}
Usable via SSH and ubiquitous. There are many alternatives, but here we use \textcolor{lg}{\texttt{nano}}:
\begin{itemize}
\item Open file:
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|nano file|
\item Save via: \keys{\ctrl + o}, \keys{\enter}
\item Exit via: \keys{\ctrl + x}
\end{itemize}
\end{frame}
\begin{frame}{Git and Social Coding}
Git needs to know who you are.
\begin{itemize}
\item On the server, run:
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|git config --global user.name "Your Name"|
\vspace{-3.1em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|git config --global user.email yourname@example.com|
\end{itemize}
GitHub is a \textbf{social coding platform} providing free accounts:
\begin{itemize}
\item Register under \textcolor{lg}{\href{https://github.com}{\texttt{github.com}}}.
\item Use a password which you can remember.
\end{itemize}
\end{frame}
\section{What?}
\subsection{What is software management?}
\begin{frame}{The Package}
\begin{center}
\textcolor{ldorange}{\Large Better organization for your research!}
\end{center}
\vspace{1.5em}
A package is a software format which is (easily):
\begin{multicols}{2}
\begin{itemize}
\item Distributable
\item Integrated
\item Testable
\item Updateable
\item Uninstallable
\item Understandable
\end{itemize}
\end{multicols}
\end{frame}
\begin{frame}{Package Management --- best done automatically}
\begin{minipage}{0.44\textwidth}
Packages interact in complex and nontrivial manners:
\begin{itemize}
\item Version-dependent behaviour
\item Optional features
\item Incompatibilities
\item Static/dynamic linking
\end{itemize}
\end{minipage}
\begin{minipage}{0.55\textwidth}
\begin{figure}
\includegraphics[height=0.83\textheight]{img/ng_mi.png}
\vspace{-1em}
\caption{Minimal neuroscience package dependency stack \cite{Ioanas2017}}
\end{figure}
\end{minipage}
\end{frame}
\begin{frame}{Binary Packages}
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{img/pm_d.png}
\caption{Rudimentary overview of binary package distribution.}
\end{figure}
\begin{columns}
\column{.5\linewidth}
Advantages:
\begin{itemize}
\item Faster installation
\item Less variable installation
\end{itemize}
\column{.5\linewidth}
Disadvantages:
\begin{itemize}
\item No access to live software
\item Man-in-the middle
\item Limited support for rolling release
\end{itemize}
\end{columns}
\end{frame}
\begin{frame}{Source-Based Packages}
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{img/pm_g.png}
\caption{Rudimentary overview of source-based package distribution.}
\end{figure}
\begin{columns}
\column{.5\linewidth}
Advantages:
\begin{itemize}
\item Live software is a first-class citizen
\item Thin wrapper for upstream
\item Acutely version and linking aware
\end{itemize}
\column{.5\linewidth}
Disadvantages:
\begin{itemize}
\item Slower installation
\item More variable installation
\end{itemize}
\end{columns}
\end{frame}
\section{Why?}
\subsection{Why does open science require package management?}
\begin{frame}{Quality}
\begin{itemize}
\item Make development more transparent.
\item Get \textbf{constructive} feedback.
\item Ask for help with concrete reproducible examples.
\item Easily manage \textcolor{lg}{\href{https://github.com/gentoo-science/sci/issues}{\texttt{bugs/issues}}} and \textcolor{lg}{\href{https://github.com/gentoo-science/sci/pulls}{\texttt{conributions}}}.
\item Implement proper version tracking.
\end{itemize}
\end{frame}
\begin{frame}{Impact}
\begin{itemize}
\item Reach more potential users.
\item Communicate with users to improve your software's usability.
\item Retain more users.
\end{itemize}
\end{frame}
\begin{frame}{Recognition}
\begin{itemize}
\item Establish proof of authorship.
\item Publicize your innovative workflows, solutions, data structures.
\item Create a handle for attribution (including DOI), e.g:
\begin{itemize}
\item BehavioPy: \textcolor{lg}{\href{http://doi.org/10.5281/zenodo.188169}{\texttt{10.5281/zenodo.188169}}}
\item Nipype: \textcolor{lg}{\href{http://doi.org/10.5281/zenodo.50186}{\texttt{10.5281/zenodo.50186}}}
\end{itemize}
\end{itemize}
\end{frame}
\begin{frame}{Sustainability}
A sustainable project \textbf{cannot} depend on environments remaining unchanged.
\begin{itemize}
\item Ensure long-term viability of your software.
\item Avoid death-by-PhD.
\item Give your funders their money's worth.
\item Develop a lean start-up.
\item Maintain a reliable and affordable infrastructure for your work.
\end{itemize}
\end{frame}
\begin{frame}{Why Not?}
\begin{itemize}
\item Don't be afraid of your software not being “good/unique enough”!
\item Don't wait until your software is “ready”!
\item A lot of research software you are already using is not written by “professional” programmers.
\end{itemize}
\end{frame}
\section{How?}
\subsection{How do I package my software?}
\begin{frame}{Choose Appropriate Technologies}
\begin{columns}
\column{.5\linewidth}
\begin{figure}
\centering
\includegraphics[width=0.6\textwidth]{img/gentoo.png}
\caption{Gentoo Linux Logo by Gentoo Foundation and Lennart Andre Rolland - CC BY-SA/2.5.}
\end{figure}
\column{.5\linewidth}
\begin{figure}
\centering
\includegraphics[width=0.6\textwidth]{img/python.png}
\caption{Python Logo by Python Software Foundation.}
\end{figure}
\end{columns}
\end{frame}
\begin{frame}{Python Package Distribution}
\begin{columns}
\column{.55\linewidth}
You can package your python software by writing \textbf{one short} file.
\begin{itemize}
\item Python provides its own limited package management, e.g. via \textcolor{lg}{\href{https://packaging.python.org/distributing/}{\texttt{setuptools}}}.
\item Package metadata saved in \texttt{setup.py}, e.g. \textcolor{lg}{\href{https://github.com/IBT-FMI/SAMRI/blob/master/setup.py}{\texttt{SAMRI/setup.py}}}.
\end{itemize}
\column{.45\linewidth}
\inputminted[bgcolor=tlg,fontsize=\Tiny,tabsize=4]{python}{samri/setup.py}
\end{columns}
\end{frame}
\begin{frame}{Gentoo Packages}
\begin{columns}
\column{.55\linewidth}
A Gentoo package is \textbf{one short} file.
\begin{itemize}
\item Regardless of the programming language
\item Can automatically interpret information contained in the package, e.g. in \textcolor{lg}{\texttt{setup.py}}
\end{itemize}
\column{.47\linewidth}
\vspace{-3em}
\inputminted[bgcolor=tlg,fontsize=\Tiny,tabsize=4,firstline=1,lastline=36]{bash}{samri/samri-0.4.ebuild}
\vspace{-3.2em}
\inputminted[bgcolor=tlg,fontsize=\Tiny,tabsize=4,firstline=47,lastline=49]{bash}{samri/samri-0.4.ebuild}
\end{columns}
\end{frame}
\begin{frame}{Reposit Your Software}
\begin{figure}
\centering
\includegraphics[width=0.4\textwidth]{img/git.png}
\caption{Git Logo by Jason Long (\href{https://creativecommons.org/licenses/by/3.0/}{CC-BY-3.0})}
\end{figure}
You can self-host, but hosting is also available via social coding platforms:
\begin{multicols}{3}
\begin{itemize}
\item GitLab
\item GitHub
\item Bitbucket
\end{itemize}
\end{multicols}
\end{frame}
\section{Demo}
\subsection{Put what you have learned into practice, and start typing...}
\begin{frame}{A Few Basic Gentoo Commands}
\begin{itemize}
\item Check available package names, versions, and details.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|eix -v nibabel|
\item See package dependencies.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|equery g nibabel|
\item See what packages depend on a said package.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|equery d nibabel|
\item See files installed by package.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|equery f nibabel|
\item Try to install a new package.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|emerge -p psychopy|
\end{itemize}
\end{frame}
\begin{frame}{Reproduce a Scientific Article}
Novel frameworks, such as RepSeP \cite{repsep} permit articles to be written as software.
\begin{itemize}
\item Get the source code for brand-new articles:
\begin{itemize}
\item Work-in-progress (reexecution time \SI{\approx 2}{\minute})
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|git clone https://gitlab.com/Chymera/nvcz.git |
\item Preprint (reexecution time \SI{\approx 11}{\minute})
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|git clone https://bitbucket.org/TheChymera/irsabi.git |
\end{itemize}
\item Switch to article directory.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|cd nvcz|
\item Attempt to reexecute.
\vspace{-0.6em}
\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|./compile.sh|
%\mint[bgcolor=tlg,fontsize=\footnotesize]{bash}|git chechout 28b5d2d1|
\end{itemize}
\end{frame}
\begin{frame}{What happened? Dependency requirements happened.}
\begin{center}
\textcolor{ldorange}{\Large But you can solve the issue yourself!}
\end{center}
\vspace{2em}
Write a new package atom for the package manager.
\begin{itemize}
\item Gentoo Linux makes this wholly autonomous.
\item Solve one problem only once:
\begin{itemize}
\item Installation will be automatic on all your further systems.
\item And on everybody else's systems!
\end{itemize}
\end{itemize}
\end{frame}