README.md

# mossutils

A collection for useful scripts for working with Moss in the context of ETH (eDoz) and Code Expert.

## Installation

```shell
$ pip install --upgrade git+https://gitlab.ethz.ch/scmalte/mossutils.git
...
Successfully installed [...] mossutils [...]
```

Argument `--upgrade` (re)installs the package, even if the version number hasn't changed. Note: if "parser error, need lxml" appears, ensure to upgrade via `pip install --upgrade lxml` command.

## Usage

**TODO:** Add instructions for how to obtain and prepare the involved data. See also `preprocessing/README.md`.

**NOTE:** Create a directory per exercise to check, and keep the submissions in a subdirectory thereof. E.g. `~/moss_ex1/ex1` (see also step 2 further down).

1. Obtain and prepare data, see `preprocessing/README.md`.

   **NOTE:** Files `details.json` (see `preprocessing/README.md`) are currently not used.

1. If you want to include additional submissions, e.g. a master solution, create a separate user directory per additional submission, and place the submission code there. E.g. create `~/moss_ex1/ex1/00-000-001/main.cpp`, where user `00-000-001` does not exist otherwise. Corresponding `details.json` files are currently not needed, since not used anyway.

1. Open a shell one level above the directory that contains the user submissions, i.e. the `<user>` directories from `preprocessing/README.md`. E.g. if you have `~/moss_ex1/ex1/<user>`, then `cd ~/moss_ex1`. Afterwards, run `mu-init` to create a directory (`_static`) with files necessary for the next steps (e.g. for the final HTML report).

1. Run `mu-moss --help` for options that can/must be configured. Afterwards, run `mu-moss` as desired, e.g. `mu-moss -u 1234 -n 300 "./ex1/*/main.cpp"`.

   `mu-moss` connects to the Moss service, uploads submissions and downloads the generated report. This may take a while, and will probably not work for large `-n` values (3000 worked for me, 10,000 didn't).

1. Run `mu-revise`.

   **NOTE:** `mu-revise` is not yet configurable and expects `mu-moss` to have used its default file and directory names!

   `mu-revise` improves the previously downloaded HTML report, e.g. by making the table sortable, and generates a CSV file with the report data.

1. Run `mu-cluster`.

   **NOTE:** `mu-cluster` is not yet *fully* configurable and expects `mu-revise` to have used its default file and directory names! Run `mu-cluster --help` for options that can already be configured.

   `mu-cluster` computes clusters of plagiarism, creates as DOT and SVG files (by running GraphViz, which *must be in the path*), and creates a summarising CSV file.

1. Run `mu-aggr`.

   **NOTE:** `mu-aggr` is not yet *fully* configurable and expects `mu-cluster` to have used its default file and directory names! Run `mu-aggr --help` for options that must/can already be configured.

   `mu-aggr` combines the cluster data with exports from eDoz and Code Expert, and generates a cluster report as an HTML file.

   * Code Expert export: *My Courses* → *Students (Student Overview)* → *Export to CSV*. Pass file to `mu-aggr` via the mandatory `--code-expert-export` option.

   * eDoz exports: *Communication/List* → *Course units* → *<Your course>* → *Export data (Zip/Text)*, then extract CSV (`.txt`) file from downloaded ZIP file. Pass files to `mu-aggr` via the mandatory `--edoz-exports` option.

1. Optional: run `mu-mails`.

   **NOTE:** `mu-mails` is not yet *fully* configurable and expects `mu-aggr` to have used its default file and directory names! Run `mu-mails --help` for options that must/can already be configured.

   `mu-mails` creates an e-mail per identified cluster, from the template file `_static/cluster_mail.txt` (e.g. `~/moss_ex1/_static/cluster_mail.txt`). The e-mails are written to an Mbox file, as used by Thunderbird. If you pass your local Thunderbird outbox' unsent messages file to `mu-mails` via the mandatory `--mbox-file` option (e.g. `~/thunderbird/xxxxxxxx.profile/Mail/Local Folders/Unsent Messages`), then you should afterwards be able to directly send the mails from inside Thunderbird via *File* → *Send Unsent Messages*. You can also review and edit the messages beforehand, they should be visible in the *Outbox* folder of your *Local Folders* account.

## Python Package Tutorials

I've used the following tutorials to get started with developing a Python package:

* https://python-packaging.readthedocs.io/en/latest/minimal.html

* https://dev.to/rf_schubert/how-to-create-a-pip-package-and-host-on-private-github-repo-58pa