diff --git a/README.md b/README.md
index fa5f93ccd0668825058f7db285ca28520c07d6ab..813fa1c3f171d9e57c8cd4b964b9ca6393a1f36a 100644
--- a/README.md
+++ b/README.md
@@ -3,13 +3,26 @@
[](https://secure.travis-ci.org/amiv-eth/amivapi)
[](https://coveralls.io/r/amiv-eth/amivapi)
-AMIV API is a Python-EVE based REST interface to manage members, events, mail forwards, job offers and study documents for a student organisation. It was created by AMIV an der ETH to restructure the existing IT infrastructure. If you are not from AMIV and think this is useful feel free to fork and modify.
+AMIV API is a [Python-EVE]((http://docs.python-eve.org)) based REST interface
+to manage members, events, mail forwards, job offers and study documents for a
+student organisation. It was created by [AMIV an der ETH](http://amiv.ethz.ch) to restructure the
+existing IT infrastructure. If you are not from AMIV and think this is useful,
+feel free to fork and modify.
-[Request Cheatsheet (Filtering etc.)](docs/Cheatsheet.md)
+If you only want to use AMIV API, check out the online documentation
+(There's a link in the github description above).
-[Python EVE Documentation](http://python-eve.org/features.html)
+If you are an administrator and wish to get the AMIV API running, keep reading!
-[How to use central OAuth2 login](docs/OAuth.md)
+If you are a developer looking to work on AMIV API, it's best to look at the
+code directly. You can start with [bootstrap.py](amivapi/bootstrap.py),
+where all modules are assembled into a single app object. From there on,
+check out the different sub-modules. Each resource is defined in a dedicated
+sub-directory, some smaller functionality are defined in a single file.
+
+You do not need to install and configure AMIV API to test it. You can skip
+the installation and configuration sections and head right to the bottom
+of the README, where testing is explained.
## Installation & Prerequisites
@@ -26,7 +39,7 @@ You only need to install Docker, nothing else is required.
For development, we recommend to clone the repository and install AMIV API
manually.
-First of all, we advice using a [virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/).
+First of all, we advise using a [virtual environment](https://docs.python.org/3/tutorial/venv.html).
If your virtual environment is ready, clone and install AMIV API:
@@ -34,6 +47,9 @@ If your virtual environment is ready, clone and install AMIV API:
git clone https://github.com/amiv-eth/amivapi.git
cd amivapi
pip install -r requirements.txt
+
+# IMPORTANT: Install amivapi in `editable mode` to get the `amivapi` command
+pip install -e .
```
This will also install the `amivapi` command, which you can use to start
@@ -56,7 +72,7 @@ The following command runs a MongoDB service available on the default port
password `amivapi`.
```sh
-# Create a network s.t. the api service can later be connected to the db
+# Create a network so that the api service can later be connected to the db
docker network create --driver overlay backend
docker service create \
--name mongodb -p 27017:27017 --network backend\
@@ -67,7 +83,7 @@ docker service create \
```
If you have a local MongoDB running, the following command might be useful
-to set up database and user quickly:
+to set up the database and user quickly:
```sh
mongo amivapi --eval \
@@ -78,7 +94,7 @@ mongo amivapi --eval \
### Configuration File
-Now it's time to configure AMIVAPI. Create a file `config.py`
+Now it's time to configure AMIV API. Create a file `config.py`
(you can choose any other name as well) with the following content:
```python
@@ -126,7 +142,7 @@ the default settings, so any value defined in
### Run using Docker
-Configuration files can be used for services easily using
+Configuration files can be used easily for services using
[Docker configs](https://docs.docker.com/engine/swarm/configs/):
```sh
@@ -180,8 +196,8 @@ amivapi run --help
## For Developers: Running The Tests
-First, create a test user `test_user` with password `test_pw` in the `test_amviapi` database, which will be used for all tests. You only need
-to do this once to prepare the database.
+First, create a test user `test_user` with password `test_pw` in the `test_amviapi` database, which will be used for all tests.
+You only need to do this once to prepare the database.
```sh
mongo test_amivapi --eval \
@@ -236,9 +252,8 @@ Set the following environment variables:
The test will return the imported user data, be sure to verify it
- `LDAP_TEST_USER_PASSWORD` (required to test user login)
-Additionally, you need to be inside the ETH network, e.g. using a VPN, otherwise the ETH LDAP server can't be reached. Furthermore be patient,
-as the LDAP tests take a little time to complete.
-
+Additionally, you need to be inside the ETH network, e.g. using a VPN, otherwise the ETH LDAP server can't be reached.
+Furthermore be patient, as the LDAP tests take a little time to complete.
#### Sentry
@@ -248,6 +263,6 @@ The test will use the `testing` environment.
## Problems or Questions?
-For any comments, bugs, feature requests please use the issue tracker, don't hasitate to create issues, if we don't like your idea we are not offended.
+For any comments, bugs, feature requests: please use the issue tracker and don't hasitate to create issues. If we don't like your idea, we will not feel offended.
-If you need help deploying the API or creating a client, feel free to message us at api@amiv.ethz.ch
+If you need help deploying the API or creating a client, feel free to message us at [api@amiv.ethz.ch](mailto:api@amiv.ethz.ch) .
diff --git a/amivapi/documentation.py b/amivapi/documentation.py
deleted file mode 100644
index 33af54195dbb45429f39008b4786e0063bc4e26a..0000000000000000000000000000000000000000
--- a/amivapi/documentation.py
+++ /dev/null
@@ -1,95 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# license: AGPLv3, see LICENSE for details. In addition we strongly encourage
-# you to buy us beer if we meet and you like the software.
-
-"""Eve Swagger initialization."""
-
-from eve_swagger import add_documentation, swagger
-from flask import Blueprint, render_template_string, request
-
-from amivapi.utils import register_validator
-
-redoc = Blueprint('redoc', __name__, static_url_path='/docs')
-
-
-@redoc.route('/docs')
-def index():
- """Output simple html that includes ReDoc's JS and sets style"""
- redoc_template = (
- '<!DOCTYPE html>'
- '<html><head>'
- '<title>ReDoc</title>'
- '<!-- needed for adaptive design -->'
- '<meta name="viewport" content="width=device-width, initial-scale=1">'
- "<!-- ReDoc doesn't change outer page styles -->"
- '<style> body {margin: 0; padding: 0; } </style>'
- '</head>'
- '<body>'
- "<redoc spec-url='{{ spec_url }}'></redoc>"
- '<script src="https://rebilly.github.io/ReDoc/releases/latest/'
- 'redoc.min.js"> </script>'
- '</body></html>')
- spec_url = request.url.rstrip('/') + '/api-docs'
- return render_template_string(redoc_template, spec_url=spec_url)
-
-
-class DocValidator(object):
- """Add a schema rule for a 'descriptions field'.
-
- This rule will do nothing, but will stop Cerberus from complaining without
- allowing all unknown fields.
- """
-
- def _validate_description(*args):
- """Do nothing.
-
- The rule's arguments are validated against this schema:
- {'type': 'string'}
- """
-
-
-def init_app(app):
- """Create a ReDoc endpoint at /docs."""
- # Generate documentation (i.e. swagger/OpenApi) to be used by any UI
- # will be exposed at /docs/api-docs
- app.register_blueprint(swagger, url_prefix="/docs")
- # host the ui (we use redoc) at /docs
- app.register_blueprint(redoc)
-
- register_validator(app, DocValidator)
-
- # replace null-type fields with ''
- # TODO: make this dynamic
- add_documentation({'definitions': {'User': {'properties': {
- 'password': {'default': ''},
- 'nethz': {'default': ''}
- }}}})
-
- add_documentation({'securityDefinitions': {
- 'AMIVauth': {
- 'type': 'apiKey',
- 'name': 'Authorization',
- 'in': 'header',
- 'description': 'Enter a session token you got with POST to '
- '/sessions, or an API key, stored in the server config'
- }
- }})
-
- # just an example of how to include code samples
- add_documentation({'paths': {'/sessions': {'post': {
- 'x-code-samples': [
- {
- 'lang': 'python',
- 'source': '\n\n'.join([
- 'import requests',
- ('login = requests.post("http://api.amiv.ethz.ch/sessions",'
- ' data={"user": "myuser", "password": "mypassword"})'),
- "token = login.json()['token']",
- ('# now use this token to authenticate a request\n'
- 'response = requests.get('
- '"https://api.amiv.ethz.ch/users/myuser", '
- 'auth=requests.auth.HTTPBasicAuth(token, ""))')])
- }
- ]
- }}}})
diff --git a/amivapi/documentation/Auth.md b/amivapi/documentation/Auth.md
new file mode 100644
index 0000000000000000000000000000000000000000..f2bac0a0a6d1c54c1937eb2ab4893ac081e0dcdd
--- /dev/null
+++ b/amivapi/documentation/Auth.md
@@ -0,0 +1,83 @@
+# Authentication and Authorization
+
+Most resources are not public and require you to *authenticate* yourself
+using an `Authorization` header. If the header is provided, the API will
+verify that you are *authorized* to execute the requested operation.
+
+## Authentication
+
+First of all, you must aquire a token which is used by the API to identify
+which [user](#tag/User) you are.
+Providing the token is possible in two ways:
+
+1. The API supports a high-level interface based on
+ [OAuth 2 Implicit Grants](https://oauth.net/2/grant-types/implicit/),
+ intended for web applications authorizing over the API.
+
+ > Take a look at the API [OAuth Guide below](#section/OAuth) to learn how
+ > to use it.
+
+2. Aside from that, the `/sessions` resource provides a low-level interface
+ to all active user sessions for scripting and management.
+
+ > Take a look at the [Sessions resource below](#tag/Session) for more info.
+
+
+After you have acquired a token, you must send it with all your requests
+in the `Authorization` header. The API supports multiple ways of
+providing a token:
+
+```
+Authorization: <token>
+
+Authorization: <keyword> <token>
+```
+
+Where `<keyword>` can be anything, e.g. `Token`, `Bearer`, etc.
+
+Furthermore, you can also use
+[HTTP basic auth](https://en.wikipedia.org/wiki/Basic_access_authentication)
+with the token as username and an empty password.
+
+## Authorization
+
+The API handles permissions on a *per resource* basis.
+There are two fundamental permissions:
+
+| Permission | Allowed Methods |
+|-------------|---------------------------------------------|
+| `read` | `GET`, `OPTIONS` |
+| `readwrite` | `GET`, `OPTIONS`, `POST`, `PATCH`, `DELETE` |
+
+> **Not all resources support every method**
+>
+> Have a look at the respective resource sections for detailed information
+
+### Permissions are granted by groups
+
+Groups can grant permissions to all their members. Concretely,
+groups contain a `permissions` field, which maps resources to
+the permissions the group grants (if any).
+
+> Have a look at the [group section](#tag/Group) for more information.
+
+After you have been authenticated, the API checks the permissions
+of all the groups you are a member of, to determine whether you are
+*authorized*.
+
+### Admins and Users
+
+If one of your groups grants you permissions for a resource and method, you
+are considered an **admin** for this resource and method.
+
+If you are athenticated, but not an admin, you are considered a **user**.
+
+### Item and field level permissions
+
+While the API in general handles permissions per resource, some resources
+implement further restrictions on a *per item* or *per field* basis, e.g. you
+can change only your own password, not that of other users; or you can
+change the details of an event signup, but you may not replace your user with
+someone else.
+
+> Have a look at the respective resource sections for detailed information
diff --git a/amivapi/documentation/Cheatsheet.md b/amivapi/documentation/Cheatsheet.md
new file mode 100644
index 0000000000000000000000000000000000000000..c7a07f0e992d140e51a2b8351e8bfd0900fc2324
--- /dev/null
+++ b/amivapi/documentation/Cheatsheet.md
@@ -0,0 +1,145 @@
+# Cheatsheet
+
+This cheatsheet show you how you can control the data fetched from the API
+using `GET` requests with filtering, sorting, etc. and how to send different
+kinds of data to the API, e.g. arrays, objects and files.
+
+## Fetching Data
+
+The API is based on the framework [Eve](http://docs.python-eve.org/en/latest/),
+which supports a wide range of filtering, sorting, embedding and pagination
+using the request [query string](https://en.wikipedia.org/wiki/Query_string).
+Below is a short overview over the most important operations. Check out
+the [Eve documentation](http://docs.python-eve.org/en/latest/features.html) for
+further details.
+
+In any case, data is returned in [JSON](https://www.json.org) format,
+which is automatically parsed by JavaScript and supported by virtually all
+other programming languages.
+
+### Filtering
+
+```
+/events?where={"title_en":"Party"}
+
+/events?where={"$time_start":{"$gt":"2018-06-06T10:00:00Z"}}
+
+/events?where={"$or":{"title_en":"Party","title_de":"Feier"}}
+
+/events?where={"img_infoscreen":{"$exists":true}}
+```
+
+Supported operators include:
+`$gt` (>), `$gte` (>=), `$lt` (<), `$lte` (<=), `$ne` (!=), `$and`, `$or`,
+`$in`, `$exists`
+
+> **The time format is `YYYY-MM-DDThh:mm:ssZ`**
+>
+> **Y**ear, **M**onth, **D**ay, **h**our, **m**inutes, **s**econds, `T` is a
+> required separator and `Z` indicates UTC (other timezones are not supported)
+
+
+### Sorting
+
+```
+/events?sort=time_start,-time_end
+```
+
+The prefix `-` inverts the sorting order.
+
+
+### Pagination
+
+There is a global maximum on the results per request, so getting all results in
+one request may not be possible. Each portion of results is called a *page*,
+and you can specify the number of results per page and page number:
+
+```
+/events?max_results=20&page=2
+```
+
+
+### Embedding
+
+By default, other resources are only included by `_id`. With *embedding*, you
+can include complete objects.
+
+```
+/groupmemberships?embedded={"user":1,"group":1}
+```
+
+### Projections
+
+Hide fields or show hidden fields. Some fields like passwords can't be shown.
+
+```
+/groupmemberships?projection={"group":0}
+```
+
+## Sending Data
+
+Data can be sent in JSON format or using
+[multipart/form-data](https://tools.ietf.org/html/rfc2388).
+
+### JSON
+
+Primarily, data is sent to the API in JSON format. This allows sending
+arrays and objects in fields without problems.
+
+Concretely, a request is sent with the header
+`Content-Type: application/json` and a JSON string as body.
+
+In most languages, sending JSON is very easy, e.g. in Javascript the
+[axios library](https://github.com/axios/axios) sends JSON by default and
+in python, the [requests package](http://docs.python-requests.org) can
+send dictionaries as JSON directly, too.
+Other languages may have similar libraries available.
+
+However, there is one caveat: With JSON, it is not possible to send files.
+For this reason, the API also accepts forms, in particular the
+*multipart/form-data format*, wich can be used for sending files.
+
+
+### multipart/form-data
+
+Using multipart/form-data, sending arrays and objects is also possible.
+When files are not required, sending JSON data might be preferrable though,
+as it is a bit simpler.
+
+- **File** sending is implemented a bit differently by each library, so it's
+ best to google quickly how to do it.
+
+ - [Javascript Example](https://stackoverflow.com/a/9397172)
+ - [Python Example](http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file)
+
+- **Arrays** can be sent by simply including the same field multiple times in
+ the request form, e.g. the API transforms the request form
+
+ ```
+ field1: value1
+ field1: value2
+ ```
+
+ into the array
+
+ ```
+ field1: [value1, value2]
+ ```
+
+
+- **Objects** can be sent using *dot notation*, e.g. the API transforms the
+ request form
+
+ ```
+ field1.subfield1: value1
+ field1.subfield2: value2
+ ```
+
+ into the object
+
+ ```
+ field1: {
+ subfield1: value1,
+ subfield2: value2
+ }
+ ```
diff --git a/amivapi/documentation/Introduction.md b/amivapi/documentation/Introduction.md
new file mode 100644
index 0000000000000000000000000000000000000000..b41b8e9365cb988ea77cad1902f220fa7c018184
--- /dev/null
+++ b/amivapi/documentation/Introduction.md
@@ -0,0 +1,46 @@
+
+# REST Introduction
+
+A [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) API
+is a certain type of web service, following a set of design principles for
+the structure of, and interaction with data.
+
+
+## Data structure
+
+In the API, data is grouped by resources, such as *Users*, which consist of
+items, e.g. a *User*, which are identified by unique ids.
+Every resource and item is reachable by an endpoint,
+e.g. `/users` for *Users* and `/users/<id>` for a *User* with a given id.
+
+
+## Interaction with data
+
+Interacting with data follows the *Create, Read, Update, Delete*
+([CRUD](https://en.wikipedia.org/wiki/Representational_state_transfer))
+principle, where each operation corresponds to
+[HTTP methods](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods).
+
+Concretely, this API supports the following methods:
+
+| Operation | Method | Endpoint |Description |
+|------------|-----------------|--|--|
+| **Create** | `POST` | Resource |Create a new item for a resource. |
+| **Read** | `GET`,`OPTIONS` | Resource, Item | Read all items of a resource, or a specific item. `OPTIONS` works like `GET`, but only returns headers. <blockquote>Additional filtering is supported, have a look at the [Cheatsheet](#section/Cheatsheet) for more info.</blockquote> |
+| **Update** | `PATCH` | Item | Modify an item.|
+| **Delete** | `DELETE` | Item | Delete an item.|
+
+
+## Security
+
+Most resources are not public, but require authentication & authorization.
+The API uses tokens for both, which need to be provided with the
+`Authorization` header:
+
+```
+Authorization: <token>
+```
+
+How to aquire a token and how permissions work is explained in the
+[Authentication and Authorization](#section/Authentication-and-Authorization)
+section below.
diff --git a/amivapi/documentation/OAuth.md b/amivapi/documentation/OAuth.md
new file mode 100644
index 0000000000000000000000000000000000000000..6a5f50fc9751e0323146a5f85efd4194577a8f21
--- /dev/null
+++ b/amivapi/documentation/OAuth.md
@@ -0,0 +1,59 @@
+# OAuth
+
+The API implements the
+[OAuth 2.0 Implicit Grant Flow](https://oauth.net/2/grant-types/implicit/)
+to allow web-services to use the API for authorization.
+
+This means that the API provides a central login page that other services
+can use instead of implementing their own.
+
+The process works like this:
+
+1. Redirect the user from your web service to the `/oauth` endpoint
+2. (The user logs in with the API)
+3. The user is redirected back to your web-service with an API token
+
+```
+ 1. REDIRECT to API
++-------------+ +-----------+
+| +----------------------------> |
+| web-service | | API OAuth | 2. Login
+| <----------------------------+ |
++-------------+ +-----------+
+ 3. REDIRECT to service
+```
+
+For increased security, web-services must be whitelisted to use OAuth.
+
+
+## Using OAuth
+
+To use the central login, you must follow the OAuth 2.0 implicit grant flow. This basically means the following:
+
+1. To get a user token you redirect the client's browser to the central login page, `/oauth` of the AMIV API. You need to provide the following query parameters with the redirect:
+
+ - ```response_type```: Must be set to ```token```.
+ - ```client_id```: Name of your application (displayed to the user)
+ - ```redirect_uri```: URL to return the user to after successful login.
+ - ```state```: A random token you generated. Used to prevent CSRF attacks.
+
+2. On the OAuth page, the user will be prompted to log in.
+
+3. After successful login, the client will be redirected to the URL you provided in the ```redirect_uri``` parameter with the following additional query parameters:
+
+ - ```access_token```: Your new login token. Provide it in the Authorization header of requests to the AMIV API.
+ - ```token_type```: Will always be ```bearer```. You can ignore it.
+ - ```scope```: Will always be ```amiv```. You can ignore it.
+ - ```state```: The CSRF token you sent in the authorization request. You must check that it is still equal to your provided value to prevent CSRF attacks.
+
+As OAuth is an open standard, it is very likely that there are libraries to automate this workflow in your preferred programming language. These libraries might expect more parameters, which we don't use. If you need to enter a token refresh URL or a ```client_secret```, make sure you are using the *implicit grant* authorization type. By the standard, an authorization request can also supply a requested ```scope```. AMIVAPI will ignore this parameter if you supply it.
+
+The ```redirect_uri``` must be an **HTTPS** URL and must not contain a fragment
+(nothing after ```#```).
+
+## Whitelisting of OAuth clients
+
+To prevent phishing attacks we use a whitelist of ```client_id``` and
+```redirect_uri```. To register a client with the API, a request to the
+```/oauthclients``` endpoint can be issued. Check out the
+[resource documentation below](#tag/Oauthclient) for more info.
diff --git a/amivapi/documentation/__init__.py b/amivapi/documentation/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..ef073b91f710eb7cc3a08e5946cd880aa4842e85
--- /dev/null
+++ b/amivapi/documentation/__init__.py
@@ -0,0 +1,62 @@
+# -*- coding: utf-8 -*-
+#
+# license: AGPLv3, see LICENSE for details. In addition we strongly encourage
+# you to buy us beer if we meet and you like the software.
+
+"""Online documentation initialization.
+
+We use ReDoc to display an OpenAPI documentation.
+The documenation is produced by Eve-Swagger, which we extend with details.
+"""
+from flask import Blueprint, render_template_string, request, current_app
+from eve_swagger import swagger
+
+
+from .update_documentation import update_documentation
+
+
+redoc = Blueprint('redoc', __name__, static_url_path='/docs')
+
+
+doc_template = ("""
+<!DOCTYPE html>
+<html>
+ <head>
+ <title>{{ title }}</title>
+
+ <!-- Responive Sizing -->
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <!-- Normalize Page Style -->
+ <style>body { margin: 0; padding: 0; }</style>
+ </head>
+
+ <body>
+ <redoc spec-url={{ spec_url }}></redoc>
+ <script src=
+ "https://cdn.jsdelivr.net/npm/redoc@next/bundles/redoc.standalone.js">
+ </script>
+ </body>
+</html>
+""")
+
+
+@redoc.route('/docs')
+def index():
+ """Output simple html that includes ReDoc's JS and sets styles."""
+ spec_url = request.url.rstrip('/') + '/api-docs'
+ title = current_app.config['SWAGGER_INFO']['title']
+ return render_template_string(doc_template,
+ spec_url=spec_url,
+ title=title)
+
+
+def init_app(app):
+ """Create a ReDoc endpoint at /docs."""
+ # Generate documentation (i.e. swagger/OpenApi) to be used by any UI
+ # will be exposed at /docs/api-docs
+ app.register_blueprint(swagger, url_prefix="/docs")
+ # host the ui (we use redoc) at /docs
+ app.register_blueprint(redoc)
+
+ update_documentation(app)
diff --git a/amivapi/documentation/update_documentation.py b/amivapi/documentation/update_documentation.py
new file mode 100644
index 0000000000000000000000000000000000000000..7acc24b9eade21c9d889c4988a5e8c4b90db49bd
--- /dev/null
+++ b/amivapi/documentation/update_documentation.py
@@ -0,0 +1,301 @@
+# -*- coding: utf-8 -*-
+#
+# license: AGPLv3, see LICENSE for details. In addition we strongly encourage
+# you to buy us beer if we meet and you like the software.
+"""Add additional documentation to the Eve-Swagger output.
+
+Eve-Swagger provides a useful baseline, but is missing many details.
+
+Furthermore, as of this commit, development seems slow with multiple open pull
+requests without responses. Thus we have decided to update the documentation
+manually. Should Eve-Swagger development gain traction again, we could try to
+integrate these changes.
+"""
+from os import path
+
+from eve_swagger import add_documentation
+
+
+def update_documentation(app):
+ """Update the API documentation provided by Eve-Swagger.
+
+ 1. Update top-level descriptions etc.
+ 2. Add definitions for responses and parameters
+ 3. Update each resource, i.e. lookups, methods, and properties
+ """
+ _update_top_level(app)
+ _update_definitions(app)
+
+ for resource, domain in app.config['DOMAIN'].items():
+ _update_properties(domain)
+ _update_paths(resource, domain)
+ _update_methods(resource, domain)
+
+
+def _update_top_level(app):
+ """Update top-level descriptions."""
+ # Extend documentation description
+ # Markdown files that will be included in the API documentation
+ doc_files = ['Introduction.md', 'Cheatsheet.md', 'Auth.md', 'OAuth.md']
+ dirname = path.dirname(path.realpath(__file__))
+ doc_paths = [path.join(dirname, filename) for filename in doc_files]
+
+ additional_docs = []
+ for filename in doc_paths:
+ with open(filename) as file:
+ additional_docs.append(file.read().strip())
+
+ # Join parts with double newlines (empty line) for markdown formatting
+ docs = app.config['SWAGGER_INFO']
+ docs['description'] = "\n\n".join((docs['description'].strip(),
+ *additional_docs))
+
+ # Add logo
+ add_documentation({'info': {'x-logo': app.config['SWAGGER_LOGO']}})
+
+ # Add servers
+ add_documentation({'servers': app.config['SWAGGER_SERVERS']})
+
+ # Required to tell online docs that we don't return xml
+ app.config['XML'] = False
+
+
+def _update_definitions(app):
+ """Update the definitions in the docs.
+
+ In particular, extend the `parameters` section with definitions
+ for the various query parameters, e.g. filter and embed.
+
+ Furthermore, add definitions for the error responses to the `definitions`
+ section.
+ """
+ add_documentation({
+ # Query parameters
+ 'parameters': {
+ 'auth': {
+ "in": "header",
+ "name": "Authorization",
+ "description": "API token.<br />(read more in "
+ "[Authentication and Authorization]"
+ "(#section/Authentication-and-Authorization))",
+ "required": True,
+ "type": "string"
+ },
+ 'filter': {
+ "in": "query",
+ "name": "filter",
+ "type": "object",
+ "description": "Apply a filter."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ 'max_results': {
+ "in": "query",
+ "name": "max_results",
+ "type": "integer",
+ "minimum": 0,
+ "maximum": app.config['PAGINATION_LIMIT'],
+ "default": app.config['PAGINATION_DEFAULT'],
+ "description": "Limit the number of results per page."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ 'page': {
+ "in": "query",
+ "name": "page",
+ "type": "integer",
+ "minimum": 1,
+ "description": "Specify result page."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ 'sort': {
+ "in": "query",
+ "name": "sort",
+ "type": "object",
+ "description": "Sort results."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ 'embed': {
+ "in": "query",
+ "name": "embedded",
+ "type": "object",
+ "description": "Control embedding of related resources."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ 'project': {
+ "in": "query",
+ "name": "projections",
+ "type": "object",
+ "description": "Show/hide fields in response."
+ "<br />[(Cheatsheet)](#section/Cheatsheet)",
+ },
+ },
+
+ # Error Responses
+ 'definitions': {
+ '404': {
+ 'description': 'No item with the provided `_id` exists',
+ },
+ '401': {
+ 'description': 'The `Authorization` header is missing or '
+ 'contains an invalid token',
+ },
+ '403': {
+ 'description': 'The `Authorization` header contains a valid '
+ 'token, but you do not have the required '
+ 'permissions',
+ },
+ '422': {
+ 'description': 'Validation of the document body failed',
+ },
+ '428': {
+ 'description': "The `If-Match` header is missing",
+ },
+ '412': {
+ 'description': "The `If-Match` header does not match the "
+ "current `_etag`",
+ }
+ }
+ })
+
+
+def _add_param_refs(path, method, references):
+ """Helper to add references to query parameters to a path method."""
+ parameters = [{'$ref': '#/parameters/%s' % ref}
+ for ref in references]
+ add_documentation({
+ 'paths': {path: {method.lower(): {'parameters': parameters}}},
+ })
+
+
+def _add_error_refs(path, method, codes):
+ """Helper to add references to error responses to a path method."""
+ errors = {str(code): {'$ref': '#/definitions/%s' % code}
+ for code in codes}
+
+ add_documentation({
+ 'paths': {path: {method.lower(): {'responses': errors}}},
+ })
+
+
+def _update_properties(domain):
+ """Update field properties.
+
+ - Properties can have a title. Use `title` if specified, otherwise
+ capitalized field name (without underscores) as default.
+ - ReDoc can mark fields as `Nullable` (x-nullable extension).
+ - OpenAPI supports writeonly properties.
+ - Fix description and example for field with related resources, which
+ are not displayed correctly
+ """
+ def _update_property(prop, key, value):
+ """Helper to update a property."""
+ add_documentation({'definitions': {
+ domain['item_title']: {'properties': {prop: {key: value}}}
+ }})
+
+ for prop, prop_def in domain['schema'].items():
+ if prop_def.get('nullable'):
+ _update_property(prop, 'x-nullable', True)
+
+ if prop_def.get('writeonly'):
+ _update_property(prop, 'writeOnly', True)
+
+ default_title = prop.title().replace('_', ' ')
+ _update_property(prop, 'title', prop_def.get('title', default_title))
+
+ if 'data_relation' in prop_def:
+ for fix in ('description', 'example'):
+ _update_property(prop, fix, prop_def.get(fix, ''))
+
+ # Contraty to description and example, the default is optional
+ if 'default' in prop_def:
+ _update_property(prop, 'default', prop_def['default'])
+
+
+def _update_paths(resource, domain):
+ """Update the lookup paths for a resource.
+
+ Re-format the default _id lookup and add additional lookups,
+ if any exist.
+ """
+ title = domain['item_title']
+ key = '%s__id' % title
+ path = '/%s/{%sId}' % (resource, title.lower())
+
+ updates = {'description': 'The `_id` field of a %s document' % title}
+ add_documentation({'parameters': {key: updates}})
+
+ try:
+ lookup = domain['additional_lookup']
+ except KeyError:
+ pass
+ else:
+ field = lookup['field']
+ params = [{
+ 'in': 'path',
+ 'name': field,
+ 'required': False,
+ 'description': '*Instead* of the `_id`, you can also use the '
+ '`%s` field as an alternative lookup when '
+ '*retrieving* a document.' % field,
+ 'type': lookup['url'],
+ }]
+ add_documentation({
+ 'paths': {path: {'get': {'parameters': params}}},
+ })
+
+
+def _update_methods(resource, domain):
+ """For each method, add the appropriate query params and responses."""
+ # 0: Check if the resource has data relation and can use `embedded`
+ has_data_relation = any('data_relation' in field_def
+ for field_def in domain['schema'].values())
+
+ # 1: Resource methods, `GET` and `POST` possible
+ resource_path = '/%s' % resource
+ for method in domain['resource_methods']:
+ parameters = []
+ errors = []
+
+ if method not in domain['public_methods']:
+ errors += [401, 403]
+ parameters.append('auth')
+
+ if method == 'GET':
+ parameters += ['filter', 'max_results', 'page', 'sort']
+
+ if method == 'POST':
+ errors.append(422)
+
+ parameters.append('project')
+ if has_data_relation:
+ parameters.append('embed')
+
+ _add_error_refs(resource_path, method, errors)
+ _add_param_refs(resource_path, method, parameters)
+
+ # 2: Item methods, `GET`, `PATCH` and `DELETE` possible
+ item_path = '/%s/{%sId}' % (resource, domain['item_title'].lower())
+ for method in domain['item_methods']:
+ parameters = []
+ errors = [404] # all item methods can result in 404 if item is missing
+
+ if method not in domain['public_item_methods']:
+ errors += [401, 403]
+ parameters.append('auth')
+
+ if method == 'GET':
+ parameters.append('filter')
+
+ if method == 'PATCH':
+ errors += [412, 422, 428]
+
+ if method in ['GET', 'PATCH']:
+ parameters.append('project')
+ if has_data_relation:
+ parameters.append('embed')
+
+ if method == 'DELETE':
+ errors += [412, 428]
+
+ _add_error_refs(item_path, method, errors)
+ _add_param_refs(item_path, method, parameters)
diff --git a/docs/Admin_Guide.md b/docs/Admin_Guide.md
deleted file mode 100644
index 1318f5f0db3df890c9c6d514be7aa9b3e98764ed..0000000000000000000000000000000000000000
--- a/docs/Admin_Guide.md
+++ /dev/null
@@ -1,29 +0,0 @@
-# Admin tasks
-
-# Cron
-
-The API requires a cron job to do tasks on a regular basis, which includes
-sending warnings about expiring permissions. You should configure a cronjob to
-run `amivapi/cron.py` once per day.
-Append something like this to your crontab:
-
- 39 3 * * * /path/to/env/python /path/to/api/amivapi/cron.py
-
-# Using manage.py
-
-The API offers a convenient management tool for various tasks. This is provided by [flask-script](https://flask-script.readthedocs.org/en/latest/)
-and all the functions can be found in manage.py.
-
-The functions can be used with:
-
- python manage.py somefunction --option=something
-
-A quick guide for the most important functions:
-
-## LDAP
-
-manage.py offers a functions to trigger ldap synchronization.
-
- python manage.py ldap_sync
-
-This imports all users missing from ldap to the api and updates existing users
diff --git a/docs/Cheatsheet.md b/docs/Cheatsheet.md
deleted file mode 100644
index ba3df18d7f6701560ef3ca35baf5f43b1c2467d2..0000000000000000000000000000000000000000
--- a/docs/Cheatsheet.md
+++ /dev/null
@@ -1,57 +0,0 @@
-# AMIVAPI Cheatsheet
-
-## Filtering
-
-```
-https://amivapi/events?where={"title_en":"Party"}
-
-https://amivapi/events?where={"$gt":{"time_start":"20180606T100000Z"}}
-
-https://amivapi/events?where={"$or":{"title_en":"Party","title_de":"Feier"}}
-```
-
-Supported operators include:
-`$gt` (>), `$gte` (>=), `$lt` (<), `$lte` (<=), `$ne` (!=), `$and`, `$or`, `$in`
-
-To get events which have an image for the inforscreen:
-```
-https://amivapi/events?where={"img_infoscreen":{"$exists":true}}
-```
-
-Time format: `YYYYMMDDThhmmssZ`
-
-**Y**ear, **M**onth, **D**ay, **h**our, **m**inutes, **s**econds, `T` is a required separator
-and `Z` indicates UTC (other timezones are not supported)
-
-## Sorting
-
-```
-https://amivapi/events?sort=time_start,-time_end
-```
-
-The prefix ```-``` inverts the sorting order.
-
-## Pagination
-
-```
-https://amivapi/events?max_results=20&page=2
-```
-
-There is a global maximum on the results per page, so getting all results in one request
-may not be possible.
-
-## Embedding
-
-By default, other resources are only included by `_id`. with embedding, you can include complete objects.
-
-```
-https://amivapi/groupmemberships?embedded={"user":1,"group":1}
-```
-
-## Projections
-
-Hide fields or show hidden fields. Passwords can't be shown.
-
-```
-https://amivapi/groupmemberships?projection={"group":0}
-```
diff --git a/docs/Developer_Guide.md b/docs/Developer_Guide.md
deleted file mode 100644
index c9c245b83fd04c6dbaca1a74b08db8151d0b4e4b..0000000000000000000000000000000000000000
--- a/docs/Developer_Guide.md
+++ /dev/null
@@ -1,378 +0,0 @@
-# General
-
-## Used Frameworks
-
-AMIV API uses the [python-eve](http://python-eve.org/) Framework which is a collection of libraries around [Flask](http://flask.pocoo.org/) and [SQLAlchemy](http://www.sqlalchemy.org/).
-The best source for information during development is the EVE Source Code at [Eve Github Repository SQL Alchemy Branch](https://github.com/nicolaiarocci/eve/tree/sqlalchemy).
-
-The main links for research about the used technologies are:
-
- * [Flask](http://flask.pocoo.org/docs/0.10/api/)
- * [SQL Alchemy](http://docs.sqlalchemy.org/en/rel_0_9/)
- * [Flask-SQL Alchemy](https://pythonhosted.org/Flask-SQLAlchemy/)
- * [Werkzeug](http://werkzeug.pocoo.org/)
- * [Eve](http://python-eve.org/)
-
-## Development status
-
-Eve is still in early development and changing a lot. That means it might be possible that we can improve our codebase as more features move into Eve's core. We are currently using a patched version of eve-sqlalchemy and eve-docs, which are forked on github here:
-
-* [eve-sqlalchemy fork by Leonidaz0r](https://github.com/Leonidaz0r/eve-sqlalchemy)
-* [eve-docs fork by hermannsblum](https://github.com/hermannsblum/eve-docs)
-
-## Installation
-
-To setup a development environment of the API we recommend using a virtual environment with the pip python package manager. Furthermore you need git.
-
-The following command works on Archlinux based systems, other distributions should provide a similar package:
-
- sudo pacman -S python2-pip git
-
-After installing pip create a working environment. First create a folder:
-
- mkdir amivapi
- cd amivapi
-
-Now create a virtualenv which will have the python package inside and activate it:
-
- virtualenv venv
- . venv/bin/activate
-
-Now get the source:
-
- git clone https://github.com/amiv-eth/amivapi.git
- cd amivapi
-
-Install requirements:
-
- pip install -r requirements.txt
-
-## Configuration
-
-Create a configuration:
-
- python2 manage.py create_config
-
-The tests will create their own database. If you configure a MySQL Server you will be asked whether the tests should also be run there. If you don't activate that
-they will create temporary databases on the fly in temporary files. Note that even if they run on a MySQL server they will create their own database, so you need
-to have the permissions for CREATE DATABASE.
-
-## Unittests
-
-To run the tests you need to install tox:
-
- pip install tox
-
-Create a config(see above). To run all tests enter
-
- tox
-
-To test just one environment use -e with py27, py34, pypi or flake8
-
- tox -e py27
-
-To run only some tests specify them in the following way(substitute your test class):
-
- tox -- amivapi.tests.forwards
-
-## Integration tests
-
-We have currently one "integration" test for LDAP. Since we have no "dummy" user for LDAP, you have to test it with your own credentials.
-
-You need to install nosetests:
-
- pip install nose
-
-Next you need to set some environment variables, since nose does not like command line arguments.
-Example for windows:
-
- $env:ldap_test_user= "YOURUSERNAMEHERE"
- $env:ldap_test_pass= "YOURPASSWORDHERE"
-
-Now you can run the test, make sure to show stdout with '-s'
-
- nosetests -s .\amivapi\tests\ldap_integration.py
-
-Important: Since the test can't know your user data, you have to check the printout if the data imported by ldap is correct.
-
-## Debugging server
-
-To play around with the API start a debug server:
-
- python2 run.py
-
-When the debug server is running it will be available at http://localhost:5000 and show all messages printed using the logger functions, print functions or exceptions thrown.
-
-# Architecture
-
-The main-directory lists following files:
-
-* authentication.py: Everything about who somebody is. Tokens are mapped to sessions and logins are handled. Also author fields are set.
-* authorization.py: Everything about what somebody can do. Permissions are implemented here.
-* bootstrap.py: The Eve-App gets created here. All blueprints and event-hooks are registered in the bootstrap.
-* confirm.py: Blueprint and event-hooks regarding the confirmation of unregistered users.
-* cron.py: Jobs run on a regular basis (sending mail about expiring permissions, cleanup)
-* documentation.py: Loads additional documentation for the blueptrints.
-* forwards.py: Hooks to implement the email-functionality of forwards and assignments to forwards.
-* localization.py: Localization of content-fields.
-* media.py: File Storage. Handles uploaded files and serves them to the user.
-* models.py: The Data-Model. As a basis of the API, in the Data-Model the different Data-Classes and their relations get defined.
-* schemas.py: Creates the basic validation-schema out of the data-model and applies custom changes.
-* ldap.py: Contains everything to connect the api with ldap and carry out user imports and updates.
-* settings.py: Constants which should not be changed by the admin, but can be changed by some developer.
-* utils.py: General helping functions.
-* validation.py: Every validation that extends the basic Cerberus-schema-definition and Hooks for special semantic checks, e.g. whether an end-time comes after a start-time.
-
-For understanding the structure of the api, the data-model in models.py is the Point to start.
-
-# Security
-
-Checking whether a request should be allowed consists of two steps,
-authentication and authorization. Authentification is the process of
-determining the user which is performing the action. Authorization is the
-process of determining which actions should be allowed for the authentificated
-user.
-
-Authentification will provide the ID of the authentificated user in
-g.logged_in_user
-
-Authorization will provide whether the user has an admin role in
-g.resource_admin
-
-Requests which are handled by eve will automatically perform authentication
-and authorization. If you implement a custom endpoint you have to call them
-yourself. However authorization really depends on what is about to happen,
-so you might have to do it yourself. To get an idea of what to do look at
-the authorization hooks(pre_xxx_permission_filter()). You can quite certainly
-reuse that code somehow.
-
-Perform authentication(will abort the request for anonymous users):
-
- if app.auth and not app.auth.authorized([], <resource>, request.method):
- return app.auth.authenticate()
-
-Replace <resource> with the respective resource name.
-
-
-## Authentification
-
-File: authentication.py
-
-The process of authentication is straight forward. A user is identified by
-his nethz (or email) and his password. He can sent those to the /sessions resource and obtain a token which can prove authenticity of subsequent requests.
-This login process is done by the process_login function. Sessions do not time
-out, but can be deleted.
-
-When a user sends a request with a token eve will create a TokenAuth object
-and call its check_auth function. That function will check the token against
-the database and set the global variable g.logged_in_user(g is the Flask g
-object) to the ID of the owner of the token.
-
-## Authorization
-
-File: authorization.py
-
-A request might be authorized based on different things. These are defined by
-the following properties of the model:
-
- __public_methods__ = [<methods>]
- __registered_methods__ = [<methods>]
- __owner_methods__ = [<methods>]
- __owner__ = [<fields>]
-
-The __xx_methods__ properties define methods which can be accessed. Also a list
-of fields can be set, which make somebody an owner of that object if he has the
-user ID corresponding to the fields content. For example a ForwardUser object
-has the list
-
- __owner__ = ['user_id', 'forward.owner_id']
-
-This defines the user referenced by the field user_id as well as the user
-referenced by owner_id of the corresponding Forward as the owner of this
-ForwardUser object.
-
-I recommend to look over the common_authorization() function. The rules created
-by it are the following, in that order:
-
-1. If the user has ID 0 allow the request.
-2. If the user has a role which allows admin access to the endpoint allow the
- request
-3. If the endpoint is public allow the request.
-4. If the endpoint is open to registered users allow the request.
-5. If the endpoint is open to object owners, perform owner checks(see below)
-6. Abort(403)
-
-The function will also set the variable g.resource_admin depending on whether
-the user has an admin role(or is root).
-
-One thing to note is that users which are not logged in are already aborted by
-eve when authentication fails for resources which are not public, therefore
-this is not checked anymore in step 4.
-
-### Roles
-
-Roles can be defined in permission_matrix.py. A role can give a user the right
-to perform any action on an endpoint. If permission is granted based on a role
-no further filters are applied, hence it is refered to as admin access and
-g.resource_admin is set.
-
-### Owner checks
-
-If the authorization check arrives at step 5 and the requested resource has
-owner fields, then those will be used to determine the results. This is the
-case for example when a registered user without an admin role performs a GET
-request on the ForwardUser resource. He can perform that query, however he is
-supposed to only see entries which forward to him or where he is the
-listowner.
-
-This is solved by two functions. When extracting data we need to create
-additional lookup filters. Those are inserted by the
-apply_lookup_filters() function which is called by the hooks below it.
-When inserting new data or changing data it gets more complicated. First we
-need to make sure that the object which is manipulated belongs to the user,
-that is achieved using the previously described function. In addition we need
-to make sure that the object afterwards still belongs to him. We do not want
-people moving EventSignups or ForwardUsers to other users. All this is done in
-the will_be_owner() function which is used by the hooks as needed.
-However to achieve this the function needs to figure out what would happen if
-the request was executed. This is currently done by the resolve_future_field()
-function, which tries to resolve relationships using SQLAlchemy meta
-attributes for the data which is not yet inserted.
-If this checks out ok, the hooks return, if not the request is aborted.
-
-To check ownership inside your own function for an existing object, you can use get_owner(resource, _id) from utils. It will return a list of user-ids who are owners of the item. A common owner check looks like this:
-
- if g.logged_in_user is in utils.get_owner(<resource>, <_id>):
-
-## API Keys
-
-Instead of a token an API key can be sent. These are generated by manage.py
-and are stored in the config file. If an API key is sent, the user ID will be
--1(the anonymous user) and all actions will be authorized based on the
-settings for that key in the config.
-
-For implementation see common_authorization() and TokenAuth.check_auth()
-
-# Localization
-
-The api includes support for several languages in four fields which contain
-language dependant content, these are:
-
-- joboffers.title
-- joboffers.description
-- events.title
-- events.desription
-
-
-The general idea is: for every instance of each field we want an unique ID that
-can be used to provide translated content.
-
-This is solved using two new resources:
-
-1. translationmappings
-This resource is internal (see schemas.py), which means that it can only be accessed by eve internally.
-To ensure that this works with eve and our modifications (like _author
-fields) we are not using SQLAlchemy relationship configurations to create this field. (Since the eve sqlalchmey extension can not handle this)
-Instead the hook "insert_localization_ids" is called whenever events and joboffers are created. It posts internally to languagemappings to create the ids which are then added to the data of post.
-The relationship in models.py ensures that all entries in the mapping
-table are deleted with the event/joboffer
-
-2. translations
-This resource contains the actual translated data and works pretty straightforward:
-Given a localization id entries can be added
-
-How is the content added when fetching the resource?
-
-The insert_localized_fields hook check the language relevant fields and has to query the database to retrieve the language content for the given localization id.
-
-Then it uses flasks request.accept_languages.best_match() function to get the best fitting option. (it compares the Accept Language header to the given languages)
-
-When none is matching it tries to return content in default language as specified in settings.py (Idea behind this: There will nearly always be german content). If this it not available, it uses an empty string)
-
-The field (title or description) is then added to the response
-
-
-## Note: Testing
-
-Both events and joboffers have the exact language fields, but job_offers have less other required fields.
-Therefore testing is done with job_offers - if there are any problems with language fields in events, ensure that the tests work AND that all language fields in events are configured EXACLY like in joboffers
-(Or extend the tests to cover events as well to be sure)
-
-## Note: Automatization
-
-Since there are only four language fields (with title and description for both events and joboffers, which is convenient) all hooks and schema updates are done manually. Should a update of the api be intended which includes several more language fields automating this should be considered.
-
-For every language field the following is necessary:
-
-- Internal post to languagemappings to create the id (locatization.py, hook)
-- Retrieving the content when fetching the resource (locatization.py, hook)
-- Adding a id (foreignkey) and relationship to translationmappings (models.py)
-- Removing id from the schema to prohibit manually setting it (schemas.py)
-
-# LDAP
-
-The ETH uses the Lightweight Directory Access Protocol (LDAP) to provide data about students etc. [(More about ldap)](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol)
-
-It is important that users can log in using their ETH credentials. But we cannot use shibboleth alone because extraordinary or honorary members without nethz credentials would not be able to log in. Therefore the api just authenticates via LDAP.
-
-We use a self-developed ETH-LDAP connector (based on the python ldap3 library) for all connections. [(Check it out)](https://github.com/NotSpecial/nethz)
-The file ldap.py contains all related code.
-
-## LDAP Connector
-
-This class is a small LDAP connector used by the api during authentication. It provides one basic function:
-Provide a nethz and password and the connector will try to authenticate the user and, if successful, query LDAP for the user data.
-
-This way the LDAP signup works and we make sure that relogging can be used to ensure the user data is up to date
-
-It is implemented as a class to be able to create the connector only once and keep it for the app.
-
-## ldap_synchronize
-
-This function is intended for periodic database updates. It queries all members from ldap and compares them to the database. Those missing are imported, those existing are updated (if necessary)
-
-This function takes a little time so it should not be executed when processing requests. It is intended to be used by manage.py (see admin docs) or cron jobs.
-
-### Why all users at the same time?
-
-Because ldap is stupid, basically. It can not handle complex logical statements very well so it is not effective to try to exclude users already existing ("Give me all users except user with names a, b or c")
-
-After a lot of testing we settled on just importing them all as the most effective solution.
-
-## About LDAP entries
-
-Most of the field ins LDAP are straightforward. Many fields exist and most of them can be ignored. See ldap.py for the relevant fields.
-
-Important is only the field "ou" which stands for "organisational unit" in LDAP.
-This field contains info about field of study and VSETH membership.
-The fields which are part of AMIV are specified in settings.py and can be changed there (if necessary)
-The list itself can be aquired from VSETH IT.
-
-
-# Files
-
-For files we wrote our own MediaStorage class as used by Eve by [extending the template](https://github.com/nicolaiarocci/eve/blob/develop/eve/io/media.py).
-The files need a folder which is created in the process of "create_config".
-
-Maybe in future releases of Eve there will be an official implementation of file system storage. Maybe it would be useful to use this instead of our implementation instead in this case.
-
-How Eve uses the MediaStorage Class can be found [here](http://python-eve.org/features.html#file-storage)
-
-To serve the information specified in EXTENDED_MEDIA_INFO the file "media.py" contains the class "ExtFile" which contains the file as well as the additional information Eve needs.
-
-As EXTENDED_MEDIA_INFO we use file name, size and a URL to the file.
-The URL can be accessed over a custom endpoint specified in "file_endpoint.py", using flask methods.
-
-
-# Validation
-
-Luckily the cerberus validator is easily extensible, so we could implement many custom rules. Those are found in validator.py and are not very complex.
-
-More information on cerberus and its merits can be found in the [Cerberus Documentation](https://cerberus.readthedocs.org/en/latest/)
-
-# Cron
-
-There are some tasks which are done on a regular basis. This includes removing
-expired permissions and unused sessions. Users who's permissions expire should
-be warned prior to this by mail. This is all done by a cronjob. The cronjob runs
-cron.py.
diff --git a/docs/OAuth.md b/docs/OAuth.md
deleted file mode 100644
index 324f93d53de1f09f503c82867ae7bfb42ad8a004..0000000000000000000000000000000000000000
--- a/docs/OAuth.md
+++ /dev/null
@@ -1,32 +0,0 @@
-# Using the central login
-
-To use the central login, you must follow the OAuth 2.0 implicit grant flow. This basically means the following:
-
-1. To get a user token you redirect the client's browser to the central login page, ```/oauth``` of the AMIV API. You need to provide the following query parameters:
-
-* ```response_type```: Must be ```token```.
-* ```client_id```: Name of your application (displayed to the user)
-* ```redirect_uri```: URL to return the user to after successful login.
-* ```state```: A random token you generated. Used to prevent CSRF attacks.
-
-2. After successful login, the client will be redirected to the URL you provided in the ```redirect_uri``` parameter with the following additional query parameters:
-
-* ```access_token```: Your new login token. Provide it in the Authorization header of requests to the AMIV API.
-* ```token_type```: Will always be ```bearer```. You can ignore it.
-* ```scope```: Will always be ```amiv```. You can ignore it.
-* ```state```: The CSRF token you sent in the authorization request. You must check that it is still equal to your provided value to prevent CSRF attacks.
-
-
-As OAuth is an open standard, it is very likely that there are libraries to automate this workflow in your preferred programming language. These libraries might expect more parameters, which we don't use. If you need to enter a token refresh URL or a ```client_secret```, make sure you are using the implicit grant authorization type. By the standard an authorization request can also supply a requested ```scope```. AMIVAPI will ignore this parameter, if you supply it.
-
-The ```redirect_uri``` must be an https URL and must not contain a fragment (anything after ```#```).
-
-# Whitelisting of OAuth clients
-
-To prevent phishing attacks we use a whitelist of ```client_id``` and ```redirect_uri```. To register a client with the API, a request to the ```oauthclients``` endpoint can be issued, e.g. with the following command:
-
-```
-curl -X POST -d "client_id=<your client ID>&redirect_uri=<your redirect URI>" -H "Authorization: <admin token>" https://<API URL>/oauthclients
-```
-
-The admin token needs to have readWrite access to the ```oauthclients``` resource.
diff --git a/docs/User_Guide.md b/docs/User_Guide.md
deleted file mode 100644
index 6597b8307104956438249e821d1ac4ca30414a46..0000000000000000000000000000000000000000
--- a/docs/User_Guide.md
+++ /dev/null
@@ -1,552 +0,0 @@
-# General
-
-[TOC]
-
-## About this document
-
-This document should help somebody who wants to develop a client for the AMIV API. It focuses on manipulating data via the public interface. For in depth information see Developer Guide, for reference visit the [API Reference](https://<base_url>/docs).
-
-## Portability
-
-The API is designed and all clients should be designed to be useable outside of AMIV. Although we will use api.amiv.ethz.ch as the base URL in this document this is not necessary and a client should provide a config entry for that.
-
-## Encryption
-
-The API is only accessible via SSL and will never be made public via an unencrypted channel, as should all your apps.
-
-## Date format, string format
-
-Date and time is always UTC in ISO format with time and without microseconds. Any different time format will result in 400 Bad Request.
-
- %Y-%m-%DT%H:%M:%SZ
-
-All strings are UTF-8.
-
-## About REST
-
-AMIV API is a [REST API](https://de.wikipedia.org/wiki/Representational_State_Transfer). REST is a stateless protocoll modelled after HTTP. The API consists of resources, which have objects. For example the resource /users provides access to the member database. Every resource has the methods GET, POST, PUT, PATCH, DELETE. These are the known regular HTTP methods, GET and POST being the most well known. The API is based on the [python-eve](http://python-eve.org/index.html) framework, so you can refer to eve for detailed information as well.
-
-There are many clients available to use REST and there are libraries for all kind of programming languages. Many HTTP libraries will also be able to communicate with a REST API.
-
-The methods meanings:
-
-Resource methods(use i.e. on /users)
- * GET - Retriving data, query information may be passed in the query string
- * POST - Creating a new entry, the new entry must be provided in the data section
-
-Item methods(use i.e. on /users/4)
- * PATCH - Changing an entry
- * DELETE - Removing an entry
- * PUT - Replacing an entry, this is like DELETE immediately followed by POST. PUT ensures no one else can perform a transaction in between those two queries
-
-## Response format
-
-The status code returned by the API are the standard [HTTP status codes](https://de.wikipedia.org/wiki/HTTP-Statuscode). Codes starting with 2 mean the operation was successfull, starting with 3 are authentication related, 4 are client errors, 5 are server errors, 6 are global errors. Most important codes are:
-
- * 200 - OK (Generic success)
- * 201 - Created (successful POST)
- * 204 - Deleted (successful DELETE)
-
- * 400 - Bad request (This means your request has created an exception in the server and the previous state was restored, if you are sure it is not your fault file a bug report)
- * 401 - Please log in
- * 403 - Logged in but not allowed (This is not for you)
- * 404 - No content (This can also mean you could retrive something here, but no object is visible to you because your account is that of a peasant)
- * 412 - The etag or confirmation-token you provieded is wrong
- * 422 - Semantic error (Your data does not make sense, e.g. dates in the past which should not be)
-
- * 500 - Generic server error
- * 501 - Not implemented (Should work after alpha)
-
-All responses by the API are in the [json format](https://de.wikipedia.org/wiki/JavaScript_Object_Notation) by default. Using the Accept header output can be switched to XML, but we encourage to use json as near to no testing has been done for XML output. If you want XML support consider reading the Developer Guide and providing unit tests for XML output.
-
-## HATEOAS
-
-The API is supposed to be human readable, meaning a human can read the responses and only knowing REST standard can perform any action available. That means it is possible to get any information about the structure of the data via the API. Starting at the root node / URL links will be provided to any object.
-
-Check [wikipedia](https://en.wikipedia.org/wiki/HATEOAS) for more info.
-
-## Example: First Request
-
-The examples will provide code in python using the [requests](http://docs.python-requests.org/en/latest/) library. If you are developing a client in the python language requests might be a possible choice to query the API.
-
-
-Request:
-
- GET /
-
-Code:
-
- response = requests.get("https://api.amiv.ethz.ch/")
-
-Response:
-
- status: 200
-
- {
- "_links": {
- "child": [
- {
- "href": "/files",
- "title": "files"
- },
- {
- "href": "/studydocuments",
- "title": "studydocuments"
- },
- {
- "href": "/forwardusers",
- "title": "forwardusers"
- },
- {
- "href": "/forwards",
- "title": "forwards"
- },
- {
- "href": "/sessions",
- "title": "sessions"
- },
- {
- "href": "/joboffers",
- "title": "joboffers"
- },
- {
- "href": "/eventsignups",
- "title": "eventsignups"
- },
- {
- "href": "/forwardaddresses",
- "title": "forwardaddresses"
- },
- {
- "href": "/users",
- "title": "users"
- },
- {
- "href": "/events",
- "title": "events"
- },
- {
- "href": "/permissions",
- "title": "permissions"
- }
- ]
- }
- }
-
-
-# Authentification
-
-Most access to the API is restricted. To perform queries you have to log in and acquire a login token. The login token is a unique string identifying you during a session. Sessions are a part of the data model as any other object and can be created in the normal way. Just send a POST request to the /sessions resource:
-
-## Example
-
-Request:
-
- POST /sessions?user=mynethzormail&password=mypassword
-
-Code:
-
- response = requests.post("https://api.amiv.ethz.ch/sessions", data={"user": "mynethzormail", "password": "mypassword"})
-
-Response:
-
- status = 201
-
- {
- u'_author': -1,
- u'_created': u'2014-12-20T11:50:06Z',
- u'_etag': u'088401622fc10cbf0d549e9282072c37829a1b81',
- u'_id': 4,
- u'_links': {u'self': {u'href': u'/sessions/4', u'title': u'Session'}},
- u'_status': u'OK',
- u'_updated': u'2014-12-20T11:50:06Z',
- u'id': 4,
- u'token': u'eyJzaWduYXR1cmUiOiAiSFlKVGhIeUVvSHNSL203M0I1RlBTckdUQlFyOUJ4QzlUMHhsZmNZY1dWQlBpQnZ6T2dvM1NXY2RSU3NiVTJhRFRpQzQ4N2VlcVFxcjN4d094YStZM1o2Zi85cnV6d1RKVHVDL1pqcnlKaXZ4cDc4RzlaejdGb1BvZ0VhTXk5Zy9DdW9LL25vb3BNYVRnd2hmUW1RZDRPV1dMV1ZDZVZkM0dYb0VKQWJZR3NEZ2F3V0Q5dlRhanVIcEhUQUYwS1FOSlp0V3prcU9ldW5nb1pseHdqUXhxdXJhK2hjaEdTNmFsWC9NT3NiWVh1d2R3TXFXaVFaMys0dTdVdHBrSmZiY04vcmJ6MS9ldWF6NFJlRCtMandoWDBMTTAvOXdLamlFNW9BbFlrajkxQW9LYnJtY0R2Q0gxcGlJaWtlRHVxL2NiZHhTT1Uvck5jOC9GR29JejRPMG13PT0iLCAidXNlcl9pZCI6IDAsICJsb2dpbl90aW1lIjogIjIwMTQtMTItMjBUMTI6NTA6MDZaIn0=',
- u'user_id': 4
- }
-
-We will look at the details of this response later. First we only notice the token field and use that token to issue an authenticated query to find out something about our user account. A token can be passed as the HTTP Basic Auth username with an empty password. The python requests library provides this functionality as does command line curl. If you can not pass such a field you can create the Authorization header which would be generated by that parameter yourself. For that you need to base64 encode the token followed by a colon. We will see examples for both methods.
-
-
-## Example: Retrieving user
-
-It is possible to retrive a user using its nethz. Normally we would use an ID to retrive an item, but in this case it is easier this way.
-
-Request:
-
- POST /sessions "nethz=mynethzormail&password=mypassword"
- GET /users/mynethz (+Authorization header)
-
-Code with good REST library:
-
- login = requests.post("http://api.amiv.ethz.ch/sessions", data={"user": "myuser", "password": "mypassword"})
- token = login.json()['token']
- response = requests.get("https://api.amiv.ethz.ch/users/myuser", auth=requests.auth.HTTPBasicAuth(token, ""))
-
-Code with bad REST library:
-
- login = requests.post("/sessions", data={"user": "myuser", "password": "mypassword"})
- token = login.json()['token']
- auth_header = b64encode(token + ":")
- response = requests.get("/users/myuser", headers={"Authorization": auth_header})
-
-Response:
-
- {
- '_author': 0,
- '_created': '2014-12-18T23:29:07Z',
- '_etag': '290234023482903482034982039482034',
- '_links': {
- 'parent': {
- 'href': '/', 'title': 'home'
- },
- 'self': {
- 'href': '/users',
- 'title': 'users'
- }
- },
- '_updated': '2014-12-18T23:29:07Z',
- 'birthday': None,
- 'department': None,
- 'email': 'kimjong@whitehouse.gov',
- 'firstname': 'Edward',
- 'gender': 'male',
- 'groups': None,
- 'id': 4,
- 'lastname': 'Nigma',
- 'ldapAddress': None,
- 'legi': None,
- 'membership': 'none',
- 'nethz': None,
- 'phone': None,
- 'rfid': None,
- }
-
-## API keys
-
-If access is not done by a user but rather by a service(cron, vending machine, info screen), user based authorization does not work. Instead an API key can be used. The API administrator can generate keys using the manage.py script and configure which endpoints can be accessed. Endpoint access via API key will give admin priviledges. The API key can be sent in the same way as a token. You can think of it as a permanent admin session for specific endpoints.
-
-##Unregistered users
-
-Next to GET operations on public data, AMIV API currently allows unregistered users in exactly two cases: Signing up for a public event or managing email-subscribtions for public email lists. In Both cases, 'is_public' of the event or forward must be True.
-
-Basically, an unregistered user can perform any GET, POST, PATCH or DELETE action on the supported resource within the usual rights. However, as the HTTP request comes without login, you need to confirm yourself and your email-address with a special token.
-After the creation of a new item with POST, the User will get an email with the Token. Your Admin might provide links in this mail to a user-friendly tool. However, here is the Workflow that always works:
-Just POST the token send to you to '/confirmations' in the following way:
-
- POST /confirmations?token=dagrfvcihk34t8xa2dasfd
-
-After this, the server knows that the given email-address is valid.
-Every further Action kann be performed as usually, but with a special Header:
-
- {
- 'Token': dagrfvcihk34t8xa2dasfd
- }
-
-The API will return 403 FORBIDDEN if you did forgot to provide a token and will return 412 PRECONDITION FAILED if the provided token is not valid for the requested item.
-
-##Public Events
-To subscribe to a public event with an email-address you simply post to "/eventsignups":
-
-Data:
-
- {
- 'event_id': 17,
- 'user_id': -1,
- 'email': "mymail@myprovider.ch",
- }
-
-You will receive a 202 Acepted. This means that the signup is not valid yet, but the server has received valid data and the user can confirm the signup by clicking on a link in an email.
-The User-ID '-1' stands for the anonymous user.
-
-##Email Forwards
-For email-lists, we know 3 resources: '/forwards', '/forwardusers', '/forwardaddresses'. '/forwards' is used to manage lists. '/forwardusers' is used to manage entries which forward to a registered user. '/forwardaddresses' is used for anonymous entries. To create a new subscription or change an existing one for an unregistered user, you need to use '/forwardaddresses'. The procedure of confirmation is exactly the same as for events.
-
-
-
-# GET queries
-
-GET queries can be customized in many ways. There is the possibility for where, projection, embedding and pagination clauses.
-
-## where clauses
-
-Using a where clause one can specify details about the object looked for. Queries can be stated in the python syntax(as if you would write an if clause). This is some kind of experimental, if any issues occur please contact api@amiv.ethz.ch or write a report in the issue tracker on github.
-
-An example (url-encoded) is:
-
- GET /events?where=title=="Testevent"+and+spots>5
-
-A more complex query would be
-
- GET /events?where=(title=="Testevent"+and+spots>5)+or+title=="Testevent2"
-
-Embedding works only for equality comparison and no recursion at the moment(to improve, commit to the eve-sqlalchemy project), for example:
-
- GET /events?where=signups.user_id==5
-
-This would return all events which the user with the id 5 is signed up for.
-
-## Projections
-
-Using the projection parameter it is possible to decide which fields should be returned. For example:
-
- GET /events?projection={"location":0,"signups":1}
-
-This will turn of the location field, but return a list of signups. The behaviour of data relations when their projection is enabled can be configured using embedding.
-
-## Embedding
-
-Turning embedding on and off will determine how relations are returned by the API. With embedding turned on the whole object will be returned, with embedding turned off only the ID will be returned.
-
- GET /users?projection={"permissions":1}&embedded={"permissions":1}
-
-This will return all the permission objects embedded in the response
-
-## Sorting
-
-Results can be sorted using the *sort* query parameter. Prepending the name with a - will sort in descending order.
-
- GET /events?sort=-start_time
-
-This will return the events sorted by descending start_time.
-
-## Pagination
-
-The number of returned results for queries to resource endpoints can be controlled using the max_results and the page parameter.
-
- GET /events?max_results=10&page=3
-
-This will return the third page of 10 items.
-
-
-# PUT, PATCH, DELETE queries
-
-## If-Match
-
-To manipulate an existing object you have to supply the If-Match header to prevent race conditions.
-When you use GET on an element you will be provided with an _etag field. The etag is a string which changes whenever the object is manipulated somehow. When issuing a PUT, PATCH or DELETE query you must supply the etag in the If-Match header to ensure that no one else changed the object in between.
-
-If no etag is provided, you will recieve 403 FORBIDDEN. If the etag is wrong, the api returns 412 PRECONDITION FAILED.
-
-### Example: Use PATCH to change a password
-
- GET /users/myuser (+Authorization header)
- PATCH /users/myuser data: "password=newpw" headers: "If-Match: a23...12b"
-
-Code:
-
- me = requests.get("/users/myuser", auth=myauth)
- etag = me.json()['_etag']
- result = requests.patch("/users/myuser", data={"password":"newpw"}, headers={"If-Match":etag})
-
-The response will be the changed user object.
-
-# Localization: Content in different languages
-
-The api supports descriptions and titles for events and job offers in different
-languages.
-If you post to one of those ressources, the response will contain a title_id
-and description_id. Those are the unique identifiers.
-To add content in various languages you can now use this id to post to the
-/translations resource
-
-## Example: Create an event with the requests library
-
-Code:
-
- import json # To properly encode event data
-
- """Usual login"""
- auth = {'user': user, 'password': pw}
- r = requests.post('http://api.amiv.ethz.ch/sessions', data=auth)
- token = r.json().get('token')
- session = requests.Session()
- session.auth = (token, '')
-
- """Some data without language relevant content"""
- data = {'time_start': '2045-01-12T12:00:00Z',
- 'time_end': '2045-01-12T13:00:00Z',
- 'time_register_start': '2045-01-11T12:00:00Z',
- 'time_register_end': '2045-01-11T13:00:00Z',
- 'location': 'AMIV Aufenthaltsraum',
- 'spots': 20,
- 'is_public': True}
-
- payload = json.dumps(data)
-
- self.session.headers['Content-Type'] = 'application/json'
- response = self.session.post('http://api.amiv.ethz.ch/events',
- data=payload).json()
- del(self.session.headers['Content-Type']) # Header not needed anymore
-
-Response:
-
- {u'_author': 0,
- u'_created': u'2015-03-05T14:12:19Z',
- u'_etag': u'8a20c7c3e035eb5a03906ce8f0f7717a4300e9de',
- u'_id': 1,
- u'_links': {u'self': {u'href': u'/events/1', u'title': u'Event'}},
- u'_status': u'OK',
- u'_updated': u'2015-03-05T14:12:19Z',
- u'description_id': 2,
- u'id': 1,
- u'is_public': True,
- u'location': u'AMIV Aufenthaltsraum',
- u'spots': 20,
- u'time_end': u'2045-01-12T13:00:00Z',
- u'time_register_end': u'2045-01-11T13:00:00Z',
- u'time_register_start': u'2045-01-11T12:00:00Z',
- u'time_start': u'2045-01-12T12:00:00Z',
- u'title_id': 1}
-
-Now extract ids to post translations
-
-Code:
-
- """Now add some titles"""
- self.session.post('http://api.amiv.ethz.ch/translations',
- data={'localization_id': r['title_id'],
- 'language': 'de',
- 'content': 'Irgendein Event'})
- self.session.post('http://api.amiv.ethz.ch/translations',
- data={'localization_id': r['title_id'],
- 'language': 'en',
- 'content': 'A random Event'})
-
- """And description"""
- self.session.post('http://api.amiv.ethz.ch/translations',
- data={'localization_id': r['description_id'],
- 'language': 'de',
- 'content': 'Hier passiert was. Komm vorbei!'})
- self.session.post('http://api.amiv.ethz.ch/translations',
- data={'localization_id': r['description_id'],
- 'language': 'en',
- 'content': 'Something is happening. Join us!'})
-
-If we now specify the 'Accept-Language' Header, we get the correct content!
-
-Code:
-
- self.session.headers['Accept-Language'] = 'en'
-
- self.session.get('http://api.amiv.ethz.ch/events/%i' % response['id']).json()
-
-Response:
-
- {u'_author': 0,
- u'_created': u'2015-03-05T14:12:19Z',
- u'_etag': u'8a20c7c3e035eb5a03906ce8f0f7717a4300e9de',
- u'_links': {u'collection': {u'href': u'/events', u'title': u'events'},
- u'parent': {u'href': u'/', u'title': u'home'},
- u'self': {u'href': u'/events/1', u'title': u'Event'}},
- u'_updated': u'2015-03-05T14:12:19Z',
- u'additional_fields': None,
- u'description': u'Something is happening. Join us!',
- u'description_id': 2,
- u'id': 1,
- u'img_1920_1080': None,
- u'img_thumbnail': None,
- u'img_web': None,
- u'is_public': True,
- u'location': u'AMIV Aufenthaltsraum',
- u'price': None,
- u'signups': [],
- u'spots': 20,
- u'time_end': u'2045-01-12T13:00:00Z',
- u'time_register_end': u'2045-01-11T13:00:00Z',
- u'time_register_start': u'2045-01-11T12:00:00Z',
- u'time_start': u'2045-01-12T12:00:00Z',
- u'title': u'A random Event',
- u'title_id': 1}
-
-Yay! The title and description are added in english as requested.
-
-# Working with files
-
-Working with files is not much different from other resources. Most resources
-contain the file, only study documents, which will be explained below.
-
-##Files in Events, Joboffers, etc.
-
-Files can be uploaded using the "multipart/form-data" type. This is supported
-by most REST clients. Example using python library "requests" and a job offer:
-(More info on requests here: http://docs.python-requests.org/en/latest/)
-
-Code:
-
- """Usual login"""
- auth = {'user': user, 'password': pw}
- r = requests.post('http://api.amiv.ethz.ch/sessions', data=auth)
- token = r.json().get('token')
- session = requests.Session()
- session.auth = (token, '')
-
- """Now uploading the file"""
- with open('somefile.pdf', 'rb') as file:
- data = {'title': 'Some Offer'}
- files = {'pdf': file}
- session.post('http://api.amiv.ethz.ch/joboffers',
- data=data, files=files)
-
-Response:
-
- {'_author': 0,
- '_created': '2015-02-19T14:46:14Z',
- '_etag': '9cd7fdf37507d2001f5902330ff38db1236bdb84',
- '_id': 1,
- '_links': {'self': {'href': '/joboffers/1', 'title': 'Joboffer'}},
- '_status': 'OK',
- '_updated': '2015-02-19T14:46:14Z',
- 'id': 1,
- 'pdf': {'content_url': '/storage/somefile.jpg',
- 'file': None,
- 'filename': 'somefile.jpg',
- 'size': 55069},
- 'title': 'Some Offer'}
-
-Note that 'file' in the response is None since returning as Base64 string is
-deactivated.
-
-##Working with study documents
-
-Study documents are a collection of files. Using them is simple:
-
-1. Create a study document (POST to /studydocuments)
-2. Save ID of the newly created document
-3. Upload files to the '/files' resource as described above, using the ID
-
-# Common Problems
-
-## PATCH, PUT or DELETE returns 403
-
-It is only possible to issue these methods on objects, not on resources.
-This will not work:
-
- DELETE /users?where=id==3
-
-Use this instead:
-
- DELETE /users/3
-
-Make sure you provided the required If-Match header. If that does not help
-make sure you can use GET on the item. If you are unable to request a GET
-then your account can not access the object.
-If you are able to GET the object, then your provided data is invalid. If
-you do not have admin priviledges for the endpoint(method on that resource)
-make sure your request will conserve your ownership of the object.
-
-## How can I send boolean etc to the server using python requests?
-
-To properly encode Integer, Boolean and such you need to properly format the
-data to json before sending, like this:
-
-Code:
-
- import json
-
- data_raw = {'spots': 42,
- 'is_public:' True}
-
- payload = json.dumps(data_raw)
-
-The payload is now ready for sending! (Be sure to set the 'Content-Type' header to 'application/json')