Merge branch 'feature/notebook_brevitas_import' into dev

Dockerfile

@@ -8,6 +8,9 @@ COPY requirements.txt .
 RUN pip install -r requirements.txt
 RUN rm requirements.txt
 RUN apt update; apt install nano
+RUN pip install jupyter
+RUN pip install netron
+RUN pip install matplotlib
 
 # Note that we expect the cloned finn directory on the host to be
 # mounted on /workspace/finn -- see run-docker.sh for an example
@@ -35,4 +38,3 @@ RUN chown -R $UNAME:$GNAME /home/$UNAME
 USER $UNAME
 
 WORKDIR /home/$UNAME/finn
-ENTRYPOINT /bin/bash

notebooks/FINN-HowToAnalysisPass.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# FINN - Analysis passes\n",
+    "--------------------------------------\n",
+    "* <font size=\"3\">traverses the graph structure and produces information about certain properties</font>\n",
+    "* <font size=\"3\">input: ModelWrapper</font>\n",
+    "* <font size=\"3\">returns dictionary of named properties that the analysis extracts</font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example - Quantity analysis of nodes in onnx graph\n",
+    "----------------------------------------------------------------------\n",
+    "<font size=\"3\">Purpose of this analysis pass is to return the number of similar nodes in a dictionary. So first an onnx model is loaded. In this example a trained brevitas model is used. It was exported from brevitas and saved as .onnx file. With the help of `import onnx` the load function can be accessed. As argument it takes the model path.</font>\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import onnx\n",
+    "onnx_model = onnx.load('LFCW1A1.onnx')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Here a picture or a call for onnx_model in netron**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<font size=\"3\">The onnx model has to be converted to a format that can be processed by FINN. This is done with ModelWrapper. As described in the short introduction, this is the format an analysis pass takes as input.</font>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from finn.core.modelwrapper import ModelWrapper\n",
+    "onnx_model = ModelWrapper(onnx_model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<font size=\"3\">The idea is to count all nodes that have the same operation type. The result should contain the operation types and the corresponding number of nodes that occur in the model. At the beginning an empty dictionary is created which is filled by the function and returned as result to the user at the end of the analysis.</font>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def count_equal_nodes(model):\n",
+    "    count_dict = {}\n",
+    "    for node in model.graph.node:\n",
+    "        if node.op_type in count_dict:\n",
+    "            count_dict[node.op_type] +=1\n",
+    "        else:\n",
+    "            count_dict[node.op_type] = 1\n",
+    "    return count_dict"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<font size=\"3\">The function takes the model as input and iterates over the nodes. Then it is checked whether there is already an entry for the operation type in the dictionary. If this is not the case, an entry is created and set to `1`. If there is already an entry, it is incremented. If all nodes in the model have been iterated, the filled dictionary is returned.</font>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'Shape': 1, 'Gather': 1, 'Unsqueeze': 5, 'Concat': 1, 'Reshape': 1, 'Mul': 5, 'Sub': 1, 'Sign': 4, 'MatMul': 4, 'BatchNormalization': 3, 'Squeeze': 3}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(count_equal_nodes(onnx_model))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

notebooks/FINN-HowToTransformationPass.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# FINN - Transformation passes\n",
+    "--------------------------------------\n",
+    "* <font size=\"3\">changes (transforms) the given graph</font>\n",
+    "* <font size=\"3\">input: ModelWrapper</font>\n",
+    "* <font size=\"3\">returns the changed model (ModelWrapper) and flag `model_was_changed`</font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## General Information\n",
+    "-----------------------------\n",
+    "<font size=\"3\">Transformation passes have a base class and must inherit from that. Transformations are meant to be applied using .transform function from the ModelWrapper. This function makes a deep copy of the input model by default. The next cell shows .transform of ModelWrapper. </font>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### .transform() from ModelWrapper\n",
+    "`def transform(self, transformation, make_deepcopy=True):     \n",
+    "     transformed_model = self\n",
+    "     if make_deepcopy:\n",
+    "         transformed_model = copy.deepcopy(self)\n",
+    "     model_was_changed = True\n",
+    "     while model_was_changed:\n",
+    "         (transformed_model, model_was_changed) = transformation.apply(\n",
+    "             transformed_model\n",
+    "         )\n",
+    "     return transformed_model`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<font size=\"3\">When the function is called, the model, the name of the transformation and, if required, the flag make_deepcopy are passed. It is also possible not to make a copy of the model. In this case `make_deepcopy` must be set to False. Then the branch `if make_deepcopy:` would not be taken and no copy of the model would be made. \n",
+    "\n",
+    "The unchanged model is first passed to the variable `transformed_model` to pass this variable on to the transformation later. \n",
+    "\n",
+    "`model_was_changed` indicates whether the transformation needs to be applied more then once. Because it needs to be applied at least one time `model_was_changed` is first set to True and then depending on the return values of the transformation function the transformation can be applied more then once. \n",
+    "\n",
+    "**Important**: Due to the structure of this function, `model_was_changed` must be set to False at some point. Otherwise the loop is infinite.\n",
+    "    \n",
+    "\n",
+    "Each new transformation must correspond to the scheme of the base class and contain at least the function `apply(model)`, which returns the changed model and a bool value for `model_was_changed`.\n",
+    "</font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### transformation base class\n",
+    "`from abc import ABC, abstractmethod`\n",
+    "\n",
+    "\n",
+    "`class Transformation(ABC):\n",
+    "    def __init__(self):\n",
+    "        super().__init__()`\n",
+    "\n",
+    " `    @abstractmethod\n",
+    "    def apply(self, model):\n",
+    "        pass`\n",
+    "        \n",
+    "<font size=\"3\"> Base class is abstract class (`import ABC`) with only one abstract method (`apply()`) which gets the model as input. To show what a transformation should look like, the following example is taken from FINN. </font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example - ConvertSubToAdd\n",
+    "-----------------------------\n",
+    "<font size=\"3\">The transformation replaces all subtraction nodes in a model with addition nodes with appropriate sign.</font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`from finn.transformation import Transformation`\n",
+    "\n",
+    "`class ConvertSubToAdd(Transformation):\n",
+    "     def apply(self, model):\n",
+    "         graph = model.graph\n",
+    "         for n in graph.node:\n",
+    "             if n.op_type == \"Sub\":\n",
+    "                 A = model.get_initializer(n.input[1])\n",
+    "                 if A is not None:\n",
+    "                     n.op_type = \"Add\"\n",
+    "                     model.set_initializer(n.input[1], -A)\n",
+    "         return (model, False)`\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<font size=\"3\">First the transformation class must be imported. Then a class can be created for the new transformation, which is derived from the base class. In this case the transformation has only the `apply()` function. \n",
+    "\n",
+    "All nodes are checked by first extracting the graph from the model and then iterating over the node list. With the help of .op_type the operation type of the node can be checked, if the node is a subtraction node the condition `if n.op_type == \"Sub\"` is true. It may be that the subtraction input of the node has no value, this is checked with `model.get_initializer(n.input[1])`. \n",
+    "    \n",
+    "**Important:** FINN always assumes a certain order of inputs, this is especially important if you want to create additional custom operation nodes.\n",
+    "\n",
+    "When the input is initialized, the operation type of the node is converted to `\"Add\"`, this can simply be done by using the equal sign. Then the sign of the initial value must be changed. For this the ModelWrapper function `.set_initializer` can be used.\n",
+    "\n",
+    "At the end the changed model is returned and `model_was_changed` is set to False, because the transformation has to be executed only once.</font>\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

run-docker.sh

-#!/bin/sh
+#!/bin/bash
 
 if [ -z "$VIVADO_PATH" ];then
 	echo "For correct implementation please set an environment variable VIVADO_PATH that contains the path to your vivado installation directory"
@@ -41,6 +41,17 @@ echo "Mounting $SCRIPTPATH/cnpy into /workspace/cnpy"
 echo "Mounting $SCRIPTPATH/finn-hlslib into /workspace/finn-hlslib"
 echo "Mounting $VIVADO_PATH/include into /workspace/vivado-hlslib"
 
+if [ "$1" = "test" ]; then
+	echo "Running test suite"
+	DOCKER_CMD="python setup.py test"
+elif [ "$1" = "notebook" ]; then
+	echo "Running Jupyter notebook server"
+	DOCKER_CMD="jupyter notebook --ip=0.0.0.0 notebooks"
+else
+	echo "Running container only"
+	DOCKER_CMD="bash"
+fi
+
 # Build the FINN Docker image
 docker build --tag=$DOCKER_TAG \
              --build-arg GID=$DOCKER_GID \
@@ -57,4 +68,5 @@ docker run --rm --name finn_dev -it \
 -v $SCRIPTPATH/cnpy:/workspace/cnpy \
 -v $SCRIPTPATH/finn-hlslib:/workspace/finn-hlslib \
 -v $VIVADO_PATH/include:/workspace/vivado-hlslib \
-$DOCKER_TAG bash
+-p 8888:8888 -p 8081:8081 \
+$DOCKER_TAG $DOCKER_CMD