diff --git a/notebooks/FINN-VerificationHLSCustomOp.ipynb b/notebooks/FINN-VerificationHLSCustomOp.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..f89f15112e84f832da0e7e4f290c5317b5f86ced
--- /dev/null
+++ b/notebooks/FINN-VerificationHLSCustomOp.ipynb
@@ -0,0 +1,421 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# FINN - Verification of an HLSCustomOp node\n",
+ "-----------------------------------------------------------------\n",
+ "This notebook is about the verification flow and options for FINN custom operation nodes. \n",
+ "\n",
+ "Following showSrc function is used to print the source code of function calls in the Jupyter notebook:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import inspect\n",
+ "\n",
+ "def showSrc(what):\n",
+ " print(\"\".join(inspect.getsourcelines(what)[0]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Outline\n",
+ "-------------\n",
+ "* Example model (sliding window function)\n",
+ "* c++ high level simulation\n",
+ "* Vivado IP synthesis and pyverilator execution flow"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example model\n",
+ "To show the possibilities we have to verify a FINN HLSCustomOp node, an example model with the [sliding window function] (https://finn-hlslib.readthedocs.io/en/latest/library/swg.html) of the finn-hlslib is used. For that a corresponding ONNX node is created. The ONNX node contains all the template parameters of the corresponding finn-hlslib function as attributes. The function is shown below."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the next step the individual parameters are defined. At first the class 'DataType' is imported from FINN to be able to use data types like bipolar. With the member function `bitwidth()` the parameter `Input_precision` can be derived directly from this data type. The other parameters are set to reasonable values. The output dimension can be calculated using the input dimension, the kernel size and the value for stride."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.core.datatype import DataType\n",
+ "idt = DataType.BIPOLAR # input data type\n",
+ "ip = idt.bitwidth() # input precision\n",
+ "k = 2 # kernel size\n",
+ "ifm_dim = 4 # input dimension\n",
+ "ifm_ch = 1 # input channels\n",
+ "stride = 2 # stride\n",
+ "simd = ifm_ch # simd\n",
+ "\n",
+ "# output dimension\n",
+ "ofm_dim = int(((ifm_dim - k) / stride) + 1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "An additional variable is defined to be able to infer the shape of the output tensor. Furthermore the output data type is set to the same value as the input data type."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "out_pix = ofm_dim * ofm_dim\n",
+ "odt = idt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To create an ONNX node, first TensorProto and helper are imported from ONNX. These can be used to create tensors, nodes, graphs and models in ONNX. After importing, the input and output tensors can be created."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from onnx import TensorProto, helper\n",
+ "\n",
+ "inp = helper.make_tensor_value_info(\n",
+ " \"inp\", TensorProto.FLOAT, [1, ifm_ch, ifm_dim, ifm_dim]\n",
+ ")\n",
+ "outp = helper.make_tensor_value_info(\n",
+ " \"outp\", TensorProto.FLOAT, [1, out_pix, k * k * ifm_ch]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now the node can be built. This node is directly integrated into a graph environment and from this the ONNX model is created. For more information about the creation and manipulation of an ONNX model, please refer to jupyter notebook [FINN-HowToWorkWithONNX](FINN-HowToWorkWithONNX.ipynb)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "SlidingWindow_node = helper.make_node(\n",
+ " \"ConvolutionInputGenerator\",\n",
+ " [\"inp\"],\n",
+ " [\"outp\"],\n",
+ " domain=\"finn\",\n",
+ " backend=\"fpgadataflow\",\n",
+ " ConvKernelDim=k,\n",
+ " IFMChannels=ifm_ch,\n",
+ " Input_precision=ip,\n",
+ " IFMDim=ifm_dim,\n",
+ " OFMDim=ofm_dim,\n",
+ " SIMD=simd,\n",
+ " Stride=stride,\n",
+ " inputDataType=idt.name,\n",
+ " outputDataType=odt.name,\n",
+ " )\n",
+ "graph = helper.make_graph(\n",
+ " nodes=[SlidingWindow_node],\n",
+ " name=\"slidingwindow_graph\",\n",
+ " inputs=[inp],\n",
+ " outputs=[outp],\n",
+ " )\n",
+ "\n",
+ "model = helper.make_model(graph, producer_name=\"slidingwindow-model\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "FINN provides a thin wrapper around the ONNX model with a lot of helper functions that can be used by importing the class `ModelWrapper`. More information about `ModelWrapper` can be found in Jupyter notebook [FINN-ModelWrapper](FINN-ModelWrapper.ipynb). Here it is used to assign data types to the tensors FINN."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.core.modelwrapper import ModelWrapper\n",
+ "\n",
+ "model = ModelWrapper(model)\n",
+ "\n",
+ "model.set_tensor_datatype(\"inp\", idt)\n",
+ "model.set_tensor_datatype(\"outp\", odt)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "What the model looks like can be visualized with netron. Netron is a visualizer for neural network, deep learning and machine learning models. For this the model is first saved."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model.save(\"original_model.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Serving 'original_model.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "import netron\n",
+ "netron.start('original_model.onnx', port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now that we have the model, we can use various features of FINN to manipulate it. The basic principle of FINN is that there are transformations and analysis passes that can be applied to a model. A transformation pass changes a given model an returns the changed model. An analysis pass traverses the graph structure and produces information about certain properties. It returns a dictionary of named properties.\n",
+ "\n",
+ "The following section describes the transformation passes that can be used to verify an HLSCustomOp node. Firstly the verification with a c++ high level simulation is shown and afterwards with a Vivado IP synthesis and pyverilator execution flow."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### c++ high level simulation\n",
+ "\n",
+ "First, an additional attribute must be set to specify which of the two verification types should be used when executing the node. This is done with the transformation pass `SetSimMode`, to which the desired mode is passed. After that the transformation pass `CodeGen_npysim` can be applied. With this transformation c++ code is generated and stored in a temporary directory. In addition, a further attribute is set, which contains the path to this directory."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.fpgadataflow.set_sim_mode import SetSimMode\n",
+ "from finn.transformation.fpgadataflow.codegen_npysim import CodeGen_npysim\n",
+ "\n",
+ "model = model.transform(SetSimMode(\"npysim\"))\n",
+ "model = model.transform(CodeGen_npysim())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If you now save the model again and display it, these changes can be seen."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving 'modified_model.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "model.save(\"modified_model.onnx\")\n",
+ "netron.start('modified_model.onnx', port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The next step is to create the executable from the .cpp file using the `Compile` transformation. The path to the executable is also stored in a new attribute."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.fpgadataflow.compile import Compile\n",
+ "model = model.transform(Compile())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "All required files are now available and we can execute the node. This is done with the `execute_onnx` function, which gets the model and an input dictionary. That means we have to create an input tensor first. For this we use a numpy array."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[[[-1. -1. 1. 1.]\n",
+ " [-1. -1. -1. -1.]\n",
+ " [ 1. -1. 1. -1.]\n",
+ " [ 1. 1. 1. -1.]]]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "import numpy as np\n",
+ "x = np.asarray([-1, -1, 1, 1, -1, -1, -1, -1, 1, -1, 1, -1, 1, 1, 1, -1], dtype=np.float32).reshape(1, ifm_ch, ifm_dim, ifm_dim)\n",
+ "print(x)\n",
+ "input_dict = {\"inp\": (x + 1) /2}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To be able to use `execute_onnx()` `onnx_exec` must be imported. Inside `execute_onnx()` the attribute `sim_mode` is read and if \"npysim\" is selected, the input array is saved in a .npy file and the previously created executable is executed. The output is saved in another .npy file and is read by `execute_onnx()` and saved as output."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[[-1. -1. -1. -1.]\n",
+ " [ 1. 1. -1. -1.]\n",
+ " [ 1. -1. 1. 1.]\n",
+ " [ 1. -1. 1. -1.]]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "import finn.core.onnx_exec as oxe\n",
+ "y = oxe.execute_onnx(model, input_dict)[\"outp\"]\n",
+ "print(y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A different transformation flow can now be used for verification. This will be discussed in the next section."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/im2col_finnhlslib.PNG b/notebooks/im2col_finnhlslib.PNG
new file mode 100755
index 0000000000000000000000000000000000000000..4df7c3041426576fe5b422aba345a52b3d3ab51c
Binary files /dev/null and b/notebooks/im2col_finnhlslib.PNG differ