diff --git a/docker/Jenkinsfile b/docker/Jenkinsfile
index e64280222a6d2e558f00d20a25a4a79d55526a97..80be261fb3da057186259598f84d915176577a5d 100644
--- a/docker/Jenkinsfile
+++ b/docker/Jenkinsfile
@@ -35,7 +35,7 @@ pipeline {
stage('Test') {
steps {
sh """
- docker run --name $DOCKER_INST_NAME \
+ docker run --name $DOCKER_INST_NAME --init \
--hostname $DOCKER_INST_NAME \
-v ${params.VIVADO_PATH}:${params.VIVADO_PATH}:ro \
-e NUM_DEFAULT_WORKERS=${params.NUM_DEFAULT_WORKERS} \
diff --git a/docs/finn/example_networks.rst b/docs/finn/example_networks.rst
index e4193668b4c77136c3acc70275260bc4e99de4e8..0f919f6df8d3bbaea8e5dc266095322567aa1401 100644
--- a/docs/finn/example_networks.rst
+++ b/docs/finn/example_networks.rst
@@ -18,19 +18,19 @@ version, this is indicated by an x mark in the table.
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
| FINN step | Basic test | TFC-w1a1 | TFC-w1a2 | TFC-w2a2 | CNV-w1a1 | CNV-w1a2 | CNV-w2a2 |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
-| Export/Import | x | x | x | x | x | | |
+| Export/Import | x | x | x | x | x | x | x |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
| Streamlining | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
| Convert to HLS layers | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
-| Stitched IP | x | x | x | x | | | |
+| Stitched IP | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
-| Hardware test | x | x | x | | | | |
+| Hardware test | x | x | x | | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
| npysim | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
-| rtlsim node-by-node | x | x | x | x | | | |
+| rtlsim node-by-node | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
-| rtlsim stitched IP | x | x | x | x | | | |
+| rtlsim stitched IP | x | x | x | x | x | | |
+-----------------------+------------+----------+----------+----------+----------+----------+----------+
diff --git a/notebooks/end2end_example/cnv-mp-fc.png b/notebooks/end2end_example/cnv-mp-fc.png
new file mode 100644
index 0000000000000000000000000000000000000000..1d1fb0045e764be2ecf6773b3701553a928fbd23
Binary files /dev/null and b/notebooks/end2end_example/cnv-mp-fc.png differ
diff --git a/notebooks/end2end_example/cnv_end2end_example.ipynb b/notebooks/end2end_example/cnv_end2end_example.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..eda58eebfd666e90e2c79c11290e06d3632dc756
--- /dev/null
+++ b/notebooks/end2end_example/cnv_end2end_example.ipynb
@@ -0,0 +1,776 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# End-to-End FINN Flow for a Simple Convolutional Net\n",
+ "-----------------------------------------------------------------\n",
+ "\n",
+ "In this notebook, we will go through the FINN steps needed to take a binarized convolutional network all the way down to a heterogeneous streaming dataflow accelerator running on the FPGA. \n",
+ "\n",
+ "It's recommended to go through the simpler [end-to-end notebook for a fully connected network](tfc_end2end_example.ipynb) first, since many steps here are very similar and we will focus on what is done differently for convolutions.\n",
+ "\n",
+ "This notebook is quite lengthy, and some of the cells (involving Vivado synthesis) may take up to an hour to finish running. To let you save and resume your progress, we will save the intermediate ONNX models that are generated in the various steps to disk, so that you can jump back directly to where you left off."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Quick Introduction to the CNV-w1a1 Network\n",
+ "\n",
+ "The particular quantized neural network (QNN) we will be targeting in this notebook is referred to as CNV-w1a1 and it classifies 32x32 RGB images into one of ten CIFAR-10 classes. All weights and activations in this network are quantized to bipolar values (either -1 or +1), with the exception of the input (which is RGB with 8 bits per channel) and the final output (which is 32-bit numbers). It first appeared in the original [FINN paper](https://arxiv.org/abs/1612.07119) from ISFPGA'17 with the name CNV, as a variant of the binarized convolutional network from the [BinaryNet paper](https://arxiv.org/abs/1602.02830), in turn inspired by the VGG-11 topology which was the runner-up for the 2014 [ImageNet Large Scale Visual Recognition Challenge](http://www.image-net.org/challenges/LSVRC/).\n",
+ "\n",
+ "\n",
+ "You'll have a chance to interactively examine the layers that make up the network in Netron in a moment, so that's enough about the network for now. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Quick Recap of the End-to-End Flow\n",
+ "\n",
+ "The FINN compiler comes with many *transformations* that modify the ONNX representation of the network according to certain patterns. This notebook will demonstrate a *possible* sequence of such transformations to take a particular trained network all the way down to hardware, as shown in the figure below."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The white fields show the state of the network representation in the respective step. The colored fields represent the transformations that are applied to the network to achieve a certain result. The diagram is divided into 5 sections represented by a different color, each of it includes several flow steps. The flow starts in top left corner with Brevitas export (green section), followed by the preparation of the network (blue section) for the Vivado HLS synthesis and Vivado IPI stitching (orange section), and finally building a PYNQ overlay bitfile and testing it on a PYNQ board (yellow section).\n",
+ "There is an additional section for functional verification (red section) on the left side of the diagram, which we will not cover in this notebook. For details please take a look in the verification notebook which you can find [here](tfc_end2end_verification.ipynb)\n",
+ "\n",
+ "\n",
+ "We will use the helper function `showInNetron` to show the ONNX model at the current transformation step. The Netron displays are interactive, but they only work when running the notebook actively and not on GitHub (i.e. if you are viewing this on GitHub you'll only see blank squares)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.util.basic import make_build_dir\n",
+ "from finn.util.visualization import showInNetron\n",
+ " \n",
+ "build_dir = \"/workspace/finn\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1. Brevitas Export, FINN Import and Tidy-Up\n",
+ "\n",
+ "Similar to what we did in the TFC-w1a1 end-to-end notebook, we will start by exporting the [pretrained CNV-w1a1 network](https://github.com/maltanar/brevitas_cnv_lfc) to ONNX, importing that into FINN and running the \"tidy-up\" transformations to have a first look at the topology."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/workspace/brevitas_cnv_lfc/training_scripts/models/CNV.py:112: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.\n",
+ " x = 2.0 * x - torch.tensor([1.0]).to(self.device)\n"
+ ]
+ }
+ ],
+ "source": [
+ "import onnx\n",
+ "from finn.util.test import get_test_model_trained\n",
+ "import brevitas.onnx as bo\n",
+ "from finn.core.modelwrapper import ModelWrapper\n",
+ "from finn.transformation.double_to_single_float import DoubleToSingleFloat\n",
+ "from finn.transformation.infer_shapes import InferShapes\n",
+ "from finn.transformation.fold_constants import FoldConstants\n",
+ "from finn.transformation.general import GiveReadableTensorNames, GiveUniqueNodeNames\n",
+ "\n",
+ "cnv = get_test_model_trained(\"CNV\", 1, 1)\n",
+ "bo.export_finn_onnx(cnv, (1, 3, 32, 32), build_dir + \"/end2end_cnv_w1a1_export.onnx\")\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_export.onnx\")\n",
+ "model = model.transform(DoubleToSingleFloat())\n",
+ "model = model.transform(InferShapes())\n",
+ "model = model.transform(FoldConstants())\n",
+ "model = model.transform(GiveUniqueNodeNames())\n",
+ "model = model.transform(GiveReadableTensorNames())\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_tidy.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now that the model is exported, let's have a look at its layer structure with Netron. Remember that the visualization below is interactive, you can click on the individual nodes and view the layer attributes, trained weights and so on."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Serving '/workspace/finn/end2end_cnv_w1a1_tidy.onnx' at http://0.0.0.0:8081\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"400\"\n",
+ " src=\"http://0.0.0.0:8081/\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x7f7b24ef8b00>"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "showInNetron(build_dir+\"/end2end_cnv_w1a1_tidy.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can see that the network is composed of a repeating convolution-convolution-maxpool layer pattern to extract features using 3x3 convolution kernels (with weights binarized) and `Sign` activations, followed by fully connected layers acting as the classifier. Also notice the initial `MultiThreshold` layer at the beginning of the network, which is quantizing float inputs to 8-bit ones."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2. How FINN Implements Convolutions: Lowering and Streamlining\n",
+ "\n",
+ "In FINN, we implement convolutions with the *lowering* approach: we convert them to matrix-matrix multiply operations, where one of the matrices is generated by sliding a window over the input image. You can read more about the sliding window operator and how convolution lowering works [in this notebook](https://github.com/maltanar/qnn-inference-examples/blob/master/3-convolutional-binarized-gtsrb.ipynb). The streaming dataflow architecture we will end up with is going to look something like this figure from the [FINN-R paper](https://arxiv.org/abs/1809.04570):\n",
+ "\n",
+ "\n",
+ "\n",
+ "Note how the convolution layer looks very similar to the fully connected one in terms of the matrix-vector-threshold unit (MVTU), but now the MVTU is preceded by a sliding window unit that produces the matrix from the input image. All of these building blocks, including the `MaxPool` layer you see in this figure, exist as templated Vivado HLS C++ functions in [finn-hlslib](https://github.com/Xilinx/finn-hlslib).\n",
+ "\n",
+ "\n",
+ "To target this kind of hardware architecture with our network we'll apply a convolution lowering transformation, in addition to streamlining. You may recall the *streamlining transformation* that we applied to the TFC-w1a1 network, which is a series of mathematical simplifications that allow us to get rid of floating point scaling operations by implementing few-bit activations as thresholding operations. **The current implementation of streamlining is highly network-specific and may not work for your network if its topology is very different than the example network here. We hope to rectify this in future releases.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.streamline import Streamline\n",
+ "from finn.transformation.lower_convs_to_matmul import LowerConvsToMatMul\n",
+ "from finn.transformation.bipolar_to_xnor import ConvertBipolarMatMulToXnorPopcount\n",
+ "import finn.transformation.streamline.absorb as absorb\n",
+ "from finn.transformation.streamline.reorder import MakeMaxPoolNHWC\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_tidy.onnx\")\n",
+ "model = model.transform(Streamline())\n",
+ "model = model.transform(LowerConvsToMatMul())\n",
+ "model = model.transform(MakeMaxPoolNHWC())\n",
+ "model = model.transform(absorb.AbsorbTransposeIntoMultiThreshold())\n",
+ "model = model.transform(ConvertBipolarMatMulToXnorPopcount())\n",
+ "model = model.transform(Streamline())\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_streamlined.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We won't go into too much detail about what happens in each transformation and why they are called in the particular order they are (feel free to visualize the intermediate steps using Netron yourself if you are curious) but here is a brief summmmary:\n",
+ "\n",
+ "* `Streamline` moves floating point scaling and addition operations closer to the input of the nearest thresholding activation and absorbs them into thresholds\n",
+ "* `LowerConvsToMatMul` converts ONNX `Conv` nodes into sequences of `Im2Col, MatMul` nodes as discussed above. `Im2Col` is a custom FINN ONNX high-level node type that implements the sliding window operator.\n",
+ "* `MakeMaxPoolNHWC` and `AbsorbTransposeIntoMultiThreshold` convert the *data layout* of the network into the NHWC data layout that finn-hlslib primitives use. NCHW means the tensor dimensions are ordered as `(N : batch, H : height, W : width, C : channels)` (assuming 2D images). The ONNX standard ops normally use the NCHW layout, but the ONNX intermediate representation itself does not dictate any data layout.\n",
+ "* You may recall `ConvertBipolarMatMulToXnorPopcount` from the TFC-w1a1 example, which is needed to implement bipolar-by-bipolar (w1a1) networks correctly using finn-hlslib.\n",
+ "\n",
+ "Let's visualize the streamlined and lowered network with Netron. Observe how all the `Conv` nodes have turned into pairs of `Im2Col, MatMul` nodes, and many nodes including `BatchNorm, Mul, Add` nodes have disappeared and replaced with `MultiThreshold` nodes."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving '/workspace/finn/end2end_cnv_w1a1_streamlined.onnx' at http://0.0.0.0:8081\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"400\"\n",
+ " src=\"http://0.0.0.0:8081/\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x7f0cb1098f28>"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "showInNetron(build_dir+\"/end2end_cnv_w1a1_streamlined.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 3. Partitioning, Conversion to HLS Layers and Folding\n",
+ "\n",
+ "The next steps will be (again) very similar to what we did for the TFC-w1a1 network. We'll first convert the layers that we can put into the FPGA into their HLS equivalents and separate them out into a *dataflow partition*:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import finn.transformation.fpgadataflow.convert_to_hls_layers as to_hls\n",
+ "from finn.transformation.fpgadataflow.create_dataflow_partition import (\n",
+ " CreateDataflowPartition,\n",
+ ")\n",
+ "from finn.transformation.move_reshape import MoveReshape\n",
+ "from finn.custom_op.registry import getCustomOp\n",
+ "\n",
+ "# choose the memory mode for the MVTU units, decoupled or const\n",
+ "mem_mode = \"decoupled\"\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_streamlined.onnx\")\n",
+ "model = model.transform(to_hls.InferBinaryStreamingFCLayer(mem_mode))\n",
+ "model = model.transform(to_hls.InferQuantizedStreamingFCLayer(mem_mode))\n",
+ "model = model.transform(to_hls.InferConvInpGen())\n",
+ "model = model.transform(to_hls.InferStreamingMaxPool())\n",
+ "# get rid of Reshape(-1, 1) operation between hlslib nodes\n",
+ "model = model.transform(MoveReshape())\n",
+ "parent_model = model.transform(CreateDataflowPartition())\n",
+ "parent_model.save(build_dir + \"/end2end_cnv_w1a1_dataflow_parent.onnx\")\n",
+ "sdp_node = parent_model.get_nodes_by_op_type(\"StreamingDataflowPartition\")[0]\n",
+ "sdp_node = getCustomOp(sdp_node)\n",
+ "dataflow_model_filename = sdp_node.get_nodeattr(\"model\")\n",
+ "# save the dataflow partition with a different name for easier access\n",
+ "dataflow_model = ModelWrapper(dataflow_model_filename)\n",
+ "dataflow_model.save(build_dir + \"/end2end_cnv_w1a1_dataflow_model.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Notice the additional `MoveReshape` transformation that was not used for TFC-w1a1. In the last Netron visualization you may have noticed a `Reshape` operation towards the end of the network where the convolutional part of the network ends and the fully-connected layers started. That `Reshape` is essentialy a tensor flattening operation, which we can remove for the purposes of hardware implementation. We can examine the contents of the dataflow partition with Netron, and observe the `ConvolutionInputGenerator`, `StreamingFCLayer_Batch` and `StreamingMaxPool_Batch` nodes that implement the sliding window, matrix multiply and maxpool operations in hlslib. *Note that the StreamingFCLayer instances following the ConvolutionInputGenerator nodes are really implementing the convolutions, despite the name. The final three StreamingFCLayer instances implement actual FC layers.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving '/workspace/finn/end2end_cnv_w1a1_dataflow_model.onnx' at http://0.0.0.0:8081\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"400\"\n",
+ " src=\"http://0.0.0.0:8081/\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x7f0cb063e208>"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "showInNetron(build_dir + \"/end2end_cnv_w1a1_dataflow_model.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we have to set the *folding factors* for certain layers to adjust the performance of our accelerator, similar to the TFC-w1a1 example. We'll also set the desired FIFO depths around those layers, which are important to achieve full throughput in the accelerator."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.fpgadataflow.insert_dwc import InsertDWC\n",
+ "from finn.transformation.fpgadataflow.insert_tlastmarker import InsertTLastMarker\n",
+ "from finn.transformation.fpgadataflow.insert_fifo import InsertFIFO\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_dataflow_model.onnx\")\n",
+ "fc_layers = model.get_nodes_by_op_type(\"StreamingFCLayer_Batch\")\n",
+ "# each tuple is (PE, SIMD, in_fifo_depth) for a layer\n",
+ "folding = [\n",
+ " (16, 3, 128),\n",
+ " (32, 32, 128),\n",
+ " (16, 32, 128),\n",
+ " (16, 32, 128),\n",
+ " (4, 32, 81),\n",
+ " (1, 32, 2),\n",
+ " (1, 4, 2),\n",
+ " (1, 8, 128),\n",
+ " (5, 1, 3),\n",
+ "]\n",
+ "for fcl, (pe, simd, ififodepth) in zip(fc_layers, folding):\n",
+ " fcl_inst = getCustomOp(fcl)\n",
+ " fcl_inst.set_nodeattr(\"PE\", pe)\n",
+ " fcl_inst.set_nodeattr(\"SIMD\", simd)\n",
+ " fcl_inst.set_nodeattr(\"inFIFODepth\", ififodepth)\n",
+ "\n",
+ "# use same SIMD values for the sliding window operators\n",
+ "swg_layers = model.get_nodes_by_op_type(\"ConvolutionInputGenerator\")\n",
+ "for i in range(len(swg_layers)):\n",
+ " swg_inst = getCustomOp(swg_layers[i])\n",
+ " simd = folding[i][1]\n",
+ " swg_inst.set_nodeattr(\"SIMD\", simd)\n",
+ "\n",
+ "model = model.transform(InsertDWC())\n",
+ "model = model.transform(InsertFIFO())\n",
+ "model = model.transform(InsertTLastMarker())\n",
+ "model = model.transform(GiveUniqueNodeNames())\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_folded.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Below we visualize in Netron to observe the `StreamingDataWidthConverter` and `StreamingFIFO` nodes that have been inserted into graph, as well as the folding factors in the `PE` and `SIMD` attributes of each `StreamingFCLayer_Batch`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving '/workspace/finn/end2end_cnv_w1a1_folded.onnx' at http://0.0.0.0:8081\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"400\"\n",
+ " src=\"http://0.0.0.0:8081/\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x7f0cb1098748>"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "showInNetron(build_dir + \"/end2end_cnv_w1a1_folded.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Our network is now ready and we can start with the hardware generation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 4. Hardware Generation\n",
+ "\n",
+ "From this point onward, the steps we have to follow do not depend on the particular network and will be exactly the same as the TFC-w1a1 example. We first proceed with HLS synthesis, **which may take 10-20 minutes depending on your host computer**."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.fpgadataflow.codegen_ipgen import CodeGen_ipgen\n",
+ "from finn.transformation.fpgadataflow.hlssynth_ipgen import HLSSynth_IPGen\n",
+ "from finn.util.basic import pynq_part_map\n",
+ "\n",
+ "test_pynq_board = \"Pynq-Z1\"\n",
+ "test_fpga_part = pynq_part_map[test_pynq_board]\n",
+ "target_clk_ns = 5\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_folded.onnx\")\n",
+ "model = model.transform(CodeGen_ipgen(test_fpga_part, target_clk_ns))\n",
+ "model = model.transform(HLSSynth_IPGen())\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_ipgen.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once the HLS synthesis is complete, we can stitch together the generated IP blocks into a larger IP that is the implementation of our network:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from finn.transformation.fpgadataflow.replace_verilog_relpaths import (\n",
+ " ReplaceVerilogRelPaths,\n",
+ ")\n",
+ "from finn.transformation.fpgadataflow.codegen_ipstitch import CodeGen_ipstitch\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_ipgen.onnx\")\n",
+ "model = model.transform(ReplaceVerilogRelPaths())\n",
+ "model = model.transform(CodeGen_ipstitch(test_fpga_part))\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_ipstitch.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Finally, we create a PYNQ project that includes the hardware \"shell\" that will support our accelerator, including the data movers, and run Vivado synthesis, **which may take around 30 minutes depending on your host computer.**\n",
+ "\n",
+ "*If you'd like to watch the progress, you can open the generated project file (printed below) with the Vivado GUI.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Vivado synthesis project is at /tmp/finn_dev_maltanar/vivado_pynq_proj_96qtjweo/resizer.xpr\n"
+ ]
+ }
+ ],
+ "source": [
+ "from finn.transformation.fpgadataflow.make_pynq_proj import MakePYNQProject\n",
+ "from finn.transformation.fpgadataflow.synth_pynq_proj import SynthPYNQProject\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_ipstitch.onnx\")\n",
+ "model = model.transform(MakePYNQProject(test_pynq_board))\n",
+ "vivado_proj = model.get_metadata_prop(\"vivado_pynq_proj\")\n",
+ "print(\"Vivado synthesis project is at %s/resizer.xpr\" % vivado_proj)\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_pynqproj.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_pynqproj.onnx\")\n",
+ "model = model.transform(SynthPYNQProject())\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_synth.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 5. Deployment and Remote Execution\n",
+ "\n",
+ "Now that we're done with the hardware generation, we can generate a Python driver for accelerator and copy the necessary files onto our PYNQ board."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from finn.transformation.fpgadataflow.make_pynq_driver import MakePYNQDriver\n",
+ "from finn.transformation.fpgadataflow.make_deployment import DeployToPYNQ\n",
+ "\n",
+ "# set up the following values according to your own environment\n",
+ "# FINN will use ssh to deploy and run the generated accelerator\n",
+ "ip = os.getenv(\"PYNQ_IP\", \"192.168.1.99\")\n",
+ "username = os.getenv(\"PYNQ_USERNAME\", \"xilinx\")\n",
+ "password = os.getenv(\"PYNQ_PASSWORD\", \"xilinx\")\n",
+ "port = os.getenv(\"PYNQ_PORT\", 22)\n",
+ "target_dir = os.getenv(\"PYNQ_TARGET_DIR\", \"/home/xilinx/finn\")\n",
+ "\n",
+ "model = ModelWrapper(build_dir + \"/end2end_cnv_w1a1_synth.onnx\")\n",
+ "model = model.transform(MakePYNQDriver())\n",
+ "model = model.transform(DeployToPYNQ(ip, port, username, password, target_dir))\n",
+ "model.save(build_dir + \"/end2end_cnv_w1a1_pynq_deploy.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "total 4260\r\n",
+ "-rw-r--r-- 1 xilinx xilinx 6380 May 7 15:14 driver.py\r\n",
+ "drwxr-xr-x 4 xilinx xilinx 4096 May 7 15:14 finn\r\n",
+ "-rw-r--r-- 1 xilinx xilinx 4045675 May 7 15:14 resizer.bit\r\n",
+ "-rw-r--r-- 1 xilinx xilinx 302015 May 7 15:14 resizer.hwh\r\n"
+ ]
+ }
+ ],
+ "source": [
+ "! sshpass -p {password} ssh {username}@{ip} -p {port} 'ls -l {target_dir}/*'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We only have two more steps to be able to remotely execute the deployed bitfile with some test data from the CIFAR-10 dataset. Let's load up some test data that comes bundled with FINN -- and before you ask, that's supposed to be a cat (CIFAR-10 class number 3)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "<matplotlib.image.AxesImage at 0x7f0c2b2c6908>"
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAe7ElEQVR4nO2daYyc13Wm31NfLb1vbLLZXEVJlBVZiSmF1tiJRpGdcaAoCWQDgccewFAAIwqCCIiBzA/BA4w9wPxwBmMb/jHwgB5rrBgeyxrbgoREyNiWgwiGHUnURi3UQnGRSDbZJJu9d+1nflTJQ2nue7vJZlfTvu8DEKy+p+/3nbr1nfqq71vnHHN3CCF+/cmttwNCiM6gYBciERTsQiSCgl2IRFCwC5EICnYhEiG/mslmdgeArwHIAPwPd/9S7Pf7u/O+YaAYPlb8PBftW0xSdHBb9FxkWvR4/Ghxo8feh2P+h20WOxmZAwAxZfbSZFvuR+xo7hd/DbSOydaD04w+6UvzI/bsmKUZcYP5OLNQx1KlEXTykoPdzDIA/w3AxwAcB/C0mT3q7q+wORsGivjCv7s+fDxv0nMVC2E3LccDolqtUFu9UePnKobfjACg0Qz76JFXxXINastl1ASv9fJjgh+zUCwHx7PIS2057n+jWae2Wp2/Zs0mCQrjftTD1ygAoMKOh+UCN+xj7E29WuXXR6MRWcfINZyLvGZVcl0t8KXHYjV8vG//5ETEh0vnFgCH3P2wu1cBPAjgrlUcTwixhqwm2LcCePuCn4+3x4QQVyBrvkFnZveY2X4z2z+/FPlcIoRYU1YT7CcAbL/g523tsXfh7vvcfa+77+3rXtV+oBBiFawm2J8GsNvMdplZEcCnADx6edwSQlxuLvlW6+51M7sXwP9BS3q7391fjs6BoUreX9yX+ESyW1kC37HOgW915/ORHfJLULyswCdVqlVqqzcjPkaktyyyi58n06zJd5hR58pFbBe5GfG/al3B8UZW4nNix2vw9bAm99GImtAVec3yxm25fES5qEXW2PifsE7W2CM6Q5aFfYwpE6v6XO3ujwF4bDXHEEJ0Bn2DTohEULALkQgKdiESQcEuRCIo2IVIhA5/y8XhLLHCufzjjfAca3CpplnjklfWHZFxwJMZmOTVjEg/xUKB2urObc1a5LlFzlevh20WyeTKRWQ+y3hikGdheQ0Alhphie3UOS5PLVS5j/PzfF7mfD36u8LrWDT+Og/0dFNbd4lLaM0cv+ZyURkt7CO/OoAaS76KaG+6swuRCAp2IRJBwS5EIijYhUgEBbsQidDR3XhzR75Bdt2zyG4xSeIoZZH8+HxsWzKS6EASDADQRJh6rFhYjvtRKPJd381XXUdts9Nnqe3sucXwufJ8Vz2HSHJKnV8iS879P3gs7KOXRuicWsYTm6p9fOd/fmaK2k5MTgfH+0r8eTVOhecAwI4xvo4b+vk6duVj5azC13Excgk3iAIRK7elO7sQiaBgFyIRFOxCJIKCXYhEULALkQgKdiESYR3KvYalAcsP8RlETqjHOnDkuCxXrfOEhWKkRlqjQWqFRRJTEJFCipE6aP/q33yM2p75+S+o7eT0ueD4QkRCqze45HXs+BlqO3KCdx8pDY0Hx7eN7aJzvNRPbdU8f10KfRuprV6eD46fmzxJ5/QMcXnw+PxpaiuTWokAMNbP01p6CuFEmEYtLKMCAGviE+nkpTu7EKmgYBciERTsQiSCgl2IRFCwC5EICnYhEmFV0puZHQUwB6ABoO7ue2O/37QcKrmwvDKz2EPnNUh7ouE+Lq8NZFwOy0fqsTUjshyTNWhdPcSz6BYXz1PbT//+EWo7Pc3r9Z2eD5/v2Al+rmMTb1Nb1tVHbY1sgNp6B0aD44Uefrx8F8+iK0VaMnXluHR4thpuKza+bQedU15aoLYjR7j0NjVTprbM+PO+amPYVmhwKc9YXcaI1Hs5dPaPuDvPuRRCXBHoY7wQibDaYHcAPzKzZ8zsnsvhkBBibVjtx/hb3f2EmW0C8GMze9Xdn7jwF9pvAvcAwHA/r/IhhFhbVnVnd/cT7f8nATwM4JbA7+xz973uvrevex2+ii+EALCKYDezXjPrf+cxgD8A8NLlckwIcXlZza12DMDD7a3+PID/5e7/GJtQbxrOLIUzfKZqPOvtiZ//c3D8N3ZzyeUj7w9LPwAwHClu2SSZbQCQI216cjme0dRw3rYooibhyLEj1Da1xDPAvGc4OJ71ceknNzxHbd1Dg9RWLXOpqUraKw0M89dsoI/bJk+dorbZ87zgZH8xfIl3dXOZ763zXFwq9G+itjOn3qK2vtN8jTcPhH3ptkimIinCioisfMnB7u6HAXzgUucLITqLpDchEkHBLkQiKNiFSAQFuxCJoGAXIhE62+stKyE/GC44uHiOv+/UiuGCglOLYSkMABarvDfYQJFntjVJ3622MTicZTxjr1zlEs8ZnryGs3NcAowVRBzeGM7mWmjO0jmj4D5mkUy0aoGvY3khLDWV57kfO8c2UNsikdAAYJJktgGAFcIy5cwUL+aISAHRpQWeEZcV+XUwOcuzDidIttzOUX5951hCXKzFITcJIX6dULALkQgKdiESQcEuRCIo2IVIhI7uxnd19+J9v/X/ZcECAI7/y2t0Xt9geDf+lg+HjwUAPdkxaquSnWIAyOV5UosVwjvTDedJPP2btlPb8wcOUVvfEN+Z3rrz/dTmufDucyGyc96shFtGAUC1GmmxFVmrjCRxvPzCATpnoBRpkdTLk2R6I3XtTp4K14yrE2UFADKygw8Aw/1cnZhp8KSn81PcduTUTHB8y9hmOifPFKVIdpXu7EIkgoJdiERQsAuRCAp2IRJBwS5EIijYhUiEjkpvuSyPnsGwpLTz6uvovCWiWuzYdS2dM1rj0sr0ES7L1SKJMI16ONHhlts+TufsuJp3xNr1m0ep7ZnnXqC24T4uyZycDNdPyzsv410qcMkLfBkxH0kKmSF14YZ7+bkip0IjIpWNbgxLswBQqYVfz7Pnw3IXAFikZVd/pE5ePuPhVC3zxJvDbx8Pjm8c4jLf7m3hNmoeuX/rzi5EIijYhUgEBbsQiaBgFyIRFOxCJIKCXYhEWFZ6M7P7AfwxgEl3v7E9NgLgewCuAnAUwCfdnRfZeudYuRyyUjhD6eTpg3Tent/+YHC8d5DX/MrmTlBbox5pkROpdXb47XC23K3D4bp6AICebdTU38vlmK48z+TqjtQ66yqSjK1IXbWtW8ap7ZU336S2YpHX+ZudC6/VVdt20znXXX8DtU1N8curb4BnHZ48NRkctxyv7zY0zGv8zURqyWURya67h/u4NBe+Dg6R6w0Auovhc9XqkSxFavl/fAvAHe8Zuw/A4+6+G8Dj7Z+FEFcwywZ7u9/6e78hcReAB9qPHwDAv1UihLgiuNS/2cfcfaL9+BRaHV2FEFcwq96gc3dH5JuOZnaPme03s/0zM7xmuBBibbnUYD9tZuMA0P4/vAsCwN33ufted987ODhwiacTQqyWSw32RwHc3X58N4BHLo87Qoi1YiXS23cB3A5g1MyOA/gCgC8BeMjMPgvgGIBPruRkZhkKXeG7e7nMCyJWKuG0t0JEgurp5Z8ieiMtjUoZz3rry4f7NX1r3zfpnD/5t/dSW2HhFLUVS5HspRz3cdfVW4Pjk1Mn6ZzyPM9e27xplNqmZrl0WKmGX8+rr+WZitdcyzMfZ557ltoW5uapbXYh7GO9wSWqpaVwOyYAGBoapLaGc6lsYIhn+9Wr4dczy/H+YMcnwh+mqyTLD1hBsLv7p4np95ebK4S4ctA36IRIBAW7EImgYBciERTsQiSCgl2IROhowUmYwbKwBLEYkX/Ki0vB8UKkJ9fcOZ7lhYxLbwXwQoTjQ+FMqTcO8p5tJ49zGxa5HHbs+FFqu2kz73G3dWe4GOWWSf6N5oVDvADnSCnSx26Iy3KHDx8Njo9vCUuDADA9y79hWYtIZafP8F51TbfguEWKQy5GpDfL8esqfKYWvZFClWiGs+yKFr7uAaB6LizbeqRsp+7sQiSCgl2IRFCwC5EICnYhEkHBLkQiKNiFSITOSm8OgPTsypxLK+Oj4f5wPV1cevvpAV4ocThSlG/3CM9O6iqFZZdinks1ZyaPUluzwosX7riGF7HMIs+7Z2A4OD46xgtfnpviWWMzkcy2RkTd3Ej6r+UjcmmZZH8B8WyupTLPDqsTJ9k4AJQrPAOzXuf3xw2jm6jNjF9XRQtfPyWL9B30cMZnIVL0Und2IRJBwS5EIijYhUgEBbsQiaBgFyIROrobbwYU8uFkksE+npwy1B+2WZPvVs46Tzw4e56nLIz28yXpLYZ3VBu5cI08ADh68ii1jQ3zemY7r+WtkMr8dHjqmXAbrRMTfOe/vy+8gw8AhQJv8fTyobe4I+Q+0ozcXyqR3fj5BZ4UMjTC2zXVSSLMxGlaEBm9/fx1yWc80aSnh9dELLK2XABQCyfyNBam6ZSxTf3B8XyBt7XSnV2IRFCwC5EICnYhEkHBLkQiKNiFSAQFuxCJsJL2T/cD+GMAk+5+Y3vsiwD+HMCZ9q993t0fW8kJMwtLIZs3hWuntZwkMk4kAWJ8G08k2R+Rw6aNS3aehevkDY7ypIrBAZ4AUegKyycAcFVEeusbDCcGAcD/vP/bwfHFyFrNLk1R2+ISrw1YiFw9m4fDz7s8xevdLZBEIwAYHOCvy6uvvUFtp0+fCY7PRlpGDQ3xJzbQ20dtmXNNtFDl65iRWoQbe/nxBrvCcZSP3L5Xcmf/FoA7AuNfdfc97X8rCnQhxPqxbLC7+xMA+Fu/EOJXgtX8zX6vmR0ws/vNjH8FSwhxRXCpwf51ANcA2ANgAsCX2S+a2T1mtt/M9k9P86//CSHWlksKdnc/7e4Nd28C+AYA2rXA3fe5+1533zs0xBsOCCHWlksKdjMbv+DHTwB46fK4I4RYK1YivX0XwO0ARs3sOIAvALjdzPagVVXuKIC/WMnJcrkczf4ZGObSW70RdrOU55lE1+3aQW37n+GS12zhWmpr2lxwfGwrl9deOfgv1PY7v/dn1PaLn/N5CwuRNknVs8HxyVNv0zmx9/z5GrflwaWh4Vw4y25rN/d95gyX0OoZ3xYa28RtjUY4k24p0uKpvMTr7i1EaujVm1zOq5VPUNumQjijb0sfz6Kr1MNzYnfvZYPd3T8dGP7mcvOEEFcW+gadEImgYBciERTsQiSCgl2IRFCwC5EIHS04mcvl0NsXzl4aHh2l8+oWdrOcK9I5XX0D1DY0xAsKvvX2KWq79YPvD/sxz9tJ9fSHs64AYOLEcWo79Prr1FZv8PZEOVJvcGF2hs7p3zBObTMzXIYa7OPFKN933Y3B8adfeJXOefbVo9R26+1/SG2FIpeoDh86FByfmePPK1YUs7zE5bWdY1zS7e7lBVVHRsLzPM8LcNar4cKXTrJKAd3ZhUgGBbsQiaBgFyIRFOxCJIKCXYhEULALkQgdld7cm2jWw5LH4Agv5LewFC5EuNjgfbeyjL+P7di+jdpef5lnXs0shiW2vl6eYbf9GmrCsdd58cUTJyeo7cMf/iC1LS6GpaH+LVvpnJEtvDjnW1NcKluqcMmx2BvuvzawcTudc1M/f13OnAn3QwOAo8deoLaFpbBMOT3DJbSNGzdS26Dz12VnH5dENw3wHmwFC2cCVmu8v10vkdhy4DGhO7sQiaBgFyIRFOxCJIKCXYhEULALkQgd3Y1v1muYOxfezeyO1PaqlMO7nNbk7pvxXcnREd4+6fXcYWqbnAq38DmX8V3pwT5eW+/6G3lCzuFjvGZcjXdJwvRsWO3YvXs3nbN7F5cMjk3wBJqXX36R2s6dDSenFEtcdRnu44kkx1/mqsCpc7yunZFkqSzSeivWOmwnzzPBjn6eGNSV40ktlXL4+mk2eW3DWp0cj1/2urMLkQoKdiESQcEuRCIo2IVIBAW7EImgYBciEVbS/mk7gL8DMIbWxv4+d/+amY0A+B6Aq9BqAfVJdw/3/GlTqVRw+FBY2tqx+zfovK5cWHprVnmiQL4rIoNEbP39XBrqGwjXtbv++vfROT/50WPUtjjD6931jGyitkPHJ6lt+7ZwUs6u991M55SK/DK4egdP8pme4i/3KwfDCUVN57rhiWmeSDJLkqEAoNzgsu3sdFiK3LSZJ928dY7XpxvZzuXScyXuB5r8uU3Xw8/N8/w6rZDjVcETblZyZ68D+Bt3vwHAhwD8lZndAOA+AI+7+24Aj7d/FkJcoSwb7O4+4e7Pth/PATgIYCuAuwA80P61BwB8fK2cFEKsnov6m93MrgJwE4AnAYy5/zK59xRaH/OFEFcoKw52M+sD8AMAn3P3d30/0d0d5It6ZnaPme03s/1zc7xggBBibVlRsJtZAa1A/467/7A9fNrMxtv2cQDBXSN33+fue919b2zzSwixtiwb7GZmaPVjP+juX7nA9CiAu9uP7wbwyOV3TwhxuVhJ1tvvAvgMgBfN7Pn22OcBfAnAQ2b2WQDHAHxyuQMtVup4/lBYNtpx4y10XhPhbDNjmT8A0OTpP7Nzc9Q2PX2W2jaM7AmO33nHR+icPR+4ntoe+uHD1GbGJZTBwWFq27olLCn1DQzROVk9vL4AMLKZXyLju2rUNtMdlo2ee4HXi5uY5yllXuDtvAY38yzG0WvCUlkWkbUazv14zcPtywDg0CkuDxYzfsylcjk4vhi5vOvN8PUx1+DZgcsGu7v/DADz9PeXmy+EuDLQN+iESAQFuxCJoGAXIhEU7EIkgoJdiEToaMHJcsPw+kx30Ha2wQsAeiEsTeSqvBiiE2kCAHI5btsyzrPN/vXvhDPHugpcctm1k7dd+qM//RS1ff/hf6C2s6f4856YCRcvLJcP0TlFcI1naonbDh3jWXuohmU5H+UZgsObwkUqAaAZqaTY+s4XmdcVPmbTwoUoAaAWaSs20+Dn6irwY3blufS2YOEsu1qBn8ub4fVtRCRb3dmFSAQFuxCJoGAXIhEU7EIkgoJdiERQsAuRCB2V3ioNw+vT4feXR37G+4bt2TkaHN9c5BlIPYVIttZm3n9tfJRnV11zNSlS6LyY4MSZc9R2/4NcXnv2+VeojfW+AwCaCOj8fd0b/HiNEl+PRo5LQ3mEJdZ6RBqq58JzAKArdqVGstTK1fDz9hyfk49kxGVN3tfPy1ymrIPPKzTDPmbGX7NqLex/pMWh7uxCpIKCXYhEULALkQgKdiESQcEuRCJ0dDe+AcN8Lpws8Pizr9N5b7wZbhl1x2/fQOdcs4W36TlyONyaCABu++CN1NZFEhPmqnyH+aF/fJrannvlJLUt1iOthCK7xblC+P27GanJlzO+ixzbtW40eQJQheww1xp8jhmvaVdBJCnE+XPL58lOd8bvcz09PKGlCO5/g2+4o2E81BpkYr3GX5dif7imoOX4eXRnFyIRFOxCJIKCXYhEULALkQgKdiESQcEuRCIsK72Z2XYAf4dWS2YHsM/dv2ZmXwTw5wDOtH/18+7+WPRk+Tw2jG4M2qbOc/lk4vx0cPznL/BWN43azognXFrZuJkkuwCwLCyHPbX/JTrnH376C2qrNHnNNeS59JbLXfx7dKPCk108Iss1I/JaTPJiLZQKeX7JWcYlTGT8NctH5mVZ+HyxJqNZZH1zzuXBRiTZqBmRDplmt3kzl4/7B8K2N0uRdeIe/JI6gL9x92fNrB/AM2b247btq+7+X1dwDCHEOrOSXm8TACbaj+fM7CAAXjJVCHFFclGfB83sKgA3AXiyPXSvmR0ws/vNjLcWFUKsOysOdjPrA/ADAJ9z91kAXwdwDYA9aN35v0zm3WNm+81sf32Jt0oWQqwtKwp2a1Xh/wGA77j7DwHA3U+7e8PdmwC+ASDYYN3d97n7Xnffm+/mjSCEEGvLssFuZgbgmwAOuvtXLhgfv+DXPgGAb0kLIdadlezG/y6AzwB40cyeb499HsCnzWwPWnLcUQB/sdyBzIzKJIUCl5rq5bCccPT0LJ1TWThIbbfdfB21dQ+NU9tMOSyR/POT++mcsvPMpVqdyzilEs9sa0bqoC0uhlsJxcgiGVnGk94Q6ciEEpG8YllZiNisxGXK7m5euy5PpL5aJKNsbmGB2hoRmbJS56/L4HC4jiIAjI2HbX2RwntLc+E/iT1ybaxkN/5nAEIveVRTF0JcWegbdEIkgoJdiERQsAuRCAp2IRJBwS5EInS04CTc0ayTLKpYxlAWlqGq4NlOk/MVanv2NV7o8c5FLq3MeVjuOHGefzOw1Mezq+qL3P9yhfvf0xORmkjbq9jxLMf9yEXaNcUy2JzIaB65vxQicuN8jWffVetcKmOyXCxjLyahLURab/UNcXltaCNvOVath4/52qs8q7NAshFrVe6f7uxCJIKCXYhEULALkQgKdiESQcEuRCIo2IVIhA5LbwBY1pBzuSPLwsX6ms5loUaOF/g7Osmlsvsf4vk9H719b3D8yMkzwXEAWGzEihBGZKguXjgwK3JbD+lhVuzmstbSHJeuYtlhHpGoCiRjK8vz1yx2rixSVDLWx25pcf6i58TONTQ8Qm0bxnjG5NlzU9Q2ffZUePwt3pPw2l27woaIpKg7uxCJoGAXIhEU7EIkgoJdiERQsAuRCAp2IRKho9Jbls8wMjQUtJXLXA5bWApn8hQznv1Vj8hCuUhxyyeeOkBtR06Gs+VmFnjhyKn5JWojyU4AgN7eSLZcpKhgqRR+bvmIXNfVzTPKskhGXL7Aj9kg95F6RPKyiM2d+9io8fWv1sKL3N3FpcjRDRuobXiUy2vVSOZmpRgpHkn6szXzXD5eKIevq2ZEwtadXYhEULALkQgKdiESQcEuRCIo2IVIhGV3482sC8ATAErt3/++u3/BzHYBeBDABgDPAPiMu0f2lwFvOipkF7EUedupNMK7rYWM7wbX+SYyPMdPluvmu+DHSMJLLpLcUa/xHeaYYlAul6ltIdKeKEeeG9ulB4DeIt/17Y4k0ORy3P9iV/h83T18fatVnghzdoonkjTB5+UL4fUYHuilc8ZGwooRAGzezBNhphd4nb+56fPUNj8zHRwfGuHnOnvmbHC8HkkmWsmdvQLgo+7+AbTaM99hZh8C8LcAvuru1wI4D+CzKziWEGKdWDbYvcU7eYKF9j8H8FEA32+PPwDg42vioRDisrDS/uxZu4PrJIAfA3gTwLT7L1uUHgewdW1cFEJcDlYU7O7ecPc9ALYBuAXA9Ss9gZndY2b7zWx/bZG3WBZCrC0XtRvv7tMA/gnAhwEMmf2ysfc2ACfInH3uvtfd9xZ6BlblrBDi0lk22M1so5kNtR93A/gYgINoBf2ftn/tbgCPrJWTQojVs5JEmHEAD5hZhtabw0Pu/vdm9gqAB83sPwN4DsA3lztQs9lEZSksKZUyo/N6iJfNGk8yiXQtQhNcMoolEjRJu6l6NZLA0eDPK9aCKGZrRhJhmPR2/jyXfqYi6zjQxyWqwUg9tgFSC68LXMprNLl0lbdIsk6Jv9iVcviYpTx/XWLnqi/ORGzc//npc9TWJMk6XSUuiZZZnTyLPC9qaePuBwDcFBg/jNbf70KIXwH0DTohEkHBLkQiKNiFSAQFuxCJoGAXIhEsJvFc9pOZnQFwrP3jKIBw6k5nkR/vRn68m181P3a6+8aQoaPB/q4Tm+1393DzNPkhP+THZfdDH+OFSAQFuxCJsJ7Bvm8dz30h8uPdyI9382vjx7r9zS6E6Cz6GC9EIqxLsJvZHWb2mpkdMrP71sOHth9HzexFM3vezPZ38Lz3m9mkmb10wdiImf3YzN5o/z+8Tn580cxOtNfkeTO7swN+bDezfzKzV8zsZTP76/Z4R9ck4kdH18TMuszsKTN7oe3Hf2qP7zKzJ9tx8z0z4xVXQ7h7R/8ByNAqa3U1gCKAFwDc0Gk/2r4cBTC6Due9DcDNAF66YOy/ALiv/fg+AH+7Tn58EcC/7/B6jAO4uf24H8DrAG7o9JpE/OjomgAwAH3txwUATwL4EICHAHyqPf7fAfzlxRx3Pe7stwA45O6HvVV6+kEAd62DH+uGuz8B4L21ke9Cq3An0KECnsSPjuPuE+7+bPvxHFrFUbaiw2sS8aOjeIvLXuR1PYJ9K4C3L/h5PYtVOoAfmdkzZnbPOvnwDmPuPtF+fArA2Dr6cq+ZHWh/zF/zPycuxMyuQqt+wpNYxzV5jx9Ah9dkLYq8pr5Bd6u73wzgDwH8lZndtt4OAa13drTeiNaDrwO4Bq0eARMAvtypE5tZH4AfAPicu7+rOmkn1yTgR8fXxFdR5JWxHsF+AsD2C36mxSrXGnc/0f5/EsDDWN/KO6fNbBwA2v9ProcT7n66faE1AXwDHVoTMyugFWDfcfcftoc7viYhP9ZrTdrnvugir4z1CPanAexu7ywWAXwKwKOddsLMes2s/53HAP4AwEvxWWvKo2gV7gTWsYDnO8HV5hPowJqYmaFVw/Cgu3/lAlNH14T50ek1WbMir53aYXzPbuOdaO10vgngP6yTD1ejpQS8AODlTvoB4LtofRysofW312fR6pn3OIA3APwEwMg6+fFtAC8COIBWsI13wI9b0fqIfgDA8+1/d3Z6TSJ+dHRNAPwWWkVcD6D1xvIfL7hmnwJwCMD/BlC6mOPqG3RCJELqG3RCJIOCXYhEULALkQgKdiESQcEuRCIo2IVIBAW7EImgYBciEf4vt7E0CnHQV6IAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "<Figure size 432x288 with 1 Axes>"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import pkg_resources as pk\n",
+ "import matplotlib.pyplot as plt\n",
+ "import numpy as np\n",
+ "\n",
+ "fn = pk.resource_filename(\"finn\", \"data/cifar10/cifar10-test-data-class3.npz\")\n",
+ "x = np.load(fn)[\"arr_0\"].astype(np.float32)\n",
+ "x = x / 255\n",
+ "plt.imshow(x.reshape(3, 32,32).transpose(1, 2, 0))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Recall that we partitioned our original network into a parent graph that contained the non-synthesizable nodes and a child graph that contained the bulk of the network, which we turned into a bitfile. We'll load up the parent graph, modify the `StreamingDataflowPartition` node so that it points to the deployed ONNX graph."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# point to the PYNQ-deployed model as the StreamingDataflowPartition in the parent\n",
+ "parent_model = ModelWrapper(build_dir+\"/end2end_cnv_w1a1_dataflow_parent.onnx\")\n",
+ "sdp_node = parent_model.get_nodes_by_op_type(\"StreamingDataflowPartition\")[0]\n",
+ "sdp_node = getCustomOp(sdp_node)\n",
+ "sdp_node.set_nodeattr(\"model\", build_dir + \"/end2end_cnv_w1a1_pynq_deploy.onnx\")\n",
+ "parent_model.save(build_dir+\"/end2end_cnv_w1a1_dataflow_parent_with_remote_bitfile_exec.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Finally, we can call `execute_onnx` on the parent graph, which will internally call remote execution with the bitfile once the `StreamingDataflowPartition` node is reached, grab the results, then continue executing the last portion of the network. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "from finn.core.onnx_exec import execute_onnx\n",
+ "iname = parent_model.graph.input[0].name\n",
+ "oname = parent_model.graph.output[0].name\n",
+ "ishape = parent_model.get_tensor_shape(iname)\n",
+ "input_dict = {iname: x.reshape(ishape)}\n",
+ "ret = execute_onnx(parent_model, input_dict, True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We'll pass the output of the network through a softmax function to interpret it as probabilities, and plot the per-class probabilities as a bar chart."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "<BarContainer object of 10 artists>"
+ ]
+ },
+ "execution_count": 28,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAABIEAAADCCAYAAADetdIQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAWR0lEQVR4nO3de9RlZX0f8O9PEDVotJFJVwroEELUUeNtQI0xMWpaDBXsCqxAjfGWEBJRrNWGLF3U0mRV44rWRhJFazReFl4S26miuIr1moAMCnKxKCIKxJohUbygIvLrH3sPHCbvzHtm5rzzvjP78/lnzt77Ofs8735m7/2c7977OdXdAQAAAGDfdpfVrgAAAAAAK08IBAAAADABQiAAAACACRACAQAAAEyAEAgAAABgAoRAAAAAABOw/2p98EEHHdTr169frY8HAAAA2OdcfPHFN3b3uqWWrVoItH79+mzevHm1Ph4AAABgn1NVX9neMo+DAQAAAEyAEAgAAABgAoRAAAAAABMgBAIAAACYACEQAAAAwASs2q+DAbB2rT/9A6tdhX3Gta84ZrWrAAAASdwJBAAAADAJQiAAAACACRACAQAAAEyAEAgAAABgAoRAAAAAABMgBAIAAACYACEQAAAAwAQIgQAAAAAmQAgEAAAAMAFCIAAAAIAJEAIBAAAATIAQCAAAAGAChEAAAAAAEyAEAgAAAJgAIRAAAADABAiBAAAAACZACAQAAAAwAXOFQFV1dFVdVVVXV9XpOyj3a1XVVbVxcVUEAAAAYHctGwJV1X5JzkrylCQbkpxUVRuWKHevJKcluXDRlQQAAABg98xzJ9BRSa7u7mu6+5Yk5yQ5boly/znJK5N8f4H1AwAAAGAB5gmBDk5y3cz09eO821XVI5Mc2t0f2NGKqurkqtpcVZu3bNmy05UFAAAAYNfs9sDQVXWXJK9O8u+XK9vdZ3f3xu7euG7dut39aAAAAADmNE8IdEOSQ2emDxnnbXWvJA9J8tGqujbJY5JsMjg0AAAAwNoxTwh0UZIjquqwqjogyYlJNm1d2N03dfdB3b2+u9cnuSDJsd29eUVqDAAAAMBOWzYE6u5bk5ya5Lwkn0/y7u6+oqrOrKpjV7qCAAAAAOy+/ecp1N3nJjl3m3lnbKfsE3a/WgAAAAAs0m4PDA0AAADA2icEAgAAAJgAIRAAAADABAiBAAAAACZACAQAAAAwAUIgAAAAgAkQAgEAAABMgBAIAAAAYAKEQAAAAAATIAQCAAAAmAAhEAAAAMAECIEAAAAAJkAIBAAAADABQiAAAACACRACAQAAAEyAEAgAAABgAoRAAAAAABMgBAIAAACYACEQAAAAwAQIgQAAAAAmQAgEAAAAMAFCIAAAAIAJEAIBAAAATIAQCAAAAGAC5gqBquroqrqqqq6uqtOXWH5KVV1WVZdU1SerasPiqwoAAADArlo2BKqq/ZKcleQpSTYkOWmJkOed3f3Q7n54kj9O8uqF1xQAAACAXTbPnUBHJbm6u6/p7luSnJPkuNkC3f2tmckDk/TiqggAAADA7tp/jjIHJ7luZvr6JI/etlBVPS/Ji5IckOSJC6kdAAAAAAuxsIGhu/us7j48ye8nedlSZarq5KraXFWbt2zZsqiPBgAAAGAZ84RANyQ5dGb6kHHe9pyT5GlLLejus7t7Y3dvXLdu3fy1BAAAAGC3zBMCXZTkiKo6rKoOSHJikk2zBarqiJnJY5J8cXFVBAAAAGB3LTsmUHffWlWnJjkvyX5J3tzdV1TVmUk2d/emJKdW1ZOT/DDJN5I8cyUrDQAAAMDOmWdg6HT3uUnO3WbeGTOvT1twvQAAAABYoIUNDA0AAADA2iUEAgAAAJgAIRAAAADABAiBAAAAACZACAQAAAAwAUIgAAAAgAkQAgEAAABMgBAIAAAAYAKEQAAAAAATIAQCAAAAmAAhEAAAAMAECIEAAAAAJkAIBAAAADABQiAAAACACRACAQAAAEyAEAgAAABgAoRAAAAAABMgBAIAAACYACEQAAAAwAQIgQAAAAAmQAgEAAAAMAFCIAAAAIAJEAIBAAAATIAQCAAAAGAChEAAAAAAEzBXCFRVR1fVVVV1dVWdvsTyF1XVlVX1uao6v6ruv/iqAgAAALCrlg2Bqmq/JGcleUqSDUlOqqoN2xT7bJKN3f1zSd6b5I8XXVEAAAAAdt08dwIdleTq7r6mu29Jck6S42YLdPf/6e6bx8kLkhyy2GoCAAAAsDvmCYEOTnLdzPT147zteW6SD+5OpQAAAABYrP0XubKq+o0kG5P80naWn5zk5CS53/3ut8iPBgAAAGAH5rkT6IYkh85MHzLOu5OqenKSlyY5trt/sNSKuvvs7t7Y3RvXrVu3K/UFAAAAYBfMEwJdlOSIqjqsqg5IcmKSTbMFquoRSd6QIQD6+8VXEwAAAIDdsWwI1N23Jjk1yXlJPp/k3d19RVWdWVXHjsVeleSeSd5TVZdU1abtrA4AAACAVTDXmEDdfW6Sc7eZd8bM6ycvuF4AAAAALNA8j4MBAAAAsJcTAgEAAABMgBAIAAAAYAKEQAAAAAATIAQCAAAAmAAhEAAAAMAECIEAAAAAJkAIBAAAADABQiAAAACACRACAQAAAEyAEAgAAABgAoRAAAAAABMgBAIAAACYACEQAAAAwAQIgQAAAAAmQAgEAAAAMAFCIAAAAIAJEAIBAAAATIAQCAAAAGAChEAAAAAAEyAEAgAAAJgAIRAAAADABAiBAAAAACZACAQAAAAwAUIgAAAAgAmYKwSqqqOr6qqqurqqTl9i+S9W1Weq6taqOn7x1QQAAABgdywbAlXVfknOSvKUJBuSnFRVG7Yp9tUkz0ryzkVXEAAAAIDdt/8cZY5KcnV3X5MkVXVOkuOSXLm1QHdfOy67bQXqCAAAAMBumudxsIOTXDczff04b6dV1clVtbmqNm/ZsmVXVgEAAADALtijA0N399ndvbG7N65bt25PfjQAAADApM0TAt2Q5NCZ6UPGeQAAAADsJeYJgS5KckRVHVZVByQ5Mcmmla0WAAAAAIu0bAjU3bcmOTXJeUk+n+Td3X1FVZ1ZVccmSVUdWVXXJzkhyRuq6oqVrDQAAAAAO2eeXwdLd5+b5Nxt5p0x8/qiDI+JAQAAALAG7dGBoQEAAABYHUIgAAAAgAkQAgEAAABMwFxjAgEAsLz1p39gtauwz7j2FcesdhUAYJ/jTiAAAACACRACAQAAAEyAEAgAAABgAowJBADAJBizaXGM2TQt9p3Fse+w2oRAwKrRoVgcHYppse8sjn0H1gbHtcVxXAN2RAi0AE5ai+OkBQAAMD/fRxdjKt9FhUDs0xwQF2cqB0UAAIB9lYGhAQAAACZACAQAAAAwAUIgAAAAgAkQAgEAAABMgBAIAAAAYAKEQAAAAAATIAQCAAAAmAAhEAAAAMAECIEAAAAAJkAIBAAAADABQiAAAACACRACAQAAAEyAEAgAAABgAuYKgarq6Kq6qqqurqrTl1h+t6p617j8wqpav+iKAgAAALDrlg2Bqmq/JGcleUqSDUlOqqoN2xR7bpJvdPfPJHlNklcuuqIAAAAA7Lp57gQ6KsnV3X1Nd9+S5Jwkx21T5rgkbx1fvzfJk6qqFldNAAAAAHbHPCHQwUmum5m+fpy3ZJnuvjXJTUnuu4gKAgAAALD7qrt3XKDq+CRHd/dvjdPPSPLo7j51pszlY5nrx+kvjWVu3GZdJyc5eZx8QJKrFvWHsKyDkty4bClWi/ZZu7TN2qZ91i5ts7Zpn7VL26xt2mft0jZrm/bZs+7f3euWWrD/HG++IcmhM9OHjPOWKnN9Ve2f5N5J/mHbFXX32UnOnqfGLFZVbe7ujatdD5amfdYubbO2aZ+1S9usbdpn7dI2a5v2Wbu0zdqmfdaOeR4HuyjJEVV1WFUdkOTEJJu2KbMpyTPH18cn+Ugvd4sRAAAAAHvMsncCdfetVXVqkvOS7Jfkzd19RVWdmWRzd29K8t+TvK2qrk7yjxmCIgAAAADWiHkeB0t3n5vk3G3mnTHz+vtJTlhs1Vgwj+Gtbdpn7dI2a5v2Wbu0zdqmfdYubbO2aZ+1S9usbdpnjVh2YGgAAAAA9n7zjAkEAAAAwF5OCLSXqapzq+o+O/met1TV8StVJ5KqelpVbVjhz1hfVZdvZ9mbtn5+VV1bVQetZF32FdvbprPbc5n3P6uqXrcytWNXVNUTqurnV7seU1FVL6+qF692Pdgx7bS2VNULqurzVfWO1a7L1O2ob8Xat70+b1UdW1Wnr0adpqKq7lNVv7egdT2hqt6/iHUxHyHQXqa7f7W7vzk7rwbacnU9LcmKhkA70t2/1d1Xrtbn72u2tz2rar/VqA875QlJhEB7kaqaa3xC2If8XpJf6e6nb51hP9j7aLO1q7s3dfcrVrse+7j7ZDiW3Yn9Yu8gOFjDqup/VNXFVXVFVZ08zru2qg4ar1xcVVV/meTyJIdW1Xeq6jVj+fOrat0S6zyjqi6qqsur6uyqqnH+R6vqlVX16ar6QlU9fpy/X1W9anzP56rqd/bkNlhN29n+35lZfvx4l9XPJzk2yauq6pKqOryqHl5VF4zb7H1V9c/G93x0bKPN41XAI6vqr6vqi1X1hzPrftHYRpdX1QtnqrV/Vb1jfO97q+rHZta7cYm/4TfGNr2kqt4gxFjSP9mms9tz3K/+pKouTfLYqnr2uI98OsnjVrfq01FVvznuT5dW1duq6qlVdWFVfbaq/ndV/fOqWp/klCT/bvw///jVrfW+qapeOu4Dn0zygHHe4VX1ofGY+YmqeuA4f11V/dV4Drmoqh43zn/52I6fSvK21ftr9l3baaftnZuOHOddMp7z3RmxQqrq9Ul+OskHq+qm2f1g7Nt9ZGyL86vqfuN7Dh/b7bKq+sPZvggLsV9VvXHs7324qu6xTD/uv1bV5iSnVdUJY1/t0qr6+Fhmsn3nlVRVB1bVB8ZtfXlV/fq46PlV9Zlx/9h67rn9Tu0a+uqvH/veX6iqf71qf8S+5RVJDh/PGxeN5/5NSa6sbe6wq6oXV9XLx9c/M/bbLh3b7fDZlY7no89uO5/FEgKtbc/p7kcl2ZjkBVV1322WH5Hkz7r7wd39lSQHJtnc3Q9O8rEk/3GJdb6uu4/s7ockuUeS2QPh/t19VJIXzrz3uUlu6u4jkxyZ5Ler6rBF/YFr3HLbP0nS3X+TZFOSl3T3w7v7S0n+Msnvd/fPJbksd26LW7p7Y5LXJ/mfSZ6X5CFJnlVV962qRyV5dpJHJ3lMhm3+iPG9D8jQ5g9K8q0skcBvVVUPSvLrSR7X3Q9P8qMkT99e+QlbbpsemOTC7n5Yki8l+U8Zwp9fyCre/TUlVfXgJC9L8sSxHU5L8skkj+nuRyQ5J8l/6O5rM+xXrxn3xU+sVp33VePx6cQkD0/yqxnOC8nwix/PH4+ZL07yZ+P812ZojyOT/FqSN82sbkOSJ3f3SXui7lOyg3ba3rnpL5L8zsy5ghXS3ack+bskv5zkNbnzfvCnSd46ts87kvy38W2vTfLa7n5okuv3fK33eUckOWvsP38zw7FqR/24A7p7Y3f/SZIzkvyr8dx07Lh8yn3nlXR0kr/r7oeN32M+NM6/sbsfmeTPM5x/lrI+yVFJjkny+qq6+0pXdgJOT/Kl8bzxkiSPTHJad//sMu97R4b97WEZ7tz+2tYFNVxYf32S48bvU6wQIdDa9oLx7oMLkhya4SQ16yvdfcHM9G1J3jW+fnuGL6nb+uXx6vllSZ6Y5MEzy/56/PfiDAfLJPmXSX6zqi5JcmGS+y5Rj33Vctt/SVV17yT36e6PjbPemuQXZ4psGv+9LMkV3f217v5BkmvGz/mFJO/r7u9293cytMvWOxqu6+5Pja+318ZbPSnJo5JcNLbfkzJcfeTOltumP0ryV+PrRyf5aHdv6e5bcsf+xsp6YpL3dPeNSdLd/5jkkCTnjceyl+TOxzJWzuMzHJ9u7u5vZTie3T1DR+4947HmDUl+aiz/5CSvG+dvSvLjVXXPcdmm7v7enq3+ZCzVTgdmiXNTDeMM3qu7/3ac/849X91Jm90PHps7tv/bcsf56LFJ3jO+1j6L9+XuvmR8fXGSw7Pjftzsuf9TSd5SVb+dZOvd1lPuO6+ky5L8Sg1PLjy+u28a5y/1/WVb7+7u27r7ixn62w9c2apO0qe7+8s7KlBV90pycHe/L0m6+/vdffO4+EEZLig9tbu/urJVxTN7a1RVPSFD5/mx3X1zVX00Q0d71neXWU1vs867Z7g6u7G7rxtvy5td5w/Gf3+UO/5vVIaru+ft7N+wN9vB9p/dprt6FWHrdr5t5vXW6eX2yV5melZluKL4BztXvclZbpt+v7tdGV97/jTJq7t707i/vnx1qzNpd0nyzfFq4FLLHtPd35+dWcOTyMudw2AK7Aerb7Yv9qMMY53syO1t1t2nVNWjM9xhcvF4F94k+84rrbu/UFWPzHB34x9W1fnjoqW+v/yTty8zze6bPZbdmjvfbDLPd6avjeUekeFuSVaQO4HWrnsn+cYYQDwww2NBy7lLkq2/AvZvMzwuMWvrDnjjeCV2nl8MOy/J71bVXZOkqn62qg6c4317u+1t/69X1YNqGIj738yU/3aSeyXJeGXiG3XHeCTPyPB43rw+keRpNYxNc+D4OVsfa7lfVT12fL1UG886P8nxVfWTSVJVP1FV99+JekzFzmzTC5P80vjY3l2TnLDitSNJPpLkhK2PZFbVT2TYR28Ylz9zpuzt+yIr4uMZjk/3GK/oPTXJzUm+XFUnJLf/WMHDxvIfTvL8rW+uqqWCIhZvqXb6bpY4N40/NvHt8YtsMjxGxur4m9yx/Z+eO879F2R4RCnRPnvC3P24qjq8uy/s7jOSbMlwR/dU+84rqqr+RZKbu/vtSV6V4fGjeZ1QVXcZx5n56SRXrUQdJ2ZH/a2vJ/nJsb98t4zDj3T3t5NcX1VPS5KquluN45tmeBTzmCT/Zby4xwpyJ9Da9aEkp1TV5zMcqC5YpnwydPCOqqqXJfn7DOPB3K67v1lVb8wwkPT/S3LRHOt8U4ZbKz9Tw6XbLRl+CWtft73tf3qS92fYDpuTbH2s4Zwkb6yqF2QI156Z4ZnjH8tw2+mz5/3g7v5MVb0lyafHWW/q7s/WMOjtVUmeV1VvTnJlhueft7eeK8f/Cx8eQ6sfZhh/6Cvz1mUiltqmT12qYHd/bbyD7m8znKwuWaoci9XdV1TVHyX5WFX9KMlnM9z5856q+kaGkGjreAv/K8l7q+q4DFdijQu0QOPx6V1JLs1wntl6Hnl6kj8fjzl3zXBMvDTJC5KcVVWfy9Dn+HiGwbtZQTtop+2dm56b4Rx2W4YvuzeF1fD8JH9RVS/J0M/Y2j4vTPL2qnpphv6J9ll58/bjXlVVR2S4++f8DPvc5zLNvvNKe2iG7X1bhj7t7yZ575zv/WqGfvWPJzll27tT2Xnd/Q9V9akaBoD+XobgZ+uyH1bVmRm2+Q1J/u/MW5+R5A3j8h9m5oJqd3+9hoG7P1hVz+nuC/fE3zJF1e1uuH1FVX2nu++5fEkAgEFV3XMcgy5VdXqSn+ru01a5WozGIOJ73d1VdWKSk7r7uNWuF+wNxgur7+/ueQMj2Oe5EwgAYNqOqao/yNAv/EqSZ61uddjGozIMsF4Z7kJ9zirXB4C9mDuBAAAAACbAwNAAAAAAEyAEAgAAAJgAIRAAAADABAiBAAAAACZACAQAAAAwAUIgAAAAgAn4/ybos9quolFvAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "<Figure size 1440x216 with 1 Axes>"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "def softmax(x):\n",
+ " \"\"\"Compute softmax values for each sets of scores in x.\"\"\"\n",
+ " e_x = np.exp(x - np.max(x))\n",
+ " return e_x / e_x.sum()\n",
+ "\n",
+ "logits = ret[oname].flatten()\n",
+ "prob = softmax(logits)\n",
+ "\n",
+ "classes = [\"airplane\", \"automobile\", \"bird\", \"cat\", \"deer\", \"dog\", \"frog\", \"horse\", \"ship\", \"truck\"]\n",
+ "\n",
+ "plt.figure(figsize=(20, 3)) \n",
+ "plt.bar(classes, prob)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We see that the network correctly predicts this as a class 3 (\"cat\") with high probability. This concludes our tutorial on how to take a convolutional BNN all the way down to hardware with FINN, and execute it remotely on a PYNQ board."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/run-docker.sh b/run-docker.sh
index 018bd9aa8c39666a1b9c0ef7f426587f265769f7..b7f844d314c5fb67e11e0933f42b3edfa4d96036 100755
--- a/run-docker.sh
+++ b/run-docker.sh
@@ -104,7 +104,9 @@ docker build -f docker/Dockerfile.finn_dev --tag=$DOCKER_TAG \
--build-arg NETRON_PORT=$NETRON_PORT \
.
# Launch container with current directory mounted
-docker run -t --rm --name $DOCKER_INST_NAME -it \
+# important to pass the --init flag here for correct Vivado operation, see:
+# https://stackoverflow.com/questions/55733058/vivado-synthesis-hangs-in-docker-container-spawned-by-jenkins
+docker run -t --rm --name $DOCKER_INST_NAME -it --init \
--hostname $DOCKER_INST_NAME \
-e "XILINX_VIVADO=$VIVADO_PATH" \
-e "SHELL=/bin/bash" \