diff --git a/notebooks/9-FINN-EndToEndFlow.ipynb b/notebooks/9-FINN-EndToEndFlow.ipynb
index 9e5761b833d5542655e323cbd5b26bc53dd1cf63..48e78453e2530f6b036ba3abd4fe49624fc7d55a 100644
--- a/notebooks/9-FINN-EndToEndFlow.ipynb
+++ b/notebooks/9-FINN-EndToEndFlow.ipynb
@@ -13,7 +13,7 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -116,7 +116,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 2,
"metadata": {},
"outputs": [
{
@@ -134,7 +134,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 3,
"metadata": {},
"outputs": [
{
@@ -164,7 +164,7 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
@@ -176,7 +176,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now the model is prepared and it can be processed in different ways. The principle of FINN are analysis and transformation passes, which can be applied to the model. An analysis pass extracts specific information about the model and returns it to the user in the form of a dictionary. For more details see [4-FINN-HowToAnalysisPass](4-FINN-HowToAnalysisPass.ipynb). A transformation pass changes the model and returns the changed model back to the FINN flow, for more information about transformation passes see notebook [5-FINN-HowToTransformationPass](5-FINN-HowToTransformationPass.ipynb).\n",
+ "Now the model is prepared and could be simulated using Python. How this works is described in subsection [Simulation using Python](#simpy) in the section about *Simulation & Emulation flows for functional verification*.\n",
+ "\n",
+ "The model can now also be processed in different ways. The principle of FINN are analysis and transformation passes, which can be applied to the model. An analysis pass extracts specific information about the model and returns it to the user in the form of a dictionary. For more details see [4-FINN-HowToAnalysisPass](4-FINN-HowToAnalysisPass.ipynb). A transformation pass changes the model and returns the changed model back to the FINN flow, for more information about transformation passes see notebook [5-FINN-HowToTransformationPass](5-FINN-HowToTransformationPass.ipynb).\n",
"\n",
"Since the goal in this notebook is to process the model to such an extent that a bitstream can be generated from it, the focus is on the transformations that are necessary for this. In the next section these are discussed in more detail."
]
@@ -187,17 +189,17 @@
"source": [
"## 2. Network preparation <a id='nw_prep'></a>\n",
"\n",
- "* Basic transformations\n",
- "* Streamlining\n",
- "* Conversion to HLS layers\n",
- "* Folding"
+ "* [Basic transformations](#basic_trafo)\n",
+ "* [Streamlining](#streamline)\n",
+ "* [Conversion to HLS layers](#hls_layers)\n",
+ "* [Folding](#folding)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Basic transformations\n",
+ "### Basic transformations <a id='basic_trafo'></a>\n",
"This section deals with the basic transformations, which are applied to the model like a kind of clean up. They do not appear in the diagram above, but they are applied in many steps in the FINN flow to postprocess the model after a transformation and/or to prepare it for the next transformation."
]
},
@@ -231,7 +233,7 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
@@ -256,7 +258,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 6,
"metadata": {},
"outputs": [
{
@@ -276,7 +278,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 7,
"metadata": {},
"outputs": [
{
@@ -301,7 +303,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Streamlining\n",
+ "### Streamlining <a id='streamline'></a>\n",
"Streamlining is a transformation containing several sub-transformations. The goal of streamlining is to eliminate floating point operations by moving them around, then collapsing them into one operation and in the last step transform them into multithresholding nodes. For the theoretical background see arXiv:1709.04060.\n",
"\n",
"In the following the streamlining transformation is shown."
@@ -430,14 +432,73 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Conversion to HLS layers"
+ "### Conversion to HLS layers <a id='hls_layers'></a>\n",
+ "Converts the nodes to HLS layers that correspond to the functions in [finn-hls library](https://finn-hlslib.readthedocs.io/en/latest/). In our case this transformation onverts pairs of binary XnorPopcountMatMul layers to StreamingFCLayer_Batch layers. Any immediately following MultiThreshold layers will also be absorbed into the MVTU.\n",
+ "\n",
+ "Below is the code for the transformation and the network is visualized using netron to create the new structure which now corresponds to the finn-hls library."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving 'lfc_w1_a1.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "import finn.transformation.fpgadataflow.convert_to_hls_layers as to_hls\n",
+ "model = model.transform(to_hls.InferBinaryStreamingFCLayer())\n",
+ "model.save(\"lfc_w1_a1.onnx\")\n",
+ "netron.start(\"lfc_w1_a1.onnx\", port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Each StreamingFCLayer_Batch node has two attributes that specify the degree of folding, PE and SIMD. In all nodes the values for these attributes are set as default to 1, which would correspond to a maximum folding. The user can now adjust the folding as desired. This is described in the next section.\n",
+ "\n",
+ "At this point the model can also be simulated using C++. The exact procedure is described in section [Simulation using C++](#npysim)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Folding"
+ "### Folding <a id='folding'></a>"
]
},
{
@@ -469,7 +530,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Conversion to HLS layers"
+ "### PYNQ shell project"
]
},
{
@@ -500,21 +561,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Simulation using Python"
+ "### Simulation using Python <a id='simpy'></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Simulation (npysim) using C++"
+ "### Simulation (npysim) using C++ <a id='npysim'></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Emulation (rtlsim) using Pyverilator"
+ "### Emulation (rtlsim) using Pyverilator <a id='rtlsim'></a>"
]
},
{