From ab8d58786c710d7c26eb3306e967af9fed513927 Mon Sep 17 00:00:00 2001
From: auphelia <jakobapk@web.de>
Date: Wed, 5 Feb 2020 13:53:51 +0000
Subject: [PATCH] [Notebook] Added section about HLS IP per layer
---
notebooks/9-FINN-EndToEndFlow.ipynb | 239 +++++++++++++++++++++++++---
1 file changed, 219 insertions(+), 20 deletions(-)
diff --git a/notebooks/9-FINN-EndToEndFlow.ipynb b/notebooks/9-FINN-EndToEndFlow.ipynb
index 83ff14252..3559ec7f9 100644
--- a/notebooks/9-FINN-EndToEndFlow.ipynb
+++ b/notebooks/9-FINN-EndToEndFlow.ipynb
@@ -13,7 +13,7 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
@@ -116,7 +116,7 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 1,
"metadata": {},
"outputs": [
{
@@ -134,7 +134,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -164,7 +164,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
@@ -233,7 +233,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
@@ -258,7 +258,7 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -278,7 +278,7 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 5,
"metadata": {},
"outputs": [
{
@@ -507,7 +507,7 @@
},
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": 14,
"metadata": {},
"outputs": [
{
@@ -541,7 +541,7 @@
},
{
"cell_type": "code",
- "execution_count": 25,
+ "execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
@@ -576,38 +576,237 @@
"metadata": {},
"source": [
"## 3. Vivado HLS and Vivado synthesis <a id='vivado'></a>\n",
- "* HLS IP per layer\n",
- "* Creation of stitched design\n",
- "* PYNQ shell project\n",
- "* Synthesis, place and route"
+ "* [HLS IP per layer](#hls_per_layer)\n",
+ "* [Creation of stitched design](#stitched_design)\n",
+ "* [PYNQ shell project](#pynq_shell)\n",
+ "* [Synthesis, place and route](#synth_pl_ro)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### HLS IP per layer <a id='hls_per_layer'></a>\n",
+ "This section deals with the generation of an IP block from the different layers. These can then be stitched to a block design that corresponds to the complete model. The single conversion into IP blocks allows a good transparency and we can check the functionality of each IP block and compare it with the behaviour of the corresponding ONNX node. The emulation of such an IP block is performed with PyVerilator and is described in detail in section [Emulation (rtlsim) using Pyverilator](#rtlsim)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Two transformations are required to generate HLS IP blocks for each layer: \n",
+ "* `CodeGen_ipgen` which generates the C++ code for the node and a tcl-script which starts the HLS synthesis and exports the design as IP. \n",
+ "* `HLSSynth_IPGen` which passes the tcl-script to Vivado and thus performs the actual IP generation. \n",
+ "\n",
+ "First the basic transformation `GiveUniqueNodeNames` is applied and then the two transformations necessary for the IP block creation are performed. This will take some time as Vivado is called and for each StreamingFCLayer_Batch node in the design HLS synthesis is performed and the node is exported as IP block. `CodeGen_ipgen` gets as arguments an FPGA part as string and the clock in ns as integer."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 39,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model = model.transform(GiveUniqueNodeNames())\n",
+ "\n",
+ "from finn.transformation.fpgadataflow.codegen_ipgen import CodeGen_ipgen\n",
+ "from finn.transformation.fpgadataflow.hlssynth_ipgen import HLSSynth_IPGen\n",
+ "\n",
+ "model = model.transform(CodeGen_ipgen(\"xc7z020clg400-1\", 5))\n",
+ "model = model.transform(HLSSynth_IPGen())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Each StreamingFCLayer_Batch node now has new attributes which can be examined more closely with netron."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving 'lfc_w1_a1.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "model.save(\"lfc_w1_a1.onnx\")\n",
+ "netron.start(\"lfc_w1_a1.onnx\", port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "There are two additional attributes: \n",
+ "* `code_gen_dir_ipgen` which contains the directory path where all the files generated by the ipgen transformations are stored\n",
+ "* `ipgen_path` which contains the path to the project directory in which the generated IP block is stored\n",
+ "\n",
+ "We can further investigate which files are produced by taking a look in this directory. For example for the first StreamingFCLayer_Batch node."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "hls_syn_StreamingFCLayer_Batch_0.tcl thresh.h\r\n",
+ "ipgen.sh\t\t\t top_StreamingFCLayer_Batch_0.cpp\r\n",
+ "params.h\t\t\t vivado_hls.log\r\n",
+ "project_StreamingFCLayer_Batch_0\r\n"
+ ]
+ }
+ ],
+ "source": [
+ "fc0 = model.graph.node[2]\n",
+ "fc0w = StreamingFCLayer_Batch(fc0)\n",
+ "code_gen_dir = fc0w.get_nodeattr(\"code_gen_dir_ipgen\")\n",
+ "!ls {code_gen_dir}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Directory *project_StreamingFCLayer_Batch_0* contains the project created by Vivado HLS into which the IP Block is exported, along with other files generated by Vivado HLS. If we compare it to the above visualization of the network with netron, this is exactly the name of the folder stored in the node attribute `ipgen_path`. The .cpp code that is passed to Vivado HLS can be found in the file *top_StreamingFCLayer_Batch_0.cpp*. The files *params.h* and *thresh.h* belong to that as well, they contain the values for the weights and thresholds. *vivado_hls.log* is the log file from Vivado HLS. Besides these files, the folder contains *ipgen.sh* and *hls_syn_StreamingFCLayer_Batch_0.tcl*. First we take a look at *ipgen.sh*."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 37,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "#!/bin/bash \r\n",
+ "cd /tmp/code_gen_ipgen_StreamingFCLayer_Batch_hkd3xuxq\r\n",
+ "vivado_hls /tmp/code_gen_ipgen_StreamingFCLayer_Batch_hkd3xuxq/hls_syn_StreamingFCLayer_Batch_0.tcl\r\n",
+ "cd /workspace/finn\r\n"
+ ]
+ }
+ ],
+ "source": [
+ "shell_script = code_gen_dir + \"/ipgen.sh\"\n",
+ "!cat {shell_script}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### HLS IP per layer"
+ "The script consists only of two framing `cd` commands and a command to pass the tcl script to *vivado_hls*. The directory has to be changed to create the files in the correct folder and will then be changed back to the original directory. \n",
+ "\n",
+ "Below is the tcl script which is passed to *vivado_hls*."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 38,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\r\n",
+ "set config_proj_name project_StreamingFCLayer_Batch_0\r\n",
+ "puts \"HLS project: $config_proj_name\"\r\n",
+ "set config_hwsrcdir \"/tmp/code_gen_ipgen_StreamingFCLayer_Batch_hkd3xuxq\"\r\n",
+ "puts \"HW source dir: $config_hwsrcdir\"\r\n",
+ "set config_proj_part \"xc7z020clg400-1\"\r\n",
+ "\r\n",
+ "set config_bnnlibdir \"/workspace/finn-hlslib\"\r\n",
+ "\r\n",
+ "set config_toplevelfxn \"StreamingFCLayer_Batch_0\"\r\n",
+ "set config_clkperiod 5\r\n",
+ "\r\n",
+ "open_project $config_proj_name\r\n",
+ "add_files $config_hwsrcdir/top_StreamingFCLayer_Batch_0.cpp -cflags \"-std=c++0x -I$config_bnnlibdir\"\r\n",
+ "\r\n",
+ "set_top $config_toplevelfxn\r\n",
+ "open_solution sol1\r\n",
+ "set_part $config_proj_part\r\n",
+ "\r\n",
+ "config_interface -m_axi_addr64\r\n",
+ "\r\n",
+ "create_clock -period $config_clkperiod -name default\r\n",
+ "csynth_design\r\n",
+ "export_design -format ip_catalog\r\n",
+ "exit 0\r\n"
+ ]
+ }
+ ],
+ "source": [
+ "tcl_script = code_gen_dir + \"/hls_syn_StreamingFCLayer_Batch_0.tcl\"\n",
+ "!cat {tcl_script}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the first part of the script the project is configured. For example the FPGA part and the clock are set. Then the project is opened and the files are added. The toplevel function is set and after creating a clock, the design is first synthesized with `csynth` and then exported as an IP block.\n",
+ "\n",
+ "Now that all IP blocks are in place, they can be stitched together to create an IP design that matches the ONNX model. This is covered in the next section."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Creation of stitched design"
+ "### Creation of stitched design <a id='stitched_design'></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### PYNQ shell project"
+ "### PYNQ shell project <a id='pynq_shell'></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Synthesis, place and route"
+ "### Synthesis, place and route <a id='synth_pl_ro'></a>"
]
},
{
@@ -622,9 +821,9 @@
"metadata": {},
"source": [
"## 5. Simulation & Emulation flows for functional verification <a id='sim'></a>\n",
- "* Simulation using Python\n",
- "* Simulation (npysim) using C++\n",
- "* Emulation (rtlsim) using PyVerilator"
+ "* [Simulation using Python](#simpy)\n",
+ "* [Simulation (npysim) using C++](#npysim)\n",
+ "* [Emulation (rtlsim) using PyVerilator](#rtlsim)"
]
},
{
--
GitLab