From cbcd38cc82cc73b29611d7191c8a5596f21dd776 Mon Sep 17 00:00:00 2001
From: auphelia <jakobapk@web.de>
Date: Thu, 5 Dec 2019 16:31:44 +0000
Subject: [PATCH] [notebook] introduction of example model to show code gen and
compilation
---
notebooks/FCLayer_graph.onnx | Bin 0 -> 822 bytes
.../FINN-CodeGenerationAndCompilation.ipynb | 213 +++++++++++++++++-
2 files changed, 212 insertions(+), 1 deletion(-)
create mode 100644 notebooks/FCLayer_graph.onnx
diff --git a/notebooks/FCLayer_graph.onnx b/notebooks/FCLayer_graph.onnx
new file mode 100644
index 0000000000000000000000000000000000000000..efefcd681bfda4d72fd9be9d15f5069c05184a37
GIT binary patch
literal 822
zcmaJ<+fKqj6r@W->v9omLJFE-@I{;W1H2d!j2MVUNJxCyEbaE9r8H$5<c)9e0}MZ6
z{55wAEe!#iOgiVx%t^a@<eXA8br&y4a5I@2#BF}(;17UV$Mc~yCyr&)nIcX@>NhF_
z8W22jJnP}U#}lZ9scvgW5cD5ac_n~|juQIPQAOU(LvhgUx07Pn@ATEoMj7!Uvelq8
z#`*{Gj7GuqEo@+lO?Nt930dtF7IgM`$RZvxoS?zdS*-$($tTf6ipMTNKBo4%e+G-}
z^F!KZ1BMHqG^AGd<+*qF%0uU{#Q)}iT8C0VW>M$_`-t&r80ZAKA-%fZ#Mh^?Z+KHQ
z9nVWn6j=fGZS4!l>?sSa_&lGMAd^}Crf{3$>3+<2bzbix?LDM%57**%>`T|2$Kd*Z
zoa<?D?{3X|F%s>q8m*Y3z_KhT1qft;0WgT;8jSXBpY~qeE@cT8SX5?_2tj>4QH4c*
c35ahGWtXqSmmgeg<{oxjZl@nKwLm%l2T(H7KL7v#
literal 0
HcmV?d00001
diff --git a/notebooks/FINN-CodeGenerationAndCompilation.ipynb b/notebooks/FINN-CodeGenerationAndCompilation.ipynb
index 922693c8e..0183ce9a6 100644
--- a/notebooks/FINN-CodeGenerationAndCompilation.ipynb
+++ b/notebooks/FINN-CodeGenerationAndCompilation.ipynb
@@ -6,7 +6,218 @@
"source": [
"# FINN - Code Generation and Compilation\n",
"-----------------------------------------------------------------\n",
- "This notebook is about code generation and compilation to enable execution of FINN "
+ "<font size=\"3\">This notebook is about code generation and compilation to enable execution of FINN custom operation nodes. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Outline\n",
+ "-------------\n",
+ "* <font size=\"3\">Example model</font>\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example model\n",
+ "<font size=\"3\">To show the code generation and compilation of a node, an example model with a streaming fclayer node is first created. To learn more about FINN custom operation nodes, please take a look in notebook *FINN-CustomOps*.\n",
+ "\n",
+ "First TensorProto and helper are imported from ONNX. These functions can be used to create tensors, nodes, graphs and models in ONNX. Additional functions from `util` and the classes `DataType` and `ModelWrapper` are needed. More information about `DataType` and `ModelWrapper` can be found in Jupyter notebook *FINN-ModelWrapper*.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from onnx import TensorProto, helper\n",
+ "import finn.core.utils as util\n",
+ "from finn.core.datatype import DataType\n",
+ "from finn.core.modelwrapper import ModelWrapper"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Then all parameters, that are needed to create a streaming fclayer, are set. To keep the example clear small values are chosen. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "idt = wdt = odt = DataType.BIPOLAR\n",
+ "mw = 8\n",
+ "mh = 8\n",
+ "pe = 4\n",
+ "simd = 4\n",
+ "wmem = mw * mh // (pe * simd)\n",
+ "nf = mh // pe\n",
+ "sf = mw // simd\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">A `tensor_value_info` is created for all tensors involved. In this case there is one tensor for the weights besides the input and output tensors. Then an input list is created containing the two inputs (`\"inp\"`and `\"weights\"`).</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "inp = helper.make_tensor_value_info(\"inp\", TensorProto.FLOAT, [1, sf, simd])\n",
+ "weights = helper.make_tensor_value_info(\"weights\", TensorProto.FLOAT, [mw, mh])\n",
+ "outp = helper.make_tensor_value_info(\"outp\", TensorProto.FLOAT, [1, nf, pe])\n",
+ "node_inp_list = [\"inp\", \"weights\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Now the node can be created. The operation type is set to `\"StreamingFCLayer_Batch\"` and the rest of the attributes are set appropriately. The relevant attributes for the activation of the code generation and compilation are:</font>\n",
+ "* <font size=\"3\">**`domain=\"finn\"`**: specifies that the created node is a FINN-Custom Op</font>\n",
+ "* <font size=\"3\">**`backend=\"fpgadataflow\"`**: specifies that it is a node that corresponds to a function in the finn-hls library</font>\n",
+ "* <font size=\"3\">**`code_gen_dir\"`**: specifies the path to the directory where the generated c++ files are (is set during code generation)</font>\n",
+ "* <font size=\"3\">**`executable_path\"`**: specifies the path to the executable created after compilation (is set during compilation)</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "FCLayer_node = helper.make_node(\n",
+ " \"StreamingFCLayer_Batch\",\n",
+ " node_inp_list,\n",
+ " [\"outp\"],\n",
+ " domain=\"finn\",\n",
+ " backend=\"fpgadataflow\",\n",
+ " code_gen_dir=\"\",\n",
+ " executable_path=\"\",\n",
+ " resType=\"ap_resource_lut()\",\n",
+ " MW=mw,\n",
+ " MH=mh,\n",
+ " SIMD=simd,\n",
+ " PE=pe,\n",
+ " WMEM=wmem,\n",
+ " TMEM=0,\n",
+ " inputDataType=idt.name,\n",
+ " weightDataType=wdt.name,\n",
+ " outputDataType=odt.name,\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\"> The node is packed into a graph environment and the inputs and outputs are set.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "graph = helper.make_graph(\n",
+ " nodes=[FCLayer_node], name=\"fclayer_graph\", inputs=[inp], outputs=[outp]\n",
+ " )"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">A model is now created from the graph, which is then converted into a ModelWrapper object for further processing in FINN. Afterwards the ModelWrapper internal functions can be used to set the FINN data types and the initializer for the weights. Since this is an example, the weights are not taken from the training, but random values are generated using the utility function `gen_finn_dt_tensor()`. This function gets a FINN datatype and a shape and generates a tensor with values of this datatype in the desired shape.</font>\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model = helper.make_model(graph, producer_name=\"fclayer-model\")\n",
+ "model = ModelWrapper(model)\n",
+ "\n",
+ "model.set_tensor_datatype(\"inp\", idt)\n",
+ "model.set_tensor_datatype(\"outp\", odt)\n",
+ "model.set_tensor_datatype(\"weights\", wdt)\n",
+ "W = util.gen_finn_dt_tensor(wdt, (mw, mh))\n",
+ "model.set_initializer(\"weights\", W)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The model is saved and then netron is used to visualize the resulting model. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model.save(\"FCLayer_graph.onnx\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Stopping http://0.0.0.0:8081\n",
+ "Serving 'FCLayer_graph.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "import netron\n",
+ "netron.start('FCLayer_graph.onnx', port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
]
},
{
--
GitLab