diff --git a/notebooks/FCLayer_graph.onnx b/notebooks/FCLayer_graph.onnx
index 4cd3fa2804a9960ed405da87d02b0b15b5909bf4..950c78a9de7224b83ff46da4920da1baa5d80d61 100644
Binary files a/notebooks/FCLayer_graph.onnx and b/notebooks/FCLayer_graph.onnx differ
diff --git a/notebooks/FINN-CodeGenerationAndCompilation.ipynb b/notebooks/FINN-CodeGenerationAndCompilation.ipynb
index e81bc5a364b67caf6cac6be78317b8e306277cdc..df28989756bde4e1984bb18f94528ef935854f3c 100644
--- a/notebooks/FINN-CodeGenerationAndCompilation.ipynb
+++ b/notebooks/FINN-CodeGenerationAndCompilation.ipynb
@@ -13,7 +13,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -30,7 +30,10 @@
"## Outline\n",
"-------------\n",
"* <font size=\"3\">Example model</font>\n",
- "* <font size=\"3\">Code generation</font>"
+ "* <font size=\"3\">Code generation</font>\n",
+ "* <font size=\"3\">Compilation</font>\n",
+ "* <font size=\"3\">CustomOp node execution</font>\n",
+ "* <font size=\"3\">Conclusion</font>"
]
},
{
@@ -38,14 +41,14 @@
"metadata": {},
"source": [
"### Example model\n",
- "<font size=\"3\">To show the code generation and compilation of a node, an example model with a streaming fclayer node is first created. To learn more about FINN custom operation nodes, please take a look in notebook *FINN-CustomOps*.\n",
+ "<font size=\"3\">To show the code generation and compilation of a node, an example model with a StreamingFCLayer_Batch node is first created. To learn more about FINN custom operation nodes, please take a look at notebook [FINN-CustomOps](FINN-CustomOps.ipynb).\n",
"\n",
- "First TensorProto and helper are imported from ONNX. These functions can be used to create tensors, nodes, graphs and models in ONNX. Additional functions from `util` and the classes `DataType` and `ModelWrapper` are needed. More information about `DataType` and `ModelWrapper` can be found in Jupyter notebook *FINN-ModelWrapper*.</font>"
+ "First TensorProto and helper are imported from ONNX. These can be used to create tensors, nodes, graphs and models in ONNX. Additional functions from `util` and the classes `DataType` and `ModelWrapper` are needed. More information about `DataType` and `ModelWrapper` can be found in Jupyter notebook [FINN-ModelWrapper](FINN-ModelWrapper.ipynb).</font>"
]
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
@@ -59,12 +62,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">Then all parameters, that are needed to create a streaming fclayer, are set. To keep the example clear small values are chosen. </font>"
+ "<font size=\"3\">Then all parameters, that are needed to create a StreamingFCLayer_Batch node, are set. To keep the example clear small values are chosen. For more information about the parameters please take a look at the documentation of the [finn-hls library](https://finn-hlslib.readthedocs.io/en/latest/).</font>"
]
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
@@ -81,12 +84,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">A `tensor_value_info` is created for all tensors involved. In this case there is one tensor for the weights besides the input and output tensors. Then an input list is created containing the two inputs (`\"inp\"`and `\"weights\"`).</font>"
+ "<font size=\"3\">A `tensor_value_info` is created for all tensors involved. In this case there is one tensor for the weights besides the input and output tensors. Then an input list is created containing the two inputs (`\"inp\"`and `\"weights\"`).\n",
+ "\n",
+ "**Note**: A StreamingFCLayer_Batch node can also have an output activation which is passed in the form of thresholds as input tensor</font>"
]
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
@@ -109,7 +114,7 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
@@ -143,7 +148,7 @@
},
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
@@ -161,7 +166,7 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
@@ -184,7 +189,7 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
@@ -193,7 +198,7 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": 9,
"metadata": {},
"outputs": [
{
@@ -211,7 +216,7 @@
},
{
"cell_type": "code",
- "execution_count": 12,
+ "execution_count": 10,
"metadata": {},
"outputs": [
{
@@ -237,14 +242,14 @@
"metadata": {},
"source": [
"### Code Generation\n",
- "<font size=\"3\">Code generation is a transformation that can be applied to the model. For more information about transformation passes, see Jupyter Notebook *FINN-HowToTransformPass*.\n",
+ "<font size=\"3\">Code generation is a transformation pass that can be applied to the model. For more information about transformation passes, see Jupyter Notebook [FINN-HowToTransformPass](FINN-HowToTransformPass.ipynb).\n",
"\n",
"The code generation transformation is shown below.</font>"
]
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": 11,
"metadata": {},
"outputs": [
{
@@ -277,12 +282,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">The transformation passes iterates over all nodes in the model and if `domain=\"finn\"` and `backend=\"fpgadataflow\"` the function `_codegen_single_node()` is executed which is also part of the transformation pass and is shown below. </font>"
+ "<font size=\"3\">The transformation pass iterates over all nodes in the model and if `domain=\"finn\"` and `backend=\"fpgadataflow\"` is True, the function `_codegen_single_node()` is executed which is also part of the transformation pass and is shown below. </font>"
]
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 12,
"metadata": {},
"outputs": [
{
@@ -322,14 +327,598 @@
"source": [
"<font size=\"3\">An instance of the node is created and checked for the attribute `code_gen_dir`. If the attribute is not set, a temporary directory is created and the attribute is set accordingly. \n",
"\n",
- "Then the `code_generation()` function of the instance is called. If an error occurs during this process, this is probably due to the fact that the selected CustomOp is not yet supported.</font>"
+ "Then the `code_generation()` function of the instance is called. If an error occurs during this process, this is probably due to the fact that the selected CustomOp is not yet supported. The following description of the code generation within the CustomOp instance may lead to overlaps with the Jupyter notebook [FINN-CustomOps](FINN-CustomOps.ipynb).\n",
+ "\n",
+ "In order to clarify the individual components involved in code generation, an instance of the node is first created, as in the `_codegen_single_node` function. This is done by looking up the op_type in the [registry](https://github.com/Xilinx/finn/blob/dev/src/finn/custom_op/registry.py) of CustomOps. The instance contains a template for code generation which is shown below.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ " #include \"cnpy.h\"\n",
+ " #include \"npy2apintstream.hpp\"\n",
+ " #include <vector>\n",
+ " #include \"bnn-library.h\"\n",
+ "\n",
+ " // includes for network parameters\n",
+ " $GLOBALS$\n",
+ "\n",
+ " // defines for network parameters\n",
+ " $DEFINES$\n",
+ "\n",
+ " int main(){\n",
+ "\n",
+ " $STREAMDECLARATIONS$\n",
+ "\n",
+ " $READNPYDATA$\n",
+ "\n",
+ " $DOCOMPUTE$\n",
+ "\n",
+ " $DATAOUTSTREAM$\n",
+ "\n",
+ " $SAVEASCNPY$\n",
+ "\n",
+ " }\n",
+ "\n",
+ " \n"
+ ]
+ }
+ ],
+ "source": [
+ "import finn.custom_op.registry as registry\n",
+ "node = FCLayer_node\n",
+ "op_type = FCLayer_node.op_type\n",
+ "inst = registry.custom_op[op_type](node)\n",
+ "print(inst.docompute_template)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The template has some general constructs, like the inclusion of bnn-library.h, which contains the references to the finn-hls library, and of cnpy.h and npy2apintstream.hpp, which support the transfer of python numpy arrays in c++. The idea of this template is to replace the variables marked with `$ $` with c++ calls during code generation. Then the template can be written into a .cpp file and be compiled. \n",
+ "\n",
+ "The sub-functions that are called during code generation are shown below.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def code_generation(self, model):\n",
+ " node = self.onnx_node\n",
+ " self.generate_params(model)\n",
+ " self.global_includes()\n",
+ " self.defines()\n",
+ " self.read_npy_data()\n",
+ " self.strm_decl()\n",
+ " self.docompute()\n",
+ " self.dataoutstrm()\n",
+ " self.save_as_npy()\n",
+ "\n",
+ " template = self.docompute_template\n",
+ "\n",
+ " for key in self.code_gen_dict:\n",
+ " # transform list into long string separated by '\\n'\n",
+ " code_gen_line = \"\\n\".join(self.code_gen_dict[key])\n",
+ " template = template.replace(key, code_gen_line)\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " f = open(os.path.join(code_gen_dir, \"execute_{}.cpp\".format(node.op_type)), \"w\")\n",
+ " f.write(template)\n",
+ " f.close()\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "from finn.custom_op.fpgadataflow.streamingfclayer_batch import StreamingFCLayer_Batch\n",
+ "showSrc(StreamingFCLayer_Batch.code_generation)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Except for the function `generate_params(model)` all functions needed to fill the template correspond to the `$ $` variable names, i.e. function `defines()` returns the part of the c++ code that replaces `$DEFINES$` in the template. The individual functions are member functions of the class HLSCustomOp and are defined in each CustomOp. The code for a StreamingFCLayer_Batch node can be looked up in the [code](https://github.com/Xilinx/finn/blob/dev/src/finn/custom_op/fpgadataflow/streamingfclayer_batch.py).</font> "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">A special function for code generation for the StreamingFCLayer_Batch node is the `generate_params(model)` function. Besides the normal input tensor, an fc layer has weight values as input and can get additional thresholds for activation. This function reads the values for the weights and thresholds via the `get_initializer` function of the ModelWrapper and writes them c++ conform in .h files, which are added to the includes. \n",
+ "\n",
+ "The `generate_params` function of the StreamingFCLayer_Batch is shown below.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def generate_params(self, model):\n",
+ " # weights\n",
+ " weights = model.get_initializer(self.onnx_node.input[1])\n",
+ " # convert weights into hlslib-compatible format\n",
+ " weight_tensor = self.get_hls_compatible_weight_tensor(weights)\n",
+ " export_wdt = self.get_weight_datatype()\n",
+ " # we have converted bipolar weights to binary for export,\n",
+ " # so use it as such for weight generation\n",
+ " if self.get_weight_datatype() == DataType.BIPOLAR:\n",
+ " export_wdt = DataType.BINARY\n",
+ " weight_hls_code = numpy_to_hls_code(\n",
+ " weight_tensor, export_wdt, \"weights\", True, True\n",
+ " )\n",
+ " # write weights into params.h\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " f_weights = open(\"{}/params.h\".format(code_gen_dir), \"w\")\n",
+ "\n",
+ " if export_wdt.bitwidth() != 1:\n",
+ " f_weights.write(\n",
+ " \"static FixedPointWeights<{},{},{},{}> weights = \".format(\n",
+ " self.get_nodeattr(\"SIMD\"),\n",
+ " export_wdt.get_hls_datatype_str(),\n",
+ " self.get_nodeattr(\"PE\"),\n",
+ " self.calc_wmem(),\n",
+ " )\n",
+ " )\n",
+ " else:\n",
+ " f_weights.write(\n",
+ " \"static BinaryWeights<{},{},{}> weights = \".format(\n",
+ " self.get_nodeattr(\"SIMD\"), self.get_nodeattr(\"PE\"), self.calc_wmem()\n",
+ " )\n",
+ " )\n",
+ " f_weights.write(weight_hls_code)\n",
+ " f_weights.close()\n",
+ " # thresholds\n",
+ " if len(self.onnx_node.input) > 2:\n",
+ " thresholds = model.get_initializer(self.onnx_node.input[2])\n",
+ " if thresholds is not None:\n",
+ " threshold_tensor = self.get_hls_compatible_threshold_tensor(thresholds)\n",
+ " tdt = DataType.INT32\n",
+ " # use UINT32 threshold export for bipolar times bipolar\n",
+ " inp_is_bipolar = self.get_input_datatype() == DataType.BIPOLAR\n",
+ " wt_is_bipolar = self.get_weight_datatype() == DataType.BIPOLAR\n",
+ " # reinterpret inp/wt as bipolar if bin_xnor_mode is iset\n",
+ " inp_is_binary = self.get_input_datatype() == DataType.BINARY\n",
+ " wt_is_binary = self.get_weight_datatype() == DataType.BINARY\n",
+ " bin_xnor_mode = self.get_nodeattr(\"binaryXnorMode\") == 1\n",
+ " inp_is_bipolar = inp_is_bipolar or (inp_is_binary and bin_xnor_mode)\n",
+ " wt_is_bipolar = wt_is_bipolar or (wt_is_binary and bin_xnor_mode)\n",
+ " if inp_is_bipolar and wt_is_bipolar:\n",
+ " tdt = DataType.UINT32\n",
+ " thresholds_hls_code = numpy_to_hls_code(\n",
+ " threshold_tensor, tdt, \"thresholds\", False, True\n",
+ " )\n",
+ " # write thresholds into thresh.h\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " f_thresh = open(\"{}/thresh.h\".format(code_gen_dir), \"w\")\n",
+ " tdt_hls = tdt.get_hls_datatype_str()\n",
+ " # use binary to export bipolar activations\n",
+ " export_odt = self.get_output_datatype()\n",
+ " if self.get_output_datatype() == DataType.BIPOLAR:\n",
+ " export_odt = DataType.BINARY\n",
+ " odt_hls = export_odt.get_hls_datatype_str()\n",
+ " f_thresh.write(\n",
+ " \"static ThresholdsActivation<{},{},{},{},{},{},{}> threshs \\\n",
+ " = \".format(\n",
+ " self.calc_tmem(),\n",
+ " self.get_nodeattr(\"PE\"),\n",
+ " threshold_tensor.shape[-1],\n",
+ " tdt_hls,\n",
+ " odt_hls,\n",
+ " self.get_nodeattr(\"ActVal\"),\n",
+ " \"std::less_equal<%s>\" % tdt_hls,\n",
+ " )\n",
+ " )\n",
+ " f_thresh.write(thresholds_hls_code)\n",
+ " f_thresh.close()\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(StreamingFCLayer_Batch.generate_params)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The generated code is written to the previously created temporary directory and the node attribute `code_gen_dir` is set. This completes the code generation for executing a single CustomOp. The next step is compilation. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Compilation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">The following description of the code generation within the CustomOp instance may lead to overlaps with the Jupyter notebook *FINN-CustomOps*. </font>"
+ "<font size=\"3\">The compilation is a transformation pass like the code generation. The code of this transformation is shown below. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "class Compile(Transformation):\n",
+ " \"\"\"Compile for all nodes in model\"\"\"\n",
+ "\n",
+ " def __init__(self):\n",
+ " super().__init__()\n",
+ "\n",
+ " def apply(self, model):\n",
+ " for node in model.graph.node:\n",
+ " op_type = node.op_type\n",
+ " if node.domain == \"finn\":\n",
+ " backend_attribute = util.get_by_name(node.attribute, \"backend\")\n",
+ " if backend_attribute is None:\n",
+ " continue\n",
+ " backend_value = backend_attribute.s.decode(\"UTF-8\")\n",
+ " if backend_value == \"fpgadataflow\":\n",
+ " try:\n",
+ " # lookup op_type in registry of CustomOps\n",
+ " inst = registry.custom_op[op_type](node)\n",
+ " # ensure that code is generated\n",
+ " assert inst.get_nodeattr(\"code_gen_dir\") != \"\"\n",
+ " # call the compilation function for this node\n",
+ " inst.compile_singlenode_code()\n",
+ " # ensure that executable path is now set\n",
+ " assert inst.get_nodeattr(\"executable_path\") != \"\"\n",
+ " except KeyError:\n",
+ " # exception if op_type is not supported\n",
+ " raise Exception(\n",
+ " \"Custom op_type %s is currently not supported.\" % op_type\n",
+ " )\n",
+ " return (model, False)\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "from finn.transformation.fpgadataflow.compile import Compile\n",
+ "showSrc(Compile)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The scheme resembles that of the code generation transformation pass. The pass iterates over all nodes in the model and if `domain=\"finn\"` and `backend=\"fpgadataflow\"` is True, the compilation is activated for that node. First an instance of the node is created and checked whether the code was generated. For this the node attribute `code_gen_dir` is checked. If it exists, the function `compile_singlenode_code()` can be executed. Then it is checked whether the path to the executable has been set. There is an exception if the custom op_type is not supported. \n",
+ "\n",
+ "The actual compilation is done with the function `compile_singlenode_code()`. What happens inside the function is shown below.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def compile_singlenode_code(self):\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " builder = CppBuilder()\n",
+ " builder.append_includes(\"-I/workspace/finn/src/finn/data/cpp\")\n",
+ " builder.append_includes(\"-I/workspace/cnpy/\")\n",
+ " builder.append_includes(\"-I/workspace/finn-hlslib\")\n",
+ " builder.append_includes(\"-I/workspace/vivado-hlslib\")\n",
+ " builder.append_includes(\"--std=c++11\")\n",
+ " builder.append_sources(code_gen_dir + \"/*.cpp\")\n",
+ " builder.append_sources(\"/workspace/cnpy/cnpy.cpp\")\n",
+ " builder.append_includes(\"-lz\")\n",
+ " builder.set_executable_path(code_gen_dir + \"/node_model\")\n",
+ " builder.build(code_gen_dir)\n",
+ " self.set_nodeattr(\"executable_path\", builder.executable_path)\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(StreamingFCLayer_Batch.compile_singlenode_code)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">To execute the compilation the class `CppBuilder` from `core.utils` is used. Subsequently the member functions of this class are used to construct the g++ command. To better understand the exact procedure the class `CppBuilder` is shown below. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "class CppBuilder:\n",
+ " def __init__(self):\n",
+ " self.include_paths = []\n",
+ " self.cpp_files = []\n",
+ " self.executable_path = \"\"\n",
+ " self.code_gen_dir = \"\"\n",
+ " self.compile_components = []\n",
+ " self.compile_script = \"\"\n",
+ "\n",
+ " def append_includes(self, library_path):\n",
+ " self.include_paths.append(library_path)\n",
+ "\n",
+ " def append_sources(self, cpp_file):\n",
+ " self.cpp_files.append(cpp_file)\n",
+ "\n",
+ " def set_executable_path(self, path):\n",
+ " self.executable_path = path\n",
+ "\n",
+ " def build(self, code_gen_dir):\n",
+ " # raise error if includes are empty\n",
+ " self.code_gen_dir = code_gen_dir\n",
+ " self.compile_components.append(\"g++ -o \" + str(self.executable_path))\n",
+ " for cpp_file in self.cpp_files:\n",
+ " self.compile_components.append(cpp_file)\n",
+ " for lib in self.include_paths:\n",
+ " self.compile_components.append(lib)\n",
+ " bash_compile = \"\"\n",
+ " for component in self.compile_components:\n",
+ " bash_compile += str(component) + \" \"\n",
+ " self.compile_script = str(self.code_gen_dir) + \"/compile.sh\"\n",
+ " with open(self.compile_script, \"w\") as f:\n",
+ " f.write(\"#!/bin/bash \\n\")\n",
+ " f.write(bash_compile + \"\\n\")\n",
+ " bash_command = [\"bash\", self.compile_script]\n",
+ " process_compile = subprocess.Popen(bash_command, stdout=subprocess.PIPE)\n",
+ " process_compile.communicate()\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(util.CppBuilder)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The class contains several member variables needed to execute the compilation command. These are reset when instantiating the class. The following functions are to fill these variables and in the build function, everything is combined into one compile command, which is then executed using the python library `subprocess`. \n",
+ " \n",
+ "After the executables have been created, the `compile_singlenode_code` function sets the `executable_path` node attribute.</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">This flow is needed for the execution of a single CustomOp node. The execution itself is represented in function `execute_node` of the respective node class. The last part of this Jupyter notebook is about this function.</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### CustomOp node execution"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The function `execute_node` of StreamingFCLayer_Batch is displayed below. The class HLSCustomOp also has an `execute_node` function, which contains the basic principle of the execution. However, for the StreamingFcLayer_Batch node further transformations are necessary. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def execute_node(self, context, graph):\n",
+ " node = self.onnx_node\n",
+ " mw = self.get_nodeattr(\"MW\")\n",
+ " mh = self.get_nodeattr(\"MH\")\n",
+ " simd = self.get_nodeattr(\"SIMD\")\n",
+ " pe = self.get_nodeattr(\"PE\")\n",
+ " sf = mw // simd\n",
+ " nf = mh // pe\n",
+ "\n",
+ " # TODO ensure codegen dir exists\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " # create a npy file fore each input of the node (in_ind is input index)\n",
+ " in_ind = 0\n",
+ " for inputs in node.input:\n",
+ " # it is assumed that the first input of the node is the data input\n",
+ " # the second input are the weights\n",
+ " # the third input are the thresholds\n",
+ " if in_ind == 0:\n",
+ " assert str(context[inputs].dtype) == \"float32\"\n",
+ " expected_inp_shape = (1, sf, simd)\n",
+ " reshaped_input = context[inputs].reshape(expected_inp_shape)\n",
+ " # flip SIMD (innermost) dimension of input tensor, there's some reversal\n",
+ " # going on somewhere with a mistmatch between npy and hls...\n",
+ " reshaped_input = np.flip(reshaped_input, -1)\n",
+ " if self.get_input_datatype() == DataType.BIPOLAR:\n",
+ " # store bipolar activations as binary\n",
+ " reshaped_input = (reshaped_input + 1) / 2\n",
+ " np.save(\n",
+ " os.path.join(code_gen_dir, \"input_{}.npy\".format(in_ind)),\n",
+ " reshaped_input,\n",
+ " )\n",
+ " elif in_ind > 2:\n",
+ " raise Exception(\"Unexpected input found for StreamingFCLayer\")\n",
+ " in_ind += 1\n",
+ " # execute the precompiled model\n",
+ " super().exec_precompiled_singlenode_model()\n",
+ " # load output npy file\n",
+ " super().npy_to_dynamic_output(context)\n",
+ " # reinterpret binary output as bipolar where needed\n",
+ " if self.get_output_datatype() == DataType.BIPOLAR:\n",
+ " out = context[node.output[0]]\n",
+ " out = 2 * out - 1\n",
+ " context[node.output[0]] = out\n",
+ " assert context[node.output[0]].shape == (1, nf, pe)\n",
+ " # reshape output to have expected shape\n",
+ " context[node.output[0]] = context[node.output[0]].reshape(1, mh)\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(StreamingFCLayer_Batch.execute_node)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">First, all parameters are extracted using the `get_nodeattr` function. It is also important to read the code generation via `code_gen_dir`. `execute_node` is divided into three parts:</font>\n",
+ "* <font size=\"3\">creation of a npy file for each input of the node</font>\n",
+ "* <font size=\"3\">execution of the precompiled model</font>\n",
+ "* <font size=\"3\">loading the output npy file</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Creation of a npy file for each input of the node\n",
+ "\n",
+ "<font size=\"3\">To transfer the input values correctly to the c++ model, the input tensor has to be reshaped and the innermost dimension (SIMD) has to be flipped. Afterwards the tensor can be stored in a .npy file. \n",
+ "\n",
+ "Since the StreamingFcLayer_Batch node only has a maximum of three inputs (input, weights, thresholds), an error will be thrown if this number is exceeded. The weights and thresholds have already been written to separate .h files and therefore only the input tensor has to be stored in a .npy file. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Execution of the precompiled model\n",
+ "<font size=\"3\">The function from class HLSCustomOp is used here. It is shown below.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def exec_precompiled_singlenode_model(self):\n",
+ " # execute precompiled executable\n",
+ " executable_path = self.get_nodeattr(\"executable_path\")\n",
+ " if executable_path == \"\":\n",
+ " raise Exception(\n",
+ " \"\"\"\n",
+ "Found no executable for this node, did you run the codegen and\n",
+ "compilation transformations?\n",
+ " \"\"\"\n",
+ " )\n",
+ " process_execute = subprocess.Popen(executable_path, stdout=subprocess.PIPE)\n",
+ " process_execute.communicate()\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(StreamingFCLayer_Batch.exec_precompiled_singlenode_model)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">After checking that the attribute `executable_path` is not empty, the executable is executed via `subprocess`. The output is written from the c++ code into a .npy file, which can be read later on. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Loading the output npy file\n",
+ "\n",
+ "<font size=\"3\">To load the output data the function `npy_to_dynamic_output` is used. It is shown below. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " def npy_to_dynamic_output(self, context):\n",
+ " # TODO support multi-output nodes as needed\n",
+ " node = self.onnx_node\n",
+ " code_gen_dir = self.get_nodeattr(\"code_gen_dir\")\n",
+ " output = np.load(\"{}/output.npy\".format(code_gen_dir))\n",
+ " context[node.output[0]] = output\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "showSrc(StreamingFCLayer_Batch.npy_to_dynamic_output)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Since the output file is stored in the same directory as the generated code, the attribute `code_gen_dir` is read first and then the output data is loaded using the numpy function `.load`. The context is set accordingly. \n",
+ "\n",
+ "Finally, the output data is manipulated in the `execute_node` function. If the data is bipolar, it is converted into binary data for further processing. Then the shape of the tensor is checked and converted into the expected output shape.\n",
+ "</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Conclusion\n",
+ "\n",
+ "<font size=\"3\">Code generation and compilation are transformation passes that must be applied before a node can be executed. They are independent of the execution of a node and can be used for further development to enable functions such as code generation for synthesis or larger models. \n",
+ "\n",
+ "All files belonging to the code generation and compilation are stored in the directory which is specified in `code_gen_dir`.\n",
+ "\n",
+ "**Important**: If the code is executed inside the docker container, the directory will be deleted after closing the container. \n",
+ " \n",
+ "For further reading please see the /tests folder of the FINN repo. The subfolder /fpgadataflow contains for example: [test_fpgadataflow_fclayer](https://github.com/Xilinx/finn/blob/dev/tests/fpgadataflow/test_fpgadataflow_fclayer.py) which tests the functionality of the flow described above.</font>"
]
},
{
diff --git a/notebooks/FINN-CustomOps.ipynb b/notebooks/FINN-CustomOps.ipynb
index 6d0e3b33ba1bf2787498ef7e85449e5db2d1e272..def670e46ff50a539df6a5e00788749c396bbd57 100644
--- a/notebooks/FINN-CustomOps.ipynb
+++ b/notebooks/FINN-CustomOps.ipynb
@@ -32,8 +32,6 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## FINN Custom Ops\n",
- "---------------------------\n",
"<font size=\"3\">FINN uses many custom operations (`op_type` in ONNX NodeProto) that are not defined in the ONNX operator schema. These custom nodes are marked with `domain=\"finn\"` in the protobuf to identify them as such. These nodes can represent specific operations that we need for low-bit networks, or operations that are specific to a particular hardware backend.\n",
"\n",
"A very abstract version of a custom op node representing a streaming fc layer is shown below. </font>"
@@ -43,45 +41,56 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "`FCLayer_node = helper.make_node(\n",
- " \"StreamingFCLayer_Batch\",\n",
- " node_inp_list,\n",
- " node_outp_list,\n",
+ "## Outline\n",
+ "---------------------------\n",
+ "* <font size=\"3\">Basic FINN-ONNX node</font>\n",
+ "* <font size=\"3\">CustomOp class</font>\n",
+ "* <font size=\"3\">HLS FINN-ONNX node</font>\n",
+ "* <font size=\"3\">HLSCustomOp class</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Basic FINN-ONNX node\n",
+ "\n",
+ "<font size=\"3\">To create a FINN-ONNX node you can use the helper function of ONNX. Because it is an ONNX NodeProtobuf, but with several additional attributes. The procedure is shown with an example for a multithreshold node. </font>\n",
+ "\n",
+ "`multithreshold_node = helper.make_node(\n",
+ " \"MultiThreshold\",\n",
+ " [\"v\", \"thresholds\"],\n",
+ " [\"out\"],\n",
" domain=\"finn\",\n",
- " backend=\"fpgadataflow\",\n",
- " code_gen_dir=\"\",\n",
- " executable_path=\"\",\n",
- " resType=\"ap_resource_lut()\",\n",
- " MW=mw,\n",
- " MH=mh,\n",
- " SIMD=simd,\n",
- " PE=pe,\n",
- " inputDataType=<FINN DataType>,\n",
- " weightDataType=<FINN DataType>,\n",
- " outputDataType=<FINN DataType>,\n",
- " ActVal=actval,\n",
- " binaryXnorMode=<0/1>,\n",
- " noActivation=<0/1>\n",
- ")`"
+ " out_scale=2.0,\n",
+ " out_bias=-1.0,\n",
+ " out_dtype=\"\",\n",
+ ")`\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- " <font size=\"3\">Unlike standard nodes, the custom op nodes has several additional attributes. The node is created using the helper function of ONNX. `\"StreamingFCLayer_Batch\"` describes the op_type, then the inputs and outputs are declared. Since this is a custom op node of FINN, the attribute `domain=\"finn\"` must be set. The streaming fc layer is a custom op from the finn-hls library, this is set in the node using the `backend` attribute. To execute a custom op from the finn-hls library, the corresponding c++ code must be created and an executable must be produced. Where the generated code is stored is specified in the `code_gen_dir` attribute and `executable_path` specifies the path to the produced executable. In addition to the data types of the input and output tensors, the node also contains various other attributes resulting from the parameters of the corresponding finn-hls library function. This will not be discussed here.</font>"
+ "<font size=\"3\">The `helper.make_node` function gets the op_type as first argument. In this case it is *MultiThreshold*. Then the inputs and outputs are passed. Beside the data input the multithreshold node has an additional input to pass the threshold values. \n",
+ "\n",
+ "The next attribute (`domain`) is to specify that it is a FINN-ONNX node. It must be set to `\"finn\"`, so that the functions that work with FINN-ONNX nodes can directly recognize that it is a CustomOp. The attributes `out_scale` and `out_bias` are special multithreshold attributes to manipulate the output value. `out_dtype` contains the output data type.\n",
+ " \n",
+ "**Note**: each FINN-ONNX node has its own special attributes, which must be set correctly to ensure proper processing.</font>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">Custom Ops are represented in FINN as ONNX nodes on the one hand and by a CustomOp class on the other hand. This allows easier access to the different attributes and introduces special custom op functions. See below for the standard CustomOp class.</font>"
+ "## CustomOp class\n",
+ "\n",
+ "<font size=\"3\">Custom Ops are represented in FINN as ONNX nodes on the one hand and by a CustomOp class on the other hand. This allows easier access to different attributes and introduces special custom op functions. See below for the standard CustomOp class.</font>"
]
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 3,
"metadata": {},
"outputs": [
{
@@ -176,14 +185,140 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">When instantiating the class, the ONNX node is passed to access all attributes of the node within the class. This is accompanied by the functions `get_nodeattr()`and `set_nodeattr()`, which each instance of this class has. Furthermore 4 abstract methods are implemented, which are described in more detail in the comments in the code. </font>"
+ "<font size=\"3\">When instantiating the class, the ONNX node is passed to access all attributes of the node within the class. This is accompanied by the functions `get_nodeattr()`and `set_nodeattr()`, which each instance of this class has. Furthermore 4 abstract methods are implemented, which are described in more detail in the commands of the code and will be exemplarily explained for the multithreshold node in the following. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "class MultiThreshold(CustomOp):\n",
+ " def get_nodeattr_types(self):\n",
+ " return {\n",
+ " \"out_dtype\": (\"s\", True, \"\"),\n",
+ " \"out_scale\": (\"f\", False, 1.0),\n",
+ " \"out_bias\": (\"f\", False, 0.0),\n",
+ " }\n",
+ "\n",
+ " def make_shape_compatible_op(self):\n",
+ " node = self.onnx_node\n",
+ " return helper.make_node(\"Relu\", [node.input[0]], [node.output[0]])\n",
+ "\n",
+ " def infer_node_datatype(self, model):\n",
+ " node = self.onnx_node\n",
+ " odt = self.get_nodeattr(\"out_dtype\")\n",
+ " model.set_tensor_datatype(node.output[0], DataType[odt])\n",
+ "\n",
+ " def execute_node(self, context, graph):\n",
+ " node = self.onnx_node\n",
+ " # save inputs\n",
+ " v = context[node.input[0]]\n",
+ " thresholds = context[node.input[1]]\n",
+ " # retrieve attributes if output scaling is used\n",
+ " out_scale = self.get_nodeattr(\"out_scale\")\n",
+ " out_bias = self.get_nodeattr(\"out_bias\")\n",
+ " # calculate output\n",
+ " output = multithreshold(v, thresholds, out_scale, out_bias)\n",
+ " # setting context according to output\n",
+ " context[node.output[0]] = output\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "from finn.custom_op.multithreshold import MultiThreshold\n",
+ "showSrc(MultiThreshold)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\"> `get_nodeattr_types`: returns a dict for the permitted attributes for node. It returns a triple with following values for each of the special multithreshold attributes. </font>\n",
+ "* <font size=\"3\">`dtype`: indicates which member of the ONNX AttributeProto will be utilized </font>\n",
+ "* <font size=\"3\">`require`: indicates whether this attribute is required </font>\n",
+ "* <font size=\"3\">`default_value`: indicates the default value that will be used if the attribute is not set </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">`make_shape_compatible_op`: To use the flow of FINN, the transformation pass [infer_shapes](https://github.com/Xilinx/finn/blob/dev/src/finn/transformation/infer_shapes.py) is applied to the graphs in various places. In order for this transformation to be applied to CustomOps, they must first be converted to standard ONNX nodes with the same shape behavior. This means, nodes where the relationship between input and output shape is the same. \n",
+ "\n",
+ "This is done at this point. Since the output shape of a multithreshold node is the same as the input shape, it can be replaced by a `\"Relu\"` node from the standard node library of onnx.</font>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">If it is a node from the finn-hls library another class is used which is derived from the CustomOp class:</font>"
+ "<font size=\"3\">`infer_node_datatype`: sets the output tensor data type accordingly to the attribute `out_dtype` </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">`execute_node`: This function allows the execution of the node, depending on the CustomOp a different functionality has to be implemented. In the case of the multithreshold node the input values and the thresholds are first extracted and after the attributes for the output scaling have been retrieved, the output is calculated with the help of a separate function. For more details regarding this function please take a look in the code [here](https://github.com/Xilinx/finn/blob/dev/src/finn/custom_op/multithreshold.py). </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">FINN has a subset of CustomOps that correspond to the [finn-hls](https://finn-hlslib.readthedocs.io/en/latest/) library. In the next part of the Jupyter notebook these are described in more detail. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## HLS FINN-ONNX node\n",
+ "\n",
+ "<font size=\"3\">The creation of an HLS FINN-ONNX node looks very similar to the creation of a basic FINN-ONNX node. But three new attributes are introduced that are necessary to enable the processing of HLS FINN-ONNX nodes in FINN.</font>\n",
+ "\n",
+ "`FCLayer_node = helper.make_node(\n",
+ " \"StreamingFCLayer_Batch\",\n",
+ " node_inp_list,\n",
+ " node_outp_list,\n",
+ " domain=\"finn\",\n",
+ " backend=\"fpgadataflow\",\n",
+ " code_gen_dir=\"\",\n",
+ " executable_path=\"\",\n",
+ " resType=\"ap_resource_lut()\",\n",
+ " MW=mw,\n",
+ " MH=mh,\n",
+ " SIMD=simd,\n",
+ " PE=pe,\n",
+ " inputDataType=<FINN DataType>,\n",
+ " weightDataType=<FINN DataType>,\n",
+ " outputDataType=<FINN DataType>,\n",
+ " ActVal=actval,\n",
+ " binaryXnorMode=<0/1>,\n",
+ " noActivation=<0/1>\n",
+ ")`"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">`\"StreamingFCLayer_Batch\"` describes the op_type, then the inputs and outputs are declared. This is still like building a default onnx node without additional attributes. But since this is a custom op node of FINN, the attribute `domain=\"finn\"` must be set. The streaming fc layer is a custom op from the [finn-hls](https://finn-hlslib.readthedocs.io/en/latest/) library, this information is set in the node using the `backend` attribute. To execute a custom op from the [finn-hls](https://finn-hlslib.readthedocs.io/en/latest/) library, the corresponding c++ code must be created and an executable must be produced. Where the generated code is stored is specified in the `code_gen_dir` attribute and `executable_path` specifies the path to the produced executable. In addition to the data types of the input and output tensors, the node also contains various other attributes resulting from the parameters of the corresponding [finn-hls](https://finn-hlslib.readthedocs.io/en/latest/) library function. More detailed information can be found in the documentation of [finn-hlslib](https://finn-hlslib.readthedocs.io/en/latest/).</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## HLSCustomOp class\n",
+ "\n",
+ "<font size=\"3\">If it is a node from the [finn-hls](https://finn-hlslib.readthedocs.io/en/latest/) library another class is used which is derived from the CustomOp class:</font>"
]
},
{
@@ -377,7 +512,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">**`compile_singlenode_code()`**: To compile the generated code, the compile command must be built. This is done in this function. It creates an instance of the `CppBuilder()` class and assembles the various components for the function. The `.build` function creates the executable and then sets the corresponding attribute. The class `CppBuilder` is a transformation and a more detailed description can be found in Jupyter notebook *FINN-CodeGenerationAndCompilation*.\n",
+ "<font size=\"3\">**`compile_singlenode_code()`**: To compile the generated code, the compile command must be built. This is done in this function. It creates an instance of the `CppBuilder()` class and assembles the various components for the function. The `.build` function creates the executable and then sets the corresponding attribute. The class `CppBuilder` is a transformation and a more detailed description can be found in Jupyter notebook [FINN-CodeGenerationAndCompilation](FINN-CodeGenerationAndCompilation.ipynb).\n",
"</font>"
]
},
@@ -385,7 +520,28 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">**`dynamic_input_to_npy(context, count)`**:</font>"
+ "<font size=\"3\">**`dynamic_input_to_npy(context, count)`**: creates a .npy file for all inputs of the node. These files will be stored in the directory specified by code_gen_dir. The argument `count` must be used to specify the number of inputs. `context` contains the values for the inputs.</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">**`npy_to_dynamic_output(context)`**: reads the output values and sets `context` dictionary accordingly. When executing the c++ executable of the node, the output values are written to a .npy file. </font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">**`exec_precompiled_singlenode_model()`**: executes precompiled executable which is specified in `executable_path`</font>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">**`execute_node(context,graph)`**: calls first `dynamic_input_to_npy()`, then executes the executable using `exec_precompiled_singlenode_model()` and at the end reads the output .npy file with `npy_to_dynamic_output`</font>"
]
},
{
@@ -494,6 +650,13 @@
"showSrc(StreamingFCLayer_Batch.generate_params)"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">First, the values for the weights are extracted with `get_initializer()` using the ModelWrapper. At this point it is assumed that the second input of the streamingfclayer specifies the weights. After a few manipulations the weights are written in `params.h`. If there are threshold values, they will be prepared and written to `thresh.h`. </font>"
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
diff --git a/notebooks/FINN-ModelWrapper.ipynb b/notebooks/FINN-ModelWrapper.ipynb
index 1d349da210ca1ee7d3ad79c2ff0494db30e43f51..ca9c4c6e43584cfcb12c795e5896e726a40fc5d6 100644
--- a/notebooks/FINN-ModelWrapper.ipynb
+++ b/notebooks/FINN-ModelWrapper.ipynb
@@ -31,8 +31,8 @@
"------------------------------\n",
"* <font size=\"3\"> wrapper around ONNX ModelProto that exposes several utility\n",
" functions for graph manipulation and exploration </font>\n",
- "* <font size=\"3\"> ModelWrapper instance takes onnx model proto and `make_deepcopy` flag as input </font>\n",
- "* <font size=\"3\"> onnx model proto can either be a string with the path to a stored .onnx file on disk, or serialized bytes </font>\n",
+ "* <font size=\"3\"> ModelWrapper instance takes ONNX ModelProto and `make_deepcopy` flag as input </font>\n",
+ "* <font size=\"3\"> ONNX ModelProto can either be a string with the path to a stored .onnx file on disk, or serialized bytes </font>\n",
"* <font size=\"3\"> `make_deepcopy` is by default False but can be set to True if a (deep) copy should be created </font>"
]
},
@@ -322,7 +322,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "<font size=\"3\">Like for the other tensor helper functions there is a `set_initializer(tensor_name, tensor_value)` function.</font>"
+ "<font size=\"3\">Like for the other tensor helper functions there is a corresponding set function (`set_initializer(tensor_name, tensor_value)`).</font>"
]
},
{
@@ -330,7 +330,7 @@
"metadata": {},
"source": [
"### More helper functions\n",
- "<font size=\"3\">ModelWrapper contains more useful functions, if you are interested please have a look at the [Python code](https://github.com/Xilinx/finn/blob/dev/src/finn/core/modelwrapper.py) directly. Additionally, in the folder notebooks/ a Jupyter notebook about transformation passes and one about analysis passes can be found.</font>"
+ "<font size=\"3\">ModelWrapper contains more useful functions, if you are interested please have a look at the [Python code](https://github.com/Xilinx/finn/blob/dev/src/finn/core/modelwrapper.py) directly. Additionally, in the folder notebooks/ a Jupyter notebook about transformation passes [FINN-HowToTransformationPass](FINN-HowToTransformationPass.ipynb) and one about analysis passes [FINN-HowToAnalysisPass](FINN-HowToAnalysisPass.ipynb) can be found.</font>"
]
},
{