From 599c7695ec79ce2535a264d1fa874a0a28f2b8fb Mon Sep 17 00:00:00 2001
From: auphelia <jakobapk@web.de>
Date: Fri, 13 Dec 2019 13:37:24 +0000
Subject: [PATCH] [notebook] Added section about how to create and execute a
simple onnx graph
---
notebooks/FINN-HowToWorkWithONNX.ipynb | 302 +++++++++++++++++++++++++
1 file changed, 302 insertions(+)
diff --git a/notebooks/FINN-HowToWorkWithONNX.ipynb b/notebooks/FINN-HowToWorkWithONNX.ipynb
index 0ee257394..c9ca6a94b 100644
--- a/notebooks/FINN-HowToWorkWithONNX.ipynb
+++ b/notebooks/FINN-HowToWorkWithONNX.ipynb
@@ -9,6 +9,308 @@
"<font size=\"3\">This notebook should give an overview of ONNX ProtoBuf and help to create and manipulate an ONNX model and use FINN functions to work with it. There may be overlaps to other notebooks, like [FINN-ModelWrapper](FINN-ModelWrapper.ipynb) and [FINN-CustomOps](FINN-CustomOps.ipynb), but this notebook should give an overview about the handling of ONNX models in FINN. </font>"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Outline\n",
+ "* #### How to create a simple model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### How to create a simple model\n",
+ "\n",
+ "<font size=\"3\">To explain how to create an ONNX graph a simple example with mathematical operations is used. All nodes are from the [standard operations library of ONNX](https://github.com/onnx/onnx/blob/master/docs/Operators.md).\n",
+ "\n",
+ "First ONNX is imported, then the helper function can be used to make a node.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import onnx\n",
+ "\n",
+ "Add1_node = onnx.helper.make_node(\n",
+ " 'Add',\n",
+ " inputs=['in1', 'in2'],\n",
+ " outputs=['sum1'],\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The first attribute of the node is the operation type. In this case it is `'Add'`, so it is an adder node. Then the input names are passed to the node and at the end a name is assigned to the output.\n",
+ " \n",
+ "For this example we want two other adder nodes, one abs node and the output shall be rounded so one round node is needed. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "Add2_node = onnx.helper.make_node(\n",
+ " 'Add',\n",
+ " inputs=['sum1', 'in3'],\n",
+ " outputs=['sum2'],\n",
+ ")\n",
+ "\n",
+ "Add3_node = onnx.helper.make_node(\n",
+ " 'Add',\n",
+ " inputs=['sum2', 'abs1'],\n",
+ " outputs=['sum3'],\n",
+ ")\n",
+ "\n",
+ "Abs_node = onnx.helper.make_node(\n",
+ " 'Abs',\n",
+ " inputs=['sum2'],\n",
+ " outputs=['abs1'],\n",
+ ")\n",
+ "\n",
+ "Round_node = onnx.helper.make_node(\n",
+ " 'Round',\n",
+ " inputs=['sum3'],\n",
+ " outputs=['out1'],\n",
+ ")\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Seeing the names of the inputs and outputs of the nodes the structure of the resulting graph can already be recognized. In order to integrate the nodes into a graph environment, the inputs and outputs of the graph have to be specified first. In ONNX all data edges are processed as tensors. So with the helper function tensor value infos are created for the input and output tensors of the graph. For the data type float from ONNX is used. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "in1 = onnx.helper.make_tensor_value_info(\"in1\", onnx.TensorProto.FLOAT, [4, 4])\n",
+ "in2 = onnx.helper.make_tensor_value_info(\"in2\", onnx.TensorProto.FLOAT, [4, 4])\n",
+ "in3 = onnx.helper.make_tensor_value_info(\"in3\", onnx.TensorProto.FLOAT, [4, 4])\n",
+ "out1 = onnx.helper.make_tensor_value_info(\"out1\", onnx.TensorProto.FLOAT, [4, 4])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Now the graph can be built. First all nodes are passed. Here it is to be noted that it requires a certain sequence. The nodes must be instantiated in their dependencies to each other. This means Add2 must not be listed before Add1, because Add2 depends on the result of Add1. A name is then assigned to the graph. This is followed by the inputs and outputs. \n",
+ "\n",
+ "`value_info` of the graph contains the remaining tensors within the graph. When creating the nodes we have already defined names for the inner edges and now these are assigned tensors of the datatype float and a certain shape.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ " graph = onnx.helper.make_graph(\n",
+ " nodes=[\n",
+ " Add1_node,\n",
+ " Add2_node,\n",
+ " Abs_node,\n",
+ " Add3_node,\n",
+ " Round_node,\n",
+ " ],\n",
+ " name=\"simple_graph\",\n",
+ " inputs=[in1, in2, in3],\n",
+ " outputs=[out1],\n",
+ " value_info=[\n",
+ " onnx.helper.make_tensor_value_info(\"sum1\", onnx.TensorProto.FLOAT, [4, 4]),\n",
+ " onnx.helper.make_tensor_value_info(\"sum2\", onnx.TensorProto.FLOAT, [4, 4]),\n",
+ " onnx.helper.make_tensor_value_info(\"abs1\", onnx.TensorProto.FLOAT, [4, 4]),\n",
+ " onnx.helper.make_tensor_value_info(\"sum3\", onnx.TensorProto.FLOAT, [4, 4]),\n",
+ " ],\n",
+ " )\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">**Important**: In our example, the shape of the tensors does not change during the calculation. This is not always the case. So you have to make sure that you specify the shape correctly.\n",
+ "\n",
+ "Now a model can be created from the graph and saved using the `.save` function. The model is saved in .onnx format and can be reloaded with `onnx.load()`. This also means that you can easily share your own model in .onnx format with others.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "onnx_model = onnx.helper.make_model(graph, producer_name=\"simple-model\")\n",
+ "onnx.save(onnx_model, 'simple_model.onnx')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size='3'>To visualize the created model, [netron](https://github.com/lutzroeder/netron) can be used. Netron is a visualizer for neural network, deep learning and machine learning models. <font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Serving 'simple_model.onnx' at http://0.0.0.0:8081\n"
+ ]
+ }
+ ],
+ "source": [
+ "import netron\n",
+ "netron.start('simple_model.onnx', port=8081, host=\"0.0.0.0\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>\n"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "%%html\n",
+ "<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Netron also allows you to interactively explore the model. If you click on a node, the node attributes will be displayed. \n",
+ "\n",
+ "In order to test the resulting model, a function is first written in Python that calculates the expected output. Because numpy arrays are to be used, numpy is imported first.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "def expected_output(in1, in2, in3):\n",
+ " sum1 = np.add(in1, in2)\n",
+ " sum2 = np.add(sum1, in3)\n",
+ " abs1 = np.absolute(sum2)\n",
+ " sum3 = np.add(abs1, sum2)\n",
+ " return np.round(sum3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">Then the values for the three inputs are calculated. Random numbers are used.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "in1_values =np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)\n",
+ "in2_values = np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)\n",
+ "in3_values = np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">We can easily pass the values to the function we just wrote to calculate the expected result. For the created model the inputs must be summarized in a dictionary, which is then passed on to the model.</font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "input_dict = {}\n",
+ "input_dict[\"in1\"] = in1_values\n",
+ "input_dict[\"in2\"] = in2_values\n",
+ "input_dict[\"in3\"] = in3_values"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">To run the model and calculate the output, [onnxruntime](https://github.com/microsoft/onnxruntime) can be used. ONNX Runtime is a performance-focused complete scoring engine for Open Neural Network Exchange (ONNX) models. The `.InferenceSession` function can be used to create a session of the model and .run can be used to execute the model. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import onnxruntime as rt\n",
+ "\n",
+ "sess = rt.InferenceSession(onnx_model.SerializeToString())\n",
+ "output = sess.run(None, input_dict)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<font size=\"3\">The input values are also transferred to the reference function. Now the output of the execution of the model can be compared with that of the reference. </font>"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "ref_output= expected_output(in1_values, in2_values, in3_values)\n",
+ "assert output[0].all() == ref_output.all()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ },
{
"cell_type": "code",
"execution_count": null,
--
GitLab