diff --git a/notebooks/FINN-HowToWorkWithONNX.ipynb b/notebooks/FINN-HowToWorkWithONNX.ipynb
index 8ce25ba19fc83dc7cc5382e6502ba73fef851490..5256f9e388eee635d51c2f224ee01f2e0c0ebabe 100644
--- a/notebooks/FINN-HowToWorkWithONNX.ipynb
+++ b/notebooks/FINN-HowToWorkWithONNX.ipynb
@@ -24,14 +24,14 @@
"source": [
"### How to create a simple model\n",
"\n",
- "To explain how to create an ONNX graph a simple example with mathematical operations is used. All nodes are from the [standard operations library of ONNX](https://github.com/onnx/onnx/blob/master/docs/Operators.md).\n",
+ "To explain how to create an ONNX model a simple example with mathematical operations is used. All nodes are from the [standard operations library of ONNX](https://github.com/onnx/onnx/blob/master/docs/Operators.md).\n",
"\n",
"First ONNX is imported, then the helper function can be used to make a node."
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -283,9 +283,21 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": 44,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "ename": "NameError",
+ "evalue": "name 'input_dict' is not defined",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
+ "\u001b[0;32m<ipython-input-44-143d22d72f78>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0msess\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mInferenceSession\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0monnx_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSerializeToString\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput_dict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[0;31mNameError\u001b[0m: name 'input_dict' is not defined"
+ ]
+ }
+ ],
"source": [
"import onnxruntime as rt\n",
"\n",
@@ -363,13 +375,27 @@
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"from finn.core.modelwrapper import ModelWrapper\n",
- "finn_model = ModelWrapper(onnx_model)\n",
- "\n",
+ "finn_model = ModelWrapper(onnx_model)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As explained in the previous section, it is important that the nodes are listed in the correct order. If a new node has to be inserted or an old node has to be replaced, it is important to do that in the appropriate position. The following function serves this purpose. It returns a dictionary, which contains the node name as key and the respective node index as value."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [],
+ "source": [
"def get_node_id(model):\n",
" node_index = {}\n",
" node_ind = 0\n",
@@ -383,12 +409,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now we design a function that searches for adder nodes in the graph and returns the found nodes. "
+ "The function scans the list of nodes and stores a run index (`node_ind`) as node index in the dictionary for every node name.\n",
+ "\n",
+ "Another helper function is being implemented that searches for adder nodes in the graph and returns the found nodes. This is needed to determine if and which adder nodes are in the given model."
]
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
@@ -397,8 +425,7 @@
" for node in model.graph.node:\n",
" if node.op_type == \"Add\":\n",
" add_nodes.append(node)\n",
- " return add_nodes\n",
- " "
+ " return add_nodes"
]
},
{
@@ -412,7 +439,7 @@
},
{
"cell_type": "code",
- "execution_count": 15,
+ "execution_count": 21,
"metadata": {},
"outputs": [
{
@@ -435,12 +462,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Among other helper functions, ModelWrapper offers two functions to determine the preceding and succeeding node of a node. However, these functions are not getting a node as input, but can determine the consumer or producer of a tensor. We write two functions that uses these helper functions to determine the previous and the next node of a node."
+ "Among other helper functions, `ModelWrapper` offers two functions that can help to determine the preceding and succeeding node of a node. However, these functions are not getting a node as input, but can determine the consumer or producer of a tensor. We write two functions that uses these helper functions to determine the previous and the next node of a node."
]
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
@@ -464,14 +491,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "The first function uses `find_producer` from ModelWrapper to create a list of the producers of the inputs of the given node. So the returned list is indirectly filled with the predecessors of the node. The second function works in a similar way, `find_consumer` from ModelWrapper is used to find the consumers of the output tensors of the node and so a list with the successors can be created. \n",
- "\n",
- "So now we can find out which adder node has an adder node as successor."
+ "The first function uses `find_producer` from `ModelWrapper` to create a list of the producers of the inputs of the given node. So the returned list is indirectly filled with the predecessors of the node. The second function works in a similar way, `find_consumer` from `ModelWrapper` is used to find the consumers of the output tensors of the node and so a list with the successors can be created. "
]
},
{
"cell_type": "code",
- "execution_count": 17,
+ "execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
@@ -494,12 +519,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "The function gets a node and the model as input. Two empty lists are created to be filled with a list of adder node pairs that can be returned as result of the function. Then the function `find_successor` is used to return all of the successors of the node. If one of the successors is an adder node, the node is saved in `node_pair` together with the successive adder node and put in the list `adder_pairs`. Then the temporary list is cleaned and can be filled with the next adder node pair."
+ "The function gets a node and the model as input. Two empty lists are created to be filled with a list of adder node pairs that can be returned as result of the function. Then the function `find_successor` is used to return all of the successors of the node. If one of the successors is an adder node, the node is saved in `node_pair` together with the successive adder node and put in the list `adder_pairs`. Then the temporary list is cleaned and can be filled with the next adder node pair. Since it is theoretically possible for an adder node to have more than one subsequent adder node, a list of lists is created. This list of the node with all its successive adder nodes is returned.\n",
+ "\n",
+ "So now we can find out which adder node has an adder node as successor. Since the model is known, one adder pair (Add1+Add2) should be found when applying the function to the previously determined adder node list (`add_nodes`)."
]
},
{
"cell_type": "code",
- "execution_count": 18,
+ "execution_count": 30,
"metadata": {},
"outputs": [
{
@@ -516,25 +543,26 @@
"for node in add_nodes:\n",
" add_pairs = adder_pair(finn_model, node)\n",
" if len(add_pairs) != 0:\n",
- " substitute_pair = add_pairs[0]\n",
- " print(\"Found following pair that could be replaced by a sum node:\")\n",
- " for node_pair in add_pairs:\n",
- " for node in node_pair:\n",
- " print(node.name)"
+ " for i in range(len(add_pairs)):\n",
+ " substitute_pair = add_pairs[i]\n",
+ " print(\"Found following pair that could be replaced by a sum node:\")\n",
+ " for node_pair in add_pairs:\n",
+ " for node in node_pair:\n",
+ " print(node.name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now that the pair to be replaced has been identified, a sum node can be instantiated and inserted into the graph at the correct position. \n",
+ "Now that the pair to be replaced has been identified (`substitute_pair`), a sum node can be instantiated and inserted into the graph at the correct position. \n",
"\n",
"First of all, the inputs must be determined. For this the adder nodes inputs are used minus the input, which corresponds to the output of the other adder node."
]
},
{
"cell_type": "code",
- "execution_count": 21,
+ "execution_count": 34,
"metadata": {},
"outputs": [
{
@@ -560,30 +588,35 @@
"print(\"The new node gets the following inputs: \\n{}\".format(input_list))"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The output of the sum node matches the output of the second adder node and can therefore be taken over directly."
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": 35,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "sum_output = substitute_pair[1].output[0]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
"metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "sum2\n"
- ]
- }
- ],
"source": [
- "print(substitute_pair[1].output[0])"
+ "The summary node can be created with this information."
]
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
- "sum_output = substitute_pair[1].output[0]\n",
"Sum_node = onnx.helper.make_node(\n",
" 'Sum',\n",
" inputs=input_list,\n",
@@ -592,9 +625,16 @@
")"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The node can now be inserted into the graph and the old nodes are removed."
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 24,
+ "execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
@@ -606,9 +646,23 @@
" graph.node.remove(node)"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To insert the node in the right place, the index of the first node of the substitute_pair is used as node index for the sum node and embedded into the graph using `.insert`. Then the two elements in `substitute_pair` are deleted using `.remove`. `.insert` and `.remove` are functions provided by ONNX."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The new graph is saved as ONNX model and can be visualized with Netron."
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 25,
+ "execution_count": 40,
"metadata": {},
"outputs": [],
"source": [
@@ -618,7 +672,7 @@
},
{
"cell_type": "code",
- "execution_count": 26,
+ "execution_count": 41,
"metadata": {},
"outputs": [
{
@@ -638,7 +692,7 @@
},
{
"cell_type": "code",
- "execution_count": 27,
+ "execution_count": 42,
"metadata": {},
"outputs": [
{
@@ -659,11 +713,30 @@
"<iframe src=\"http://0.0.0.0:8081/\" style=\"position: relative; width: 100%;\" height=\"400\"></iframe>"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Through the visualization it can already be seen that the insertion was successful, but it is still to be checked whether the result remains the same. Therefore the result of the reference function written in the previous section is used and the new model with the input values is simulated. At this point onnxruntime can be used again. The simulation is analogous to the one of the first model in the previous section."
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 28,
+ "execution_count": 43,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "ename": "NameError",
+ "evalue": "name 'rt' is not defined",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
+ "\u001b[0;32m<ipython-input-43-72bdabe6c93b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msess\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mInferenceSession\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0monnx_model1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSerializeToString\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput_dict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+ "\u001b[0;31mNameError\u001b[0m: name 'rt' is not defined"
+ ]
+ }
+ ],
"source": [
"sess = rt.InferenceSession(onnx_model1.SerializeToString())\n",
"output = sess.run(None, input_dict)"