Skip to content
Snippets Groups Projects
Commit cb53b018 authored by auphelia's avatar auphelia
Browse files

[notebook] Added new content and changed model so it is easier to use... 

[notebook] Added new content and changed model so it is easier to use find_consumer and find_producer on the model
parent 599c7695
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
notebooks/FINN-HowToWorkWithONNX.ipynb
+ 227
− 18
View file @ cb53b018
%% Cell type:markdown id: tags:
# FINN - How to work with ONNX
<font size="3">This notebook should give an overview of ONNX ProtoBuf and help to create and manipulate an ONNX model and use FINN functions to work with it. There may be overlaps to other notebooks, like [FINN-ModelWrapper](FINN-ModelWrapper.ipynb) and [FINN-CustomOps](FINN-CustomOps.ipynb), but this notebook should give an overview about the handling of ONNX models in FINN. </font>
%% Cell type:markdown id: tags:
## Outline
* #### How to create a simple model
* #### How to manipulate an ONNX model
%% Cell type:markdown id: tags:
### How to create a simple model
<font size="3">To explain how to create an ONNX graph a simple example with mathematical operations is used. All nodes are from the [standard operations library of ONNX](https://github.com/onnx/onnx/blob/master/docs/Operators.md).
First ONNX is imported, then the helper function can be used to make a node.</font>
%% Cell type:code id: tags:
``` python
import onnx
Add1_node = onnx.helper.make_node(
'Add',
inputs=['in1', 'in2'],
outputs=['sum1'],
)
```
%% Cell type:markdown id: tags:
<font size="3">The first attribute of the node is the operation type. In this case it is `'Add'`, so it is an adder node. Then the input names are passed to the node and at the end a name is assigned to the output.
For this example we want two other adder nodes, one abs node and the output shall be rounded so one round node is needed. </font>
%% Cell type:code id: tags:
``` python
Add2_node = onnx.helper.make_node(
'Add',
inputs=['sum1', 'in3'],
outputs=['sum2'],
)
Add3_node = onnx.helper.make_node(
'Add',
inputs=['sum2', 'abs1'],
inputs=['abs1', 'abs1'],
outputs=['sum3'],
)
Abs_node = onnx.helper.make_node(
'Abs',
inputs=['sum2'],
outputs=['abs1'],
)
Round_node = onnx.helper.make_node(
'Round',
inputs=['sum3'],
outputs=['out1'],
)
```
%% Cell type:markdown id: tags:
<font size="3">Seeing the names of the inputs and outputs of the nodes the structure of the resulting graph can already be recognized. In order to integrate the nodes into a graph environment, the inputs and outputs of the graph have to be specified first. In ONNX all data edges are processed as tensors. So with the helper function tensor value infos are created for the input and output tensors of the graph. For the data type float from ONNX is used. </font>
%% Cell type:code id: tags:
``` python
in1 = onnx.helper.make_tensor_value_info("in1", onnx.TensorProto.FLOAT, [4, 4])
in2 = onnx.helper.make_tensor_value_info("in2", onnx.TensorProto.FLOAT, [4, 4])
in3 = onnx.helper.make_tensor_value_info("in3", onnx.TensorProto.FLOAT, [4, 4])
out1 = onnx.helper.make_tensor_value_info("out1", onnx.TensorProto.FLOAT, [4, 4])
```
%% Cell type:markdown id: tags:
<font size="3">Now the graph can be built. First all nodes are passed. Here it is to be noted that it requires a certain sequence. The nodes must be instantiated in their dependencies to each other. This means Add2 must not be listed before Add1, because Add2 depends on the result of Add1. A name is then assigned to the graph. This is followed by the inputs and outputs.
`value_info` of the graph contains the remaining tensors within the graph. When creating the nodes we have already defined names for the inner edges and now these are assigned tensors of the datatype float and a certain shape.</font>
%% Cell type:code id: tags:
``` python
graph = onnx.helper.make_graph(
nodes=[
Add1_node,
Add2_node,
Abs_node,
Add3_node,
Round_node,
],
name="simple_graph",
inputs=[in1, in2, in3],
outputs=[out1],
value_info=[
onnx.helper.make_tensor_value_info("sum1", onnx.TensorProto.FLOAT, [4, 4]),
onnx.helper.make_tensor_value_info("sum2", onnx.TensorProto.FLOAT, [4, 4]),
onnx.helper.make_tensor_value_info("abs1", onnx.TensorProto.FLOAT, [4, 4]),
onnx.helper.make_tensor_value_info("sum3", onnx.TensorProto.FLOAT, [4, 4]),
],
)
```
%% Cell type:markdown id: tags:
<font size="3">**Important**: In our example, the shape of the tensors does not change during the calculation. This is not always the case. So you have to make sure that you specify the shape correctly.
Now a model can be created from the graph and saved using the `.save` function. The model is saved in .onnx format and can be reloaded with `onnx.load()`. This also means that you can easily share your own model in .onnx format with others.</font>
%% Cell type:code id: tags:
``` python
onnx_model = onnx.helper.make_model(graph, producer_name="simple-model")
onnx.save(onnx_model, 'simple_model.onnx')
```
%% Cell type:markdown id: tags:
<font size='3'>To visualize the created model, [netron](https://github.com/lutzroeder/netron) can be used. Netron is a visualizer for neural network, deep learning and machine learning models. <font>
%% Cell type:code id: tags:
``` python
import netron
netron.start('simple_model.onnx', port=8081, host="0.0.0.0")
```
%% Output
Stopping http://0.0.0.0:8081
Serving 'simple_model.onnx' at http://0.0.0.0:8081
%% Cell type:code id: tags:
``` python
%%html
<iframe src="http://0.0.0.0:8081/" style="position: relative; width: 100%;" height="400"></iframe>
```
%% Output
%% Cell type:markdown id: tags:
<font size="3">Netron also allows you to interactively explore the model. If you click on a node, the node attributes will be displayed.
In order to test the resulting model, a function is first written in Python that calculates the expected output. Because numpy arrays are to be used, numpy is imported first.</font>
%% Cell type:code id: tags:
``` python
import numpy as np
def expected_output(in1, in2, in3):
sum1 = np.add(in1, in2)
sum2 = np.add(sum1, in3)
abs1 = np.absolute(sum2)
sum3 = np.add(abs1, sum2)
sum3 = np.add(abs1, abs1)
return np.round(sum3)
```
%% Cell type:markdown id: tags:
<font size="3">Then the values for the three inputs are calculated. Random numbers are used.</font>
%% Cell type:code id: tags:
``` python
in1_values =np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)
in2_values = np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)
in3_values = np.asarray(np.random.uniform(low=-5, high=5, size=(4,4)), dtype=np.float32)
```
%% Cell type:markdown id: tags:
<font size="3">We can easily pass the values to the function we just wrote to calculate the expected result. For the created model the inputs must be summarized in a dictionary, which is then passed on to the model.</font>
%% Cell type:code id: tags:
``` python
input_dict = {}
input_dict["in1"] = in1_values
input_dict["in2"] = in2_values
input_dict["in3"] = in3_values
```
%% Cell type:markdown id: tags:
<font size="3">To run the model and calculate the output, [onnxruntime](https://github.com/microsoft/onnxruntime) can be used. ONNX Runtime is a performance-focused complete scoring engine for Open Neural Network Exchange (ONNX) models. The `.InferenceSession` function can be used to create a session of the model and .run can be used to execute the model. </font>
%% Cell type:code id: tags:
``` python
import onnxruntime as rt
sess = rt.InferenceSession(onnx_model.SerializeToString())
output = sess.run(None, input_dict)
```
%% Cell type:markdown id: tags:
<font size="3">The input values are also transferred to the reference function. Now the output of the execution of the model can be compared with that of the reference. </font>
%% Cell type:code id: tags:
``` python
ref_output= expected_output(in1_values, in2_values, in3_values)
assert output[0].all() == ref_output.all()
print("The output of the ONNX model is: \n{}".format(output[0]))
print("\nThe output of the reference function is: \n{}".format(ref_output))
if (output[0] == ref_output).all():
print("\nThe results are the same!")
else:
raise Exception("Something went wrong, the output of the model doesn't match the expected output!")
```
%% Output
The output of the ONNX model is:
[[24. 2. 6. 2.]
[ 5. 1. 9. 5.]
[ 6. 1. 8. 8.]
[14. 4. 0. 1.]]
The output of the reference function is:
[[24. 2. 6. 2.]
[ 5. 1. 9. 5.]
[ 6. 1. 8. 8.]
[14. 4. 0. 1.]]
The results are the same!
%% Cell type:markdown id: tags:
<font size="3">Now that we have verified that the model works as we expected it to, we can continue working with the graph. </font>
%% Cell type:markdown id: tags:
### How to manipulate an ONNX model
<font size="3">In the model there are two successive adder nodes. An adder node in ONNX can only add two inputs, but there is also the [**sum**](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Sum) node, which can process more than one input. So it would be a reasonable change of the graph to combine the two successive adder nodes to one sum node. </font>
%% Cell type:markdown id: tags:
<font size="3">In the following we assume that we do not know the appearance of the model, so we first try to identify whether there are two consecutive adders in the graph and then convert them into a sum node.
Here we make use of FINN. FINN provides a thin wrapper around the model which provides several additional helper functions to manipulate the graph. The code can be found [here](https://github.com/Xilinx/finn/blob/dev/src/finn/core/modelwrapper.py) and you can find a more detailed description in the notebook [FINN-ModelWrapper](FINN-ModelWrapper.ipynb).</font>
%% Cell type:code id: tags:
``` python
from finn.core.modelwrapper import ModelWrapper
finn_model = ModelWrapper(onnx_model)
```
%% Cell type:markdown id: tags:
<font size="3">Now we design a function that searches for adder nodes in the graph and returns the found nodes. </font>
%% Cell type:code id: tags:
``` python
def identify_adder_nodes(model):
add_nodes = []
for node in model.graph.node:
if node.op_type == "Add":
add_nodes.append(node)
return add_nodes
```
%% Cell type:markdown id: tags:
<font size="3">The function iterates over all nodes of the model and if the operation type is "Add" the node will be stored in `add_nodes`. At the end `add_nodes` is returned.
If we apply this to our model, three nodes should be returned.</font>
%% Cell type:code id: tags:
``` python
add_nodes = identify_adder_nodes(finn_model)
for node in add_nodes:
print("Found adder node: \n{}".format(node))
```
%% Output
Found adder node:
input: "in1"
input: "in2"
output: "sum1"
op_type: "Add"
Found adder node:
input: "sum1"
input: "in3"
output: "sum2"
op_type: "Add"
Found adder node:
input: "abs1"
input: "abs1"
output: "sum3"
op_type: "Add"
%% Cell type:markdown id: tags:
<font size="3">Among other helper functions, ModelWrapper offers two functions to determine the preceding and succeeding node of a node. However, these functions are not getting a node as input, but can determine the consumer or producer of a tensor. We write two functions that uses these helper functions to determine the previous and the next node of a node.</font>
%% Cell type:code id: tags:
``` python
def find_predecessor(model, node):
predecessors = []
for i in range(len(node.input)):
producer = model.find_producer(node.input[i])
predecessors.append(producer)
return predecessors
def find_successor(model, node):
successors = []
for i in range(len(node.output)):
consumer = model.find_consumer(node.output[i])
successors.append(consumer)
return successors
```
%% Cell type:markdown id: tags:
<font size="3">The first function uses `find_producer` from ModelWrapper to create a list of the producers of the inputs of the given node. So the returned list is indirectly filled with the predecessors of the node. The second function works in a similar way, `find_consumer` from ModelWrapper is used to find the consumers of the output tensors of the node and so a list with the successors can be created.
So now we can find out which adder node has an adder node as successor</font>
%% Cell type:code id: tags:
``` python
def adder_pair(model, node):
successor_list = find_successor(model, node)
node_pair = []
for successor in successor_list:
if successor.op_type == "Add":
node_pair.append(node)
node_pair.append(successor)
return node_pair
```
%% Cell type:code id: tags:
``` python
i = 0
for node in add_nodes:
add_pair = adder_pair(finn_model, node)
if len(add_pair) != 0:
print(add_pair)
i+=1
```
%% Output
[input: "in1"
input: "in2"
output: "sum1"
op_type: "Add"
, input: "sum1"
input: "in3"
output: "sum2"
op_type: "Add"
]
%% Cell type:code id: tags:
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment