Skip to content
Snippets Groups Projects
Commit 491d882a authored by auphelia's avatar auphelia
Browse files

[notebook] Fixed some grammar mistakes

parent 9515ba56
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
notebooks/FINN-VerificationHLSCustomOp.ipynb
+ 37
− 41
View file @ 491d882a
%% Cell type:raw id: tags:
%% Cell type:markdown id: tags:
# FINN - Verification of an HLSCustomOp node
-----------------------------------------------------------------
This notebook is about the verification flow and options for FINN custom operation nodes.
Following showSrc function is used to print the source code of function calls in the Jupyter notebook:
%% Cell type:code id: tags:
``` python
import inspect
def showSrc(what):
print("".join(inspect.getsourcelines(what)[0]))
```
%% Cell type:markdown id: tags:
## Outline
-------------
* Example model (sliding window function)
* c++ high level simulation
* Vivado IP synthesis and pyverilator execution flow
%% Cell type:markdown id: tags:
### Example model
To show the possibilities we have to verify a FINN HLSCustomOp node, an example model with the [sliding window function] (https://finn-hlslib.readthedocs.io/en/latest/library/swg.html) of the finn-hlslib is used. For that a corresponding ONNX node is created. The ONNX node contains all the template parameters of the corresponding finn-hlslib function as attributes. The function is shown below.
To show the possibilities of how to verify a FINN HLSCustomOp node, an example model with the [sliding window function](https://finn-hlslib.readthedocs.io/en/latest/library/swg.html) of the finn-hlslib is used. For that a corresponding ONNX node is created. The ONNX node contains all the template parameters of the corresponding finn-hlslib function as attributes. The function is shown below.
%% Cell type:markdown id: tags:
![title](im2col_finnhlslib.PNG)
%% Cell type:markdown id: tags:
In the next step the individual parameters are defined. At first the class 'DataType' is imported from FINN to be able to use data types like bipolar. With the member function `bitwidth()` the parameter `Input_precision` can be derived directly from this data type. The other parameters are set to reasonable values. The output dimension can be calculated using the input dimension, the kernel size and the value for stride.
%% Cell type:code id: tags:
``` python
from finn.core.datatype import DataType
idt = DataType.BIPOLAR # input data type
ip = idt.bitwidth() # input precision
k = 2 # kernel size
ifm_dim = 4 # input dimension
ifm_ch = 1 # input channels
stride = 2 # stride
simd = ifm_ch # simd
# output dimension
ofm_dim = int(((ifm_dim - k) / stride) + 1)
```
%% Cell type:markdown id: tags:
An additional variable is defined to be able to infer the shape of the output tensor. Furthermore the output data type is set to the same value as the input data type.
%% Cell type:code id: tags:
``` python
out_pix = ofm_dim * ofm_dim
odt = idt
```
%% Cell type:markdown id: tags:
To create an ONNX node, first TensorProto and helper are imported from ONNX. These can be used to create tensors, nodes, graphs and models in ONNX. After importing, the input and output tensors can be created.
%% Cell type:code id: tags:
``` python
from onnx import TensorProto, helper
inp = helper.make_tensor_value_info(
"inp", TensorProto.FLOAT, [1, ifm_ch, ifm_dim, ifm_dim]
)
outp = helper.make_tensor_value_info(
"outp", TensorProto.FLOAT, [1, out_pix, k * k * ifm_ch]
)
```
%% Cell type:markdown id: tags:
Now the node can be built. This node is directly integrated into a graph environment and from this the ONNX model is created. For more information about the creation and manipulation of an ONNX model, please refer to jupyter notebook [FINN-HowToWorkWithONNX](FINN-HowToWorkWithONNX.ipynb).
%% Cell type:code id: tags:
``` python
SlidingWindow_node = helper.make_node(
"ConvolutionInputGenerator",
["inp"],
["outp"],
domain="finn",
backend="fpgadataflow",
ConvKernelDim=k,
IFMChannels=ifm_ch,
Input_precision=ip,
IFMDim=ifm_dim,
OFMDim=ofm_dim,
SIMD=simd,
Stride=stride,
inputDataType=idt.name,
outputDataType=odt.name,
)
graph = helper.make_graph(
nodes=[SlidingWindow_node],
name="slidingwindow_graph",
inputs=[inp],
outputs=[outp],
)
model = helper.make_model(graph, producer_name="slidingwindow-model")
```
%% Cell type:markdown id: tags:
FINN provides a thin wrapper around the ONNX model with a lot of helper functions that can be used by importing the class `ModelWrapper`. More information about `ModelWrapper` can be found in Jupyter notebook [FINN-ModelWrapper](FINN-ModelWrapper.ipynb). Here it is used to assign data types to the tensors FINN.
FINN provides a thin wrapper around the ONNX model with a lot of helper functions that can be used by importing the class `ModelWrapper`. More information about `ModelWrapper` can be found in Jupyter notebook [FINN-ModelWrapper](FINN-ModelWrapper.ipynb). Here it is used to assign FINN data types to the tensors.
%% Cell type:code id: tags:
``` python
from finn.core.modelwrapper import ModelWrapper
model = ModelWrapper(model)
model.set_tensor_datatype("inp", idt)
model.set_tensor_datatype("outp", odt)
```
%% Cell type:markdown id: tags:
What the model looks like can be visualized with netron. Netron is a visualizer for neural network, deep learning and machine learning models. For this the model is first saved.
%% Cell type:code id: tags:
``` python
model.save("original_model.onnx")
```
%% Cell type:code id: tags:
``` python
import netron
netron.start('original_model.onnx', port=8081, host="0.0.0.0")
```
%% Output
Serving 'original_model.onnx' at http://0.0.0.0:8081
%% Cell type:code id: tags:
``` python
%%html
<iframe src="http://0.0.0.0:8081/" style="position: relative; width: 100%;" height="400"></iframe>
```
%% Output
%% Cell type:markdown id: tags:
Now that we have the model, we can use various features of FINN to manipulate it. The basic principle of FINN is that there are transformations and analysis passes that can be applied to a model. A transformation pass changes a given model an returns the changed model. An analysis pass traverses the graph structure and produces information about certain properties. It returns a dictionary of named properties.
Now that we have the model, we can use various features of FINN to manipulate it. The basic principle of FINN is that there are transformation and analysis passes that can be applied to a model. A transformation pass changes a given model and returns the changed model. An analysis pass traverses the graph structure and produces information about certain properties. It returns a dictionary of named properties.
The following section describes the transformation passes that can be used to verify an HLSCustomOp node. Firstly the verification with a c++ high level simulation is shown and afterwards with a Vivado IP synthesis and pyverilator execution flow.
%% Cell type:markdown id: tags:
### c++ high level simulation
First, an additional attribute must be set to specify which of the two verification types should be used when executing the node. This is done with the transformation pass `SetSimMode`, to which the desired mode is passed. After that the transformation pass `CodeGen_npysim` can be applied. With this transformation c++ code is generated and stored in a temporary directory. In addition, a further attribute is set, which contains the path to this directory.
%% Cell type:code id: tags:
``` python
from finn.transformation.fpgadataflow.set_sim_mode import SetSimMode
from finn.transformation.fpgadataflow.codegen_npysim import CodeGen_npysim
model = model.transform(SetSimMode("npysim"))
model = model.transform(CodeGen_npysim())
```
%% Cell type:markdown id: tags:
If you now save the model again and display it, these changes can be seen.
%% Cell type:code id: tags:
``` python
model.save("modified_model.onnx")
netron.start('modified_model.onnx', port=8081, host="0.0.0.0")
```
%% Output
Stopping http://0.0.0.0:8081
Serving 'modified_model.onnx' at http://0.0.0.0:8081
%% Cell type:code id: tags:
``` python
%%html
<iframe src="http://0.0.0.0:8081/" style="position: relative; width: 100%;" height="400"></iframe>
```
%% Output
%% Cell type:markdown id: tags:
The next step is to create the executable from the .cpp file using the `Compile` transformation. The path to the executable is also stored in a new attribute.
%% Cell type:code id: tags:
``` python
from finn.transformation.fpgadataflow.compile import Compile
model = model.transform(Compile())
```
%% Cell type:markdown id: tags:
All required files are now available and we can execute the node. This is done with the `execute_onnx` function, which gets the model and an input dictionary. That means we have to create an input tensor first. For this we use a numpy array.
%% Cell type:code id: tags:
``` python
import numpy as np
x = np.asarray([-1, -1, 1, 1, -1, -1, -1, -1, 1, -1, 1, -1, 1, 1, 1, -1], dtype=np.float32).reshape(1, ifm_ch, ifm_dim, ifm_dim)
print(x)
input_dict = {"inp": (x + 1) /2}
```
%% Output
[[[[-1. -1. 1. 1.]
[-1. -1. -1. -1.]
[ 1. -1. 1. -1.]
[ 1. 1. 1. -1.]]]]
%% Cell type:markdown id: tags:
To be able to use `execute_onnx()` `onnx_exec` must be imported. Inside `execute_onnx()` the attribute `sim_mode` is read and if "npysim" is selected, the input array is saved in a .npy file and the previously created executable is executed. The output is saved in another .npy file and is read by `execute_onnx()` and saved as output.
%% Cell type:code id: tags:
``` python
import finn.core.onnx_exec as oxe
y_npysim = oxe.execute_onnx(model, input_dict)["outp"]
print(y_npysim)
```
%% Output
[[[-1. -1. -1. -1.]
[ 1. 1. -1. -1.]
[ 1. -1. 1. 1.]
[ 1. -1. 1. -1.]]]
%% Cell type:markdown id: tags:
A different transformation flow can be used for verification. This will be discussed in the next section.
%% Cell type:markdown id: tags:
### Vivado IP synthesis and pyverilator execution flow
In this verification a .cpp code is generated from the node, which is synthesized to an IP block using Vivado. Afterwards the functionality can be simulated with [pyverilator](https://github.com/maltanar/pyverilator). Pyverilator is a tool which makes it possible to simulate verilog files using verilator via a python interface.
%% Cell type:markdown id: tags:
In the first step `sim_mode` must be set to "rtlsim" in order to select the corresponding functionality when executing the node. In addition, the nodes in the model are assigned unique names using the `GiveUniqueNodeNames()` transformation. Then the transformation `CodeGen_ipgen()` can be executed. Two arguments are passed to this transformation, one is an fpga part and the other is a value for the clock.
%% Cell type:code id: tags:
``` python
from finn.transformation.general import GiveUniqueNodeNames
from finn.transformation.fpgadataflow.codegen_ipgen import CodeGen_ipgen
model = model.transform(SetSimMode("rtlsim"))
model = model.transform(GiveUniqueNodeNames())
model = model.transform(CodeGen_ipgen("xc7z020clg400-1", 5))
```
%% Cell type:markdown id: tags:
During the transformation a new attribute with the temporary directory is set, in which the .cpp and a .tcl script are stored, with which the synthesis can be started. This can be seen in the following using netron.
%% Cell type:code id: tags:
``` python
model.save("modified_model.onnx")
netron.start('modified_model.onnx', port=8081, host="0.0.0.0")
```
%% Output
Stopping http://0.0.0.0:8081
Serving 'modified_model.onnx' at http://0.0.0.0:8081
%% Cell type:code id: tags:
``` python
%%html
<iframe src="http://0.0.0.0:8081/" style="position: relative; width: 100%;" height="400"></iframe>
```
%% Output
%% Cell type:markdown id: tags:
The next step is to perform the synthesis using the `HLSSynth_IPGen()` transformation and set another attribute with the project directory, which contains the IP block.
So that the execution can run without errors, two env variables must be set inside the jupyter notebook.
%% Cell type:code id: tags:
``` python
# env variable has to be set because it is used inside the trafo
%env PWD=/workspace/finn/notebooks
%env VIVADO_PATH=/home/maltanar/Xilinx/Vivado/2017.4
from finn.transformation.fpgadataflow.hlssynth_ipgen import HLSSynth_IPGen
model = model.transform(HLSSynth_IPGen())
```
%% Output
env: PWD=/workspace/finn/notebooks
env: VIVADO_PATH=/home/maltanar/Xilinx/Vivado/2017.4
%% Cell type:markdown id: tags:
Now the execution can run again and pyverilator is used in the background to simulate the generated verilog files.
%% Cell type:code id: tags:
``` python
y_rtlsim = oxe.execute_onnx(model, input_dict)["outp"]
y_rtlsim
```
%% Output
array([[[-1., -1., -1., -1.],
[ 1., 1., -1., -1.],
[ 1., -1., 1., 1.],
[ 1., -1., 1., -1.]]], dtype=float32)
%% Cell type:markdown id: tags:
In the last step it can be checked whether the two results from the simulations match.
%% Cell type:code id: tags:
``` python
assert (y_npysim == y_rtlsim).all()
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment