Skip to content
Snippets Groups Projects
Commit e03c89e9 authored by auphelia's avatar auphelia
Browse files

[notebook - customOps] Finished first version of CustomOp notebook

parent f7f1ada0
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
notebooks/FINN-CustomOps.ipynb
+ 52
− 7
View file @ e03c89e9
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# FINN - CustomOps # FINN - CustomOps
----------------------------------------------------------------- -----------------------------------------------------------------
<font size="3">This notebook should give a more detailed insight into FINN custom operation nodes. </font> <font size="3">This notebook should give a more detailed insight into FINN custom operation nodes. </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">Following showSrc function is used to print the source code of function calls in the Jupyter notebook: </font> <font size="3">Following showSrc function is used to print the source code of function calls in the Jupyter notebook: </font>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import inspect import inspect
def showSrc(what): def showSrc(what):
print("".join(inspect.getsourcelines(what)[0])) print("".join(inspect.getsourcelines(what)[0]))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## FINN Custom Ops
---------------------------
<font size="3">FINN uses many custom operations (`op_type` in ONNX NodeProto) that are not defined in the ONNX operator schema. These custom nodes are marked with `domain="finn"` in the protobuf to identify them as such. These nodes can represent specific operations that we need for low-bit networks, or operations that are specific to a particular hardware backend. <font size="3">FINN uses many custom operations (`op_type` in ONNX NodeProto) that are not defined in the ONNX operator schema. These custom nodes are marked with `domain="finn"` in the protobuf to identify them as such. These nodes can represent specific operations that we need for low-bit networks, or operations that are specific to a particular hardware backend.
A very abstract version of a custom op node representing a streaming fc layer is shown below. </font> A very abstract version of a custom op node representing a streaming fc layer is shown below. </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Outline
---------------------------
* <font size="3">FINN-ONNX node</font>
* <font size="3">CustomOp class</font>
* <font size="3">HLSCustomOp class</font>
%% Cell type:markdown id: tags:
## FINN-ONNX node
<font size="3">To create a FINN-ONNX node you can use the helper function of ONNX. Because it is an ONNX NodeProtobuf, but with several additional attributes. </font>
`FCLayer_node = helper.make_node( `FCLayer_node = helper.make_node(
"StreamingFCLayer_Batch", "StreamingFCLayer_Batch",
node_inp_list, node_inp_list,
node_outp_list, node_outp_list,
domain="finn", domain="finn",
backend="fpgadataflow", backend="fpgadataflow",
code_gen_dir="", code_gen_dir="",
executable_path="", executable_path="",
resType="ap_resource_lut()", resType="ap_resource_lut()",
MW=mw, MW=mw,
MH=mh, MH=mh,
SIMD=simd, SIMD=simd,
PE=pe, PE=pe,
inputDataType=<FINN DataType>, inputDataType=<FINN DataType>,
weightDataType=<FINN DataType>, weightDataType=<FINN DataType>,
outputDataType=<FINN DataType>, outputDataType=<FINN DataType>,
ActVal=actval, ActVal=actval,
binaryXnorMode=<0/1>, binaryXnorMode=<0/1>,
noActivation=<0/1> noActivation=<0/1>
)` )`
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">Unlike standard nodes, the custom op nodes has several additional attributes. The node is created using the helper function of ONNX. `"StreamingFCLayer_Batch"` describes the op_type, then the inputs and outputs are declared. Since this is a custom op node of FINN, the attribute `domain="finn"` must be set. The streaming fc layer is a custom op from the finn-hls library, this is set in the node using the `backend` attribute. To execute a custom op from the finn-hls library, the corresponding c++ code must be created and an executable must be produced. Where the generated code is stored is specified in the `code_gen_dir` attribute and `executable_path` specifies the path to the produced executable. In addition to the data types of the input and output tensors, the node also contains various other attributes resulting from the parameters of the corresponding finn-hls library function. This will not be discussed here.</font> <font size="3">`"StreamingFCLayer_Batch"` describes the op_type, then the inputs and outputs are declared. This is still like building a default onnx node without additional attributes. But since this is a custom op node of FINN, the attribute `domain="finn"` must be set. The streaming fc layer is a custom op from the finn-hls library, this information is set in the node using the `backend` attribute. To execute a custom op from the finn-hls library, the corresponding c++ code must be created and an executable must be produced. Where the generated code is stored is specified in the `code_gen_dir` attribute and `executable_path` specifies the path to the produced executable. In addition to the data types of the input and output tensors, the node also contains various other attributes resulting from the parameters of the corresponding finn-hls library function. More detailed information can be found in the documentation of [finn-hlslib](github.com/Xilinx/finn-hlslib).</font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">Custom Ops are represented in FINN as ONNX nodes on the one hand and by a CustomOp class on the other hand. This allows easier access to the different attributes and introduces special custom op functions. See below for the standard CustomOp class.</font> ## CustomOp class
<font size="3">Custom Ops are represented in FINN as ONNX nodes on the one hand and by a CustomOp class on the other hand. This allows easier access to different attributes and introduces special custom op functions. See below for the standard CustomOp class.</font>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from finn.custom_op import CustomOp from finn.custom_op import CustomOp
showSrc(CustomOp) showSrc(CustomOp)
``` ```
%% Output %% Output
class CustomOp(ABC): class CustomOp(ABC):
def __init__(self, onnx_node): def __init__(self, onnx_node):
super().__init__() super().__init__()
self.onnx_node = onnx_node self.onnx_node = onnx_node
def get_nodeattr(self, name): def get_nodeattr(self, name):
"""Get a node attribute by name. Data is stored inside the ONNX node's """Get a node attribute by name. Data is stored inside the ONNX node's
AttributeProto container. Attribute must be part of get_nodeattr_types. AttributeProto container. Attribute must be part of get_nodeattr_types.
Default value is returned if attribute is not set.""" Default value is returned if attribute is not set."""
try: try:
(dtype, req, def_val) = self.get_nodeattr_types()[name] (dtype, req, def_val) = self.get_nodeattr_types()[name]
attr = get_by_name(self.onnx_node.attribute, name) attr = get_by_name(self.onnx_node.attribute, name)
if attr is not None: if attr is not None:
# dtype indicates which ONNX Attribute member to use # dtype indicates which ONNX Attribute member to use
# (such as i, f, s...) # (such as i, f, s...)
ret = attr.__getattribute__(dtype) ret = attr.__getattribute__(dtype)
if dtype == "s": if dtype == "s":
# decode string attributes # decode string attributes
ret = ret.decode("utf-8") ret = ret.decode("utf-8")
return ret return ret
else: else:
# not set, return default value # not set, return default value
return def_val return def_val
except KeyError: except KeyError:
raise AttributeError("Op has no such attribute: " + name) raise AttributeError("Op has no such attribute: " + name)
def set_nodeattr(self, name, value): def set_nodeattr(self, name, value):
"""Set a node attribute by name. Data is stored inside the ONNX node's """Set a node attribute by name. Data is stored inside the ONNX node's
AttributeProto container. Attribute must be part of get_nodeattr_types.""" AttributeProto container. Attribute must be part of get_nodeattr_types."""
try: try:
(dtype, req, def_val) = self.get_nodeattr_types()[name] (dtype, req, def_val) = self.get_nodeattr_types()[name]
attr = get_by_name(self.onnx_node.attribute, name) attr = get_by_name(self.onnx_node.attribute, name)
if attr is not None: if attr is not None:
# dtype indicates which ONNX Attribute member to use # dtype indicates which ONNX Attribute member to use
# (such as i, f, s...) # (such as i, f, s...)
if dtype == "s": if dtype == "s":
# encode string attributes # encode string attributes
value = value.encode("utf-8") value = value.encode("utf-8")
attr.__setattr__(dtype, value) attr.__setattr__(dtype, value)
else: else:
# not set, create and insert AttributeProto # not set, create and insert AttributeProto
attr_proto = helper.make_attribute(name, value) attr_proto = helper.make_attribute(name, value)
self.onnx_node.attribute.append(attr_proto) self.onnx_node.attribute.append(attr_proto)
except KeyError: except KeyError:
raise AttributeError("Op has no such attribute: " + name) raise AttributeError("Op has no such attribute: " + name)
@abstractmethod @abstractmethod
def get_nodeattr_types(self): def get_nodeattr_types(self):
"""Returns a dict of permitted attributes for node, where: """Returns a dict of permitted attributes for node, where:
returned_dict[attribute_name] = (dtype, require, default_value) returned_dict[attribute_name] = (dtype, require, default_value)
- dtype indicates which member of the ONNX AttributeProto - dtype indicates which member of the ONNX AttributeProto
will be utilized will be utilized
- require indicates whether this attribute is required - require indicates whether this attribute is required
- default_val indicates the default value that will be used if the - default_val indicates the default value that will be used if the
attribute is not set attribute is not set
""" """
pass pass
@abstractmethod @abstractmethod
def make_shape_compatible_op(self): def make_shape_compatible_op(self):
"""Returns a standard ONNX op which is compatible with this CustomOp """Returns a standard ONNX op which is compatible with this CustomOp
for performing shape inference.""" for performing shape inference."""
pass pass
@abstractmethod @abstractmethod
def infer_node_datatype(self, model): def infer_node_datatype(self, model):
"""Set the DataType annotations corresponding to the outputs of this """Set the DataType annotations corresponding to the outputs of this
node.""" node."""
pass pass
@abstractmethod @abstractmethod
def execute_node(self, context, graph): def execute_node(self, context, graph):
"""Execute this CustomOp instance, given the execution context and """Execute this CustomOp instance, given the execution context and
ONNX graph.""" ONNX graph."""
pass pass
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">When instantiating the class, the ONNX node is passed to access all attributes of the node within the class. This is accompanied by the functions `get_nodeattr()`and `set_nodeattr()`, which each instance of this class has. Furthermore 4 abstract methods are implemented, which are described in more detail in the comments in the code. </font> <font size="3">When instantiating the class, the ONNX node is passed to access all attributes of the node within the class. This is accompanied by the functions `get_nodeattr()`and `set_nodeattr()`, which each instance of this class has. Furthermore 4 abstract methods are implemented, which are described in more detail in the comments in the code. </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## HLSCustomOp class
<font size="3">If it is a node from the finn-hls library another class is used which is derived from the CustomOp class:</font> <font size="3">If it is a node from the finn-hls library another class is used which is derived from the CustomOp class:</font>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from finn.custom_op.fpgadataflow import HLSCustomOp from finn.custom_op.fpgadataflow import HLSCustomOp
showSrc(HLSCustomOp) showSrc(HLSCustomOp)
``` ```
%% Output %% Output
class HLSCustomOp(CustomOp): class HLSCustomOp(CustomOp):
def __init__(self, onnx_node): def __init__(self, onnx_node):
super().__init__(onnx_node) super().__init__(onnx_node)
# template for single node execution # template for single node execution
self.docompute_template = """ self.docompute_template = """
#include "cnpy.h" #include "cnpy.h"
#include "npy2apintstream.hpp" #include "npy2apintstream.hpp"
#include <vector> #include <vector>
#include "bnn-library.h" #include "bnn-library.h"
// includes for network parameters // includes for network parameters
$GLOBALS$ $GLOBALS$
// defines for network parameters // defines for network parameters
$DEFINES$ $DEFINES$
int main(){ int main(){
$STREAMDECLARATIONS$ $STREAMDECLARATIONS$
$READNPYDATA$ $READNPYDATA$
$DOCOMPUTE$ $DOCOMPUTE$
$DATAOUTSTREAM$ $DATAOUTSTREAM$
$SAVEASCNPY$ $SAVEASCNPY$
} }
""" """
self.code_gen_dict = {} self.code_gen_dict = {}
def get_nodeattr_types(self): def get_nodeattr_types(self):
return {"code_gen_dir": ("s", False, ""), "executable_path": ("s", False, "")} return {"code_gen_dir": ("s", False, ""), "executable_path": ("s", False, "")}
def code_generation(self, model): def code_generation(self, model):
node = self.onnx_node node = self.onnx_node
self.generate_params(model) self.generate_params(model)
self.global_includes() self.global_includes()
self.defines() self.defines()
self.read_npy_data() self.read_npy_data()
self.strm_decl() self.strm_decl()
self.docompute() self.docompute()
self.dataoutstrm() self.dataoutstrm()
self.save_as_npy() self.save_as_npy()
template = self.docompute_template template = self.docompute_template
for key in self.code_gen_dict: for key in self.code_gen_dict:
# transform list into long string separated by '\n' # transform list into long string separated by '\n'
code_gen_line = "\n".join(self.code_gen_dict[key]) code_gen_line = "\n".join(self.code_gen_dict[key])
template = template.replace(key, code_gen_line) template = template.replace(key, code_gen_line)
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
f = open(os.path.join(code_gen_dir, "execute_{}.cpp".format(node.op_type)), "w") f = open(os.path.join(code_gen_dir, "execute_{}.cpp".format(node.op_type)), "w")
f.write(template) f.write(template)
f.close() f.close()
def compile_singlenode_code(self): def compile_singlenode_code(self):
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
builder = CppBuilder() builder = CppBuilder()
builder.append_includes("-I/workspace/finn/src/finn/data/cpp") builder.append_includes("-I/workspace/finn/src/finn/data/cpp")
builder.append_includes("-I/workspace/cnpy/") builder.append_includes("-I/workspace/cnpy/")
builder.append_includes("-I/workspace/finn-hlslib") builder.append_includes("-I/workspace/finn-hlslib")
builder.append_includes("-I/workspace/vivado-hlslib") builder.append_includes("-I/workspace/vivado-hlslib")
builder.append_includes("--std=c++11") builder.append_includes("--std=c++11")
builder.append_sources(code_gen_dir + "/*.cpp") builder.append_sources(code_gen_dir + "/*.cpp")
builder.append_sources("/workspace/cnpy/cnpy.cpp") builder.append_sources("/workspace/cnpy/cnpy.cpp")
builder.append_includes("-lz") builder.append_includes("-lz")
builder.set_executable_path(code_gen_dir + "/node_model") builder.set_executable_path(code_gen_dir + "/node_model")
builder.build(code_gen_dir) builder.build(code_gen_dir)
self.set_nodeattr("executable_path", builder.executable_path) self.set_nodeattr("executable_path", builder.executable_path)
def dynamic_input_to_npy(self, context, count): def dynamic_input_to_npy(self, context, count):
node = self.onnx_node node = self.onnx_node
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
if code_gen_dir == "": if code_gen_dir == "":
raise Exception( raise Exception(
""" """
Found no codegen dir for this node, did you run the codegen transformation? Found no codegen dir for this node, did you run the codegen transformation?
""" """
) )
# create a npy file for each input of the node (in_ind is input index) # create a npy file for each input of the node (in_ind is input index)
# assuming dynamic inputs start from 0 # assuming dynamic inputs start from 0
for in_ind in range(count): for in_ind in range(count):
current_input_name = node.input[in_ind] current_input_name = node.input[in_ind]
np.save( np.save(
os.path.join(code_gen_dir, "input_{}.npy".format(in_ind)), os.path.join(code_gen_dir, "input_{}.npy".format(in_ind)),
context[current_input_name], context[current_input_name],
) )
def npy_to_dynamic_output(self, context): def npy_to_dynamic_output(self, context):
# TODO support multi-output nodes as needed # TODO support multi-output nodes as needed
node = self.onnx_node node = self.onnx_node
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
output = np.load("{}/output.npy".format(code_gen_dir)) output = np.load("{}/output.npy".format(code_gen_dir))
context[node.output[0]] = output context[node.output[0]] = output
def exec_precompiled_singlenode_model(self): def exec_precompiled_singlenode_model(self):
# execute precompiled executable # execute precompiled executable
executable_path = self.get_nodeattr("executable_path") executable_path = self.get_nodeattr("executable_path")
if executable_path == "": if executable_path == "":
raise Exception( raise Exception(
""" """
Found no executable for this node, did you run the codegen and Found no executable for this node, did you run the codegen and
compilation transformations? compilation transformations?
""" """
) )
process_execute = subprocess.Popen(executable_path, stdout=subprocess.PIPE) process_execute = subprocess.Popen(executable_path, stdout=subprocess.PIPE)
process_execute.communicate() process_execute.communicate()
def execute_node(self, context, graph): def execute_node(self, context, graph):
# save input(s) # save input(s)
self.dynamic_input_to_npy(context, 1) self.dynamic_input_to_npy(context, 1)
# execute the precompiled model # execute the precompiled model
self.exec_precompiled_singlenode_model() self.exec_precompiled_singlenode_model()
# load output npy file # load output npy file
self.npy_to_dynamic_output(context) self.npy_to_dynamic_output(context)
def generate_params(self, model): def generate_params(self, model):
pass pass
@abstractmethod @abstractmethod
def global_includes(self): def global_includes(self):
pass pass
@abstractmethod @abstractmethod
def defines(self): def defines(self):
pass pass
@abstractmethod @abstractmethod
def read_npy_data(self): def read_npy_data(self):
pass pass
@abstractmethod @abstractmethod
def strm_decl(self): def strm_decl(self):
pass pass
@abstractmethod @abstractmethod
def docompute(self): def docompute(self):
pass pass
@abstractmethod @abstractmethod
def dataoutstrm(self): def dataoutstrm(self):
pass pass
@abstractmethod @abstractmethod
def save_as_npy(self): def save_as_npy(self):
pass pass
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">When creating an instance of this class, a template is introduced, which forms the layout for the c++ code to execute the node. It has some general constructs, like the inclusion of bnn-library.h, which contains the references to the finn-hls library, and of cnpy.h and npy2apintstream.hpp, which support the transfer of python numpy arrays in c++. The idea of this template is to replace the variables marked with `$ $` with c++ calls during code generation. Then the template can be written into a .cpp file and be compiled. <font size="3">When creating an instance of this class, a template is introduced, which forms the layout for the c++ code to execute the node. It has some general constructs, like the inclusion of bnn-library.h, which contains the references to the finn-hls library, and of cnpy.h and npy2apintstream.hpp, which support the transfer of python numpy arrays in c++. The idea of this template is to replace the variables marked with `$ $` with c++ calls during code generation. Then the template can be written into a .cpp file and be compiled.
**`get_nodeattr_types()`**: each instance of the HLSCustomOp class must have the attributes `code_gen_dir` and `executable_path`, since to execute these nodes c++ code must be generated and correspondingly the executables. **`get_nodeattr_types()`**: each instance of the HLSCustomOp class must have the attributes `code_gen_dir` and `executable_path`, since to execute these nodes c++ code must be generated and correspondingly the executables.
</font> </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">**`code_generation(model)`**: all functions required for code generation are called and the `$ $` variables in the template are replaced accordingly and written into a .cpp file. Almost all of these subfunctions are implemented as abstract methods in the class, so they are completely customized for each custom op node. A special function is `generate_params()`. This is not implemented as an abstract method, but as a normal function, but contains by default only `pass`. This is because some custom op nodes do not have parameters that need to be generated and in this way the function is skipped. For example for a streaming fc layer node a parameter generation is necessary. How such a parameter generation can look like is described in more detail in the course of this notebook. <font size="3">**`code_generation(model)`**: all functions required for code generation are called and the `$ $` variables in the template are replaced accordingly and written into a .cpp file. Almost all of these subfunctions are implemented as abstract methods in the class, so they are completely customized for each custom op node. A special function is `generate_params()`. This is not implemented as an abstract method, but as a normal function, but contains by default only `pass`. This is because some custom op nodes do not have parameters that need to be generated and in this way the function is skipped. For example for a streaming fc layer node a parameter generation is necessary. How such a parameter generation can look like is described in more detail in the course of this notebook.
</font> </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">**`compile_singlenode_code()`**: To compile the generated code, the compile command must be built. This is done in this function. It creates an instance of the `CppBuilder()` class and assembles the various components for the function. The `.build` function creates the executable and then sets the corresponding attribute. The class `CppBuilder` is a transformation and a more detailed description can be found in Jupyter notebook *FINN-CodeGenerationAndCompilation*. <font size="3">**`compile_singlenode_code()`**: To compile the generated code, the compile command must be built. This is done in this function. It creates an instance of the `CppBuilder()` class and assembles the various components for the function. The `.build` function creates the executable and then sets the corresponding attribute. The class `CppBuilder` is a transformation and a more detailed description can be found in Jupyter notebook [FINN-CodeGenerationAndCompilation](github.com/Xilinx/finn/blob/dev/notebooks/FINN-CodeGenerationAndCompilation.ipynb).
</font> </font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<font size="3">**`dynamic_input_to_npy(context, count)`**:</font> <font size="3">**`dynamic_input_to_npy(context, count)`**: creates a .npy file for all inputs of the node. These files will be stored in the directory specified by code_gen_dir. The argument `count` must be used to specify the number of inputs. `context` contains the values for the inputs.</font>
%% Cell type:markdown id: tags:
<font size="3">**`npy_to_dynamic_output(context)`**: reads the output values and sets `context` dictionary accordingly. When executing the c++ executable of the node, the output values are written to a .npy file. </font>
%% Cell type:markdown id: tags:
<font size="3">**`exec_precompiled_singlenode_model()`**: executes precompiled executable which is specified in `executable_path`</font>
%% Cell type:markdown id: tags:
<font size="3">**`execute_node(context,graph)`**: calls first `dynamic_input_to_npy()`, then executes the executable using `exec_precompiled_singlenode_model()` and at the end reads the output .npy file with `npy_to_dynamic_output`</font>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
#### Generate Parameter #### Generate Parameter
<font size="3">Parameters have to be generated for specific types of HLSCustomOps. For example if the node is a streaming fc layer, there are weights and activation values, which are written to separate .h files and added to the template using `#include`. For streaming fc layer the parameter generation looks like this: <font size="3">Parameters have to be generated for specific types of HLSCustomOps. For example if the node is a streaming fc layer, there are weights and activation values, which are written to separate .h files and added to the template using `#include`. For streaming fc layer the parameter generation looks like this:
</font> </font>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from finn.custom_op.fpgadataflow.streamingfclayer_batch import StreamingFCLayer_Batch from finn.custom_op.fpgadataflow.streamingfclayer_batch import StreamingFCLayer_Batch
showSrc(StreamingFCLayer_Batch.generate_params) showSrc(StreamingFCLayer_Batch.generate_params)
``` ```
%% Output %% Output
def generate_params(self, model): def generate_params(self, model):
# weights # weights
weights = model.get_initializer(self.onnx_node.input[1]) weights = model.get_initializer(self.onnx_node.input[1])
# convert weights into hlslib-compatible format # convert weights into hlslib-compatible format
weight_tensor = self.get_hls_compatible_weight_tensor(weights) weight_tensor = self.get_hls_compatible_weight_tensor(weights)
export_wdt = self.get_weight_datatype() export_wdt = self.get_weight_datatype()
# we have converted bipolar weights to binary for export, # we have converted bipolar weights to binary for export,
# so use it as such for weight generation # so use it as such for weight generation
if self.get_weight_datatype() == DataType.BIPOLAR: if self.get_weight_datatype() == DataType.BIPOLAR:
export_wdt = DataType.BINARY export_wdt = DataType.BINARY
weight_hls_code = numpy_to_hls_code( weight_hls_code = numpy_to_hls_code(
weight_tensor, export_wdt, "weights", True, True weight_tensor, export_wdt, "weights", True, True
) )
# write weights into params.h # write weights into params.h
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
f_weights = open("{}/params.h".format(code_gen_dir), "w") f_weights = open("{}/params.h".format(code_gen_dir), "w")
if export_wdt.bitwidth() != 1: if export_wdt.bitwidth() != 1:
f_weights.write( f_weights.write(
"static FixedPointWeights<{},{},{},{}> weights = ".format( "static FixedPointWeights<{},{},{},{}> weights = ".format(
self.get_nodeattr("SIMD"), self.get_nodeattr("SIMD"),
export_wdt.get_hls_datatype_str(), export_wdt.get_hls_datatype_str(),
self.get_nodeattr("PE"), self.get_nodeattr("PE"),
self.calc_wmem(), self.calc_wmem(),
) )
) )
else: else:
f_weights.write( f_weights.write(
"static BinaryWeights<{},{},{}> weights = ".format( "static BinaryWeights<{},{},{}> weights = ".format(
self.get_nodeattr("SIMD"), self.get_nodeattr("PE"), self.calc_wmem() self.get_nodeattr("SIMD"), self.get_nodeattr("PE"), self.calc_wmem()
) )
) )
f_weights.write(weight_hls_code) f_weights.write(weight_hls_code)
f_weights.close() f_weights.close()
# thresholds # thresholds
if len(self.onnx_node.input) > 2: if len(self.onnx_node.input) > 2:
thresholds = model.get_initializer(self.onnx_node.input[2]) thresholds = model.get_initializer(self.onnx_node.input[2])
if thresholds is not None: if thresholds is not None:
threshold_tensor = self.get_hls_compatible_threshold_tensor(thresholds) threshold_tensor = self.get_hls_compatible_threshold_tensor(thresholds)
tdt = DataType.INT32 tdt = DataType.INT32
# use UINT32 threshold export for bipolar times bipolar # use UINT32 threshold export for bipolar times bipolar
inp_is_bipolar = self.get_input_datatype() == DataType.BIPOLAR inp_is_bipolar = self.get_input_datatype() == DataType.BIPOLAR
wt_is_bipolar = self.get_weight_datatype() == DataType.BIPOLAR wt_is_bipolar = self.get_weight_datatype() == DataType.BIPOLAR
# reinterpret inp/wt as bipolar if bin_xnor_mode is iset # reinterpret inp/wt as bipolar if bin_xnor_mode is iset
inp_is_binary = self.get_input_datatype() == DataType.BINARY inp_is_binary = self.get_input_datatype() == DataType.BINARY
wt_is_binary = self.get_weight_datatype() == DataType.BINARY wt_is_binary = self.get_weight_datatype() == DataType.BINARY
bin_xnor_mode = self.get_nodeattr("binaryXnorMode") == 1 bin_xnor_mode = self.get_nodeattr("binaryXnorMode") == 1
inp_is_bipolar = inp_is_bipolar or (inp_is_binary and bin_xnor_mode) inp_is_bipolar = inp_is_bipolar or (inp_is_binary and bin_xnor_mode)
wt_is_bipolar = wt_is_bipolar or (wt_is_binary and bin_xnor_mode) wt_is_bipolar = wt_is_bipolar or (wt_is_binary and bin_xnor_mode)
if inp_is_bipolar and wt_is_bipolar: if inp_is_bipolar and wt_is_bipolar:
tdt = DataType.UINT32 tdt = DataType.UINT32
thresholds_hls_code = numpy_to_hls_code( thresholds_hls_code = numpy_to_hls_code(
threshold_tensor, tdt, "thresholds", False, True threshold_tensor, tdt, "thresholds", False, True
) )
# write thresholds into thresh.h # write thresholds into thresh.h
code_gen_dir = self.get_nodeattr("code_gen_dir") code_gen_dir = self.get_nodeattr("code_gen_dir")
f_thresh = open("{}/thresh.h".format(code_gen_dir), "w") f_thresh = open("{}/thresh.h".format(code_gen_dir), "w")
tdt_hls = tdt.get_hls_datatype_str() tdt_hls = tdt.get_hls_datatype_str()
# use binary to export bipolar activations # use binary to export bipolar activations
export_odt = self.get_output_datatype() export_odt = self.get_output_datatype()
if self.get_output_datatype() == DataType.BIPOLAR: if self.get_output_datatype() == DataType.BIPOLAR:
export_odt = DataType.BINARY export_odt = DataType.BINARY
odt_hls = export_odt.get_hls_datatype_str() odt_hls = export_odt.get_hls_datatype_str()
f_thresh.write( f_thresh.write(
"static ThresholdsActivation<{},{},{},{},{},{},{}> threshs \ "static ThresholdsActivation<{},{},{},{},{},{},{}> threshs \
= ".format( = ".format(
self.calc_tmem(), self.calc_tmem(),
self.get_nodeattr("PE"), self.get_nodeattr("PE"),
threshold_tensor.shape[-1], threshold_tensor.shape[-1],
tdt_hls, tdt_hls,
odt_hls, odt_hls,
self.get_nodeattr("ActVal"), self.get_nodeattr("ActVal"),
"std::less_equal<%s>" % tdt_hls, "std::less_equal<%s>" % tdt_hls,
) )
) )
f_thresh.write(thresholds_hls_code) f_thresh.write(thresholds_hls_code)
f_thresh.close() f_thresh.close()
%% Cell type:markdown id: tags:
<font size="3">First, the values for the weights are extracted with `get_initializer()` using the ModelWrapper. At this point it is assumed that the second input of the streamingfclayer specifies the weights. After a few manipulations the weights are written in `params.h`. If there are threshold values, they will be prepared and written to `thresh.h`. </font>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
``` ```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment