quark.onnx.finetuning.create_torch.create_model

Module Contents

Classes

class quark.onnx.finetuning.create_torch.create_model.TorchModel(onnx_model: onnx.ModelProto)

A torch model converted from a onnx model.

forward(inputs: torch.Tensor) → Any

Support the models with single input and single output

set_weight(weight: numpy.typing.NDArray[numpy.float32]) → None

Set the original float weight for the compute module

get_weight() → Any

Get the optimized quantized weight of the compute module

set_bias(bias: numpy.typing.NDArray[numpy.float32]) → None

Set the original float bias for the compute module

get_bias() → Any

Get the optimized quantized bias of the compute module