:orphan:

:py:mod:`neural_compressor.onnxrt.algorithms.weight_only.rtn`
=============================================================

.. py:module:: neural_compressor.onnxrt.algorithms.weight_only.rtn


Module Contents
---------------


Functions
~~~~~~~~~

.. autoapisummary::

   neural_compressor.onnxrt.algorithms.weight_only.rtn.rtn_quantize
   neural_compressor.onnxrt.algorithms.weight_only.rtn.apply_rtn_on_model



.. py:function:: rtn_quantize(model: Union[onnx.ModelProto, neural_compressor.onnxrt.utils.onnx_model.ONNXModel, pathlib.Path, str], weight_config: dict = {}, num_bits: int = 4, group_size: int = 32, scheme: str = 'asym', ratios: dict = {}, accuracy_level: int = 0, providers: List[str] = ['CPUExecutionProvider'], return_modelproto: bool = True)

   Quantize the model with round to nearst method.

   :param model: onnx model
   :type model: Union[onnx.ModelProto, ONNXModel, Path, str]
   :param weight_config: quantization config
                         For example,
                         weight_config = {
                             '(fc2, "MatMul")':
                                 {
                                     'weight_dtype': 'int',
                                     'weight_bits': 4,
                                     'weight_group_size': 32,
                                     'weight_sym': True,
                                     'accuracy_level': 0
                                 }
                         }. Defaults to {}.
   :type weight_config: dict, optional
   :param num_bits: number of bits used to represent weights. Defaults to 4.
   :type num_bits: int, optional
   :param group_size: size of weight groups. Defaults to 32.
   :type group_size: int, optional
   :param scheme: indicates whether weights are symmetric. Defaults to "asym".
   :type scheme: str, optional
   :param ratios: percentile of clip. Defaults to {}.
   :type ratios: dict, optional
   :param accuracy_level: accuracy level. Support 0 (unset), 1(fp32 compute type of jblas kernel),
                          2 (fp16 compute type of jblas kernel), 3 (bf16 compute type of jblas kernel),
                          4 (int8 compute type of jblas kernel). Defaults to 0.
   :type accuracy_level: int, optional
   :param providers: providers to use. Defaults to ["CPUExecutionProvider"].
   :type providers: list, optional
   :param return_modelproto: whether to return onnx.Modelproto. set False for layer-wise quant.
                             Default to True
   :type return_modelproto: bool, optionmal

   :returns: quantized onnx model.
   :rtype: onnx.ModelProto


.. py:function:: apply_rtn_on_model(model: Union[onnx.ModelProto, neural_compressor.onnxrt.utils.onnx_model.ONNXModel, pathlib.Path, str], quant_config: dict) -> onnx.ModelProto

   Apply RTN on onnx model.

   :param model: onnx model.
   :type model: Union[onnx.ModelProto, ONNXModel, Path, str]
   :param quant_config: quantization config.
   :type quant_config: dict

   :returns: quantized onnx model.
   :rtype: onnx.ModelProto