neural_compressor.torch.utils.utility
=====================================

.. py:module:: neural_compressor.torch.utils.utility

.. autoapi-nested-parse::

   Intel Neural Compressor PyTorch utilities.



Functions
---------

.. autoapisummary::

   neural_compressor.torch.utils.utility.register_algo
   neural_compressor.torch.utils.utility.fetch_module
   neural_compressor.torch.utils.utility.set_module
   neural_compressor.torch.utils.utility.get_model_info
   neural_compressor.torch.utils.utility.get_double_quant_config_dict
   neural_compressor.torch.utils.utility.get_quantizer
   neural_compressor.torch.utils.utility.postprocess_model
   neural_compressor.torch.utils.utility.dump_model_op_stats
   neural_compressor.torch.utils.utility.get_model_device
   neural_compressor.torch.utils.utility.get_processor_type_from_user_config
   neural_compressor.torch.utils.utility.dowload_hf_model
   neural_compressor.torch.utils.utility.load_empty_model


Module Contents
---------------

.. py:function:: register_algo(name)

   Decorator function to register algorithms in the algos_mapping dictionary.

   Usage example:
       @register_algo(name=example_algo)
       def example_algo(model: torch.nn.Module, quant_config: RTNConfig) -> torch.nn.Module:
           ...

   :param name: The name under which the algorithm function will be registered.
   :type name: str

   :returns: The decorator function to be used with algorithm functions.
   :rtype: decorator


.. py:function:: fetch_module(model, op_name)

   Get module with a given op name.

   :param model: the input model.
   :type model: object
   :param op_name: name of op.
   :type op_name: str

   :returns: module (object).


.. py:function:: set_module(model, op_name, new_module)

   Set module with a given op name.

   :param model: the input model.
   :type model: object
   :param op_name: name of op.
   :type op_name: str
   :param new_module: the input model.
   :type new_module: object

   :returns: module (object).


.. py:function:: get_model_info(model: torch.nn.Module, white_module_list: List[Callable]) -> List[Tuple[str, str]]

   Get model info according to white_module_list.


.. py:function:: get_double_quant_config_dict(double_quant_type='BNB_NF4')

   Query config dict of double_quant according to double_quant_type.

   :param double_quant_type: double_quant type. Defaults to "BNB_NF4".
   :type double_quant_type: str, optional


.. py:function:: get_quantizer(model, quantizer_cls, quant_config=None, *args, **kwargs)

   Get the quantizer.

   Initialize a quantizer or get `quantizer` attribute from model.

   :param model: pytorch model.
   :type model: torch.nn.Module
   :param quantizer_cls: quantizer class of a specific algorithm.
   :type quantizer_cls: Quantizer
   :param quant_config: Specifies how to apply the algorithm on the given model.
                        Defaults to None.
   :type quant_config: dict, optional

   :returns: quantizer object.


.. py:function:: postprocess_model(model, mode, quantizer)

   Process `quantizer` attribute of model according to current phase.

   In `prepare` phase, the `quantizer` is set as an attribute of the model
   to avoid redundant initialization during `convert` phase.

   In 'convert' or 'quantize' phase, the unused `quantizer` attribute is removed.

   :param model: pytorch model.
   :type model: torch.nn.Module
   :param mode: The mode of current phase, including 'prepare', 'convert' and 'quantize'.
   :type mode: Mode
   :param quantizer: quantizer object.
   :type quantizer: Quantizer


.. py:function:: dump_model_op_stats(mode, tune_cfg)

   Dump quantizable ops stats of model to user.

   :param mode: quantization mode.
   :type mode: object
   :param tune_cfg: quantization config
   :type tune_cfg: dict


.. py:function:: get_model_device(model: torch.nn.Module)

   Get the device.

   :param model: the input model.
   :type model: torch.nn.Module

   :returns: a string.
   :rtype: device (str)


.. py:function:: get_processor_type_from_user_config(user_processor_type: Optional[Union[str, neural_compressor.common.utils.ProcessorType]] = None)

   Get the processor type.

   Get the processor type based on the user configuration or automatically detect it based on the hardware.

   :param user_processor_type: The user-specified processor type. Defaults to None.
   :type user_processor_type: Optional[Union[str, ProcessorType]]

   :returns: The detected or user-specified processor type.
   :rtype: ProcessorType

   :raises AssertionError: If the user-specified processor type is not supported.
   :raises NotImplementedError: If the processor type is not recognized.


.. py:function:: dowload_hf_model(repo_id, cache_dir=None, repo_type=None, revision=None)

   Download hugging face model from hf hub.


.. py:function:: load_empty_model(pretrained_model_name_or_path, cls=None, **kwargs)

   Load a empty model.