:orphan:

:py:mod:`train`
===============

.. py:module:: train


Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   train.SupervisedDataset
   train.LazySupervisedDataset



Functions
~~~~~~~~~

.. autoapisummary::

   train.safe_save_model_for_hf_trainer
   train.make_supervised_data_module



.. py:function:: safe_save_model_for_hf_trainer(trainer: transformers.Trainer, output_dir: str)

   Collects the state dict and dump to disk.


.. py:class:: SupervisedDataset(raw_data, tokenizer: transformers.PreTrainedTokenizer)




   Dataset for supervised fine-tuning.


.. py:class:: LazySupervisedDataset(raw_data, tokenizer: transformers.PreTrainedTokenizer)




   Dataset for supervised fine-tuning.


.. py:function:: make_supervised_data_module(tokenizer: transformers.PreTrainedTokenizer, data_args) -> Dict

   Make dataset and collator for supervised fine-tuning.