neural_compressor.data

Built-in dataloaders, datasets, transforms, filters for multiple framework backends.

Subpackages

• neural_compressor.data.datasets
  • neural_compressor.data.datasets.bert_dataset
  • neural_compressor.data.datasets.coco_dataset
  • neural_compressor.data.datasets.dataset
  • neural_compressor.data.datasets.dummy_dataset
  • neural_compressor.data.datasets.dummy_dataset_v2
  • neural_compressor.data.datasets.imagenet_dataset
  • neural_compressor.data.datasets.style_transfer_dataset
• neural_compressor.data.filters
  • neural_compressor.data.filters.coco_filter
  • neural_compressor.data.filters.filter
• neural_compressor.data.transforms
  • neural_compressor.data.transforms.imagenet_transform
  • neural_compressor.data.transforms.postprocess
  • neural_compressor.data.transforms.tokenization
  • neural_compressor.data.transforms.transform

Package Contents

Classes

Datasets	A base class for all framework datasets.
Dataset	The base class of dataset.
IterableDataset	An iterable Dataset.
TensorflowImageRecord	Tensorflow imageNet database in tf record format.
COCORecordDataset	Tensorflow COCO dataset in tf record format.
DataLoader	Entrance of all configured DataLoaders.
DefaultDataLoader	DefaultDataLoader for multiple framework backends.
TRANSFORMS	Transforms collection class.
BaseTransform	The base class for transform.
ComposeTransform	Composes several transforms together.
Postprocess	Just collect the infos to construct a Postprocess.
LabelShift	Convert label to label - label_shift.
BilinearImagenetTransform	Combination of a series of transforms which is applicable to images in Imagenet.
TensorflowResizeCropImagenetTransform	Combination of a series of transforms which is applicable to images in Imagenet.
TFSquadV1PostTransform	Postprocess the predictions of bert on SQuAD.
TFSquadV1ModelZooPostTransform	Postprocess the predictions of bert on SQuADV1.1.
TensorflowResizeWithRatio	Resize image with aspect ratio and pad it to max shape(optional).
ResizeTFTransform	Resize the input image to the given size.
RescaleTFTransform	Scale the values of image to [0,1].
NormalizeTFTransform	Normalize a image with mean and standard deviation.
ParseDecodeCocoTransform	Coco decoding will be performed automatically from Neural Compressor v1.4.
TensorflowShiftRescale	Label shift by 1 and rescale.
FILTERS	The filter register for all frameworks.
Filter	The base class for transform.
LabelBalanceCOCORecordFilter	The label balance filter for COCO Record.

Functions

dataset_registry(dataset_type, framework[, dataset_format])	Register dataset subclasses.
check_dataloader(dataloader)	Check if the dataloader meets requirement of neural_compressor.
transform_registry(transform_type, process, framework)	Class decorator used to register all transform subclasses.
filter_registry(filter_type, framework)	Register all transform subclasses.

Attributes

DATALOADERS