`neural_compressor.experimental.data.transforms.transform`¶

Neural Compressor built-in Transforms on multiple framework backends.

Module Contents¶

Classes¶

`Transforms`	INC supports built-in preprocessing, postprocessing and general methods on different framework backends.
`TensorflowTransforms`	Tensorflow Transforms subclass.
`MXNetTransforms`	Mxnet Transforms subclass.
`PyTorchTransforms`	Pytorch Transforms subclass.
`ONNXRTQLTransforms`	Onnxrt_qlinearops Transforms subclass.
`ONNXRTITTransforms`	Onnxrt_integerops Transforms subclass.
`TRANSFORMS`	Transforms collection class.
`BaseTransform`	The base class for transform.
`TensorflowWrapFunction`	Tensorflow wrapper function class.
`TensorflowTransform`	Tensorflow transform class, the subclass of BaseTransform.
`PytorchMxnetWrapFunction`	Pytorch and MXNet wrapper function class.
`PytorchMxnetTransform`	Pytorch and Mxnet transform class, the subclass of BaseTransform.
`ComposeTransform`	Composes several transforms together.
`CropToBoundingBox`	Crops an image to a specified bounding box.
`MXNetCropToBoundingBox`	Crops an image to a specified bounding box.
`ONNXRTCropToBoundingBox`	Crops an image to a specified bounding box.
`TensorflowCropToBoundingBox`	Crops an image to a specified bounding box.
`ResizeWithRatio`	Resize image with aspect ratio and pad it to max shape(optional).
`TensorflowResizeWithRatio`	Resize image with aspect ratio and pad it to max shape(optional).
`Transpose`	Transpose image according to perm.
`TensorflowTranspose`	Transpose image according to perm.
`MXNetTranspose`	Transpose image according to perm.
`PyTorchTranspose`	Transpose image according to perm.
`RandomVerticalFlip`	Vertically flip the given image randomly.
`TensorflowRandomVerticalFlip`	Vertically flip the given image randomly.
`RandomHorizontalFlip`	Horizontally flip the given image randomly.
`TensorflowRandomHorizontalFlip`	Horizontally flip the given image randomly.
`ToArray`	Convert PIL Image or NDArray to numpy array.
`CastTFTransform`	Convert image to given dtype.
`CastONNXTransform`	Convert image to given dtype.
`CastPyTorchTransform`	Convert image to given dtype.
`CenterCropTFTransform`	Crops the given image at the center to the given size.
`PaddedCenterCropTransform`	Crops the given image at the center to the given size with padding.
`ResizeTFTransform`	Resize the input image to the given size.
`ResizePytorchTransform`	Resize the input image to the given size.
`RandomCropTFTransform`	Crop the image at a random location to the given size.
`RandomResizedCropPytorchTransform`	Crop the given image to random size and aspect ratio.
`RandomResizedCropMXNetTransform`	Crop the given image to random size and aspect ratio.
`RandomResizedCropTFTransform`	Crop the given image to random size and aspect ratio.
`NormalizeTFTransform`	Normalize a image with mean and standard deviation.
`RescaleKerasPretrainTransform`	Scale the values of image to [0,1].
`RescaleTFTransform`	Scale the values of image to [0,1].
`RescaleTransform`	Scale the values of image to [0,1].
`AlignImageChannelTransform`	Align image channel, now just support [H,W]->[H,W,dim], [H,W,4]->[H,W,3] and [H,W,3]->[H,W].
`PyTorchAlignImageChannel`	Align image channel, now just support [H,W,4]->[H,W,3] and [H,W,3]->[H,W].
`ToNDArrayTransform`	Convert np.array to NDArray.
`ResizeMXNetTransform`	Resize the input image to the given size.
`ResizeTransform`	Resize the input image to the given size.
`CropResizeTFTransform`	Crop the input image with given location and resize it.
`PyTorchCropResizeTransform`	Crop the input image with given location and resize it.
`MXNetCropResizeTransform`	Crop the input image with given location and resize it.
`CropResizeTransform`	Crop the input image with given location and resize it.
`CenterCropTransform`	Crops the given image at the center to the given size.
`MXNetNormalizeTransform`	Normalize a image with mean and standard deviation.
`PyTorchNormalizeTransform`	Normalize a image with mean and standard deviation.
`NormalizeTransform`	Normalize a image with mean and standard deviation.
`RandomCropTransform`	Crop the image at a random location to the given size.
`RandomResizedCropTransform`	Crop the given image to random size and aspect ratio.
`SquadExample`	A single training/test example for simple sequence classification.
`InputFeatures`	A single set of features of data.
`CollectTransform`	Postprocess the predictions, collect data.
`TFSquadV1PostTransform`	Postprocess the predictions of bert on SQuAD.
`TFModelZooCollectTransform`	Postprocess the predictions of model zoo, collect data.
`TFSquadV1ModelZooPostTransform`	Postprocess the predictions of bert on SQuADV1.1.
`ParseDecodeVocTransform`	Parse features in Example proto.

Functions¶

`transform_registry`(transform_type, process, framework)	Class decorator used to register all transform subclasses.
`get_torchvision_map`(interpolation)	Get torchvision interpolation map.
`get_final_text`(pred_text, orig_text, do_lower_case)	Project the tokenized prediction back to the original text.
`read_squad_examples`(input_file)	Read a SQuAD json file into a list of SquadExample.
`convert_examples_to_features`(examples, tokenizer, ...)	Load a data file into a list of `InputBatch`s.

class neural_compressor.experimental.data.transforms.transform.Transforms(process, concat_general=True)¶

Bases: object

INC supports built-in preprocessing, postprocessing and general methods on different framework backends.

Transforms base class provides the abstract methods. Users can also register their own Transforms classes by inheriting this base class.

class neural_compressor.experimental.data.transforms.transform.TensorflowTransforms(process, concat_general=True)¶

Bases: Transforms

Tensorflow Transforms subclass.

class neural_compressor.experimental.data.transforms.transform.MXNetTransforms(process, concat_general=True)¶

Bases: Transforms

Mxnet Transforms subclass.

class neural_compressor.experimental.data.transforms.transform.PyTorchTransforms(process, concat_general=True)¶

Bases: Transforms

Pytorch Transforms subclass.

class neural_compressor.experimental.data.transforms.transform.ONNXRTQLTransforms(process, concat_general=True)¶

Bases: Transforms

Onnxrt_qlinearops Transforms subclass.

class neural_compressor.experimental.data.transforms.transform.ONNXRTITTransforms(process, concat_general=True)¶

Bases: Transforms

Onnxrt_integerops Transforms subclass.

class neural_compressor.experimental.data.transforms.transform.TRANSFORMS(framework, process)¶

Bases: object

Transforms collection class.

Provide register method to register new Transforms and provide __getitem__ method to get Transforms according to Transforms type.

register(name, transform_cls)¶

Register new Transform according to Transforms type.

Parameters:

name (str) – process name
transform_cls (class) – process function wrapper class

neural_compressor.experimental.data.transforms.transform.transform_registry(transform_type, process, framework)¶

Class decorator used to register all transform subclasses.

Parameters:

transform_type (str) – Transform registration name
process (str) – support 3 process including ‘preprocess’, ‘postprocess’, ‘general’
framework (str) – support 4 framework including ‘tensorflow’, ‘pytorch’, ‘mxnet’, ‘onnxrt’
cls (class) – The class of register.

Returns:

The class of register.

Return type:

cls

class neural_compressor.experimental.data.transforms.transform.BaseTransform¶

Bases: object

The base class for transform.

class neural_compressor.experimental.data.transforms.transform.TensorflowWrapFunction(transform_func)¶

Bases: object

Tensorflow wrapper function class.

class neural_compressor.experimental.data.transforms.transform.TensorflowTransform(transform_func, **kwargs)¶

Bases: BaseTransform

Tensorflow transform class, the subclass of BaseTransform.

class neural_compressor.experimental.data.transforms.transform.PytorchMxnetWrapFunction(transform_func)¶

Bases: object

Pytorch and MXNet wrapper function class.

class neural_compressor.experimental.data.transforms.transform.PytorchMxnetTransform(transform_func)¶

Bases: BaseTransform

Pytorch and Mxnet transform class, the subclass of BaseTransform.

neural_compressor.experimental.data.transforms.transform.get_torchvision_map(interpolation)¶: Get torchvision interpolation map.

class neural_compressor.experimental.data.transforms.transform.ComposeTransform(transform_list)¶

Bases: BaseTransform

Composes several transforms together.

Parameters:: transform_list (list of Transform objects) – list of transforms to compose
Returns:: tuple of processed image and label
Return type:: sample (tuple)

class neural_compressor.experimental.data.transforms.transform.CropToBoundingBox(offset_height, offset_width, target_height, target_width)¶

Bases: BaseTransform

Crops an image to a specified bounding box.

Parameters:

offset_height (int) – Vertical coordinate of the top-left corner of the result in the input
offset_width (int) – Horizontal coordinate of the top-left corner of the result in the input
target_height (int) – Height of the result
target_width (int) – Width of the result

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.MXNetCropToBoundingBox(offset_height, offset_width, target_height, target_width)¶

Bases: CropToBoundingBox

Crops an image to a specified bounding box.

Parameters:

offset_height (int) – Vertical coordinate of the top-left corner of the result in the input
offset_width (int) – Horizontal coordinate of the top-left corner of the result in the input
target_height (int) – Height of the result
target_width (int) – Width of the result

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ONNXRTCropToBoundingBox(offset_height, offset_width, target_height, target_width)¶

Bases: CropToBoundingBox

Crops an image to a specified bounding box.

Parameters:

offset_height (int) – Vertical coordinate of the top-left corner of the result in the input
offset_width (int) – Horizontal coordinate of the top-left corner of the result in the input
target_height (int) – Height of the result
target_width (int) – Width of the result

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.TensorflowCropToBoundingBox(offset_height, offset_width, target_height, target_width)¶

Bases: CropToBoundingBox

Crops an image to a specified bounding box.

Parameters:

offset_height (int) – Vertical coordinate of the top-left corner of the result in the input
offset_width (int) – Horizontal coordinate of the top-left corner of the result in the input
target_height (int) – Height of the result
target_width (int) – Width of the result

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ResizeWithRatio(min_dim=800, max_dim=1365, padding=False, constant_value=0)¶

Bases: BaseTransform

Resize image with aspect ratio and pad it to max shape(optional).

If the image is padded, the label will be processed at the same time. The input image should be np.array.

Parameters:

min_dim (int, default=800) – Resizes the image such that its smaller dimension == min_dim
max_dim (int, default=1365) – Ensures that the image longest side doesn’t exceed this value
padding (bool, default=False) – If true, pads image with zeros so its size is max_dim x max_dim

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.TensorflowResizeWithRatio(min_dim=800, max_dim=1365, padding=False, constant_value=0)¶

Bases: BaseTransform

Resize image with aspect ratio and pad it to max shape(optional).

If the image is padded, the label will be processed at the same time. The input image should be np.array or tf.Tensor.

Parameters:

min_dim (int, default=800) – Resizes the image such that its smaller dimension == min_dim
max_dim (int, default=1365) – Ensures that the image longest side doesn’t exceed this value
padding (bool, default=False) – If true, pads image with zeros so its size is max_dim x max_dim

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.Transpose(perm)¶

Bases: BaseTransform

Transpose image according to perm.

Parameters:: perm (list) – A permutation of the dimensions of input image
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.TensorflowTranspose(perm)¶

Bases: Transpose

Transpose image according to perm.

Parameters:: perm (list) – A permutation of the dimensions of input image
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.MXNetTranspose(perm)¶

Bases: Transpose

Transpose image according to perm.

Parameters:: perm (list) – A permutation of the dimensions of input image
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.PyTorchTranspose(perm)¶

Bases: Transpose

Transpose image according to perm.

Parameters:: perm (list) – A permutation of the dimensions of input image
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomVerticalFlip¶

Bases: BaseTransform

Vertically flip the given image randomly.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.TensorflowRandomVerticalFlip¶

Bases: BaseTransform

Vertically flip the given image randomly.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomHorizontalFlip¶

Bases: BaseTransform

Horizontally flip the given image randomly.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.TensorflowRandomHorizontalFlip¶

Bases: BaseTransform

Horizontally flip the given image randomly.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ToArray¶

Bases: BaseTransform

Convert PIL Image or NDArray to numpy array.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CastTFTransform(dtype='float32')¶

Bases: BaseTransform

Convert image to given dtype.

Parameters:: dtype (str, default='float32') – A dtype to convert image to
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CastONNXTransform(dtype='float32')¶

Bases: BaseTransform

Convert image to given dtype.

Parameters:: dtype (str, default='float32') – A dtype to convert image to
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CastPyTorchTransform(dtype='float32')¶

Bases: BaseTransform

Convert image to given dtype.

Parameters:: dtype (str, default='float32') – A dtype to convert image to
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CenterCropTFTransform(size)¶

Bases: BaseTransform

Crops the given image at the center to the given size.

Parameters:: size (list or int) – Size of the result
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.PaddedCenterCropTransform(size, crop_padding=0)¶

Bases: BaseTransform

Crops the given image at the center to the given size with padding.

Parameters:

size (list or int) – Size of the result
crop_padding (int) – crop padding number

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ResizeTFTransform(size, interpolation='bilinear')¶

Bases: BaseTransform

Resize the input image to the given size.

Parameters:

size (list or int) – Size of the result
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ResizePytorchTransform(size, interpolation='bilinear')¶

Bases: BaseTransform

Resize the input image to the given size.

Parameters:

size (list or int) – Size of the result
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomCropTFTransform(size)¶

Bases: BaseTransform

Crop the image at a random location to the given size.

Parameters:: size (list or tuple or int) – Size of the result
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomResizedCropPytorchTransform(size, scale=(0.08, 1.0), ratio=(3.0 / 4.0, 4.0 / 3.0), interpolation='bilinear')¶

Bases: BaseTransform

Crop the given image to random size and aspect ratio.

Parameters:

size (list or int) – Size of the result
scale (tuple or list, default=(0.08, 1.0)) – range of size of the origin size cropped
ratio (tuple or list, default=(3. / 4., 4. / 3.)) – range of aspect ratio of the origin aspect ratio cropped
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomResizedCropMXNetTransform(size, scale=(0.08, 1.0), ratio=(3.0 / 4.0, 4.0 / 3.0), interpolation='bilinear')¶

Bases: BaseTransform

Crop the given image to random size and aspect ratio.

Parameters:

size (list or int) – Size of the result
scale (tuple or list, default=(0.08, 1.0)) – range of size of the origin size cropped
ratio (tuple or list, default=(3. / 4., 4. / 3.)) – range of aspect ratio of the origin aspect ratio cropped
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomResizedCropTFTransform(size, scale=(0.08, 1.0), ratio=(3.0 / 4.0, 4.0 / 3.0), interpolation='bilinear')¶

Bases: BaseTransform

Crop the given image to random size and aspect ratio.

Parameters:

size (list or int) – Size of the result
scale (tuple or list, default=(0.08, 1.0)) – range of size of the origin size cropped
ratio (tuple or list, default=(3. / 4., 4. / 3.)) – range of aspect ratio of the origin aspect ratio cropped
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’

Returns:

tuple of processed image and label

get_params(image, scale, ratio)¶: Get the image prameters: position, height and width.

class neural_compressor.experimental.data.transforms.transform.NormalizeTFTransform(mean=[0.0], std=[1.0], rescale=None)¶

Bases: BaseTransform

Normalize a image with mean and standard deviation.

Parameters:

mean (list, default=[0.0]) – means for each channel, if len(mean)=1, mean will be broadcasted to each channel, otherwise its length should be same with the length of image shape
std (list, default=[1.0]) – stds for each channel, if len(std)=1, std will be broadcasted to each channel, otherwise its length should be same with the length of image shape

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RescaleKerasPretrainTransform(rescale=None)¶

Bases: BaseTransform

Scale the values of image to [0,1].

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RescaleTFTransform¶

Bases: BaseTransform

Scale the values of image to [0,1].

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RescaleTransform¶

Bases: BaseTransform

Scale the values of image to [0,1].

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.AlignImageChannelTransform(dim=3)¶

Bases: BaseTransform

Align image channel, now just support [H,W]->[H,W,dim], [H,W,4]->[H,W,3] and [H,W,3]->[H,W].

Input image must be np.ndarray.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.PyTorchAlignImageChannel(dim=3)¶

Bases: BaseTransform

Align image channel, now just support [H,W,4]->[H,W,3] and [H,W,3]->[H,W].

Input image must be PIL Image.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ToNDArrayTransform¶

Bases: BaseTransform

Convert np.array to NDArray.

Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ResizeMXNetTransform(size, interpolation='bilinear')¶

Bases: BaseTransform

Resize the input image to the given size.

Parameters:

size (list or int) – Size of the result
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.ResizeTransform(size, interpolation='bilinear')¶

Bases: BaseTransform

Resize the input image to the given size.

Parameters:

size (list or int) – Size of the result
interpolation (str, default='bilinear') – Desired interpolation type,
'bilinear' (support) –
'nearest' –
'bicubic'. –

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CropResizeTFTransform(x, y, width, height, size, interpolation='bilinear')¶

Bases: BaseTransform

Crop the input image with given location and resize it.

Parameters:

x (int) – Left boundary of the cropping area
y (int) – Top boundary of the cropping area
width (int) – Width of the cropping area
height (int) – Height of the cropping area
size (list or int) – resize to new size after cropping
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.PyTorchCropResizeTransform(x, y, width, height, size, interpolation='bilinear')¶

Bases: BaseTransform

Crop the input image with given location and resize it.

Parameters:

x (int) – Left boundary of the cropping area
y (int) – Top boundary of the cropping area
width (int) – Width of the cropping area
height (int) – Height of the cropping area
size (list or int) – resize to new size after cropping
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.MXNetCropResizeTransform(x, y, width, height, size, interpolation='bilinear')¶

Bases: BaseTransform

Crop the input image with given location and resize it.

Parameters:

x (int) – Left boundary of the cropping area
y (int) – Top boundary of the cropping area
width (int) – Width of the cropping area
height (int) – Height of the cropping area
size (list or int) – resize to new size after cropping
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CropResizeTransform(x, y, width, height, size, interpolation='bilinear')¶

Bases: BaseTransform

Crop the input image with given location and resize it.

Parameters:

x (int) – Left boundary of the cropping area
y (int) – Top boundary of the cropping area
width (int) – Width of the cropping area
height (int) – Height of the cropping area
size (list or int) – resize to new size after cropping
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’, ‘bicubic’

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.CenterCropTransform(size)¶

Bases: BaseTransform

Crops the given image at the center to the given size.

Parameters:: size (list or int) – Size of the result
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.MXNetNormalizeTransform(mean=[0.0], std=[1.0])¶

Bases: BaseTransform

Normalize a image with mean and standard deviation.

Parameters:

mean (list, default=[0.0]) – means for each channel, if len(mean)=1, mean will be broadcasted to each channel, otherwise its length should be same with the length of image shape
std (list, default=[1.0]) – stds for each channel, if len(std)=1, std will be broadcasted to each channel, otherwise its length should be same with the length of image shape

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.PyTorchNormalizeTransform(mean=[0.0], std=[1.0])¶

Bases: MXNetNormalizeTransform

Normalize a image with mean and standard deviation.

Parameters:

mean (list, default=[0.0]) – means for each channel, if len(mean)=1, mean will be broadcasted to each channel, otherwise its length should be same with the length of image shape
std (list, default=[1.0]) – stds for each channel, if len(std)=1, std will be broadcasted to each channel, otherwise its length should be same with the length of image shape

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.NormalizeTransform(mean=[0.0], std=[1.0])¶

Bases: BaseTransform

Normalize a image with mean and standard deviation.

Parameters:

mean (list, default=[0.0]) – means for each channel, if len(mean)=1, mean will be broadcasted to each channel, otherwise its length should be same with the length of image shape
std (list, default=[1.0]) – stds for each channel, if len(std)=1, std will be broadcasted to each channel, otherwise its length should be same with the length of image shape

Returns:

tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomCropTransform(size)¶

Bases: BaseTransform

Crop the image at a random location to the given size.

Parameters:: size (list or tuple or int) – Size of the result
Returns:: tuple of processed image and label

class neural_compressor.experimental.data.transforms.transform.RandomResizedCropTransform(size, scale=(0.08, 1.0), ratio=(3.0 / 4.0, 4.0 / 3.0), interpolation='bilinear')¶

Bases: BaseTransform

Crop the given image to random size and aspect ratio.

Parameters:

size (list or int) – Size of the result
scale (tuple or list, default=(0.08, 1.0)) – range of size of the origin size cropped
ratio (tuple or list, default=(3. / 4., 4. / 3.)) – range of aspect ratio of the origin aspect ratio cropped
interpolation (str, default='bilinear') – Desired interpolation type, support ‘bilinear’, ‘nearest’

Returns:

tuple of processed image and label

get_params(image, scale, ratio)¶: Get the image prameters: position, height and width.

neural_compressor.experimental.data.transforms.transform.get_final_text(pred_text, orig_text, do_lower_case)¶: Project the tokenized prediction back to the original text.

class neural_compressor.experimental.data.transforms.transform.SquadExample(qas_id, question_text, doc_tokens, orig_answer_text=None, start_position=None, end_position=None, is_impossible=False)¶

Bases: object

A single training/test example for simple sequence classification.

For examples without an answer, the start and end position are -1.

class neural_compressor.experimental.data.transforms.transform.InputFeatures(unique_id, example_index, doc_span_index, tokens, token_to_orig_map, token_is_max_context, input_ids, input_mask, segment_ids, start_position=None, end_position=None, is_impossible=None)¶

Bases: object

A single set of features of data.

neural_compressor.experimental.data.transforms.transform.read_squad_examples(input_file)¶: Read a SQuAD json file into a list of SquadExample.

neural_compressor.experimental.data.transforms.transform.convert_examples_to_features(examples, tokenizer, max_seq_length, doc_stride, max_query_length, output_fn)¶: Load a data file into a list of `InputBatch`s.

class neural_compressor.experimental.data.transforms.transform.CollectTransform(length=10833)¶

Bases: BaseTransform

Postprocess the predictions, collect data.

class neural_compressor.experimental.data.transforms.transform.TFSquadV1PostTransform(label_file, vocab_file, n_best_size=20, max_seq_length=384, max_query_length=64, max_answer_length=30, do_lower_case=True, doc_stride=128)¶

Bases: BaseTransform

Postprocess the predictions of bert on SQuAD.

Parameters:

label_file (str) – path of label file
vocab_file (str) – path of vocabulary file
n_best_size (int, default=20) – The total number of n-best predictions to generate in nbest_predictions.json
max_seq_length (int, default=384) – The maximum total input sequence length after WordPiece tokenization. Sequences longer than this will be truncated, shorter than this will be padded
max_query_length (int, default=64) – The maximum number of tokens for the question. Questions longer than this will be truncated to this length
max_answer_length (int, default=30) – The maximum length of an answer that can be generated. This is needed because the start and end predictions are not conditioned on one another
do_lower_case (bool, default=True) – Whether to lower case the input text. Should be True for uncased models and False for cased models
doc_stride (int, default=128) – When splitting up a long document into chunks, how much stride to take between chunks

Returns:

tuple of processed prediction and label

process_result(results)¶: Get the processed results.

get_postprocess_result(sample)¶: Get the post processed results.

class neural_compressor.experimental.data.transforms.transform.TFModelZooCollectTransform(length=10833)¶

Bases: CollectTransform

Postprocess the predictions of model zoo, collect data.

class neural_compressor.experimental.data.transforms.transform.TFSquadV1ModelZooPostTransform(label_file, vocab_file, n_best_size=20, max_seq_length=384, max_query_length=64, max_answer_length=30, do_lower_case=True, doc_stride=128)¶

Bases: TFSquadV1PostTransform

Postprocess the predictions of bert on SQuADV1.1.

See class TFSquadV1PostTransform for more details

class neural_compressor.experimental.data.transforms.transform.ParseDecodeVocTransform¶

Bases: BaseTransform

Parse features in Example proto.

Returns:: tuple of parsed image and labels

neural_compressor.experimental.data.transforms.transform¶

Module Contents¶

Classes¶

Functions¶

`neural_compressor.experimental.data.transforms.transform`¶