neural_compressor.experimental.data.datasets.coco_dataset

Built-in COCO datasets class for multiple framework backends.

Module Contents

Classes

ParseDecodeCoco	Helper function for TensorflowModelZooBertDataset.
COCORecordDataset	Tensorflow COCO dataset in tf record format.
COCORaw	Coco raw dataset.
COCONpy	COCO npy dataset.

class neural_compressor.experimental.data.datasets.coco_dataset.ParseDecodeCoco

Helper function for TensorflowModelZooBertDataset.

Parse the features from sample.

class neural_compressor.experimental.data.datasets.coco_dataset.COCORecordDataset

Bases: neural_compressor.experimental.data.datasets.dataset.IterableDataset

Tensorflow COCO dataset in tf record format.

Root is a full path to tfrecord file, which contains the file name. Please use Resize transform when batch_size > 1

Args: root (str): Root directory of dataset.

num_cores (int, default=28):The number of input Datasets to interleave from in parallel. transform (transform object, default=None): transform to process input data. filter (Filter objects, default=None): filter out examples according

to specific conditions.

class neural_compressor.experimental.data.datasets.coco_dataset.COCORaw(root, img_dir='val2017', anno_dir='annotations/instances_val2017.json', transform=None, filter=filter)

Bases: neural_compressor.experimental.data.datasets.dataset.Dataset

Coco raw dataset.

Please arrange data in this way:

/root/img_dir/1.jpg /root/img_dir/2.jpg … /root/img_dir/n.jpg /root/anno_dir

Please use Resize transform when batch_size > 1

Args: root (str): Root directory of dataset.

img_dir (str, default=’val2017’): image file directory. anno_dir (str, default=’annotations/instances_val2017.json’): annotation file directory. transform (transform object, default=None): transform to process input data. filter (Filter objects, default=None): filter out examples according

to specific conditions.

class neural_compressor.experimental.data.datasets.coco_dataset.COCONpy(root, npy_dir='val2017', anno_dir='annotations/instances_val2017.json', transform=None, filter=None)

Bases: neural_compressor.experimental.data.datasets.dataset.Dataset

COCO npy dataset.

Please arrange data in this way:

/root/npy_dir/1.jpg.npy /root/npy_dir/2.jpg.npy … /root/npy_dir/n.jpg.npy /root/anno_dir

Args: root (str): Root directory of dataset.

npy_dir (str, default=’val2017’): npy file directory. anno_dir (str, default=’annotations/instances_val2017.json’): annotation file directory. transform (transform object, default=None): transform to process input data. filter (Filter objects, default=None): filter out examples according

to specific conditions.