neural_compressor.data.dataloaders.sampler

Definitions of the methods to sample data.

Classes

Sampler	Base class for all Samplers.
IterableSampler	Internally samples elements.
SequentialSampler	Sequentially samples elements, used for datasets retrieved element by index.
BatchSampler	Yield a batch of indices and number of batches.

Module Contents

class neural_compressor.data.dataloaders.sampler.Sampler(data_source)

Base class for all Samplers.

__iter__ is needed no matter whether you use IterableSampler or Squential sampler, if you want implement your own sampler, make clear what the type is your Dataset, if IterableDataset(method __iter__ implemented), try to use IterableSampler, else if you have an IndexDataset(method __getitem__ implemented), your dataset should have method __len__ implemented.

class neural_compressor.data.dataloaders.sampler.IterableSampler(dataset)

Internally samples elements.

Used for datasets retrieved element by iterator. Yield None to act as a placeholder for each iteration.

class neural_compressor.data.dataloaders.sampler.SequentialSampler(dataset, distributed)

Sequentially samples elements, used for datasets retrieved element by index.

class neural_compressor.data.dataloaders.sampler.BatchSampler(sampler, batch_size, drop_last=True)

Yield a batch of indices and number of batches.