neural_compressor.data.dataloaders.sampler

Definitions of the methods to sample data.

Module Contents

Classes

Sampler	Base class for all Samplers.
IterableSampler	Interally samples elements.
SequentialSampler	Sequentially samples elements, used for datasets retrieved element by index.
BatchSampler	Yield a batch of indices and number of batches.

class neural_compressor.data.dataloaders.sampler.Sampler(data_source)

Bases: object

Base class for all Samplers.

__iter__ is needed no matter whether you use IterableSampler or Squential sampler, if you want implement your own sampler, make clear what the type is your Dataset, if IterableDataset(method __iter__ implemented), try to use IterableSampler, else if you have an IndexDataset(method __getitem__ implemented), your dataset should have method __len__ implemented.

class neural_compressor.data.dataloaders.sampler.IterableSampler(dataset)

Bases: Sampler

Interally samples elements.

Used for datasets retrieved element by interator. Yield None to act as a placeholder for each iteration.

class neural_compressor.data.dataloaders.sampler.SequentialSampler(dataset, distributed)

Bases: Sampler

Sequentially samples elements, used for datasets retrieved element by index.

class neural_compressor.data.dataloaders.sampler.BatchSampler(sampler, batch_size, drop_last=True)

Bases: Sampler

Yield a batch of indices and number of batches.