Multithreading

SVS throughput can be highly improved by running multiple queries in parallel, as its performance scales very well with the number of threads (see System utilization - Multithreading for benchmarking results). Larger query batch sizes will benefit more from multi-threaded search.

The number of threads can be specified when loading the index

In Python

# We can reload an index from a previously saved set of files.
index = svs.Vamana(
    os.path.join(test_data_dir, "example_config"),
    svs.GraphLoader(os.path.join(test_data_dir, "example_graph")),
    svs.VectorDataLoader(
        os.path.join(test_data_dir, "example_data"), svs.DataType.float32
    ),
    svs.DistanceType.L2,
    num_threads = 4,
)

In C++

Click to display

// We can reload an index from a previously saved set of files.
index = svs::Vamana::assemble<float>(
    "example_config",
    svs::GraphLoader("example_graph"),
    svs::VectorDataLoader<float>("example_data"),
    svs::DistanceType::L2,
    4 // num_threads
);

or at runtime

In Python

index.num_threads = 4

In C++

Click to display

index.set_num_threads(4);

Make sure to set the number of threads according to the query batch size. Smaller batch sizes may benefit from using fewer threads, as the overhead of handling multiple threads may be too large.

Warning

To avoid performance degradation make sure to set the number of threads to 1 if running one query at a time.