Experimental Setup

Datasets

To cover a wide range of use cases, we evaluate SVS on standard datasets of diverse dimensionalities ($d=25$ to $d=768$), number of elements ($n={10}^6$ to $n={10}^9$), data types and metrics as described in the table below.

Dataset	d	n	Encoding	Similarity	n queries	Space (GiB)
gist-960-1M	960	1M	float32	L2	1000	3.6
sift-128-1M	128	1M	float32	L2	10000	0.5
deep-96-10M	96	10M	float32	cosine similarity	10000	3.6
glove-50-1.2M	50	1.2M	float32	cosine similarity	10000	0.2
glove-25-1.2M	25	1.2M	float32	cosine similarity	10000	0.1
t2i-200-100M	200	100M	float32	inner product	10000	74.5
deep-96-100M	96	100M	float32	cosine similarity	10000	35.8
deep-96-1B	96	1B	float32	cosine similarity	10000	257.6
sift-128-1B	128	1B	uint8	L2	10000	119.2

Evaluation Metrics

In all benchmarks and experimental results, search accuracy is measured by k-recall at k, defined by $|S\cap G_t|/k$, where $S$ are the ids of the $k$ retrieved neighbors and $G_t$ is the ground-truth. We use $k=10$ in all experiments. Search performance is measured by queries per second (QPS).

Query Batch Size

The size of the query batch, which will depend on the use case, can have a big impact on performance. Therefore, we evaluate batch sizes: 1 (one query at a time or single query), 128 (typical batch size for deep learning training and inference) and full batch (determined by the number of queries in the dataset, see Datasets).