-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
My system is currently under pressure while training a new model. Benchmarking the search critical path, it's the coarse quantizer that
takes the most time at a whopping 80%.
HSNW-IVF will use HSNW to find the closest centroids and cut down on the latency.
Note: This work depends on removing the fully constructed distance matrix used in KnnNearestNeighbors.
Context: KnnNearestNeighbors helps PLAID by containing a knn distance matrix with all of the centroids. PLAID relies on this to take each document's code and lookup that code's distance.
We can always use a sparse matrix for this instead, or perhaps XTR's imputing missing distances method would work well.
Metadata
Metadata
Assignees
Labels
No labels