Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[auto-merge] branch-24.10 to branch-24.12 [skip ci] [bot] #776

Merged
merged 1 commit into from
Nov 6, 2024

Conversation

nvauto
Copy link
Collaborator

@nvauto nvauto commented Nov 6, 2024

auto-merge triggered by github actions on branch-24.10 to create a PR keeping branch-24.12 up-to-date. If this PR is unable to be merged due to conflicts, it will remain open until manually fix.

…ices for all ann algorithms. (#757)

* squashed and rebased

support derived class and cuvs ivf_pq

add testing cosine for ivf_pq

replace cuml ivfpq with cuvs ivf_pq

fix less than k items probed and support long label dtype in create spark dataframe

normalize dataset to unit norms for inner_product distances to avoid mg failure

increase ivf_pq quantization to make its recall more stable

remove normalization as it transform the dataset that leads to lower recall

add case when less than k items are probed

* rebased and second squash:

improve test case for fewer k items probed

fix bug relates to CPUNN

revise per comments

fix create_pyspark_dataframe to get it works for cp arrays as input

fix bug on label of create_pyspark_dataframe

fix bug tested in CPUNearestNeighbors model

add refine to the knn.py for ivfpq

in progress for checkout

add debug info

get ivf_pq cosine passed by increasing dataset std to make it separable

get ivf_pq working after using refine

remove unnecessary test for refine

get refine work for less than k itmes probed

replace df.withColumn with df.select to fix slowdown for df that was initialized with wide pd.DataFrame

revise comment to make it more clear

* ensure spark returns are consistent with cuvs when handling less than k items probed

listening for future updates to consolidate behaviors of ivfflat, ivfpq and refine
@nvauto nvauto merged commit 6a77b73 into branch-24.12 Nov 6, 2024
@nvauto
Copy link
Collaborator Author

nvauto commented Nov 6, 2024

SUCCESS - auto-merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants