Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expected behaviour of retrieval when using a single query with a line break? #1817

Open
kgourgou opened this issue Nov 18, 2024 · 2 comments

Comments

@kgourgou
Copy link

kgourgou commented Nov 18, 2024

A colleague discovered this line in DSPy that is a bit sus.

queries = [query.strip().split("\n")[0].strip() for query in queries]

https://github.com/stanfordnlp/dspy/blob/332d4c31cf9f101682e386fb488c56a724b4239d/dspy/retrieve/retrieve.py#L70C9-L70C78

What is the expected behaviour here when we have something like queries=['Hello\n can I have some RAG help?']? Because of the above line, DSPy will remap the list to

queries = ['Hello']

which would then impact retrieval.

@okhat
Copy link
Collaborator

okhat commented Nov 18, 2024

Hey @kgourgou ! Good catch. This is worth fixing as part of #1739

@kgourgou kgourgou changed the title Expected behaviour of retrieval when using a single query with one more line breaks? Expected behaviour of retrieval when using a single query with a line break? Nov 18, 2024
@arnavsinghvi11
Copy link
Collaborator

Thanks for raising this @kgourgou ! As we migrate to using the dspy.Retriever interface in #1739 , we will support custom retriever composability to avoid such hard-coded behavior!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants