Low performance hotspots #1160
Open
Low performance hotspots #1160
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* exposed AzureOpenAI provider * added docs * Update CONTRIBUTING.md * typo in mkdocs.yml --------- Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
* first * typos * typehint
* import llama only if needed * use optional imports instead --------- Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* fix * typo * don't print external if internal is available
Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* adjust docstring for select_context * langchain select_context, update quickstarts * undo app name change * remove dev cell * generalized langchain select_context (truera#711) * generalized langchain select_context * typo * typo in string * update langchain example to pass app in select_context --------- Co-authored-by: Josh Reini <joshreini1@gmail.com> * comments, clarity updates to quickstarts * add lib-independent select_context * update lc li quickstarts --------- Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* add optional * bug class_info fix
* update configs * bugfix * dont add class info to dicts
* Fix correctness prompt Fixes truera#718 * Update base.py
* Bump suggested notebook versions * Combine notebooks and py files --------- Co-authored-by: Shayak Sen <shayak@truera.com>
* Bump suggested notebook versions * Combine notebooks and py files * Update __init__.py --------- Co-authored-by: Shayak Sen <shayak@truera.com>
* debug * display python version * python version * PromptTemplate update import * bad escape fix * add msg to exception * pass kwargs in Groundedness * pass kwargs with GroundTruthAgreement * give default value to ground_truth_imp * migrate db on reset
* fix example notebook * fixes * remove commented out
* always use prompt instead of messages * use messages in base * use prompt in bedrock * move score to top of cot template, request entire template be used * remove dev * add TODO
* update langchain instrumentation page * include instrumented methods * llama-index instrumentation updates * update the overview * change path to instrumentation overview * add some more info in appendices and line space --------- Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* add instructions and text wrapping * format * clean up github scripts and update README sources * typo --------- Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
Co-authored-by: joshreini1 <joshreini1@users.noreply.github.com>
* fix * remove redundant --------- Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
* adjusted * fix typo
Co-authored-by: joshreini1 <joshreini1@users.noreply.github.com>
* add instructions and text wrapping * format * debugging * making obj arg no longer required * remove obj and add documentation for WithClassInfo * remove IPython from most notebooks and organize imports * fix test errors * forgot warning --------- Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
* update notebooks to test * rehack * update langchain requirement * add core lowerbound
* fix instance check on generic types * remove original check --------- Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* check for langchain legacy * few more * typo * rephrase * rephrase
* pass bedrock provider to ground truth eval * Update bedrock_finetuning_experiments.ipynb --------- Co-authored-by: Piotr Mardziel <piotrm@gmail.com>
* first * temp format * minor * more notes * minor fixes
retreivers -> retrievers
* unify groundedness start * remove groundedness.py * groundedness nli moves * remove custom aggregator * groundedness aggregator to user code * move agg to trulens side by default (groundedness) * remove extra code * remove hf key setting * remove hf import * add comment about aggregation for context relevance * update init * remove unneeded import * use generate_score_and_reasons for groundedness internally * f-strings for groundedness prompts * docstring * docstrings formatting * groundedness reasons template * remove redundant prompt * update quickstarts * llama-index notebooks * rag triad helper update * oai assistant nb * update readme * models notebooks updates * iterate nbs * mongo, pinecone nbs * update huggingface docstring * remove outdated docstring selector notes * more docstring cleaning
* open ai streaming adjustments in cost tracking * notes * delete outputs
Co-authored-by: joshreini1 <joshreini1@users.noreply.github.com> Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
* Update selecting_components.md * Update MultiQueryRetrievalLangchain.ipynb * Update random_evaluation.ipynb * Update canopy_quickstart.ipynb
Co-authored-by: joshreini1 <joshreini1@users.noreply.github.com>
* update comprehensiveness + nb * nb expansion * fix typo * meetingbank transcript data * oss models in app * test * benchmarking gpt-3.5-turbo, gpt-4-turbo, and gpt-4o * update path * comprehensiveness benchmark * updated summarization_eval nb * fix normalization * show improvement in comprehensiveness feedback functions --------- Co-authored-by: Daniel <dah0417912@gmail.com>
* version bump * simpler lc quickstart * update installs and imports * update langchain instrumentation docs * remove groundedness ref from providers.md * build docs fixes * remove key cell * fix docs build * firx formatting for stock.md * remove extra spaces * undo format change * update docstrings for hugs and base provider * openai docstring updates * hugs docstring update * update context relevance hugs docstring * more docstring updates * remove can be changed messages from openai provider docstrings
Co-authored-by: joshreini1 <joshreini1@users.noreply.github.com>
* add to glossary * finish some terms
* add httpx to requirements * undo httpx add * test add langchain-community * change langchain-community required version
…nai endpoint (truera#1154) * first * adjust docs * logic * add langhain_community to pinecone reqs too * adjust trubot example with new reqs * more reqs * remove extra text * add langchain_community to a few imports in notebooks --------- Co-authored-by: Piotr Mardziel <piotrm@Piotrs-Laptop.local> Co-authored-by: Josh Reini <60949774+joshreini1@users.noreply.github.com>
Co-authored-by: Josh Reini <joshua.reini@snowflake.com>
Co-authored-by: Josh Reini <joshua.reini@snowflake.com>
dosubot
bot
added
the
size:XL
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
joshreini1
reviewed
May 29, 2024
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
joshreini1
reviewed
May 29, 2024
| @@ -0,0 +1,13086 @@ | |||
| { | |||
There was a problem hiding this comment.
Can you streamline this setup? The key point to get across to the reader is that they need to bring their own record metadata (the title here) and then your change finds hotspots for them.
Reply via ReviewNB
|
Can you focus this change on the hotspots and drop the category and topic analysis pages? For the UI, this could probably use some design thoughts - @walnutdust can you give some feedback here? |
dosubot
bot
added
size:L
joshreini1
changed the title
|
Hey @bodhisaha are you still working on this? |
|
No, I'm not currently working on this.
…On Mon, Aug 5, 2024 at 5:06 PM Prudhvi Dharmana ***@***.***> wrote:
Hey @bodhisaha <https://github.com/bodhisaha> are you still working on
this?
—
Reply to this email directly, view it on GitHub
<#1160 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AP564PR5BPSMHHQU54N7QV3ZQAHRDAVCNFSM6AAAAABIJCEEFGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZQGEZDMMRYGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Secondary pull request without Open AI that adds the notebook that analyzes topic hotspots given the topics, as well as the pages on streamline to display this analysis.
Use case - supervised query classification and analysis.