merge upstream #2
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pported by exllama (#729) Refactored the exl2 function in exllamav2.py. The new version offers the following benefits: 1. auto split support. You no longer need to split a large model over 2 GPUs manually, exllama will do it for you 2. 8 bit cache support. Supports the 8 bit cache, can squeeze more context into the same GPU 3. Additional exllamav2 improvements. Supports low_mem, fasttensors. 4. No longer need to pass in num_experts, it is optional. 5. Future support for 4 bit cache. Whenever turbo updates the pip package, uncomment the 4 bit lines for 4 bit support. 6. Refactored the function parameters. Changed the model_kwargs dictionary to individual parameters. Combined with documentation this makes it easier for new users to understand what options they can select.
Fixes [this issue](#743)
…r individual bytes
Some model use bytes as their tokens, such as Qwen (see: https://huggingface.co/Qwen/Qwen-7B/blob/ef3c5c9c57b252f3149c1408daf4d649ec8b6c85/tokenization_qwen.py#L136 )
Switched order of AzureAsyncOpenAI to AsyncAzureOpenAI to match name in repo https://github.com/openai/openai-python/blob/main/src/openai/lib/azure.py
- Will's structured generation workflow cookbook example was not in the mkdocs index, so it was not being displayed. - Same with the LM Studio serving docs. - The brand color was also slightly off:  It has been fixed to this: 
Request received in discord to add an example for the new transformers vision capability. # Vision-Language Models with Outlines This guide demonstrates how to use Outlines with vision-language models, leveraging the new transformers_vision module. Vision-language models can process both text and images, allowing for tasks like image captioning, visual question answering, and more. We will be using the Pixtral-12B model from Mistral to take advantage of some of its visual reasoning capabilities and a workflow to generate a multistage atomic caption. --------- Signed-off-by: jphillips <josh.phillips@fearnworks.com>
accross -> across
This is a condensed version of the demo for [extracting earnings reports](https://github.com/dottxt-ai/demos/tree/main/earnings-reports) to CSV. Overview: - Shows how to use Outlines to structure CSV output - Provides simple tools for converting a table specification to regular expressions - Includes a tuned extraction prompt that performs reasonably well on income statements
Adds a cookbook on extracting structured output from PDFs. I included some extra bells and whistles here by showing how to do JSON, regex, and `choice`, which should help provide inspiration to people working with PDFs.
Forgot to add the earnings report cookbook to the cookbook index (#1235), this fixes it.
I added a receipt processing cookbook. - Uses Qwen or Pixtral - General purpose message templating, no messy model-specific token adding - Easy function for compressing images down for lower processing/memory requirements Should help illustrate a simple use case for vision models.
Fix that error NameError: name 'rng' is not defined
`[Outlines model](../models)` does not return the link correctly. Tried switching to `[Outlines model](../models/models.md)`
This PR adds a JAX compatible API, refer issue #1027
This PR aims at solving #1217
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.