Merge branch 'dev' into async-only

.github/workflows/ci.yml

@@ -40,11 +40,25 @@ jobs:
     - name: Install dependencies
       run: poetry install --all-extras
 
+    - name: Verify version match
+      run: |
+        toml_version=$(grep -oP '(?<=version = ")[^"]+' pyproject.toml | head -1)
+        package_version=$(grep -oP '(?<=__version__ = ")[^"]+' rigging/__init__.py)
+        
+        if [ "$toml_version" != "$package_version" ]; then
+          echo "Version mismatch detected!"
+          echo "Version in pyproject.toml: $toml_version"
+          echo "Version in package: $package_version"
+          exit 1
+        else
+          echo "Versions match: $toml_version"
+        fi
+
     - name: Linting
-      run: poetry run ruff -v rigging/
+      run: poetry run ruff rigging/
 
     - name: Typecheck
-      run: poetry run mypy -v rigging/
+      run: poetry run mypy rigging/
 
     - name: Tests
-      run: poetry run pytest -v
+      run: poetry run pytest

README.md

@@ -1,12 +1,29 @@
-# Rigging
+<p align="center">
+    <img src="docs/assets/rigging.png" alt="rigging" width="300" align='center'/>
+</p>
 
-Rigging is a lightweight LLM interaction framework built on Pydantic XML. The goal is to make leveraging LLMs in production pipelines as simple and effictive as possible. Here are the highlights:
+<h3 align="center">
+Simplify using LLMs in code
+</h3>
+
+<h4 align="center">
+    <img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/rigging">
+    <img alt="PyPI - Version" src="https://img.shields.io/pypi/v/rigging">
+    <img alt="GitHub License" src="https://img.shields.io/github/license/dreadnode/rigging">
+    <img alt="GitHub Actions Workflow Status" src="https://img.shields.io/github/actions/workflow/status/dreadnode/rigging/ci.yml">
+</h4>
+
+</br>
+
+Rigging is a lightweight LLM framework built on Pydantic XML. The goal is to make leveraging language models in production code as simple and effictive as possible. Here are the highlights:
 
 - **Structured Pydantic models** can be used interchangably with unstructured text output.
 - LiteLLM as the default generator giving you **instant access to a huge array of models**.
-- Add easy **tool calling** abilities to models which don't natively support it.
+- Simple **tool calling** abilities for models which don't natively support it.
 - Store different models and configs as **simple connection strings** just like databases.
 - Chat templating, forking, continuations, generation parameter overloads, stripping segments, etc.
+- Async batching and fast iterations for **large scale generation**.
+- Metadata, callbacks, and data format conversions.
 - Modern python with type hints, async support, pydantic validation, serialization, etc.
 
 ```py
@@ -24,9 +41,10 @@ print(answer.items)
 # ['J. R. R. Tolkien', 'Stephen King', 'George Orwell']
 ```
 
-Rigging is built and maintained by [dreadnode](https://dreadnode.io) where we use it daily for our work.
+Rigging is built by [**dreadnode**](https://dreadnode.io) where we use it daily.
 
 ## Installation
+
 We publish every version to Pypi:
 ```bash
 pip install rigging
@@ -38,6 +56,80 @@ cd rigging/
 poetry install
 ```
 
+## Supported LLMs
+
+Rigging will run just about any language model:
+
+- Any model from [**LiteLLM**](https://litellm.vercel.app/docs/providers)
+- Any model from [**vLLM**](https://docs.vllm.ai/en/latest/models/supported_models.html)
+- Any model from [**transformers**](https://huggingface.co/docs/transformers/)
+
+### API Keys
+
+Pass the `api_key` in an generator id or use standard environment variables.
+
+```py
+rg.get_generator("gpt-4-turbo,api_key=...")
+```
+
+```bash
+export OPENAI_API_KEY=...
+export MISTRAL_API_KEY=...
+export ANTHROPIC_API_KEY=...
+...
+```
+
+Check out [the docs](https://rigging.dreadnode.io/topics/generators/#api-keys) for more.
+
 ## Getting Started
 
-Head over to **[our documentation](https://rigging.dreadnode.io)** for more information.
+**Check out the guide [in the docs](https://rigging.dreadnode.io/#getting-started)**
+
+1. **Get a generator** using a connection string.
+2. Build a **chat** or **completion** pipeline
+3. **Run** the pipeline and get the output.
+
+```py
+import rigging as rg 
+
+# 1 - Get a generator
+generator = rg.get_generator("claude-3-sonnet-20240229")
+
+# 2 - Build a chat pipeline
+pending = generator.chat([
+    {"role": "system", "content": "Talk like a pirate."},
+    {"role": "user", "content": "Say hello!"},
+])
+
+# 3 - Run the pipeline
+chat = pending.run()
+print(chat.conversation)
+
+# [system]: Talk like a pirate.
+# [user]: Say hello!
+# [assistant]: Ahoy, matey! Here be the salty sea dog ready to trade greetings wit' ye. Arrr!
+```
+
+Want more?
+
+- Use [structured pydantic parsing](https://rigging.dreadnode.io/#basic-parsing)
+- Check out [raw completions](https://rigging.dreadnode.io/topics/completions/)
+- Give the LLM [access to tools](https://rigging.dreadnode.io/topics/tools/)
+- Play with [generation params](https://rigging.dreadnode.io/topics/generators/#overload-generation-params)
+- Use [callbacks in the pipeline](https://rigging.dreadnode.io/topics/callbacks-and-mapping/)
+- Scale up with [async and batching](https://rigging.dreadnode.io/topics/async-and-batching/)
+- Save your work with [serialization](https://rigging.dreadnode.io/topics/serialization/)
+
+## Examples
+
+- OverTheWire Bandit Agent: [**bandit.py**](examples/bandit.py)
+- Damn Vulnerable Restaurant Agent: [**dvra.py**](examples/dvra.py)
+- RAG Pipeline: [**rag.py**](examples/rag.py) (from [kyleavery](https://github.com/kyleavery/))
+
+## Documentation
+
+**[rigging.dreadnode.io](https://rigging.dreadnode.io)** has everything you need.
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=dreadnode/rigging&type=Date)](https://star-history.com/#dreadnode/rigging&Date)

docs/api/generator.md

@@ -1 +1,3 @@
-::: rigging.generator
+::: rigging.generator
+::: rigging.generator.vllm_
+::: rigging.generator.transformers_

docs/index.md

@@ -1,10 +1,12 @@
-Rigging is a lightweight LLM interaction framework built on Pydantic XML. The goal is to make leveraging LLMs in production pipelines as simple and effictive as possible. Here are the highlights:
+Rigging is a lightweight LLM framework built on Pydantic XML. The goal is to make leveraging language models in production code as simple and effictive as possible. Here are the highlights:
 
 - **Structured Pydantic models** can be used interchangably with unstructured text output.
 - LiteLLM as the default generator giving you **instant access to a huge array of models**.
-- Add easy **tool calling** abilities to models which don't natively support it.
+- Simple **tool calling** abilities for models which don't natively support it.
 - Store different models and configs as **simple connection strings** just like databases.
 - Chat templating, forking, continuations, generation parameter overloads, stripping segments, etc.
+- Async batching and fast iterations for **large scale generation**.
+- Metadata, callbacks, and data format conversions.
 - Modern python with type hints, async support, pydantic validation, serialization, etc.
 
 ```py
@@ -22,7 +24,7 @@ print(answer.items)
 # ['J. R. R. Tolkien', 'Stephen King', 'George Orwell']
 ```
 
-Rigging is built and maintained by [dreadnode](https://dreadnode.io) where we use it daily for our work.
+Rigging is built by [**dreadnode**](https://dreadnode.io) where we use it daily.
 
 ## Installation
 

docs/topics/async-and-batching.md

@@ -174,4 +174,70 @@ will scale either the generate parameters or the input messages if either is a s
     [assistant]: Sendai is the largest city in the Tohoku region of
     Japan and is known for its incredible natural scenery, such as the
     nearby Sendai Bay and Zuihoden mausoleum.
+    ```
+
+## Iterating over Models
+
+The [`.run_over`][rigging.chat.PendingChat.run_over] and [`.arun_over`][rigging.chat.PendingChat.arun_over] functions
+allow you to process a pending chat over a set of generators. This is a helper for iterating manually over models
+and calling `.chat()` on them individually.
+
+Generators can be passed as string identifiers or full instances of [Generator][rigging.generator.Generator].
+In the case of async, operations occur in parallel across the models.
+By default the original generator associated with the [`PendingChat`][rigging.chat.PendingChat] is included in the
+iteration, configurable with the `include_original` parameter. Much like the [`run_many`][rigging.chat.PendingChat.run_many]
+and [`run_batch`][rigging.chat.PendingChat.run_batch] functions, you can control the handling of failures with either
+`skip_failed` or `include_failed`.
+
+=== "Run Over Code"
+
+    ```py
+    import rigging as rg
+    from rigging.model import Answer
+
+    QUESTION = "What is the capital of France?"
+    ANSWER = "paris"
+
+    async def score_output(chats: list[rg.Chat]) -> list[rg.Chat]:
+        return [
+            chat.meta(correct=chat.last.parse(Answer).content.lower() == ANSWER)
+            for chat in chats
+        ]
+
+    chats = await (
+        rg.get_generator("openai/gpt-3.5-turbo")
+        .chat([
+            {"role": "system", "content": f"Always respond in one word between {Answer.xml_tags()} tags."},
+            {"role": "user", "content": QUESTION}
+        ])
+        .until_parsed_as(Answer, max_rounds=3)
+        .map(score_output)
+        .arun_over("gpt-4-turbo", "claude-3-haiku-20240307,temperature=0.5", "claude-3-sonnet-20240229")
+    )
+
+    for chat in chats:
+        print("Model:", chat.generator.model)
+        print("Msg:  ", chat.last.content)
+        print("Meta: ", chat.metadata)
+        print()
+    ```
+
+=== "Outputs"
+
+    ```
+    Model: gpt-4-turbo
+    Msg:   <answer>Paris</answer>
+    Meta:  {'correct': True}
+
+    Model: claude-3-haiku-20240307
+    Msg:   <answer>Paris</answer>
+    Meta:  {'correct': True}
+
+    Model: claude-3-sonnet-20240229
+    Msg:   <answer>Paris</answer>
+    Meta:  {'correct': True}
+
+    Model: openai/gpt-3.5-turbo
+    Msg:   <answer>Paris</answer>
+    Meta:  {'correct': True}
     ```