Skip to content

Commit

Permalink
Prepping for 1.0.0rc0
Browse files Browse the repository at this point in the history
  • Loading branch information
@monoxgas
monoxgas committed May 6, 2024
1 parent 347c460 commit 7c33d12
Show file tree
Hide file tree
Showing 13 changed files with 242 additions and 513 deletions.
334 changes: 25 additions & 309 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,327 +1,43 @@
# Rigging

Rigging is a lightweight LLM interaction framework built on Pydantic XML and LiteLLM. It supports useful primitives for validating LLM output and adding tool calling abilities to models that don't natively support it. It also has various helpers for common tasks like structured object parsing, templating chats, overloading generation parameters, stripping chat segments, and continuing conversations.
Rigging is a lightweight LLM interaction framework built on Pydantic XML. The goal is to make leveraging LLMs in production pipelines as simple and effictive as possible. Here are the highlights:

Modern python with type hints, pydantic validation, native serialization support, etc.
- **Structured Pydantic models** can be used interchangably with unstructured text output.
- LiteLLM as the default generator giving you **instant access to a huge array of models**.
- Add easy **tool calling** abilities to models which don't natively support it.
- Store different models and configs as **simple connection strings** just like databases.
- Chat templating, forking, continuations, generation parameter overloads, stripping segments, etc.
- Modern python with type hints, async support, pydantic validation, serialization, etc.

```
pip install rigging
```

### Overview

The basic flow in rigging is:

1. Get a generator object
2. Call `.chat()` to produce a `PendingChat`
3. Call `.run()` on a `PendingChat` to get a `Chat`

`PendingChat` objects hold any messages waiting to be delivered to an LLM in exchange
for a new response message. Afterwhich it is converted into a `Chat` which holds
all messages prior to generation (`.prev`) and after generation (`.next`).

You should think of `PendingChat` objects like the configurable pre-generation step
with calls like `.overload()`, `.apply()`, `.until()`, `.using()`, etc. Once you call
`.run()` the generator is used to produce the next message based on the prior context
and any constraints you have in place. Once you have a `Chat` object, the interation
is "done" and you can inspect/parse the messages.

You'll often see us use functional styling chaining as most of our
utility functions return the object back to you.

```python
chat = generator.chat(...).using(...).until(...).overload(...).run()
```

### Basic Chats

```python
```py
import rigging as rg
from rigging.model import CommaDelimitedAnswer as Answer

generator = rg.get_generator("claude-2.1")
chat = generator.chat(
[
{"role": "system", "content": "You are a wizard harry."},
{"role": "user", "content": "Say hello!"},
]
).run()

print(chat.last)
# [assistant]: Hello!

print(f"{chat.last!r}")
# Message(role='assistant', parts=[], content='Hello!')

print(chat.prev)
# [
# Message(role='system', parts=[], content='You are a wizard harry.'),
# Message(role='user', parts=[], content='Say hello!'),
# ]

print(chat.json)
# [{ ... }]

```

### Model Parsing

```python
import rigging as rg

class Answer(rg.Model):
content: str

chat = (
rg.get_generator("claude-3-haiku-20240307")
.chat([
{"role": "user", "content": f"Say your name between {Answer.xml_tags()}."},
])
.until_parsed_as(Answer)
answer = rg.get_generator('gpt-4') \
.chat(f"Give me 3 famous authors between {Answer.xml_tags()} tags.") \
.until_parsed_as(Answer) \
.run()
)

answer = chat.last.parse(Answer)
print(answer.content)

# "Claude"

print(f"{chat.last!r}")
print(answer.items)

# Message(role='assistant', parts=[
# ParsedMessagePart(model=Answer(content='Claude'), ref='<answer>Claude</answer>')
# ], content='<Answer>Claude</Answer>')

chat.last.content = "new content" # Updating content strips parsed parts
print(f"{chat.last!r}")

# Message(role='assistant', parts=[], content='new content')
```

### Mutliple Models

```python
import rigging as rg

class Joke(rg.Model):
content: str

chat = (
rg.get_generator("claude-2.1")
.chat([{
"role": "user",
"content": f"Provide 3 short jokes each wrapped with {Joke.xml_tags()} tags."},
])
.run()
)

jokes = chat.last.parse_set(Joke)

# [
# Joke(content="Why don't eggs tell jokes? They'd crack each other up!"),
# Joke(content='What do you call a bear with no teeth? A gummy bear!'),
# Joke(content='What do you call a fake noodle? An Impasta!')
# ]
```

### Tools

```python
from typing import Annotated
import rigging as rg

class WeatherTool(rg.Tool):
@property
def name(self) -> str:
return "weather"

@property
def description(self) -> str:
return "A tool to get the weather for a location"

def get_for_city(self, city: Annotated[str, "The city name to get weather for"]) -> str:
print(f"[=] get_for_city('{city}')")
return f"The weather in {city} is nice today"

chat = (
rg.get_generator("mistral/mistral-tiny")
.chat(
[
{"role": "user", "content": "What is the weather in London?"},
]
)
.using(WeatherTool(), force=True)
.run()
)

# [=] get_for_city('London')

print(chat.last.content)

# "Based on the information I've received, the weather in London is nice today."
# ['J. R. R. Tolkien', 'Stephen King', 'George Orwell']
```

### Continuing Chats

```python
import rigging as rg

generator = rg.get_generator("gpt-3.5-turbo")
chat = generator.chat([
{"role": "user", "content": "Hello, how are you?"},
])
Rigging is built and maintained by [dreadnode](https://dreadnode.io) where we use it daily for our work.

# We can fork (continue_) before generation has occured
specific = chat.fork("Be specific please.").run()
poetic = chat.fork("Be as poetic as possible").overload(temperature=1.5).run()

# We can also fork (continue_) after generation
next_chat = poetic.fork(
{"role": "user", "content": "That's good, tell me a joke"}
)

update = next_chat.run()
```

### Basic Templating

```python
import rigging as rg

template = rg.get_generator("gpt-4").chat([
{"role": "user", "content": "What is the capitol of $country?"},
])

for country in ["France", "Germany"]:
print(template.apply(country=country).run().last)

# The capital of France is Paris.
# The capital of Germany is Berlin.
```

### Overload Generation Params

```python
import rigging as rg

pending = rg.get_generator("gpt-3.5-turbo,max_tokens=50").chat([
{"role": "user", "content": "Say a haiku about boats"},
])

for temp in [0.1, 0.5, 1.0]:
print(pending.overload(temperature=temp).run().last.content)

```

### Complex Models

```python
import rigging as rg

class Inner(rg.Model):
type: str = rg.attr()
content: str

class Outer(rg.Model):
name: str = rg.attr()
inners: list[Inner] = rg.element()

outer = Outer(name="foo", inners=[
Inner(type="cat", content="meow"),
Inner(type="dog", content="bark")
])

print(outer.to_pretty_xml())

# <outer name="foo">
# <inner type="cat">meow</inner>
# <inner type="dog">bark</inner>
# </outer>
```

### Strip Parsed Sections

```python
import rigging as rg

class Reasoning(rg.Model):
content: str

meaning = rg.get_generator("claude-2.1").chat([
{
"role": "user",
"content": "What is the meaning of life in one sentence? "
f"Document your reasoning between {Reasoning.xml_tags()} tags.",
},
]).run()

# Gracefully handle mising models
reasoning = meaning.last.try_parse(Reasoning)
if reasoning:
print("reasoning:", reasoning.content.strip())

# Strip parsed content to avoid sharing
# previous thoughts with the model.
without_reasons = meaning.strip(Reasoning)
print("meaning of life:", without_reasons.last.content.strip())

# follow_up = without_thoughts.continue_(...)
```

### Custom Generator

Any custom generator simply needs to implement a `complete` function, and
then it can be used anywhere inside rigging.

```python
class Custom(Generator):
# model: str
# api_key: str
# params: GeneratorParams

custom_field: bool

def complete(
self,
messages: t.Sequence[rg.Message],
overloads: GenerateParams = GenerateParams(),
) -> rg.Message:
# Access self vars where needed
api_key = self.api_key
model_id = self.model

# Merge in args for API overloads
marged: dict[str, t.Any] = self._merge_params(overloads)

# response: str = ...

return rg.Message("assistant", response)


generator = Custom(model='foo', custom_field=True)
generator.chat(...)
## Installation
We publish every version to Pypi:
```bash
pip install rigging
```

*Note: we currently don't have anyway to "register" custom generators for `get_generator`.*

### Logging

By default rigging disables it's logger with loguru. To enable it run:

```python
from loguru import logger

logger.enable('rigging')
If you want to build from source:
```bash
cd rigging/
poetry install
```

To configure loguru terminal + file logging format overrides:

```python
from rigging.logging import configure_logging
## Getting Started

configure_logging(
'info', # stderr level
'out.log', # log file (optional)
'trace' # log file level
)
```
*(This will remove existing handlers, so you might prefer to configure them yourself)*
Head over to **[our documentation](https://rigging.dreadnode.io) for more information.
Loading

0 comments on commit 7c33d12

Please sign in to comment.