-
Notifications
You must be signed in to change notification settings - Fork 44.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(platform): Support manually setting up webhooks #8750
Closed
Closed
Changes from 85 commits
Commits
Show all changes
87 commits
Select commit
Hold shift + click to select a range
857ae69
Add `Graph.on_update`, `Block.on_node_update`, `Block.on_node_delete`…
Pwuts 2f405c3
Add credentials injection to `Graph.on_update` etc hooks
Pwuts 921f614
Add webhooks and their links to nodes to DB schema
Pwuts 946b35e
update webhooks db schema
Pwuts dfe36c1
Introduce `PLATFORM_BASE_URL` config attribute
Pwuts 321ef9d
Add `WebhooksManager` base and GitHub implementation + graph lifecycl…
Pwuts f4ce0f4
Add webhook ingress endpoint
Pwuts 40e846e
fix(blocks): Allow having an input and output pin with the same name
Pwuts 6dc9e0c
fix bootstrapping issues
Pwuts ab5b336
hide input pins on webhook blocks
Pwuts 41ccae7
dedup `generateInputHandles`
Pwuts a82acc7
refactor(backend): Make `RedisEventQueue` generic
Pwuts 36b87eb
apply event type filter in webhook ingress endpoint
Pwuts 13e6a75
add webhook ping endpoint
Pwuts f117d3f
hide `payload` input
Pwuts c604d27
fix graph creation
Pwuts 34d82bf
fix enum usage
Pwuts 98aed6d
smol cleanup
Pwuts 9f37671
fix webhook resource string formatting
Pwuts e5c95f6
add debug stuff for webhooks
Pwuts 5e995f9
fix webhook ingress URL
Pwuts aa12a5a
fix github events
Pwuts 707df04
Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into re…
majdyz 5158b87
[Reduce code change size] Revert CreatableGraph/CreatableNode -> Grap…
majdyz 5e70973
[Reduce code change size] Take Redis generic refactor out of the PR
majdyz 6885fa8
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
majdyz 2734d83
Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into re…
majdyz 19b38fb
Fix frontend_base_url / platform_base_url clash
majdyz 1a23591
Set default platform_base_url & fix agent output block bug
majdyz 0343402
Fix CORS issue, make ValueError 400, toast saveAgent HTTP error
majdyz 90d1fc7
Propagate clean error message from github error to user
majdyz dfeecf4
Propagate clean error message from github error to user
majdyz 089b236
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
majdyz 1ebadb7
Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into re…
majdyz 35f044d
Lint
majdyz 3e14f45
Merge remote-tracking branch 'origin/reinier/open-1961-implement-gith…
majdyz 94d7567
Refactor to use new RedisEventBus
majdyz e9d1b1b
Skip webhook activation on no events
majdyz 4b8cd25
Fix webhook changes breakage
majdyz 2b45c28
Remove already done note
majdyz 843d1cb
Unify channel name format
majdyz c357ea7
undo removing subgraph stuff
Pwuts 57b532d
fix Node <-> Webhook reference and NodeModel usages
Pwuts 9e0695f
smol clean
Pwuts 02e952c
remove user auth middleware from webhook ingress endpoint
Pwuts d818af6
fix `add_execution` webhook mechanism
Pwuts 4c68a1c
fix GitHub PR trigger block propagating output from base implementation
Pwuts e7dc870
move event output from base to PR trigger block
Pwuts 5c82edc
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts dd446a7
change event filter input to a MultiSelect
Pwuts a525ce6
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts ca942cc
fix registration of pyro (de)serializers on `DatabaseManager`
Pwuts 00882c0
format
Pwuts a573848
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts 1de18a8
format frontend
Pwuts b1fac14
fix `NodeMultiSelectInput` click behavior
Pwuts d4c5e63
fix `NodeMultiSelectInput` scroll behavior
Pwuts ce56136
add docs for webhook-triggered blocks
Pwuts 2a32b3b
fix collapsers?
Pwuts 90418e0
docs: wording & formatting
Pwuts f40aef8
formatting
Pwuts 8f708a2
fix code block titles
Pwuts 35ffa09
add instruction to add event filter
Pwuts 4048cc8
formatting
Pwuts 060b1bc
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts b7c7028
fix pre-commit isort
Pwuts 9e4f657
fix CI
Pwuts bdc3590
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts 63a41f7
make `GithubPullRequestTriggerBlock` test an actual test
Pwuts 865ac4b
address feedback
Pwuts fd0a3c2
add instructions for `PLATFORM_BASE_URL` to .env.example
Pwuts 3ae2128
move github example payload
Pwuts 5bb9e91
fix(backend): Add migrations to fix credentials inputs
Pwuts 5ec5834
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts 9e16e74
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts bc79d3b
Remove `ProviderName` enum
Pwuts 4a5b783
Improve input descriptions and fix tooltips on GitHub Pull Request Tr…
Pwuts 4c53eb3
Rename `sender` output to `triggered_by_user`
Pwuts 7c0b336
Update description of `triggered_by_user`
Pwuts 0f7fa4b
Fix description of `payload` output to be generic
Pwuts 2dcbcfe
Add `pull_request_url` output
Pwuts 397ae0b
Merge branch 'dev' into reinier/open-1961-implement-github-on-pull-re…
Pwuts 8761c03
Remove webhooks before deleting credentials
Pwuts 6eb643d
Prevent use of webhook system if `PLATFORM_BASE_URL` is not set
Pwuts 0ac0ac9
feat: wip eod commit
ntindle 463c8a6
Update triggers.py
ntindle 23ab904
Merge branch 'dev' into exploration/other-types-of-hooks
ntindle File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
75 changes: 75 additions & 0 deletions
75
autogpt_platform/backend/backend/blocks/compass/triggers.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
from typing import Literal | ||
from backend.data.block import ( | ||
Block, | ||
BlockCategory, | ||
BlockOutput, | ||
BlockSchema, | ||
BlockWebhookConfig, | ||
) | ||
from backend.data.model import CredentialsField, CredentialsMetaInput, SchemaField | ||
from backend.integrations.webhooks.simple_webhook_manager import CompassWebhookType | ||
from pydantic import BaseModel | ||
|
||
|
||
class Transcription(BaseModel): | ||
text: str | ||
speaker: str | ||
end: float | ||
start: float | ||
duration: float | ||
|
||
|
||
class TranscriptionDataModel(BaseModel): | ||
date: str | ||
transcription: str | ||
transcriptions: list[Transcription] | ||
|
||
|
||
class CompassAITriggerBlock(Block): | ||
class Input(BlockSchema): | ||
class EventsFilter(BaseModel): | ||
all: bool = True | ||
|
||
payload: TranscriptionDataModel = SchemaField(hidden=True) | ||
events: EventsFilter = SchemaField( | ||
description="Filter the events to be triggered on." | ||
) | ||
|
||
credentials: CredentialsMetaInput[Literal["compass"], Literal["api_key"]] = ( | ||
CredentialsField( | ||
provider="compass", | ||
supported_credential_types={"api_key"}, | ||
) | ||
) | ||
|
||
class Output(BlockSchema): | ||
transcription: str = SchemaField( | ||
description="The transcription of the compass transcription." | ||
) | ||
|
||
def __init__(self): | ||
super().__init__( | ||
id="9464a020-ed1d-49e1-990f-7f2ac924a2b7", | ||
description="This block forwards an input value as output, allowing reuse without change.", | ||
categories={BlockCategory.HARDWARE}, | ||
input_schema=CompassAITriggerBlock.Input, | ||
output_schema=CompassAITriggerBlock.Output, | ||
webhook_config=BlockWebhookConfig( | ||
provider="simple_hook_manager", | ||
webhook_type=CompassWebhookType.TRANSCRIPTION, | ||
resource_format="{transcription_id}", | ||
event_filter_input="events", | ||
event_format="transcription.{event}", | ||
), | ||
# test_input=[ | ||
# {"input": "Hello, World!"}, | ||
# {"input": "Hello, World!", "data": "Existing Data"}, | ||
# ], | ||
# test_output=[ | ||
# ("output", "Hello, World!"), # No data provided, so trigger is returned | ||
# ("output", "Existing Data"), # Data is provided, so data is returned. | ||
# ], | ||
) | ||
|
||
def run(self, input_data: Input, **kwargs) -> BlockOutput: | ||
yield "transcription", input_data.payload.transcription |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have any real concept of credentials that I need here. If we generate a secret on the platform side, that would be useful so we can ensure its a trusted source who sent the hook. Maybe we should store that in the credential?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For that we already have the
Webhook.secret
. This mechanism doesn't need credentials, so we should just omit thecredentials
field.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll have to amend this check in
on_node_activate
to only enforcecredentials
if there is an actualcredentials
field onblock.input_schema
:AutoGPT/autogpt_platform/backend/backend/integrations/webhooks/graph_lifecycle_hooks.py
Lines 114 to 128 in 68f4aea
This is also one of the two places where the presence of an event filter is enforced.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically the only thing it should do for your case is create a
Webhook
and store it in the DB. The easy way is to just skip parts of the process and then display the webhook ingress URL and webhook secret on the block in the UI. The neat way would be to have a picker for the webhooks for a given provider, on blocks that can't auto-create their webhook (like GitHub).