feat(cursor): Add CursorReplayStrategy with red dot painting and self-correction #823
+225
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #760
/claim #760
What kind of change does this PR introduce?
Feature
Summary
This PR introduces the
CursorReplayStrategy
feature, which allows the AI model to suggest coordinates for a red dot on a screenshot representing the intended action target location. This new strategy provides the ability to visualize and self-correct initial suggestions, improving replay accuracy. It also includes an update to the screenshot capturing functionality to support capturing all monitors.Checklist
How can your code be run and tested?
CursorReplayStrategy
by running the AI model and verifying the red dot is correctly suggested and visualized on the screenshot.Other information
No additional context needed.