Merge branch 'main' into rizzz

docs/access-control-and-security/applications.md

@@ -1,3 +1,8 @@
+---
+slug: "/access-control-and-security/applications"
+description: "Use applications to manage secure access for all connections to your Conductor cluster."
+---
+
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
@@ -56,8 +61,10 @@ Configure the application’s roles and permissions to control what your applica
     The application has been created. You can proceed to add roles or permissions to the application.
 2. Add roles to the application.
     1. In the Application Roles or Unrestricted Roles section, toggle the different application roles for your application.
-3. Add permissions to grant application-level access to resources.
-    1. In the Permissions section, select **+ Add Permission**.
+3. Generate access keys.
+    1. In the Access Keys section, select **+ Create access key** to generate a unique keyId and keySecret. The Key Secret is shown only once, so make sure to copy and store it securely.
+4. Add permissions to grant application-level access to resources.
+    1. In the Permissions section, select **+ Add permission**.
     2. Toggle between each resource type and select the resources to provide access to.
     3. Toggle the access levels for your selected resource:
         * **Read**—The application will be able to view the resource.

docs/access-control-and-security/rbac-overview.md

@@ -1,5 +1,6 @@
 ---
 slug: /category/access-control-and-security
+description: "This guide focuses on how to configure Conductor features for access control and security, including applications, users, groups, and tags."
 ---
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
@@ -78,6 +79,7 @@ If multiple roles are granted, they will have all granted role-level permissions
 | Metadata Manager | Can create, update, delete, and grant permissions to any workflow or task definition in the cluster. <br/> <br/> This role can only be granted by an Admin.                                                                             |
 | Workflow Manager | Can start, pause, resume, rerun, and delete any workflow execution in the cluster. <br/> <br/> This role can only be granted by an Admin.                                                                             |
 | Application Manager | Can create, update, and delete any application in the cluster. <br/> <br/> This role can only be granted by an Admin.                             |
+| Admin | Can create, read, modify, delete, and execute this particular application. <br/> <br/> This role can only be granted by an Admin.              |
 
 </TabItem>
 </Tabs>
@@ -90,10 +92,10 @@ Besides the role-based permissions, you can add granular permissions to **groups
 * Tasks
 * Secrets
 * Environment variables
-* Integrations
-* Prompts
 * Tags
 * Domains
+* Integrations
+* Prompts
 
 Unlike other permission targets, **tags** and **domains** provide bulk access to multiple resources. Tags can be used to grant resources across almost every resource type. When you grant access for a tag “x”, all resources with the tag “x” will be made available to the group or application.
 

docs/access-control-and-security/tags.md

@@ -1,3 +1,8 @@
+---
+slug: "/access-control-and-security/tags"
+description: "Use tags to group tasks, workflows, schedulers, or secrets and easily share these tags with a group of users with the correct permissions."
+---
+
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
@@ -90,7 +95,7 @@ The application now has access to all resources under the tag. You can select th
 </TabItem>
 </Tabs>
 
-Watch how to use tags to manage permissions in Orkes Platform.
+Watch how to use tags to manage permissions in Orkes Conductor.
 
 <center><iframe width="510" height="300" src="https://www.youtube.com/embed/qFHDwh2a_l4?si=DqWsT7_NgMv5ZZgx" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="allowfullscreen"
 mozallowfullscreen="mozallowfullscreen"

docs/access-control-and-security/users-and-groups.md

@@ -1,3 +1,8 @@
+---
+slug: "/access-control-and-security/users-and-groups"
+description: "Manage the users and groups on your Conductor cluster to provide the appropriate level of access and permissions. "
+---
+
 # Managing Users and Groups
 
 :::info
@@ -18,7 +23,7 @@ Users must be added to your Orkes Conductor cluster before they can sign up or l
 
 **To add a user to your cluster:**
 1. In the left navigation menu, go to **Access Control** > **Users**.
-2. Select **(+) Create User**.
+2. Select **(+) Create user**.
 3. Enter the following details:
 
 | Field | Description                                                                                                                                                                                                                                                     |

docs/conceptual-guides/workflow-and-task-status.md

@@ -0,0 +1,42 @@
+---
+slug: "/developer-guides/task-and-workflow-status-in-conductor"
+description: "Find out the different workflow and task status in Conductor."
+---
+
+# Workflow and Task Status
+
+The workflow and task statuses are crucial to understanding the execution mechanism in Conductor. You can check a workflow execution’s current status and task status [from the Conductor UI](debugging-workflows#searching-and-viewing-workflow-executions) or [using API](/reference-docs/api/workflow/search-workflow-executions).
+
+## Workflow status
+
+Once started, a workflow execution will either be in an ongoing or terminal status.
+
+| Status | Description |
+| --------------------------------- | --------------------------------- |
+| RUNNING | The workflow is in progress. |
+| PAUSED | The workflow is paused by a user or external event, and is waiting for a manual action to resume. |
+| COMPLETED | Terminal status where all the tasks in the workflow are completed. |
+| TIMED_OUT | Terminal status where one of the workflow tasks has timed out. |
+| TERMINATED | Terminal status where an incomplete workflow has been terminated by a user, event, or another workflow. |
+| FAILED | Terminal status where the workflow has encountered an error and failed. You can [retry the workflow execution from the failed task](debugging-workflows#recovering-from-failures). |
+
+## Task state transitions
+
+During a workflow execution, each task will go through the state transitions illustrated in the figure below. These states roughly correspond to the nine task statuses in Conductor.
+
+<p align="center"><img src="/content/img/conceptual-guides/workflow_lifecyle-task_states.png" alt="Task state transitions in Conductor." width="100%" height="auto"></img></p>
+
+### Task status
+Each task will be in a scheduled, ongoing, or terminal status.
+
+| Status | Description |
+| --------------------------------- | --------------------------------- |
+| SCHEDULED | The task has been scheduled to be picked up by a worker. |
+| IN_PROGRESS | The task is being executed by a worker. | 
+| SKIPPED | The task is skipped without executing, and the workflow continues to the subsequent tasks. <br/><br/> Occurs if the [Skip Task API](/reference-docs/api/workflow/skip-task-from-workflow) is used during a currently running workflow. |
+| TIMED_OUT | The task times out without being completed. <br/><br/> Occurs if the task has been configured with the following timeout parameters in its task definition: <ul><li>timeOutPolicy.</li><li>timeOutSeconds</li><li>pollTimeoutSeconds</li><li>responseTimeoutSeconds</li></ul> |
+| CANCELED | The scheduled task has been canceled without being completed because the workflow has been terminated. |
+| FAILED | The task failed to complete due to an error, and the workflow can be [retried from the failed task](debugging-workflows#recovering-from-failures). |
+| FAILED_WITH_TERMINAL_ERROR | The task failed to complete due to an error and cannot be retried. |
+| COMPLETED_WITH_ERRORS | The task encountered some errors but is completed. <br/><br/> Occurs due to a variety of reasons. For example, if a task is set as optional and fails during execution. |
+| COMPLETED | The task has been successfully completed without any errors. | 

docs/developer-guides/building-workflows.mdx

@@ -1,3 +1,8 @@
+---
+slug: "/developer-guides/building-workflows"
+description: "This guide focuses on the basics of building workflows in Orkes Conductor, including configuring task inputs and outputs, using workers, managing AI and human tasks, failure handling, and more."
+---
+
 # Building Workflows
 
 In Conductor, all workflow definitions are stored as JSON. There are three ways to create a Conductor workflow:
@@ -71,11 +76,11 @@ In most common cases, you can make use of existing Conductor features instead of
 | Orchestrate human input in the loop        | [Human](../reference-docs/operators/human)      |
 | Query data from Conductor Search API or Metrics         | [Query Processor](../reference-docs/system-tasks/query-processor)      |
 | Send alerts to Opsgenie                    | [Opsgenie](../reference-docs/system-tasks/opsgenie)      |
-| Retrieve text or media content from a URL  | [Get Document](../reference-docs/system-tasks/opsgenie)      |
+| Retrieve text or JSON content from a URL  | [Get Document](../reference-docs/ai-tasks/llm-get-document)      |
 | Generate text embeddings                   | [Generate Embeddings](../reference-docs/ai-tasks/llm-generate-embeddings)      |
 | Store text embeddings in a vector database | [Store Embeddings](../reference-docs/ai-tasks/llm-store-embeddings)      |
 | Generate and store text embeddings in a vector database  | [Index Text](../reference-docs/ai-tasks/llm-index-text)      |
-| Chunk, generate, and store text or media embeddings in a vector database   | [Index Document](../reference-docs/ai-tasks/llm-index-document)      |
+| Chunk, generate, and store text embeddings in a vector database   | [Index Document](../reference-docs/ai-tasks/llm-index-document)      |
 | Retrieve data from a vector database       | [Get Embeddings](../reference-docs/ai-tasks/llm-get-embeddings)      |
 | Retrieve data from a vector database based on a search query     | [Search Index](../reference-docs/ai-tasks/llm-search-index)      |
 | Generate text from an LLM based on a defined prompt              | [Text Complete](../reference-docs/ai-tasks/llm-text-complete)      |
@@ -137,7 +142,7 @@ The workflow definition contains configuration parameters that define the workfl
 
 | Parameter     | Description                                                                                                                                                                                                | Required/ Optional |
 | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
-| name               | Workflow name. <ul><li>For v4.0.1 and prior: Only letters, digits, hyphens (-), and underscores (\_) are allowed.</li>   <li>For v4.0.2 and later: Only letters, digits, spaces, hyphens (-), underscores (\_), and certain special characters (<, >, {, }, #) are allowed.</li></ul> | Required. |
+| name               | Workflow name. <ul><li>For v4.0.1 and prior: Only letters, digits, hyphens (-), and underscores (\_) are allowed.</li>   <li>For v4.0.2 and later: Only letters, digits, spaces, hyphens (-), underscores (\_), and certain special characters (`<`, `>`, `{`, `}`, `#`) are allowed.</li></ul> | Required. |
 | description        | ​Workflow description.                                       | Required. |
 | version            | Workflow version.      | Required. |
 | tasks              | Array of [task configuration](building-workflows#task-configuration) objects.           | Required. |
@@ -146,7 +151,7 @@ The workflow definition contains configuration parameters that define the workfl
 | schemaVersion      | Version of Conductor’s JSON format. Must be 2.    | Required. |
 | restartable        | Whether the workflow is allowed to restart. The default value is true. <br/><br/> Set to false if restarting a completed workflow causes side effects.            | Optional. |
 | workflowStatusListenerEnabled  | Whether to enable a workflow status listener. The default value is false.                                       | Optional. |
-| ownerEmail         | The Conductor ID of the workflow creator.                   | Required. |
+| ownerEmail         | The email ID of the workflow creator.                   | Required. |
 | timeoutPolicy      | The workflow timeout policy.                                | Optional. |
 | timeoutSeconds     | The timeout, in seconds, after which the workflow will be set as TIMED_OUT if it hasn't reached a terminal state. <br/><br/> Set to 0 for no timeouts.                                    | Optional. |
 | failureWorkflow        | The compensation flow upon workflow failure. Useful for cleanup, notifications, or post actions.                                    | Optional. |

docs/developer-guides/caching-task-outputs.md

@@ -1,5 +1,6 @@
 ---
 slug: "../faqs/task-cache-output"
+description: "Learn how to cache task outputs for quick access."
 ---
 
 # Caching Task Outputs
@@ -21,22 +22,116 @@ Configure the following parameters to enable task caching.
 | Parameter | Description | Required/ Optional | 
 | --------- | ----------- | ------------------ |
 | cacheConfig. **ttlInSecond** | The time to live in seconds, which is the duration for the output to be cached. | Required. | 
-| cacheConfig. **key** | The cache key, which is a unique identifier for the cached output. <br/><br/> The key should be constructed as a string of dynamic task inputs, such as `${uri}-${method}`. | Required. | 
+| cacheConfig. **key** | The cache key is a unique identifier for the cached output and must be constructed exclusively from the task’s input parameters. <br/><br/> It can be a string concatenation that contains the task’s input keys, such as `${uri}-${method}` or `re_${uri}_${method}`. | Required. | 
 
 **Example**
 
-``` json
-//task configuration
+Here is a straightforward example of task caching using the task’s input keys, `url` and `mediaType`.
+
+```json
+// workflow definition
+{
+  ...
+  "tasks": [
+    {
+      "name": "get_document",
+      "taskReferenceName": "get_document_ref",
+      "inputParameters": {
+        "url": "${workflow.input.url}",
+        "mediaType": "application/pdf"
+      },
+      "type": "GET_DOCUMENT",
+      "cacheConfig": {
+        "key": "${url}-${mediaType}",
+        "ttlInSecond": 60
+      }
+    }
+  ],
+  "inputParameters": [
+    "url"
+  ],
+  "outputParameters": {},
+  "schemaVersion": 2
+}
+```
+
+In advanced cases where other parameters (such as workflow inputs or global variables) must be included in the `cacheConfig.key`, you must pass these parameters into the task’s `inputParameters` first.
+
+```json
+{
+  ...
+  "tasks": [
+    {
+      "name": "get_document",
+      "taskReferenceName": "get_document_ref",
+      "inputParameters": {
+        "url": "${workflow.input.url}",
+        "mediaType": "application/pdf",
+        "id": "${workflow.input.userId}"
+      },
+      "type": "GET_DOCUMENT",
+      "cacheConfig": {
+        "key": "${url}-${id}",
+        "ttlInSecond": 60
+      }
+    }
+  ],
+  "inputParameters": [
+    "url",
+    "userId"
+  ],
+  "outputParameters": {},
+  "schemaVersion": 2
+}
+```
 
-  "cacheConfig": {
-    "ttlInSecond": 36000,
-    "key": "${uri}-${method}"
-  }
+Example of passing as a variable through the Set Variable task:
+
+```json
+// workflow definition
+{
+  ...
+  "tasks": [
+    {
+      "name": "set_variable",
+      "taskReferenceName": "set_variable_ref",
+      "type": "SET_VARIABLE",
+      "inputParameters": {
+        "region": "${workflow.input.location}"
+      }
+    },
+    {
+      "name": "http",
+      "taskReferenceName": "http_ref",
+      "inputParameters": {
+        "uri": "https://orkes-api-tester.orkesconductor.com/api",
+        "method": "GET",
+        "accept": "application/json",
+        "contentType": "application/json",
+        "encode": true,
+        "body": {
+          "cacheKey": "${workflow.variables.region}",
+          "someUnrelated": "Some-val-qvw4g"
+        }
+      },
+      "type": "HTTP",
+      "cacheConfig": {
+        "key": "re_${body.cacheKey}",
+        "ttlInSecond": 60
+      }
+    }
+  ],
+  "inputParameters": [
+    "location"
+  ],
+  "outputParameters": {},
+  "schemaVersion": 2
+}
 ```
 
 ## Task behavior with caching
 
-Before a task is scheduled, the server checks if there is cached output for the given task definition name by matching the cache key. If a match is found, the task does not get scheduled, and the cached output is used to complete the task.
+Before a task is scheduled, the server checks if there is cached output for the given task definition name by matching the cache key. If a match is found, the task is not scheduled, and the cached output is used to complete it.
 
 If no cached output is found, the task is scheduled as usual. When the task is completed successfully, the output is cached against the cache key for the duration specified.
 

docs/developer-guides/creating-and-managing-gen-ai-prompt-templates.md

@@ -1,3 +1,8 @@
+---
+slug: "/developer-guides/creating-and-managing-gen-ai-prompt-templates"
+description: "Learn how to create prompt templates and use them in Orkes' system LLM tasks."
+---
+
 # Using GenAI Prompt Templates
 
 In this guide, we’ll provide a quick overview of Generative AI Prompt Templates and how Orkes makes it easy to leverage the power of Large Language Models (LLMs) natively in their applications.
@@ -15,9 +20,9 @@ The AI prompts can be created in the Orkes Conductor cluster and can be used in
 ## Creating AI Prompts in Orkes Conductor
 
 :::note
-Before creating AI prompts, ensure you have integrated your Orkes Conductor cluster with the required AI / LLM models. 
+Before creating AI prompts, ensure you have integrated your Orkes Conductor cluster with the required AI/LLM models. 
 
-If you haven’t configured your AI / LLM provider on your Orkes Conductor cluster, navigate to the **Integrations** tab and configure your required provider. Refer to this doc on [how to integrate the LLM providers with Orkes Conductor](/content/category/integrations/ai-llm).
+If you haven’t configured your AI/LLM provider on your Orkes Conductor cluster, navigate to the **Integrations** tab and configure your required provider. Refer to this doc on [how to integrate the LLM providers with Orkes Conductor](/content/category/integrations/ai-llm).
 :::
 
 To create AI prompts,
@@ -60,7 +65,7 @@ To provide permissions to Groups:
 
 1. Navigate to **Access Control > Groups** from the left menu on your Orkes Conductor cluster.
 2. Create a new group or choose an existing one with the members needing access to prompts.
-3. Under **Permissions** section, click +Add Permission.
+3. Under **Permissions** section, click **+Add Permission**.
 4. Under the **Prompt** tab, choose the required prompts with the required permissions. 
 
 <p align="center"><img src="/content/img/rbac-for-prompts.png" alt="RBAC for prompts" width="60%" height="auto"></img></p>

docs/developer-guides/debugging-workflows.md

@@ -1,3 +1,8 @@
+---
+slug: "/developer-guides/debugging-workflows"
+description: "Learn how to use Orkes Platform to debug workflows and recover from failed executions."
+---
+
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 import Install from '@site/src/components/install.mdx';
@@ -46,7 +51,7 @@ The page consists of the following sub-tabs:
 * **Diagram** - Visual representation of the workflow's status; tasks are color-coded (green for successful, red for failed, orange for completed with errors).
 * **Task List** - Details of tasks within the workflow.
 * **Timeline** -  Chronological timeline of task execution times, with zoom functionality for accuracy.
-* **Summary** - Includes key details such as workflow ID, status, version, start/end time, and duration.
+* **Summary** - Includes key details such as workflow ID, status, version, start/end time, duration, and reason for incompletion.
 * **Workflow Input/Output** - Lists inputs and outputs of the workflow.
 * **JSON** - Complete JSON representation of the workflow.
 * **Variables** - Displays variables passed using the set variable task.