Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Apps - update docs - new initiatives (PRD-128) #657

Merged
merged 15 commits into from
Oct 29, 2024
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified components/data-apps/data_apps-packages.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added components/data-apps/data_apps-theming-custom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
146 changes: 100 additions & 46 deletions components/data-apps/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,24 @@ title: Data Apps
permalink: /components/data-apps/
---



* TOC
{:toc}

## Overview
Data apps are simple, interactive web applications that use data to deliver insight or automatically take action.
Data apps are simple, interactive web applications that use data to deliver insights or automatically take action.
They are usually custom-tailored to tackle a specific problem and enable a dynamic, purpose-built user experience.

Examples of data apps include recommendation engines, interactive segmentation, AI integration, data visualization,
customized internal reporting tools for business teams, financial apps to get insights on your spend patterns, etc.
customized internal reporting tools for business teams, and financial apps for analyzing spending patterns.

Data apps may be written in any language. However, for now Keboola only supports apps written in [Streamlit](https://streamlit.io/),
which is a Python framework for rapid development of such applications.
Data apps may be written in any language. However, for now Keboola only supports apps written in [Streamlit](https://streamlit.io/), a Python framework for the rapid development of such applications.

As mentioned above, a data app is a simple web application, which can be deployed inside a Keboola project
and also publicly accessed from outside the project.
It means that the users that will access your data app don't need access to a Keboola project.
This means that users accessing your data app do not need access to a Keboola project.

## Create a Data App

First, enter a custom prefix for your data app, which you will later share with your app users.
First, enter a custom prefix for your data app, which you will share with your users later.

{: .image-popup}
![Code - main menu](/components/data-apps/data_apps-custom-data-app-url.png)
Expand All @@ -37,22 +33,22 @@ There are two ways to create a data app in Keboola. Select a deployment type tha
![Code - main menu](/components/data-apps/data_apps-main_menu.png)

### Code
For simple use cases, where your Streamlit code fits into one page, paste the code directly into a text area.
This deployment type is ideal for very simple apps or for testing. Check out our [Titanic Demo App](https://demo.keboola.com/app/data-apps/45663441) or [this example from Streamlit docs](https://docs.streamlit.io/library/get-started/create-an-app#lets-put-it-all-together).
For simple use cases where your Streamlit code fits on one page, paste the code directly into a text area.
This deployment type is ideal for simple apps or for testing. Check out our [Titanic Demo App](https://demo.keboola.com/app/data-apps/45663441) or [this example from Streamlit docs](https://docs.streamlit.io/library/get-started/create-an-app#lets-put-it-all-together).

{: .image-popup}
![Code - code](/components/data-apps/data_app-development-type-code.png)
{: .image-popup}
![Code - code](/components/data-apps/data_apps-hello_world-code.png)

#### Packages
To use additional Python packages that are not already in our [Streamlit Base Image](#base-image), enter them into the field Packages.
To use additional Python packages that are not already included in our [Streamlit Base Image](#base-image), enter them into the `Packages` field.

{: .image-popup}
![Packages](/components/data-apps/data_apps-packages.png)

### Git Repository
{% include warning.html content="In BETA, we only support GitHub repositories." %}
{% include warning.html content="Currently in beta, we only support GitHub repositories." %}

To provide feedback, use the feedback button in your project.
If you have a complex application, push your app sources into GitHub and link the repository in this section.
Expand All @@ -69,22 +65,22 @@ If you are using a private repository, you have two options to authenticate:

Follow these steps to authenticate using your GitHub username and personal access token:

1. Generate a personal access token on GitHub by going to your GitHub account settings, selecting **Developer settings > Personal access tokens**, and clicking **Generate new token**. Make sure to give the token appropriate permissions for accessing the repository.
1. Generate a personal access token on GitHub by going to your GitHub account settings, selecting **Developer settings > Personal access tokens**, and clicking **Generate new token**. Ensure the token has the necessare permissions to access the repository.

2. In Keboola, navigate to the **Data App Repository** in your Data App Configuration, check the `Private` option, and enter your GitHub username and the personal access token you generated in step 1.
2. In Keboola, navigate to the **Data App Repository** in your data app configuration, check the `Private` option, and enter your GitHub username and the personal access token you generated in step 1.

3. Click **Save** to authenticate with the private repository.

To authenticate using your SSL private key, follow the instructions in the [GitHub manual](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent). After generating your key, enter your SSH private key into the appropriate configuration field and click **Save**.
To authenticate using your SSH private key, follow the instructions in the [GitHub manual](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent). After generating your key, enter your SSH private key into the appropriate configuration field and click **Save**.

Now, you can deploy your data app from the private repository and access it within your Keboola project.

{: .image-popup}
![Git repository](/components/data-apps/data_apps-git_repository_private_SSH.png)

## Secrets
To provide your app with environment variables or sensitive information like credentials, API keys etc., enter them as key value pairs in the section Secrets.
These secrets will be injected into the secrets.toml file upon deployment of the app.
To provide your app with environment variables or sensitive information like credentials, API keys, etc., enter them as key value pairs in the section Secrets.
These secrets will be injected into the `secrets.toml` file upon deployment of the app.
[Read more about the Streamlit secrets](https://docs.streamlit.io/streamlit-community-cloud/get-started/deploy-an-app/connect-to-data-sources/secrets-management).

### Default Secrets
Expand All @@ -104,14 +100,14 @@ client = Client(url, token)

## Loading Data from Storage
To load data from the storage of a Keboola project into the app, use the [input mapping](https://help.keboola.com/transformations/mappings/#input-mapping) section.
Just select your table in the input mapping section and navigate to that by /data/in/table/your_data.csv or /data/in/files/fileID_FileName.* in your code.
Just select your table in the input mapping section and navigate to that by `/data/in/tables/your_data.csv` or `/data/in/files/fileID_FileName.*` in your code.
Note that, while in BETA, the app needs to be redeployed to fetch up-to-date data.
Or you can use [Keboola Storage Python Client](https://github.com/keboola/sapi-python-client) in the app to load the data as needed.
See the [examples](#Examples) below for usage of the Keboola Storage Python Client.
Or you can use the [Keboola Storage Python Client](https://github.com/keboola/sapi-python-client) in the app to load the data as needed.
See the [examples](#example-data-apps) below for usage of the Keboola Storage Python Client.

## Writing Back to Storage
For writing data back to Keboola Project Storage, use our [Keboola Storage Python Client](https://github.com/keboola/sapi-python-client).
See the [examples](#Examples) below for usage of the Keboola Storage Python Client.
See the [examples](#example-data-apps) below for usage of the Keboola Storage Python Client.

## Deployment and App Management
{% include warning.html content="Once the data app is deployed, its URL will be publicly available! Keboola provides two authorization methods." %}
Expand All @@ -122,12 +118,12 @@ We recommend using the authorization methods provided by Keboola.
{: .image-popup}
![Code - code](/components/data-apps/data_app-authentication.png)

1. **Simple Authorization**: This method allows you to authenticate a user using a username and password.
1. **Basic Authorization**: This method allows you to authenticate a user using a password generated by Keboola.

{: .image-popup}
![Code - code](/components/data-apps/data_app-authentication-basic.png)
![Code - code](/components/data-apps/data_app-authentication-basic-proxy.png)

2. **OIDC (OpenID Connect) Authorization**: This enables users to log into your app using your Single Sign-On (SSO) providers providers.
2. **OIDC (OpenID Connect) Authorization**: This enables users to log into your app using your Single Sign-On (SSO) providers.

{: .image-popup}
![Code - code](/components/data-apps/data_app-authentication-oidc.png)
Expand All @@ -144,14 +140,75 @@ If you enter an app with OIDC, you'll be asked to select an `Authentication Prov

Choose the authorization method that best suits your app's requirements and security needs.

## Theming
To configure theming in your data app, you can select from predefined themes or create a custom theme. Predefined themes include `Keboola`, `Light Red`, `Light Purple`, `Light Blue`, `Dark Green`, `Dark Amber`, and `Dark Orange`. Each theme has a specified primary color, background color, secondary background color, text color, and font. Users choosing `Custom` can manually set these values.

{: .image-popup}
![Code - code](/components/data-apps/data_apps-theming-predefined.png)

For `Custom`, users can select colors using the color pickers and choose the desired font from a list.

{: .image-popup}
![Code - code](/components/data-apps/data_apps-theming-custom.png)

### Predefined Themes:
1. **Keboola**
- Primary Color: `#1F8FFF`
- Background Color: `#FFFFFF`
- Secondary Background Color: `#E6F2FF`
- Text Color: `#222529`
- Font: Sans Serif

2. **Light Red**
- Primary Color: `#FF5D5D`
- Background Color: `#FFFFFF`
- Secondary Background Color: `#FFE6E6`
- Text Color: `#222529`
- Font: Sans Serif

3. **Light Purple**
- Primary Color: `#9A6DD7`
- Background Color: `#FFFFFF`
- Secondary Background Color: `#F2E6FF`
- Text Color: `#222529`
- Font: Sans Serif

4. **Light Blue**
- Primary Color: `#0000B2`
- Background Color: `#FFFFFF`
- Secondary Background Color: `#E6E6FF`
- Text Color: `#222529`
- Font: Sans Serif

5. **Dark Green**
- Primary Color: `#4CAF50`
- Background Color: `#222529`
- Secondary Background Color: `#3D4F41`
- Text Color: `#FFFFFF`
- Font: Sans Serif

6. **Dark Amber**
- Primary Color: `#FFC107`
- Background Color: `#222529`
- Secondary Background Color: `#4A3A24`
- Text Color: `#FFFFFF`
- Font: Sans Serif

7. **Dark Orange**
- Primary Color: `#FFA500`
- Background Color: `#222529`
- Secondary Background Color: `#4A3324`
- Text Color: `#FFFFFF`
- Font: Sans Serif

## Sleep and Resume
Our Suspend/Resume feature helps you save resources by automatically putting your app to sleep after an hour of inactivity. Here's how it works:

**Activity Monitoring**: The app monitors for HTTP requests. If no activity is detected for one hour, the app automatically suspends.

**Automatic Resumption**: As soon as a new request is made to the app, it wakes up and resumes operation. While the resume process is designed to be smooth, the first request upon waking may take slightly longer to process.

**Cost Efficiency**: For example, if your app is active for two hours and then experiences no further activity, it will go to sleep after an additional hour. You'll only be billed for the three hours when the app was active and waiting to suspend.
**Cost Efficiency**: For example, if your app is active for two hours and then becomes inactive, it will go to sleep after one additional hour of inactivity. You'll only be billed for the three hours when the app was active or waiting to suspend.

This feature is not only efficient but also intelligent—ensuring you pay only for what you use, while keeping the app ready for when you need it next.

Expand All @@ -160,14 +217,19 @@ If you enter the URL of a sleeping app, it will trigger its wakeup, and you'll s
{: .image-popup}
![Code - code](/components/data-apps/data_apps-proxy-wakeup.png)

Should anything unexpected occur, a **wakeup error** page will appear, and you can click on **Show More** to view the error details.
Should anything unexpected occur, a **wakeup error** page will appear, and you can click **Show More** to view the error details.

{: .image-popup}
![Code - code](/components/data-apps/data_apps-proxy-error-wakeing-up.png)

### How to Set Up Inactivity Timeout
When you click **Deploy** or **Redeploy** for your app, a wizard will appear, prompting you to specify the backend size and the auto-sleep timeout. You can set the duration of inactivity after which the app will go to sleep, with options ranging from five minutes to 24 hours. The default is set to five minutes.

{: .image-popup}
![Code - code](/components/data-apps/data_apps-deploy-timeout-backedsize.png)

### Base Image
When the app is deployed, the code specified in one of the deployment methods will be injected into our base Streamlit docker image.
When the app is deployed, the code specified in one of the deployment methods will be injected into our base Streamlit Docker image.
This image already has Streamlit and a few other basic packages pre-installed:

```
Expand Down Expand Up @@ -264,7 +326,7 @@ wheel 0.40.0
zipp 3.16.2
```

Please note that the versions of these packages may change, as the newest version of the Streamlit package is used upon deployment unless explicitly specified in the Packages field.
Please note that the versions of these packages may change, as the newest version of the Streamlit package is used upon deployment unless explicitly specified in the `Packages` field.

### Actions Menu
{: .image-popup}
Expand All @@ -277,8 +339,8 @@ Please note that the versions of these packages may change, as the newest versio
- **Delete Data App** – stops the data app deployment and deletes its configuration.

### Debugging App Deployment
If the data app's deployment job fails, you can see the logs from its container in the events log of the deployment job.
For example, a conflict of the specified packages:
If the data app's deployment job fails, you can see the logs from its container in the event log of the deployment job.
For example, there may be a conflict with the specified packages:

{: .image-popup}
![Secrets](/components/data-apps/data_apps-job_error_log.png)
Expand All @@ -291,7 +353,7 @@ Author: Jordan Burger
This demo data app shows how to create a data app with Streamlit Python code from a code directly in Keboola.

- [Configuration](https://demo.keboola.com/app/data-apps/75298630)
- [Live App](https://hello-world-75299519.hub.north-europe.azure.keboola.com)
- [Live app](https://hello-world-75299519.hub.north-europe.azure.keboola.com)

### Titanic Demo App
Author: Monika Feigler
Expand All @@ -302,17 +364,18 @@ This demo data app shows how to create a data app with Streamlit Python code and
- [Source](https://github.com/keboola/titanic-data-app)
- [Live app](https://titanic-demo-app-deployed-from-a-github-repository-49752295.hub.north-europe.azure.keboola.com/)

### AI Created Content Checker
### AI-Created Content Checker
Author: Petr Huňka

This demo app offers a cutting-edge solution that leverages Shopify data to supercharge your campaigns. By harnessing the power of artificial intelligence (AI), we create tailor-made SMS messages and deliver them through Twilio's platform. The result? A seamlessly personalized approach that captivates your audience, ensuring your marketing efforts are not only effective but also driven by AI precision.
This demo app offers a cutting-edge solution that leverages Shopify data to supercharge your campaigns. By harnessing the power of artificial intelligence (AI), tailor-made SMS messages are created and delivered
through Twilio's platform. The result? A seamlessly personalized approach that captivates your audience, ensuring your marketing efforts are not only effective but also driven by AI precision.

- [Configuration](https://demo.keboola.com/app/data-apps/51362322)
- [Documentation](https://help.keboola.com/templates/ai-campaign-executer/)
- [Documentation](https://help.keboola.com/templates/ai-sms-campaign/)
- [Source](https://github.com/keboola/ai_campaign_executer)
- [Live app](https://ai-created-content-checker-ai-campaign-executer-51814454.hub.north-europe.azure.keboola.com)

This data app, along with the complete workflow, can be implemented using the AI SMS Campaign template.
The complete workflow for this data app can be implemented using the AI SMS Campaign template.

### Interactive Keboola Sheets
Author: Petr Huňka
Expand All @@ -331,7 +394,7 @@ Author: Ondřej Svoboda
This data app provides an interactive display of several business metrics with integrated Slack notifications.
- [Live app](https://interactive-kpi-report-kpi-app-71250158.hub.north-europe.azure.keboola.com)
- [Configuration](https://demo.keboola.com/app/data-apps/51361334)
- [Documentation](https://help.keboola.com/templates/interactive-kpi-reporting/)
- [Documentation](https://help.keboola.com/templates/ecomm-kpi-dashboard/)
- [Source](https://github.com/keboola/interactive-kpi-reporting)

This app, along with the complete workflow, can be implemented using the eCommerce KPI Dashboard template.
Expand All @@ -358,15 +421,6 @@ This data app is designed to provide a quick and comprehensive overview of the d

This app, along with the complete workflow, can be implemented using the UA and GA4 Comparison template.

### Kai PromptLab
Author: Jordan Burger and Pavel Chocholouš

Streamline your AI prompting process! Use this Streamlit app to refine, test, and compare multiple prompts, ensuring optimal results. Dive into AI with enhanced efficiency!
- [Documentation](https://help.keboola.com/templates/kai-promptlab/)
- [Source](https://github.com/keboola/kai-promptlab)

This data app, along with the complete workflow, can be implemented using the Kai PromptLab template.

### Kai SQL Bot
Author: Jordan Burger and Pavel Chocholouš

Expand Down