Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean up how we determine libretranslate api keys #337

Merged
merged 8 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
LibreTranslate is a self-hosted language translation tool.
[LibreTranslate](https://libretranslate.com/) is a self-hosted language translation tool. We provide an ArgoCD ApplicationSet based on our [fork](https://github.com/small-hack/libretranslate-helm-chart) of the [official helm chart](https://github.com/libretranslate/helm).

## Example config

```yaml
apps:
libre_translate:
description: |
📖 [link=https://libretranslate.com/]libretranslate[/link] is a self-hosted translation tool.
📖 [link=https://libretranslate.com/]libretranslate[/link] is a self-hosted language translation tool.
This is useful as both a web interface like translate.google.com, but also as an API for use with other apps such as Mastodon

If init is enabled, we'll generate an api key for you.
enabled: false
init:
enabled: true
Expand Down
4 changes: 3 additions & 1 deletion docs/k8s_apps/mastodon.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ You can export the following env vars and we'll use them for your sensitive data
- `MASTODON_S3_BACKUP_ACCESS_ID`
- `MASTODON_S3_BACKUP_SECRET_KEY`
- `MASTODON_RESTIC_REPO_PASSWORD`
- `MASTODON_LIBRETRANSLATE_API_KEY`

## Example Config

Expand All @@ -53,7 +54,7 @@ apps:
description: |
[link=https://joinmastodon.org/]Mastodon[/link] is an open source self hosted social media network.

smol-k8s-lab supports initializing mastodon, by setting up your hostname, SMTP credentials, valkey credentials, postgresql credentials, and an admin user credentials. We pass all credentials as secrets in the namespace and optionally save them to Bitwarden.
smol-k8s-lab supports initializing mastodon, by setting up your hostname, SMTP credentials, valkey credentials, postgresql credentials, libretranslate, and an admin user credentials. We pass all credentials as Secrets in the namespace and optionally save them to Bitwarden.

smol-k8s-lab also creates a local s3 endpoint and as well as S3 bucket and credentials if you enable set mastodon.argo.secret_keys.s3_provider to "minio" or "seaweedfs". Both seaweedfs and minio require you to specify a remote s3 endpoint, bucket, region, and accessID/secretKey so that we can make sure you have remote backups.

Expand All @@ -62,6 +63,7 @@ apps:
- MASTODON_S3_BACKUP_ACCESS_ID
- MASTODON_S3_BACKUP_SECRET_KEY
- MASTODON_RESTIC_REPO_PASSWORD
- MASTODON_LIBRETRANSLATE_API_KEY
enabled: false
init:
enabled: true
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ nav:
- Kepler: k8s_apps/experimental/kepler.md
- Kyverno: k8s_apps/experimental/kyverno.md
- Kubevirt: k8s_apps/experimental/kubevirt.md
- LibreTranslate: k8s_apps/experimental/libretranslate.md
- Longhorn: k8s_apps/experimental/longhorn.md
- Openbao: k8s_apps/experimental/openbao.md
- MinIO: k8s_apps/experimental/minio.md
Expand All @@ -99,6 +98,7 @@ nav:
- Ingress Nginx: k8s_apps/ingress_nginx.md
- K8tz: k8s_apps/k8tz.md
- K8up: k8s_apps/k8up.md
- LibreTranslate: k8s_apps/libretranslate.md
- Mastodon: k8s_apps/mastodon.md
- Matrix: k8s_apps/matrix.md
- MetalLB: k8s_apps/metallb.md
Expand Down
581 changes: 279 additions & 302 deletions poetry.lock

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "smol_k8s_lab"
version = "5.19.1"
version = "5.19.2"
description = "CLI and TUI to quickly install slimmer Kubernetes distros and then manage apps declaratively using Argo CD"
authors = ["Jesse Hitch <jessebot@linux.com>",
"Max Roby <emax@cloudydev.net>"]
Expand Down Expand Up @@ -36,14 +36,14 @@ cryptography = "^43.0"
kubernetes = "^31"
minio = "^7.2"
pyfiglet = "^1.0"
pyjwt = "^2.9"
pyjwt = "^2.10"
python = ">=3.11,<3.13"
pyyaml = "^6.0"
requests = "^2.32"
rich = "^13.9"
ruamel-yaml = "^0.18"
ruamel-yaml-string = "^0.1"
textual = "^0.85.0"
textual = "^0.86"
xdg-base-dirs = "^6.0"
pygame = "^2.5"
python-ulid = "^3.0"
Expand Down
7 changes: 6 additions & 1 deletion smol_k8s_lab/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,11 @@ def main(config: str = "",
# set up self hosted translation
libre_translate_dict = apps.pop('libre_translate', {})
if libre_translate_dict:
configure_libretranslate(argocd, libre_translate_dict, bw)
libretranslate_api_key = configure_libretranslate(argocd,
libre_translate_dict,
bw)
else:
libretranslate_api_key = ""

# setup nextcloud, home assistant, mastodon, and matrix
setup_federated_apps(
Expand All @@ -319,6 +323,7 @@ def main(config: str = "",
pvc_storage_class,
zitadel_hostname,
oidc_obj,
libretranslate_api_key,
bw
)

Expand Down
15 changes: 13 additions & 2 deletions smol_k8s_lab/config/default_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -752,12 +752,22 @@ apps:

libre_translate:
description: |
📖 [link=https://libretranslate.com/]libretranslate[/link] is a self-hosted translation tool.
📖 [link=https://libretranslate.com/]libretranslate[/link] is a self-hosted language translation tool.
This is useful as both a web interface like translate.google.com, but also as an API for use with other apps such as Mastodon

If init is enabled, we'll generate an api key for you.
enabled: false
init:
enabled: true
# we may supporrt restores down the line
# restore:
# enabled: true
# restic_snapshot_ids:
# db_volume: latest
# we may support backups down the line
# backups:
# # cronjob syntax schedule to run libretranslate pvc backups
# pvc_schedule: 10 0 * * *
argo:
# secret keys to provide for the argocd secret plugin app, none by default
secret_keys:
Expand Down Expand Up @@ -818,7 +828,7 @@ apps:
description: |
[link=https://joinmastodon.org/]Mastodon[/link] is an open source self hosted social media network.

smol-k8s-lab supports initializing mastodon, by setting up your hostname, SMTP credentials, valkey credentials, postgresql credentials, and an admin user credentials. We pass all credentials as secrets in the namespace and optionally save them to Bitwarden.
smol-k8s-lab supports initializing mastodon, by setting up your hostname, SMTP credentials, valkey credentials, postgresql credentials, libretranslate, and an admin user credentials. We pass all credentials as Secrets in the namespace and optionally save them to Bitwarden.

smol-k8s-lab also creates a local s3 endpoint and as well as S3 bucket and credentials if you enable set mastodon.argo.secret_keys.s3_provider to "minio" or "seaweedfs". Both seaweedfs and minio require you to specify a remote s3 endpoint, bucket, region, and accessID/secretKey so that we can make sure you have remote backups.

Expand All @@ -827,6 +837,7 @@ apps:
- MASTODON_S3_BACKUP_ACCESS_ID
- MASTODON_S3_BACKUP_SECRET_KEY
- MASTODON_RESTIC_REPO_PASSWORD
- MASTODON_LIBRETRANSLATE_API_KEY
enabled: false
init:
enabled: true
Expand Down
3 changes: 2 additions & 1 deletion smol_k8s_lab/k8s_apps/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ def setup_federated_apps(argocd: ArgoCD,
pvc_storage_class: str = "local-path",
zitadel_hostname: str = "",
zitadel_obj: Zitadel = None,
libretranslate_api_key: str = "",
bw: BwCLI = None) -> None:
"""
Setup any federated apps with initialization supported
Expand All @@ -243,7 +244,7 @@ def setup_federated_apps(argocd: ArgoCD,
zitadel_obj, bw)

if mastodon_dict.get('enabled', False):
configure_mastodon(argocd, mastodon_dict, pvc_storage_class, bw)
configure_mastodon(argocd, mastodon_dict, pvc_storage_class, libretranslate_api_key, bw)

if matrix_dict.get('enabled', False):
configure_matrix(argocd, matrix_dict, pvc_storage_class, zitadel_obj, bw)
45 changes: 34 additions & 11 deletions smol_k8s_lab/k8s_apps/social/libre_translate.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

def configure_libretranslate(argocd: ArgoCD,
cfg: dict,
bitwarden: BwCLI = None) -> None:
bitwarden: BwCLI = None) -> str:
"""
creates a libretranslate app and initializes it with secrets if you'd like :)

Expand All @@ -20,6 +20,8 @@ def configure_libretranslate(argocd: ArgoCD,

optional:
bitwarden - BwCLI() object with session token to create bitwarden items

Returns api key for libretranslate for programatic access
"""
# check immediately if this app is installed
app_installed = argocd.check_if_app_exists('libretranslate')
Expand Down Expand Up @@ -49,16 +51,19 @@ def configure_libretranslate(argocd: ArgoCD,
# we need namespace no matter the install type
libre_translate_namespace = cfg['argo']['namespace']

# api key for programatic access to libretranslate: set it to blank just in case
api_key = ""

# if the user has chosen to use smol-k8s-lab initialization
if not app_installed and init_enabled:
# immediately create namespace
argocd.k8s.create_namespace(libre_translate_namespace)

# if bitwarden is enabled, we create login items for each set of credentials
if bitwarden and not restore_enabled:
setup_bitwarden_items(argocd,
libretranslate_hostname,
bitwarden)
api_key = setup_bitwarden_items(argocd,
libretranslate_hostname,
bitwarden)
# these are standard k8s secrets
else:
# libretranslate admin credentials and smtp credentials
Expand All @@ -77,42 +82,60 @@ def configure_libretranslate(argocd: ArgoCD,
# if bitwarden and init are enabled, make sure we populate appset secret
# plugin secret with bitwarden item IDs
if bitwarden and init_enabled:
refresh_bitwarden(argocd, libretranslate_hostname, bitwarden)
api_key = refresh_bitwarden(argocd, libretranslate_hostname, bitwarden)

return api_key


def setup_bitwarden_items(argocd: ArgoCD,
libretranslate_hostname: str,
bitwarden: BwCLI) -> None:
bitwarden: BwCLI) -> str:
"""
setup initial bitwarden items for home assistant
setup initial bitwarden items for libretranslate

returns the api key used for libretranslate so you can use it in other apps
"""
sub_header("Creating libretranslate items in Bitwarden")
api_key = bitwarden.generate()

# admin credentials for initial owner user
origin = create_custom_field('origin', libretranslate_hostname)
api_id = bitwarden.create_login(
name=f'libretranslate-credentials-{libretranslate_hostname}',
item_url=libretranslate_hostname,
user="n/a",
password=bitwarden.generate(),
password=api_key,
fields=[origin]
)

# update the libretranslate values for the argocd appset
argocd.update_appset_secret({'libretranslate_credentials_bitwarden_id': api_id})

return api_key


def refresh_bitwarden(argocd: ArgoCD,
libretranslate_hostname: str,
bitwarden: BwCLI) -> None:
bitwarden: BwCLI) -> str:
"""
refresh bitwardens item in the appset secret plugin

returns the api key used for libretranslate so you can use it in other apps
"""
log.debug("Making sure libretranslate Bitwarden item IDs are in appset "
"secret plugin secret")

api_id = bitwarden.get_item(
api_item = bitwarden.get_item(
f"libretranslate-credentials-{libretranslate_hostname}"
)[0]['id']
)[0]
api_id = api_item['id']

argocd.update_appset_secret({'libretranslate_credentials_bitwarden_id': api_id})

print(" 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 ")
print(api_item)
print(" 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 ")
api_key = api_item['login']['password']
print(" 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 🦇 ")

return api_key
Loading