[BUG] Failed Translation breaks webui

[fritz-fritz]

Description

Understand this is probably a will not fix due to development on v4.

Using docker-compose.gpu.yml with NVIDIA 1650 and CUDA 12.2 works to generate the initial transcription but as soon as you attempt to translate, the browser logs a XHR GET request to /api/translate/66a6673ee59384e2d03a907b/fr that immediately returns a 500 error. The frontend does not catch the error and simply continues happily waiting forever.

To Reproduce

Steps to reproduce the behavior:

1. Enable GPU during initial ./get-whishper.sh
2. Upload and transcribe something
3. Click translate
4. See error in dev tools and uncaught error in frontend

Expected behavior

The frontend should catch ANY error here, display it to the user, and restore access to options for the transcription.

Presume the underlying issue is due to the CUDA version as I saw it requires 12.4 per another issue thread. Currently, this is not available in default debian repos. Regardless of the cause, the failure should be caught by the frontend UI.

Environment

• OS: Linux
• Browser: Mullvad
• Version: 3.1.3
• Hosting: local

Logs and Configuration

Docker Compose Logs

Run the following command in the project folder, force the error, and paste the logs below: docker compose logs -f --tail 50

$ docker compose logs -f --tail 50
WARN[0000] /home/user/AI/Whishper/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
whisper-libretranslate  | Downloading French → English (1.9) ...
whisper-libretranslate  | Downloading Spanish → English (1.0) ...
whisper-libretranslate  | Loaded support for 3 languages (4 models total)!
whisper-libretranslate  | Running on http://*:5000
whisper-libretranslate  | ERROR:libretranslate.app:Exception on /translate [POST]
whisper-libretranslate  | Traceback (most recent call last):
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2529, in wsgi_app
whisper-libretranslate  |     response = self.full_dispatch_request()
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1825, in full_dispatch_request
whisper-libretranslate  |     rv = self.handle_user_exception(e)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1823, in full_dispatch_request
whisper-libretranslate  |     rv = self.dispatch_request()
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1799, in dispatch_request
whisper-libretranslate  |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 349, in time_func
whisper-libretranslate  |     return func(*a, **kw)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 323, in func
whisper-libretranslate  |     return f(*a, **kw)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 699, in translate
whisper-libretranslate  |     raise e
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 686, in translate
whisper-libretranslate  |     hypotheses = translator.hypotheses(q, num_alternatives + 1)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/argostranslate/translate.py", line 296, in hypotheses
whisper-libretranslate  |     translated_paragraph = self.underlying.hypotheses(
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/argostranslate/translate.py", line 167, in hypotheses
whisper-libretranslate  |     self.translator = ctranslate2.Translator(model_path, device=settings.device)
whisper-libretranslate  | RuntimeError: CUDA failed with error forward compatibility was attempted on non supported HW
whisper-libretranslate  | ERROR:libretranslate.app:Exception on /translate [POST]
whisper-libretranslate  | Traceback (most recent call last):
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2529, in wsgi_app
whisper-libretranslate  |     response = self.full_dispatch_request()
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1825, in full_dispatch_request
whisper-libretranslate  |     rv = self.handle_user_exception(e)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1823, in full_dispatch_request
whisper-libretranslate  |     rv = self.dispatch_request()
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1799, in dispatch_request
whisper-libretranslate  |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 349, in time_func
whisper-libretranslate  |     return func(*a, **kw)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 323, in func
whisper-libretranslate  |     return f(*a, **kw)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 699, in translate
whisper-libretranslate  |     raise e
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/libretranslate/app.py", line 686, in translate
whisper-libretranslate  |     hypotheses = translator.hypotheses(q, num_alternatives + 1)
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/argostranslate/translate.py", line 296, in hypotheses
whisper-libretranslate  |     translated_paragraph = self.underlying.hypotheses(
whisper-libretranslate  |   File "/usr/local/lib/python3.8/dist-packages/argostranslate/translate.py", line 167, in hypotheses
whisper-libretranslate  |     self.translator = ctranslate2.Translator(model_path, device=settings.device)
whisper-libretranslate  | RuntimeError: CUDA failed with error forward compatibility was attempted on non supported HW
mongo-1                 | {"t":{"$date":"2024-07-28T15:31:51.229Z"},"s":"I",  "c":"CONTROL",  "id":20697,   "ctx":"main","msg":"Renamed existing log file","attr":{"oldLogPath":"/var/log/mongodb/mongod.log","newLogPath":"/var/log/mongodb/mongod.log.2024-07-28T15-31-51"}}
whishper                | 2024-07-28 15:34:51,288 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
whishper                | 2024-07-28 15:34:51,289 INFO supervisord started with pid 1
whishper                | 2024-07-28 15:34:52,291 INFO spawned: 'backend' with pid 7
whishper                | 2024-07-28 15:34:52,292 INFO spawned: 'frontend' with pid 8
whishper                | 2024-07-28 15:34:52,294 INFO spawned: 'nginx' with pid 9
whishper                | 2024-07-28 15:34:52,295 INFO spawned: 'transcription' with pid 10
whishper                | 2024-07-28 15:34:53,365 INFO success: backend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
whishper                | 2024-07-28 15:34:53,365 INFO success: frontend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
whishper                | 2024-07-28 15:34:53,365 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
whishper                | 2024-07-28 15:34:53,365 INFO success: transcription entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Docker Compose File

$ cat docker-compose.yml 
version: "3.9"

services:
  mongo:
    image: mongo
    env_file:
      - .env
    restart: unless-stopped
    volumes:
      - ./whishper_data/db_data:/data/db
      - ./whishper_data/db_data/logs/:/var/log/mongodb/
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${DB_USER:-whishper}
      MONGO_INITDB_ROOT_PASSWORD: ${DB_PASS:-whishper}
    expose:
      - 27017
    command: ['--logpath', '/var/log/mongodb/mongod.log']

  translate:
    container_name: whisper-libretranslate
    image: libretranslate/libretranslate:latest-cuda
    restart: unless-stopped
    volumes:
      - ./whishper_data/libretranslate/data:/home/libretranslate/.local/share
      - ./whishper_data/libretranslate/cache:/home/libretranslate/.local/cache
    env_file:
      - .env
    user: root
    tty: true
    environment:
      LT_DISABLE_WEB_UI: True
      LT_LOAD_ONLY: ${LT_LOAD_ONLY:-en,fr,es}
      LT_UPDATE_MODELS: True
    expose:
      - 5000
    networks:
      default:
        aliases:
          - translate
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

  whishper:
    pull_policy: always
    image: pluja/whishper:${WHISHPER_VERSION:-latest-gpu}
    env_file:
      - .env
    volumes:
      - ./whishper_data/uploads:/app/uploads
      - ./whishper_data/logs:/var/log/whishper
    container_name: whishper
    restart: unless-stopped
    networks:
      default:
        aliases:
          - whishper
    ports:
      - 8082:80
    depends_on:
      - mongo
      - translate
    environment:
      PUBLIC_INTERNAL_API_HOST: "http://127.0.0.1:80"
      PUBLIC_TRANSLATION_API_HOST: ""
      PUBLIC_API_HOST: ${WHISHPER_HOST:-}
      PUBLIC_WHISHPER_PROFILE: gpu
      WHISPER_MODELS_DIR: /app/models
      UPLOAD_DIR: /app/uploads
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]