Transcription of 1 minute file takes 20 - 40 seconds

[martin-opensky]

Hello,

I am using the faster-whisper-server on a Mac M1 with the following start command:

docker run --publish 8000:8000 --volume ~/.cache/huggingface:/root/.cache/huggingface fedirz/faster-whisper-server:latest-cpu

And I'm receiving no performance improvements over the original Open AI Model with the following command:
curl http://localhost:8000/v1/audio/transcriptions -F "file=@test.wav" -F "stream=false" -F "language=en" -F "model=Systran/faster-whisper-small"

This is a 1 minute file which takes between 20s - 40s to transcribe depending on the model size.

To transcribe the same audio file using the Systran faster-whisper directly it takes around a 1-3s.

I'm really unsure why this would be the case. Can anyone shed some light onto what may be causing this?

Many thanks
Martin

[fedirz]

Try the running the following command instead

docker run --publish 8000:8000 --env WHISPER__INFERENCE_DEVICE=auto --volume ~/.cache/huggingface:/root/.cache/huggingface fedirz/faster-whisper-server:latest-cpu

Please let me know if that improves the inference speed or not.

[yimejky]

I believe this is happening because the CPU version in docker cannot utilize more than one thread. I even tried overriding the config via ENV, and it's still the same :(

docker run \
    -p 8000:8000 \
    --cpus=10 \
    -e WHISPER__INFERENCE_DEVICE=auto \
    -e WHISPER__CPU_THREADS=10 \
    -e OMP_NUM_THREADS=10 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --name faster-whisper-server fedirz/faster-whisper-server:latest-cpu