Implement Basic Text to Speech #1143

ShaopengLin · 2024-07-18T07:11:12Z

Changes:

Reading the entire or selected text on a web page in different languages.
The reading can be triggered in the Edit Menu, Shortcut, and right-click on a page.
TTS UI to select languages and voices, and stop the reading.
Voice configurations are remembered between sessions.
Language determined by Zim's Locale.

Shortcut:

ALT + SHIFT + A: read article
ALT + SHIFT + S: read selection
ALT + SHIFT + X: stop reading
ALT + SHIFT + V: open voice selector
ALT + SHIFT + L: open language selector

ShaopengLin · 2024-07-18T07:34:13Z

I wasn't able to implement PAUSE, RESUME, and volume.

PAUSE/RESUME: Qt relies on the underlying system's audio engines to perform these operations and Qt has quite some buggy behavior on those engines. I tried for a long time to find solutions but none is compatible with all engines and Qt versions. This leads to the compromise of simply having Speak & STOP.

Volume: From what I tested, the Linux "speech" engines' audio doesn't change during the reading, which in my opinion makes it useless.

Considering we are shipping cross-platform, it is unreliable to have features that work on one but not on another. If we can find a way to package Qt with a working speech engine, we might be able to implement PAUSE/RESUME. However, I have no idea how to do that. @veloman-yunkan @mgautierfr Do you by any chance have an idea of how hard it would be to package an external library like Flite with Kiwix-Desktop? If it's a huge hurdle and we only get the PAUSE/RESUME, then I wouldn't say the return on the effort is great.

The priority for this PR would be to have Kiwix-Desktop be compatible with Qt6.4+, CI builds, and a satisfactory UI/UX given the existing feature.

kelson42 · 2024-07-19T06:14:05Z

@ShaopengLin Thank you very much for your PR. For the moment I'm not able to compile it on my Ubuntu 22.04 but should move soon to 24.04 and hope this will allow me to have Qt 6.4. In the meantime we need to secure the backward compatibility with older version of Qt (both version 5 and 6). Please fix the compilation script and the code to secure that this feature is only activated if Qt6.4 or higher is available, otherwise just deactive the feature in a way both compilation and run are fine.

ShaopengLin · 2024-08-05T07:57:12Z

@kelson42 The compilation is now for Qt 6.4 +.

kelson42 · 2024-09-08T22:03:51Z

@ShaopengLin I have moved to 24.04. Would you be able please to rebase on main so I can finally move forward with the testing?

ShaopengLin · 2024-09-09T06:57:28Z

@kelson42 Done.

From my own testing, TTS from Qt is rather unstable and sometimes crashes the application when used for no apparent reason/pattern at all. I have been trying my best to find out if there is a solution to the core problem, though little information really floats around the internet on this topic. I honestly hope it's just my own computer's issue... Please try it on your machine and see if any crashes or misbehaviors exist. If I cannot figure out why those things occur, rather than relying on Qt's flawed implementation, the goal should be improving screen reader compatibilities.

When you test, the dfki packages(if exists) are likely not going to work so select the other ones.

ShaopengLin · 2024-09-11T17:21:56Z

@kelson42 I have found the root cause to the crashes. Will put up a fixup commit soon...

ShaopengLin · 2024-09-13T20:01:19Z

@kelson42 Should be ready for testing now.

kelson42 · 2024-09-14T12:22:58Z

@ShaopengLin I see the menu entry but nothing happens if I click on it: no sound not bottom bar visible. I use Ubuntu 24.04 wieth Qt 5.15.13.

ShaopengLin · 2024-09-14T12:24:45Z

@kelson42 As mentioned in this comment we moved compilation to Qt6.4+. I will make a change so those menu entries are invisible in Qt 6.4 and below.

kelson42 · 2024-09-14T12:31:22Z

@kelson42 As mentioned in this comment we moved compilation to Qt6.4+. I will make a change so those menu entries are invisible in Qt 6.4 and below.

@ShaopengLin Then the option should not be available (or grayed)

kelson42 · 2024-09-14T12:49:48Z

@ShaopengLin For me, does not compile with Qt 6.4.2:

g++ -c -pipe -std=c++17 -Werror -I/usr/local/include/ -I/usr/local/include -I/usr/include/x86_64-linux-gnu -I/usr/include/p11-kit-1 -O2 -std=gnu++1z -Wall -Wextra -fPIC -D_REENTRANT -DVERSION=2.3.1 -DQT_DEPRECATED_WARNINGS -DQT_DISABLE_DEPRECATED_BEFORE=0x060000 -DQT_NO_DEBUG -DQT_WEBENGINEWIDGETS_LIB -DQT_PRINTSUPPORT_LIB -DQT_WIDGETS_LIB -DQT_WEBENGINECORE_LIB -DQT_QUICK_LIB -DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_QMLMODELS_LIB -DQT_WEBCHANNEL_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_QMLINTEGRATION_LIB -DQT_POSITIONING_LIB -DQT_CORE_LIB -I. -Isubprojects/QtSingleApplication/src -I/usr/include/x86_64-linux-gnu/qt6 -I/usr/include/x86_64-linux-gnu/qt6/QtWebEngineWidgets -I/usr/include/x86_64-linux-gnu/qt6/QtPrintSupport -I/usr/include/x86_64-linux-gnu/qt6/QtWidgets -I/usr/include/x86_64-linux-gnu/qt6/QtWebEngineCore -I/usr/include/x86_64-linux-gnu/qt6/QtQuick -I/usr/include/x86_64-linux-gnu/qt6/QtOpenGL -I/usr/include/x86_64-linux-gnu/qt6/QtGui -I/usr/include/x86_64-linux-gnu/qt6/QtQmlModels -I/usr/include/x86_64-linux-gnu/qt6/QtWebChannel -I/usr/include/x86_64-linux-gnu/qt6/QtQml -I/usr/include/x86_64-linux-gnu/qt6/QtNetwork -I/usr/include/x86_64-linux-gnu/qt6/QtQmlIntegration -I/usr/include/x86_64-linux-gnu/qt6/QtPositioning -I/usr/include/x86_64-linux-gnu/qt6/QtCore -I. -I. -I/usr/lib/x86_64-linux-gnu/qt6/mkspecs/linux-g++ -o contenttypefilter.o src/contenttypefilter.cpp
src/contenttypefilter.cpp: In constructor ‘ContentTypeFilter::ContentTypeFilter(QString, QWidget*)’:
src/contenttypefilter.cpp:16:35: error: ‘checkStateChanged’ is not a member of ‘QCheckBox’
   16 |         connect(this, &QCheckBox::checkStateChanged, this, &ContentTypeFilter::onStateChanged);
      |                                   ^~~~~~~~~~~~~~~~~
make: *** [Makefile:1437: contenttypefilter.o] Error 1

ShaopengLin · 2024-09-14T12:52:42Z

@kelson42 That is not my change. I believe this is from #1205 where I believe @sgourdas should've made the deprecation for only Qt 6.7+, and not Qt 6+. Source

kelson42 · 2024-09-17T09:31:16Z

@sgourdas Can you please fix compilation regression we have now with Qt6? This is blocking review of this PR.

sgourdas · 2024-09-17T09:52:31Z

@sgourdas Can you please fix compilation regression we have now with Qt6? This is blocking review of this PR.

@kelson42 was this not fixed with #1212 by @ShaopengLin?

kelson42 · 2024-09-17T12:30:23Z

Actually I don't have Qt6.7+ on my Ubuntu 24.04. How should i test this? @ShaopengLin @sgourdas Do you use a PPA to get the most recent version?

sgourdas · 2024-09-17T15:33:36Z

Actually I don't have Qt6.7+ on my Ubuntu 24.04. How should i test this? @ShaopengLin @sgourdas Do you use a PPA to get the most recent version?

If I remember correctly I had built it from the git source.

Would you like me to test something?

kelson42 · 2024-11-13T19:50:16Z

@veloman-yunkan Also, Qt at CI / Linux (focal) does not have texttospeech. Will we be adding the package or will I have to disable TTS if the package does not exist?

What does that technically means "does not have texttospeech"? It means no boice is available?

ShaopengLin · 2024-11-13T19:54:54Z

@kelson42 The Qt TextToSpeech module is missing in that CI.

kelson42 · 2024-11-13T20:10:49Z

@ShaopengLin The problem is not really the CI but what will happen if for some reason we have Qt5 without all the necessary. My best guess is that we should be prepared to such a scenario and hide or desactivate the feature if this lack is detected at start. Or is that depending to a certain version of Qt5?

ShaopengLin · 2024-11-13T20:12:49Z

@kelson42 Don't believe it should depend on a certain version of Qt5, but I suppose having safeguards against this would be ideal. I will change this shortly.

veloman-yunkan

Considering TTS as an accessibility feature for people with vision problems, it should be primarily accessible via keyboard. I know that kiwix-desktop's accessibility in that regard is quite poor, however introducing TTS without a way of controlling it via keyboard doesn't seem wise.

src/kiwixapp.cpp

src/texttospeechbar.cpp

veloman-yunkan · 2024-11-14T07:42:27Z

src/texttospeechbar.cpp

+    if (m_speech.state() == QTextToSpeech::Speaking)
+        speak(m_text);


IMHO, that won't be good UX. If possible (given the limitation of pause/resuming functionality of QTextToSpeech) it would be better to pause the speech, change the voice and resume the speech.

My first thought as well, unfortunately, QTextToSpeech's pause and resume barely work.

Basic Reading of Articles or Selected Texts.

Added the bar at bottom of webpage. Intoduced the stop speaking button.

Add shortcut on keyboard to stop reading for accessibility.

Bar is default closed and shown on any speech activation.

The line edit should not be edit/highlight/selectable.

Rather than only the arrow, clicking any place on the combobox should popup the selector

Tabs are by default Zim's Locale

Keep track of the voices so user do not need to reconfigure.

Show the TTS bar and popup the respective selector on shortcut for accessibility.

Some systems or QT versions might not have or the module is not available.

ShaopengLin · 2024-11-15T07:54:18Z

Added a new set of shortcuts for TTS listed on the PR description

veloman-yunkan

LGTM. Didn't test.

ShaopengLin requested a review from kelson42 July 18, 2024 07:34

ShaopengLin force-pushed the Issue#44-tts branch 8 times, most recently from e06256d to ba921f6 Compare August 5, 2024 01:46

ShaopengLin force-pushed the Issue#44-tts branch from ba921f6 to f4ad9f8 Compare September 9, 2024 06:43

ShaopengLin force-pushed the Issue#44-tts branch 2 times, most recently from 6937c65 to 1b9265c Compare September 13, 2024 19:57

ShaopengLin mentioned this pull request Sep 14, 2024

Corrected version for checkStateChanged #1212

Merged

ShaopengLin force-pushed the Issue#44-tts branch from 1b9265c to 5ce8df9 Compare September 15, 2024 11:19

ShaopengLin force-pushed the Issue#44-tts branch from 52fac5b to c62418d Compare November 13, 2024 18:26

ShaopengLin force-pushed the Issue#44-tts branch 2 times, most recently from 7b629ff to 45a1139 Compare November 14, 2024 00:29

veloman-yunkan requested changes Nov 14, 2024

View reviewed changes

ShaopengLin added 16 commits November 15, 2024 01:33

Introduce texttospeechbar.{h,cpp}

b858c0d

Basic Reading of Articles or Selected Texts.

TTS Actions Added to Right Click ContextMenu

815debd

Introduce texttospeechbar.ui

29bbc44

Added the bar at bottom of webpage. Intoduced the stop speaking button.

Introduce KiwixApp::ReadStopAction

9acefb1

Add shortcut on keyboard to stop reading for accessibility.

Add Close Button to TTS Bar

c95528e

Bar is default closed and shown on any speech activation.

ESC Key Closes TTS Bar

8f4064b

Added Language Selector to TTS Bar

0b37a51

TTS Lanugage Selector Entries Are Read-Only

edb9641

The line edit should not be edit/highlight/selectable.

TTS Language Selector Popup On-Click Anywhere

4c30141

Rather than only the arrow, clicking any place on the combobox should popup the selector

Added Voice Selector to TTS Bar

8df319e

TTS Lanugage Matches Zim Locale

9765987

Tabs are by default Zim's Locale

Save Voice Selected For Each Lanugage

414b8a7

Keep track of the voices so user do not need to reconfigure.

Restart Speech When TTS Language/Voice Changes

f992bc5

Introduce KiwixApp::ToggleReading{Voice,Language}

5b298dd

Show the TTS bar and popup the respective selector on shortcut for accessibility.

Proper Styling for TSS

de1b7bb

Disable TTS If No Such QT Module

e3106ea

Some systems or QT versions might not have or the module is not available.

ShaopengLin force-pushed the Issue#44-tts branch from 45a1139 to e3106ea Compare November 15, 2024 07:51

ShaopengLin requested a review from veloman-yunkan November 15, 2024 07:52

veloman-yunkan approved these changes Nov 15, 2024

View reviewed changes

kelson42 merged commit 5d7eb1a into main Nov 15, 2024
6 checks passed

kelson42 deleted the Issue#44-tts branch November 15, 2024 16:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Basic Text to Speech #1143

Implement Basic Text to Speech #1143

ShaopengLin commented Jul 18, 2024 •

edited

Loading

ShaopengLin commented Jul 18, 2024

kelson42 commented Jul 19, 2024

ShaopengLin commented Aug 5, 2024

kelson42 commented Sep 8, 2024

ShaopengLin commented Sep 9, 2024 •

edited

Loading

ShaopengLin commented Sep 11, 2024 •

edited

Loading

ShaopengLin commented Sep 13, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 •

edited

Loading

kelson42 commented Sep 14, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 •

edited

Loading

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

veloman-yunkan left a comment

veloman-yunkan Nov 14, 2024

ShaopengLin Nov 14, 2024

ShaopengLin commented Nov 15, 2024 •

edited

Loading

veloman-yunkan left a comment

		if (m_speech.state() == QTextToSpeech::Speaking)
		speak(m_text);

Implement Basic Text to Speech #1143

Implement Basic Text to Speech #1143

Conversation

ShaopengLin commented Jul 18, 2024 • edited Loading

ShaopengLin commented Jul 18, 2024

kelson42 commented Jul 19, 2024

ShaopengLin commented Aug 5, 2024

kelson42 commented Sep 8, 2024

ShaopengLin commented Sep 9, 2024 • edited Loading

ShaopengLin commented Sep 11, 2024 • edited Loading

ShaopengLin commented Sep 13, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 • edited Loading

kelson42 commented Sep 14, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 • edited Loading

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

veloman-yunkan left a comment

Choose a reason for hiding this comment

veloman-yunkan Nov 14, 2024

Choose a reason for hiding this comment

ShaopengLin Nov 14, 2024

Choose a reason for hiding this comment

ShaopengLin commented Nov 15, 2024 • edited Loading

veloman-yunkan left a comment

Choose a reason for hiding this comment

ShaopengLin commented Jul 18, 2024 •

edited

Loading

ShaopengLin commented Sep 9, 2024 •

edited

Loading

ShaopengLin commented Sep 11, 2024 •

edited

Loading

ShaopengLin commented Sep 14, 2024 •

edited

Loading

ShaopengLin commented Sep 14, 2024 •

edited

Loading

ShaopengLin commented Nov 15, 2024 •

edited

Loading