Keyword search performance and RemoteQuery community

[ichorid]

Currently, RemoteQuery Community does not cache any remote requests. Caching typical requests like queries for subscribed channels should help with database performance.

[triblerteam]

Could you please post a performance analysis of expensive queries? I prefer to keep Tribler code as simple as possible.

[synctext]

Adding performance results here for keyword search: 8 seconds in some cases. In other cases it is a snappy fast sub-second result. Something is seemingly blocking in some corner cases. Or the database is simply 4.4 GByte big 👍

Ubuntu 18.04 with 7.5.0RC6 and 1953 discovered channels.

-rw-r--r-- 1 pouwelse pouwelse 4.4G Jun 14 14:07 metadata.db
-rw-r--r-- 1 pouwelse pouwelse 273M Feb 12  2019 dispersy.db
-rw-r--r-- 1 pouwelse pouwelse 335M Jan 15 12:37 noodle.db
-rw-r--r-- 1 pouwelse pouwelse 400M Feb 12  2019 tribler.sdb
-rw-r--r-- 1 pouwelse pouwelse 342M Jun 14 14:03 trustchain.db

ToDo: investigate, conduct profiling , and add to performance dashboard of Tribler #4999. Not suggesting a fix..! If we know how much development time a likely fix costs, then we can make an informed decision.

[ichorid]

@kozlovsky solved most of DB performance problems! Thank, you Alexander! 👍