Task #13238

Avoid fetching multiple times the languages enabled from the database on the search populate task

Added by José Raddaoui Marín about 1 month ago. Updated about 10 hours ago.

Status:QA/ReviewStart date:01/13/2020
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:CLI tools
Target version:Release 2.6.0
Google Code Legacy ID: Tested version:
Sponsored:No Requires documentation:

Description

Analyzing the search index population process with Percona Monitoring and Management, one of the most demanding queries to the database seems to be one to fetch the enabled languages:

This query is executed here multiple times per document (once for the main document and, depending on the related objects, a few times more). For example, on a description with one creator, one repository and 2 related subjects, the query will be executed 5 times. Moreover, it's not very efficient as it filters using the scope column, which requires a full table scan on each run. From the metrics above (taken indexing the demo data, a really small database), it has been executed more than 10000 times and it took more than 12% of the total query time from the process.

I'm not sure if we already have the language settings cached in sfConfig but we should try to follow that path to only execute this query once on the entire search index population process.

language_settings.png (100 KB) José Raddaoui Marín, 01/13/2020 02:54 PM


Related issues

Related to Access to Memory (AtoM) - Task #13224: Improve hierarchy management queries Code Review 12/09/2019
Related to Access to Memory (AtoM) - Task #13261: Improve Elasticsearch indexing process In progress 02/16/2020

History

#1 Updated by José Raddaoui Marín about 1 month ago

  • Related to Task #13224: Improve hierarchy management queries added

#2 Updated by José Raddaoui Marín 10 days ago

  • Status changed from New to Code Review
  • Target version set to Release 2.6.0

#3 Updated by José Raddaoui Marín 10 days ago

  • Related to Task #13261: Improve Elasticsearch indexing process added

#4 Updated by José Raddaoui Marín about 10 hours ago

  • Status changed from Code Review to QA/Review

Merged in qa/2.6.x. The query has been removed from the search populate process entirely, so that part could be tested alongside #13261.

Additionally, other places that were using the same process to get the enabled languages have been changed and should be tested too:

- Languages header menu.
- Static page edit and delete.
- User edit, translate field.
- An ES query, fro example from the search box.

Also available in: Atom PDF