Bug #13274

Publication status update job can stall when trying to update large hierarchies

Added by Dan Gillean over 1 year ago. Updated 3 months ago.

Status:NewStart date:03/17/2020
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:Job scheduling
Target version:Release 2.7.0
Google Code Legacy ID: Tested version:2.5, 2.6
Sponsored:No Requires documentation:No

Description

We have seen instances where the job scheduler will stall when updating hierarchies with ~30,000 descendants for example. In this case, the code that loops over each record exhausts all available memory, causing the job to stall.

To reproduce

  • Find or create a very large hierarchy, with tens of thousands of descendants
  • Launch the update publication status job via the More menu, and include descendants

Resulting error

  • Job appears to run indefinitely, will not terminate

Expected result

  • All records in the descriptive hierarchy are published successfully
  • Job scheduler does not stall

Notes

Radda has explored how the ES query API could be used to handle updates, leading to much better performance and requiring less memory. See for example:


Related issues

Related to Access to Memory (AtoM) - Task #13273: Use Elasticsearch's "update by query API" to update relat... New 03/13/2020

History

#1 Updated by Dan Gillean over 1 year ago

  • Related to Task #13273: Use Elasticsearch's "update by query API" to update related resources added

#2 Updated by José Raddaoui Marín over 1 year ago

  • Target version deleted (Release 2.6.0)

#3 Updated by Dan Gillean 3 months ago

  • Target version set to Release 2.7.0

Also available in: Atom PDF