Task #13384
Reduce memory usage on nested set build task
| Status: | Verified | Start date: | 07/16/2020 | |
|---|---|---|---|---|
| Priority: | Medium | Due date: | ||
| Assignee: | Miguel Angel Medinilla Luque | % Done: | 0% | |
| Category: | Performance / scalability | |||
| Target version: | Release 2.6.3 | |||
| Google Code Legacy ID: | Tested version: | |||
| Sponsored: | No | Requires documentation: |
Description
The nested set build task saves all statements on memory leading to a high memory usage in databases with a considerable amount of resources (500K IOs -> ~500 MB). Moreover, the task is executed automatically on CSV imports after #13355, see Miguel Angel's analysis from that ticket:
Related issues
History
#1 Updated by José Raddaoui Marín almost 2 years ago
- Related to Task #13355: Reduce memory usage on descriptions CSV import added
#2 Updated by José Raddaoui Marín over 1 year ago
- Status changed from New to Code Review
- Assignee set to Steve Breker
Ready for code review in https://github.com/artefactual/atom/pull/1241.
I tested these changes with a considerable big database (915875 IOs, 347624 terms and 64 menus):
- Max memory usage was reduced from 670 to 175 MB.
- Execution time was increased from 100 to 132 seconds.
- CPU usage was a bit more even between MySQL and the PHP task.
#3 Updated by Steve Breker over 1 year ago
- Status changed from Code Review to Feedback
- Assignee changed from Steve Breker to José Raddaoui Marín
CR complete - looks great Radda!
Ran locally with no issues.
#4 Updated by José Raddaoui Marín over 1 year ago
- Status changed from Feedback to QA/Review
- Assignee changed from José Raddaoui Marín to Miguel Angel Medinilla Luque
Thanks Steve!
Merged in qa/2.x (7fedc9c). Miguel Angel. could you repeat the test from #13355 to verify this changes? Testing a CSV import over a considerable big database would be great, but if you can only test the nested set task, I think that would be okay.
#5 Updated by Redmine Admin over 1 year ago
- Target version changed from Release 2.6.2 to Release 2.6.3
#6 Updated by Sarah Mason about 1 year ago
Verified with rebuild-nested-set task. Spoke to Miguel about testing over a large database.
-Sarah M.
#7 Updated by Sarah Mason about 1 year ago
- Status changed from QA/Review to Verified