Reduce memory usage on nested set build task
|Assignee:||Miguel Angel Medinilla Luque||% Done:|
|Category:||Performance / scalability|
|Target version:||Release 2.6.3|
|Google Code Legacy ID:||Tested version:|
The nested set build task saves all statements on memory leading to a high memory usage in databases with a considerable amount of resources (500K IOs -> ~500 MB). Moreover, the task is executed automatically on CSV imports after #13355, see Miguel Angel's analysis from that ticket:
#2 Updated by José Raddaoui Marín 8 months ago
- Status changed from New to Code Review
- Assignee set to Steve Breker
Ready for code review in https://github.com/artefactual/atom/pull/1241.
I tested these changes with a considerable big database (915875 IOs, 347624 terms and 64 menus):
- Max memory usage was reduced from 670 to 175 MB.
- Execution time was increased from 100 to 132 seconds.
- CPU usage was a bit more even between MySQL and the PHP task.
#4 Updated by José Raddaoui Marín 8 months ago
- Status changed from Feedback to QA/Review
- Assignee changed from José Raddaoui Marín to Miguel Angel Medinilla Luque
Merged in qa/2.x (7fedc9c). Miguel Angel. could you repeat the test from #13355 to verify this changes? Testing a CSV import over a considerable big database would be great, but if you can only test the nested set task, I think that would be okay.