Investigate options to reduce memory usage working with related models
|Category:||Data model / ORM|
|Google Code Legacy ID:||Tested version:||1.2, 1.3, 1.3.1, 2.0.0, 2.0.1, 2.1, 2.1.1, 2.1.2, 2.2, 2.2.1, 2.3, 2.4, 2.5, 2.6, pre-1.2|
In the latest scalability enhancements we are adding calls to
Qubit::clearClassCaches(); in places where the related resources (
$this->informationObjects for example) are looped. This is added to avoid high memory usage while the related resources are hydrated. E.g.:
I wonder what are the side effects of calling this function several times. Does it cause a performance hit? Do we want to keep other classes in cache? It's possible that the answers to those questions don't justify the changes required to implement this is a different manner (specially the one outlined below), but we should investigate if it's possible to do this better/cleaner.
Laravel 6 introduced lazy collections (source code), using PHP's generators to work with large datasets while keeping the memory usage low. Implementing this in the good old Symfony and Propel 1 may not be an easy task, but the initial ideal would be to include it in the Qubit model classes (through the models builder). It doesn't have to replace the current implementation entirely, it could add an extra method/property (e.g.:
$this->informationObjectsCursor) to be used in loops only, while keeping
$this->informationObjects for other uses where the generator may be an issue.