Bug #12880

Migration 163 memory error

Added by Steve Breker 2 months ago. Updated about 1 month ago.

Status:VerifiedStart date:03/12/2019
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:Migration task
Target version:Release 2.5.0
Google Code Legacy ID: Tested version:
Sponsored:No Requires documentation:

Description

The AtoM migration #163 runs the i18nCustomLinkToMarkdownTask. During testing I ran the sql upgrade CLI task against a large database on my local VM and it hit the configured PHP memory limit of ~4GB causing the migration to fail.

Refactor the i18nCustomLinkToMarkdownTask to use less memory so that it can complete without out of memory errors.

Error encountered was:

PHP Fatal error: Allowed memory size of 4294967296 bytes exhausted (tried to allocate 65536 bytes) in /usr/share/nginx/atom/lib/task/i18n/i18nCustomLinkToMarkdownTask.class.php(83) : runtime-created function on line 9

History

#1 Updated by Steve Breker 2 months ago

  • Category set to Migration task

#2 Updated by Steve Breker 2 months ago

Full error and stack trace:

php symfony tools:upgrade-sql
  Plugin rbcmArchivesPlugin no longer exists. Remove it (Y/n)?  
y
>> upgrade-sql Removing plugin from settings: rbcmArchivesPlugin
>> upgrade-sql There is not a valid theme set currently.
  Would you like to choose a new theme (Y/n)?  
y
>> upgrade-sql Please enter a new theme choice:
1) arDominionPlugin
2) arArchivesCanadaPlugin
Select theme number: 1
>> upgrade-sql AtoM theme changed to arDominionPlugin.
  WARNING: Your database has not been backed up!             
  Please back-up your database manually before you proceed.  
  If this task fails you may lose your data.                 
  Have you done a manual backup and wish to proceed? (y/N)   
y

PHP Fatal error:  Allowed memory size of 4294967296 bytes exhausted (tried to allocate 65536 bytes) in /usr/share/nginx/atom/lib/task/i18n/i18nCustomLinkToMarkdownTask.class.php(83) : runtime-created function on line 9

PHP Stack trace:
PHP   1. {main}() /usr/share/nginx/atom/symfony:0
PHP   2. include() /usr/share/nginx/atom/symfony:14
PHP   3. sfSymfonyCommandApplication->run() /usr/share/nginx/atom/vendor/symfony/lib/command/cli.php:20
PHP   4. sfTask->runFromCLI() /usr/share/nginx/atom/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php:76
PHP   5. sfBaseTask->doRun() /usr/share/nginx/atom/vendor/symfony/lib/task/sfTask.class.php:97
PHP   6. QubitUpgradeSqlTask->execute() /usr/share/nginx/atom/vendor/symfony/lib/task/sfBaseTask.class.php:68
PHP   7. arMigration0163->up() /usr/share/nginx/atom/lib/task/migrate/arUpgradeSqlTask.class.php:214
PHP   8. i18nTransformBaseTask->execute() /usr/share/nginx/atom/lib/task/migrate/migrations/arMigration0163.class.php:35
PHP   9. i18nCustomLinkToMarkdownTask->processRow() /usr/share/nginx/atom/lib/task/i18n/i18nTransformBaseTask.class.php:138
PHP  10. create_function() /usr/share/nginx/atom/lib/task/i18n/i18nCustomLinkToMarkdownTask.class.php:83
PHP Fatal error:  Allowed memory size of 4294967296 bytes exhausted (tried to allocate 65536 bytes) in /usr/share/nginx/atom/vendor/symfony/lib/database/sfDatabaseManager.class.php on line 127

PHP Stack trace:
PHP   1. {main}() /usr/share/nginx/atom/symfony:0
PHP   2. include() /usr/share/nginx/atom/symfony:14
PHP   3. sfSymfonyCommandApplication->run() /usr/share/nginx/atom/vendor/symfony/lib/command/cli.php:20
PHP   4. sfTask->runFromCLI() /usr/share/nginx/atom/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php:76
PHP   5. sfBaseTask->doRun() /usr/share/nginx/atom/vendor/symfony/lib/task/sfTask.class.php:97
PHP   6. QubitUpgradeSqlTask->execute() /usr/share/nginx/atom/vendor/symfony/lib/task/sfBaseTask.class.php:68
PHP   7. arMigration0163->up() /usr/share/nginx/atom/lib/task/migrate/arUpgradeSqlTask.class.php:214
PHP   8. i18nTransformBaseTask->execute() /usr/share/nginx/atom/lib/task/migrate/migrations/arMigration0163.class.php:35
PHP   9. i18nCustomLinkToMarkdownTask->processRow() /usr/share/nginx/atom/lib/task/i18n/i18nTransformBaseTask.class.php:138
PHP  10. create_function() /usr/share/nginx/atom/lib/task/i18n/i18nCustomLinkToMarkdownTask.class.php:83
PHP  11. sfDatabaseManager->shutdown() /usr/share/nginx/atom/vendor/symfony/lib/database/sfDatabaseManager.class.php:0
[~/atom] (qa/2.5.x)

#4 Updated by Steve Breker about 1 month ago

  • Status changed from New to Code Review

This fix along with the change to remove create_function() will help reduce the memory usage of migration 163 so that large databases can be upgraded successfully.

https://github.com/artefactual/atom/pull/865

#5 Updated by Mike Cantelon about 1 month ago

  • Status changed from Code Review to Feedback
  • Assignee set to Steve Breker

Looks good to me!

#6 Updated by Steve Breker about 1 month ago

Merged qa/2.5.x. Ready for QA.

#7 Updated by Steve Breker about 1 month ago

  • Status changed from Feedback to QA/Review
  • Assignee deleted (Steve Breker)

#8 Updated by Dan Gillean about 1 month ago

  • Status changed from QA/Review to Verified

Also available in: Atom PDF