Bug #4182

AtoM gearman worker fails after running for a long time

Added by Jesús García Crespo over 7 years ago. Updated over 6 years ago.

Status:VerifiedStart date:
Priority:HighDue date:
Assignee:Jesús García Crespo% Done:

100%

Category:Job scheduling
Target version:Release 1.3.1
Google Code Legacy ID:atom-2234 Tested version:
Sponsored:No Requires documentation:

Description

To reproduce this error:
1) Start qubit-sword upstart service
2) Leave it running it and not using it for one day
3) Try to deposit a package using SWORD

Resulting error:

MySQL server has gone away.

The database connection should be kept alive or ensure that it is connected?

[31316] Running new fork...
[31316] A package was deposited by reference.
[31316] Location: file:///27 Budgets-55136a22-9ac9-4778-828b-907bb4b4fc22
[31316] Processing...
[31316] Object slug: series-3
[31316] Exception: exception 'PropelException' with message 'Unable to execute UPDATE statement. [wrapped: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away]' in /var/www/qubit/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/propel/util/BasePeer.php:420
Stack trace:
#0 /var/www/qubit/lib/model/om/BaseObject.php(686): BasePeer::doUpdate(Object(Criteria), Object(Criteria), Object(PropelPDO))
#1 /var/www/qubit/lib/model/om/BaseInformationObject.php(449): BaseObject->update(NULL)
#2 /var/www/qubit/lib/model/om/BaseObject.php(530): BaseInformationObject->update(NULL)
#3 /var/www/qubit/lib/model/QubitObject.php(101): BaseObject->save(NULL)
#4 /var/www/qubit/lib/model/om/BaseInformationObject.php(360): QubitObject->save(NULL)
#5 /var/www/qubit/lib/model/QubitInformationObject.php(196): BaseInformationObject->save(NULL)
#6 /var/www/qubit/plugins/qtSwordPlugin/lib/qtPackageExtractorMETSArchivematicaDIP.class.php(200): QubitInformationObject->save()
#7 /var/www/qubit/plugins/qtSwordPlugin/lib/qtPackageExtractorBase.class.php(72): qtPackageExtractorMETSArchivematicaDIP->process()
#8 /var/www/qubit/plugins/qtSwordPlugin/lib/qtSwordPluginWorker.class.php(95): qtPackageExtractorBase->run()
#9 [internal function]: qtSwordPluginWorker::depositSwordPackage(Object(GearmanJob), Object(sfGearmanWorker))
#10 /var/www/qubit/plugins/sfGearmanPlugin/lib/sfGearmanWorker.class.php(148): GearmanWorker->work()
#11 /var/www/qubit/plugins/sfGearmanPlugin/lib/task/gearmanWorkerTask.class.php(65): sfGearmanWorker->loop()
#12 /var/www/qubit/vendor/symfony/lib/task/sfBaseTask.class.php(68): gearmanWorkerTask->execute(Array, Array)
#13 /var/www/qubit/vendor/symfony/lib/task/sfTask.class.php(97): sfBaseTask->doRun(Object(sfCommandManager), NULL)
#14 /var/www/qubit/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php(76): sfTask->runFromCLI(Object(sfCommandManager), NULL)
#15 /var/www/qubit/vendor/symfony/lib/command/cli.php(20): sfSymfonyCommandApplication->run()
#16 /var/www/qubit/symfony(14): include('/var/www/qubit/...')
#17 {main}
[31316] Fork finished
Job finished

History

#1 Updated by Jesús García Crespo over 7 years ago

Temporary fix, run:
$ sudo restart qubit-sword

#2 Updated by Jesús García Crespo over 7 years ago

r11071 may fix it

#3 Updated by Anonymous over 7 years ago

  • Priority changed from Critical to High
  • Target version changed from Release 1.2.1 to Release 1.3

[g] Labels added: Milestone-Release-1.3, Priority-High
[g] Labels removed: Milestone-Release-1.2.1, Priority-Critical

#4 Updated by Jesús García Crespo about 7 years ago

  • Status changed from New to Verified

#5 Updated by Jesús García Crespo over 6 years ago

  • Subject changed from qubit-sword daemon stops working after a long time being running (MySQL server has gone away) to AtoM gearman worker fails after running for a long time
  • Category set to Job scheduling
  • Status changed from Verified to In progress
  • Target version changed from Release 1.3 to Release 1.4.0
  • Sponsored set to No

Unfortunately, this is still a problem. AtoM's default behaviour is to create PDO persistent connections and after a while they get closed automatically by the MySQL server, producing the "MySQL has gone away error". Also, I found that the ZSL engine wasn't getting closed keeping the index directory locked; that was producing some other problems in the integrity of the index.

#6 Updated by Jesús García Crespo over 6 years ago

  • Description updated (diff)

#7 Updated by Jesús García Crespo over 6 years ago

  • Status changed from In progress to QA/Review
  • % Done changed from 0 to 100

Applied in changeset atom|commit:eea2bd3ed422414c80ff93efc2f5b4df46a0e276.

#8 Updated by Jesús García Crespo over 6 years ago

Applied in changeset atom|commit:85dc4d6d0600552c146bb967855eaaf3bafb5228.

#9 Updated by Jesús García Crespo over 6 years ago

  • Status changed from QA/Review to Verified

#10 Updated by David Juhasz over 6 years ago

  • Target version changed from Release 1.4.0 to Release 1.3.1

Also available in: Atom PDF