Bug #9648

Gearman job names are not compatible with the memcached backend

Added by Jesús García Crespo about 6 years ago. Updated about 6 years ago.

Status:VerifiedStart date:04/01/2016
Priority:MediumDue date:
Assignee:Sara Allain% Done:

0%

Category:Internals
Target version:Release 2.3.0
Google Code Legacy ID: Tested version:2.2, 2.3
Sponsored:No Requires documentation:No

Description

I am not sure who is to blame but submitting jobs with names that contain characters like spaces or tabs may break the ability of Gearman to store these jobs in a memcached backend. I suspect the problem is in gearmand using the text protocol instead of the binary protocol when communicating with memcached, but I'm not entirely sure.

I'm planning to update QubitJob::getJobPrefix() to stop using filesystem paths, which falls into the category of problematic strings as the memcached keys are named after it. Instead, I'm using the first 16 characters of the md5 checksum of a string containing the site title and site url. This will also make possible to reliably implement job queues in distributed scenarios where AtoM is not always installed in the same location and avoid potential clashes with other instances of AtoM running simultaneously and sharing the same memcached back-end for job queues.

Example:

set gear_fdd4764376d2f763-arGenerateFindingAidJob-43c77cfa32a2364004aa0685f0c91702 1 0 111


Related issues

Related to Access to Memory (AtoM) - Task #7423: Make job scheduling workers AtoM instance specific Verified 10/21/2014
Related to Access to Memory (AtoM) - Task #9858: Improve job worker prefix uniqueness Verified 05/17/2016

History

#2 Updated by Jesús García Crespo about 6 years ago

  • Status changed from In progress to Code Review
  • Assignee deleted (Jesús García Crespo)

#3 Updated by Jesús García Crespo about 6 years ago

  • Related to Task #7423: Make job scheduling workers AtoM instance specific added

#4 Updated by José Raddaoui Marín about 6 years ago

  • Status changed from Code Review to Feedback
  • Assignee set to Jesús García Crespo

Looks good!

#5 Updated by Jesús García Crespo about 6 years ago

  • Status changed from Feedback to QA/Review
  • Assignee changed from Jesús García Crespo to Dan Gillean

This can be verified testing AtoM deployed via Docker Compose (presenting that on April 6th, see #9406).

#6 Updated by Dan Gillean about 6 years ago

  • Assignee changed from Dan Gillean to Sara Allain

#7 Updated by Sara Allain about 6 years ago

  • Status changed from QA/Review to Verified

Looks good to me!

#8 Updated by Mike Gale about 6 years ago

  • Related to Task #9858: Improve job worker prefix uniqueness added

Also available in: Atom PDF