Feature #239

Job scheduling support

Added by Jesús García Crespo over 10 years ago. Updated about 7 years ago.

Status:VerifiedStart date:12/05/2011
Priority:HighDue date:
Assignee:Dan Gillean% Done:

0%

Category:Performance / scalabilityEstimated time:100.00 hours
Target version:Release 2.2.0
Google Code Legacy ID: Tested version:
Sponsored:No Requires documentation:

Description

This feature will add a Jobs page to the Manage menu, and will introduce the ability for resource-intensive tasks to run asynchronously in the background, allowing users to continue using the site while they are performed, rather than relying on tasks executing through the browser - known to have timeout limits preventing large operations from completing.

A jobs page, where the status of current and completed jobs can be viewed and managed, will also be introduced (available via Manage > Jobs). A couple screenshots of this page have been attached to this issue. The Jobs page will also allow users to download a simple CSV report of completed jobs and their status.

For now in 2.2, the job scheduler is being introduced to support Finding aid generation - described in #7462, #7723, and #7615. In the future, other resource intensive tasks known to timeout have the potential to be reimplemented as jobs, including:

  • Bulk operations - move, delete, edit, search/replace, import, export, update publication status
  • Large tasks: large delete/move/edit/publication update tasks for nodes with many children
  • The generation of other kinds of reports
  • Maintenance activities (regular re-indexes on a schedule, or cache clears, etc)

jobs-cleared.png (15.3 KB) Dan Gillean, 12/17/2014 10:56 AM

jobs-csv-output.png (14.3 KB) Dan Gillean, 12/17/2014 10:56 AM

manage-jobs-tab.png (16.8 KB) Dan Gillean, 12/17/2014 10:56 AM

manage-jobs.png (66.3 KB) Dan Gillean, 12/17/2014 10:56 AM


Related issues

Related to Access to Memory (AtoM) - Feature #7462: Add ability to generate a full PDF finding aid from the E... Verified 11/03/2014
Blocks AtoM Wishlist - Feature #4231: Staged import New
Blocks AtoM Wishlist - Feature #3206: Changing maximum image width (pixels) should change refer... New
Blocks AtoM Wishlist - Feature #6087: Improve the upload calculation widget in Admin > Settings... New 12/05/2013

History

#1 Updated by David Juhasz over 9 years ago

  • Target version changed from Release 1.2.1 to Release 2.1.0

#2 Updated by Jesús García Crespo over 9 years ago

  • Subject changed from Qubit + Gearman to Job scheduling support
  • Priority changed from Medium to Critical
  • Target version deleted (Release 2.1.0)
  • Sponsored set to No

#3 Updated by Jesús García Crespo over 9 years ago

  • Priority changed from Critical to High

#4 Updated by Jesús García Crespo over 9 years ago

  • Tracker changed from Bug to Feature

#5 Updated by Jesús García Crespo over 9 years ago

  • Category set to Performance / scalability
  • Target version set to Release 2.1.0
  • Estimated time set to 100.00

#6 Updated by Dan Gillean almost 8 years ago

  • Target version changed from Release 2.1.0 to Release 2.2.0

#7 Updated by Dan Gillean over 7 years ago

  • Requires documentation set to Yes

This will implement a new "Jobs" page in AtoM, where admin users can see the status of current jobs. This will require new documentation.

#8 Updated by Mike Gale over 7 years ago

  • Status changed from New to QA/Review
  • Assignee changed from Jesús García Crespo to Dan Gillean

Documentation
New dependencies: Gearman job server and memcached

Dan, I suppose it'll be easiest to check if this is confirmed after all the finding aid gen / premis rights stuff has settled into 2.2 by next week

#9 Updated by Dan Gillean over 7 years ago

#10 Updated by Dan Gillean over 7 years ago

  • Related to Feature #7462: Add ability to generate a full PDF finding aid from the EAD output added

#11 Updated by Dan Gillean about 7 years ago

  • Status changed from QA/Review to Verified

#12 Updated by Dan Gillean about 7 years ago

  • Requires documentation deleted (Yes)

Also available in: Atom PDF