Feature #12840

Add ability for digital object load task to discard master DO after derivative creation

Added by Dan Gillean over 3 years ago. Updated about 3 years ago.

Status:VerifiedStart date:01/17/2019
Priority:MediumDue date:
Assignee:Michelle Curran% Done:


Category:CLI toolsEstimated time:20.00 hours
Target version:Release 2.5.0
Google Code Legacy ID: Tested version:
Sponsored:Yes Requires documentation:


Background and summary

The digital object load task is described in our documentation here:

The task expects a CSV as input, with 2 columns: a filepath, and either an information_object_id or an identifier.

Though undocumented, previous testing has shown that adding an HTTP web link in the filepath column will make the DO load task behave similarly to using a URL to link an external object when performed via the user interface. When external HTTP web URIs are passed to AtoM as digital objects to load, AtoM generally will:

  • Follow the URI to fetch the object
  • Make a copy in a tmp directory
  • Process the temp copy to generate the derivatives
  • Discard the temp copy

Currently there is no way to replicate this behavior with local file paths, where the master digital objects are not kept. Users who store master objects in a separate local repository might not want to maintain 2 copies of every digital object. This task will allow such users to load digital objects from a local file path, but have the master DO not be stored at the end of the process. Instead, we will store the filepath, just as we store the URI for externally linked objects.

Feature request

  • Add a new option to the DO load task (e.g. --link-source or something) that will modify the behavior of the DO load task to behave like an external DO being uploaded via URI (i.e. store the path to source file in the database, and don't copy the source "master" file to uploads directory)
  • When used, local derivatives should still be generated and saved in the uploads directory as usual.


#2 Updated by Mike Cantelon about 3 years ago

  • Status changed from New to QA/Review
  • Assignee changed from Mike Cantelon to Dan Gillean

#3 Updated by Michelle Curran about 3 years ago

  • Assignee changed from Dan Gillean to Michelle Curran

#4 Updated by Michelle Curran about 3 years ago

Verified in 16.04 env but waiting for a fix and redeployment of atom25-18.04 vagrant update.

#5 Updated by Michelle Curran about 3 years ago

  • Status changed from QA/Review to Verified

#6 Updated by Dan Gillean about 3 years ago

  • Requires documentation deleted (Yes)

Also available in: Atom PDF