Bug #13236

Investigate memory usage in PHP during digital object upload

Added by Dan Gillean over 2 years ago. Updated about 2 years ago.

Status:VerifiedStart date:01/09/2020
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:Digital object
Target version:Release 2.5.4
Google Code Legacy ID: Tested version:2.5, 2.6
Sponsored:No Requires documentation:

Description

It seems that when a digital object is uploaded (regardless of method, UI vs CLI), AtoM is loading the entire bitstream into memory. This can exhaust available memory with large upload attempts (especially when also factoring swap space memory), as seen in the following recent user forum post:

This issue is to track this behavior, and so we can investigate the rationale, and possible alternatives or improvements so that uploading digital objects is not so resource-intensive.

Related issues

Related to Access to Memory (AtoM) - Bug #13245: Checksum validation fails when importing large digital ob... Verified 01/17/2020
Related to Access to Memory (AtoM) - Task #13283: Avoid loading the file in memory to copy digital objects New 03/24/2020
Related to AtoM Wishlist - Feature #10195: Improve efficiency of QubitDigitalObject processing of di... Duplicate 08/10/2016

History

#1 Updated by José Raddaoui Marín over 2 years ago

  • Related to Bug #13245: Checksum validation fails when importing large digital objects added

#3 Updated by José Raddaoui Marín about 2 years ago

  • Related to Task #13283: Avoid loading the file in memory to copy digital objects added

#4 Updated by José Raddaoui Marín about 2 years ago

  • Category set to Digital object
  • Status changed from New to Code Review

The PR above fixes the issue in the digital objects load task. We'll use #13283 to fix it in other places involving temporary files.

#5 Updated by Mike Cantelon about 2 years ago

  • Status changed from Code Review to Feedback
  • Assignee set to José Raddaoui Marín

Looks good to me... nice fix!

#6 Updated by José Raddaoui Marín about 2 years ago

  • Status changed from Feedback to QA/Review
  • Assignee deleted (José Raddaoui Marín)

Thanks Mike, merged in qa/2.6.xand ready for review. Remember, it only affects the digital objects load task.

#7 Updated by Dan Gillean about 2 years ago

  • Target version set to Release 2.5.4

#8 Updated by Dan Gillean about 2 years ago

  • Status changed from QA/Review to Verified

#9 Updated by Dan Gillean almost 2 years ago

  • Related to Feature #10195: Improve efficiency of QubitDigitalObject processing of digital assets added

Also available in: Atom PDF