Bug #6034

Rebuild Elasticsearch needs to be updated for AIP creation and storage changes

Added by Holly Becker over 8 years ago. Updated almost 6 years ago.

Status:In progressStart date:11/28/2013
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:Index/Search
Target version:-
Google Code Legacy ID: Pull Request:
Sponsored:No Requires documentation:

Description

Rebuilding the Elasticsearch index needs to be updated for recent changes around the creation and storing of AIPs

  • AIPs are stored through the storage service, and not all put in /var/archivematica/sharedDirectory/www/AIPsStore/ on the local filesystem
  • AIPs can be a .7z or a .tar.bz2 - and potentially other packages, or even uncompressed, in the future

Additionally, the number of AIPs stored in an AIC (#5803) needs to be indexed. This is stored in the Dublin Core tag 'extent' in the AIC METS file, in the metadata/ directory.

Options for implementing this include:
  • A directory is provided as a parameter, and all files in there are considered to be AIPs that need to be re-indexed (Possibly filtering on file extensions)
    Advantage: flexible, doesn't need to be updated for storage service changes
    Disadvantage: if AIPs are not already stored in one location (eg some are in LOCKSS), could be hard to get them all in one place
  • Query the storage service for all AIPs it knows of, and index those
    Advantage: Uses existing system that is supposed to track this, handles files scattered across different storage locations
    Disadvantages: Relies on storage service knowing all the AIPs (would not handle storage service recreation), needs to be updated for SS API changes

Related issues

Related to Archivematica Storage Service - Feature #5873: add method to allow aip reindexing Duplicate 10/25/2013
Related to Archivematica Storage Service - Task #7250: Script to recreate SS DB of AIPS New

History

#1 Updated by Holly Becker over 7 years ago

  • Related to Task #7250: Script to recreate SS DB of AIPS added

#2 Updated by Courtney Mumma over 7 years ago

  • Status changed from New to Invalid

ElasticSearch handled by other tickets in other features

#3 Updated by Courtney Mumma over 7 years ago

  • Status changed from Invalid to In progress
  • Target version changed from post 1.0 to Release 1.4.0

#4 Updated by Sarah Romkey almost 6 years ago

  • Target version deleted (Release 1.4.0)

Also available in: Atom PDF