Bug #6034
Rebuild Elasticsearch needs to be updated for AIP creation and storage changes
Status: | In progress | Start date: | 11/28/2013 | |
---|---|---|---|---|
Priority: | Medium | Due date: | ||
Assignee: | - | % Done: | 0% | |
Category: | Index/Search | |||
Target version: | - | |||
Google Code Legacy ID: | Pull Request: | |||
Sponsored: | No | Requires documentation: |
Description
Rebuilding the Elasticsearch index needs to be updated for recent changes around the creation and storing of AIPs
- AIPs are stored through the storage service, and not all put in /var/archivematica/sharedDirectory/www/AIPsStore/ on the local filesystem
- AIPs can be a .7z or a .tar.bz2 - and potentially other packages, or even uncompressed, in the future
Additionally, the number of AIPs stored in an AIC (#5803) needs to be indexed. This is stored in the Dublin Core tag 'extent' in the AIC METS file, in the metadata/ directory.
Options for implementing this include:- A directory is provided as a parameter, and all files in there are considered to be AIPs that need to be re-indexed (Possibly filtering on file extensions)
Advantage: flexible, doesn't need to be updated for storage service changes
Disadvantage: if AIPs are not already stored in one location (eg some are in LOCKSS), could be hard to get them all in one place
- Query the storage service for all AIPs it knows of, and index those
Advantage: Uses existing system that is supposed to track this, handles files scattered across different storage locations
Disadvantages: Relies on storage service knowing all the AIPs (would not handle storage service recreation), needs to be updated for SS API changes
Related issues
History
#1 Updated by Holly Becker over 7 years ago
- Related to Task #7250: Script to recreate SS DB of AIPS added
#2 Updated by Courtney Mumma over 7 years ago
- Status changed from New to Invalid
ElasticSearch handled by other tickets in other features
#3 Updated by Courtney Mumma over 7 years ago
- Status changed from Invalid to In progress
- Target version changed from post 1.0 to Release 1.4.0
#4 Updated by Sarah Romkey almost 6 years ago
- Target version deleted (
Release 1.4.0)