DC metadata not imported into AtoM when DIP is created from a transfer placed in backlog
|Assignee:||Sarah Romkey||% Done:|
|Target version:||Release 2.5.2|
|Google Code Legacy ID:||Tested version:||2.4|
Issue reported by Evelyn - tested in: AM 1.9.1, AtoM 2.4.1
- In Archivematica, create a transfer and place it in transfer backlog
- In the Appraisal Tab, retrieve the transfer and create a SIP from it
- Enter DC metadata via the GUI
- Create a DIP and upload it to AtoM
No Information Object is created and none of the DC metadata appear in AtoM.
Note that the error may be caused by the presence of a second structMap in the METS file when a transfer is placed in backlog. This is a logical structMap with LABEL="Hierarchical". For transfers run straight through Archivematica without being placed in backlog, there is only the default physical structMap. It's possible that AtoM doesn't know what to do with the fact that the dmdSec is referenced from two structMaps.
AtoM should create an information object out of the Dublin Core metadata entered via the GUI in Archivmatica.
See the related AM issue and conversation for more details: https://github.com/archivematica/Issues/issues/570
One of the AM team said this might be related: https://github.com/archivematica/Issues/issues/586
Radda noted the following:
Probably the PREMIS 3 upgrade requires changes in AtoM's METS parsing process ...
#1 Updated by Evelyn McLellan 4 months ago
I don't think https://github.com/archivematica/Issues/issues/586 is related to this. That issue has to do with the transfer METS file, which doesn't affect DIP upload to AtoM.
#3 Updated by José Raddaoui Marín 4 months ago
My note about the PREMIS 3 upgrade was thinking that this issue happened using the "qa" branches of AM (to be 1.10), not using the 1.9.1 version, so that's probably not the issue in this case.
Nevertheless, we should take in consideration the PREMIS upgrade too, to avoid issues in the future, as we're using the version 2 namespace in several places:
https://github.com/artefactual/atom/blob/qa/2.6.x/lib/QubitMetsParser.class.php (look for: info:lc/xmlns/premis-v2)
#5 Updated by José Raddaoui Marín about 1 month ago
Hi Dan and Evelyn,
Fixing the PREMIS issue solved most of the problems with the hierarchical DIP upload on the AtoM side. However, AtoM only creates an intermediate archival description from the Dublin Core metadata entered via the GUI in Archivematica on the non hierarchical DIP upload.
When the import is performed from the hierarchical structMap, the "objects" DMD section is not considered and AtoM only uses the
TYPE attribute from its structMap div to update the level of description of the target description. This seems to have been always like this in the code but I could not find an explanation in the documentation. We could easily change that behavior if you want but I'm not sure if that's something we want to do.
Please, let me know what do you think about it.
#7 Updated by Sarah Romkey about 1 month ago
I'm not totally sure I follow... is the remaining problem that you can't combine the hierarchical structMap workflow with having DC metadata entered via the Archivematica GUI? If that is the case, I would consider it an edge case and not worry about a fix at this moment.
#8 Updated by Sarah Romkey about 1 month ago
Edit- I think I understand this now.
My understanding is that the "top" description created in the hierarchy gets assigned to the objects directory in the structMap. Ideally, the DC metadata would also get assigned to the objects folder and displayed at that level in AtoM. However, if that's not currently supported functionality, it's not a crucial fix at this moment IMO.
Radda, let me know if that helps!!
#9 Updated by Dan Gillean about 1 month ago
- Status changed from In progress to Feedback
Hi Radda and Evelyn,
After some discussion with the AM team, mostly Sarah Romkey, we've decided to leave this as is for now. Sarah thinks that creating a new intermediary description to capture the aggregate metadata may be confusing/frustrating to users. However, we can't overwrite the existing metadata that may be on the target description either... so for now, Sarah thinks we should just ignore the aggregate DC metadata from Archivemtatica's GUI DC template, and they will add a note about this in their documentation. In the future we can think of a better solution, but right now it feels like there are a number of edge cases that might require more consideration.
So: that part of the ticket can be marked as invalid for now! Not sure if there are other parts that mean we don't want to mark this whole ticket as such or not.
#10 Updated by José Raddaoui Marín about 1 month ago
Thanks Sarah and Dan,
I'd leave this open to verify that the hierarchical DIP upload works, as it was not creating any archival description before the fix. It should create descriptions now, except for the intermediate level with the metadata from the GUI. I could not test the directories creation due to an AM issue that seems to be fixed now, so I'll give it a final try with that fix before marking this for code review.
#11 Updated by José Raddaoui Marín about 1 month ago
- Status changed from Feedback to Code Review
Hi Sarah and Evelyn,
I've tested the AtoM fixes with Douglas' changes on the AM side and everything seems to be working as expected. However, I went a bit further and tested a partial re-ingest from an arranged SIP and what I noticed is that the logical structMap is present on the first METS but missing on the re-ingested one. Is that expected or should the logical structMap be maintained after the re-ingest?
Besides that (which would be an issue on the AM side), this is ready for code review:
#15 Updated by Sara Allain 30 days ago
- Assignee changed from Dan Gillean to Sarah Romkey
The items appear in AtoM with object-level metadata. Aggregate metadata (added via the GUI in Archivematica) is not uploaded to AtoM.
However, while testing this I noticed that on upload the DIP loses its hierarchical-ness. I created a two-level arrangement:
arranged/ └── objects ├── Landing_zone.jpg ├── level-2/ │ ├── bird.mp3 │ └── MARBLES.TGA └── View_from_lookout_over_Queenstown_towards_the_Remarkables_in_spring.jpg
The upload flattened the structure (see https://atom25.1804.qa.accesstomemory.net/weve-got-metadata). I think this is a bug, but I'll pass it on to Sarah Romkey for her opinion.
The hierarchical logical structmap is present in the AIP METS.