Feature #13148
Add support for command-line bulk authority record CSV export task (including relations and aliases)
Status: | Verified | Start date: | 08/13/2019 | |
---|---|---|---|---|
Priority: | Medium | Due date: | ||
Assignee: | - | % Done: | 100% | |
Category: | CSV export | Estimated time: | 28.00 hours | |
Target version: | Release 2.6.0 | |||
Google Code Legacy ID: | Tested version: | 2.6 | ||
Sponsored: | Yes | Requires documentation: |
Description
AtoM supports bulk CSV export of authority records via the user interface, using the clipboard. However, no equivalent command-line task currently exists. This enhancement will add a new command-line CSV export task for authority records. When alias and/or relations data is present in the exported authority records, this data will be included in the export as well in the additional CSV templates.
History
#2 Updated by Mike Cantelon almost 3 years ago
- Status changed from New to In progress
- Assignee set to Mike Cantelon
#3 Updated by Mike Cantelon almost 3 years ago
- Status changed from In progress to Code Review
- Assignee deleted (
Mike Cantelon)
PR for CR: https://github.com/artefactual/atom/pull/961
#4 Updated by Steve Breker over 2 years ago
- Status changed from Code Review to Feedback
- Assignee set to Mike Cantelon
CR complete - looks great!
#5 Updated by Mike Cantelon over 2 years ago
- Status changed from Feedback to QA/Review
- Assignee deleted (
Mike Cantelon)
Thanks Steve!
Merged into qa/2.6.x.
#6 Updated by Dan Gillean over 2 years ago
- File aliases.csv
added
- File authorities.csv
added
- File relations.csv
added
- Status changed from QA/Review to Feedback
- Assignee set to Mike Cantelon
As far as I can tell, the export is working. However, I really wanted to test roundtripping them. I noticed for example that relations appear in the CSV twice - once for each direction - and wanted to confirm this wouldn't lead to duplicate entries on re-import. Unfortunately, my import consistently fails.
I'm attaching the files I was using for testing. The command I have run to import them was:
php symfony csv:authority-import --alias-file=/vagrant/aliases.csv --relation-file=/vagrant/relations.csv /vagrant/authorities.csv
The import error I get is:
Could not find "Family" in type of entity terms array.
I've confirmed that the term exists in AtoM in both the actor entity type, and the actor relation type taxonomies. I even created new (duplicate) terms just in case, but the error persists - not sure why.
It's separate from this issue, but if an entity type is missing, AtoM should ideally make a note in the console log and then create the term and continue with the import.
In the meantime, can anyone see a reason why I can't roundtrip these records? They were exported using the new task described here.
#7 Updated by Dan Gillean over 2 years ago
Ok, ignore the above - there were apparently Finnish rows in the master authority record CSV, and that was causing the import error. If I remove them, everything imports as expected.
However, my theory appears to be correct - right now, on import each relationship is being created twice, because in the export CSV, the relation file lists each relation two times - one for each entity.
#8 Updated by Mike Cantelon over 2 years ago
- Status changed from Feedback to Code Review
- Assignee deleted (
Mike Cantelon)
#9 Updated by Mike Cantelon over 2 years ago
- Status changed from Code Review to QA/Review
Fix to duplication issue merged into qa/2.6.x.
#10 Updated by Dan Gillean over 2 years ago
- Status changed from QA/Review to Verified
Looks good! Roundtripping worked without duplication. Thanks Mike!
For reference, the bulk export command:
php symfony csv:authority-export /path/to/export/directory
#11 Updated by Dan Gillean about 2 years ago
- % Done changed from 0 to 100
- Requires documentation deleted (
Yes) - Tested version 2.6 added
Docs added to 2.6 branch in https://github.com/artefactual/atom-docs/commit/a2184c6f84d9a442d949da12ef5ade84688d423a