Feature #13148

Add support for command-line bulk authority record CSV export task (including relations and aliases)

Added by Dan Gillean 10 months ago. Updated about 1 month ago.

Status:VerifiedStart date:08/13/2019
Priority:MediumDue date:
Assignee:-% Done:

100%

Category:CSV exportEstimated time:28.00 hours
Target version:Release 2.6.0
Google Code Legacy ID: Tested version:2.6
Sponsored:Yes Requires documentation:

Description

AtoM supports bulk CSV export of authority records via the user interface, using the clipboard. However, no equivalent command-line task currently exists. This enhancement will add a new command-line CSV export task for authority records. When alias and/or relations data is present in the exported authority records, this data will be included in the export as well in the additional CSV templates.

aliases.csv Magnifier (4.92 KB) Dan Gillean, 11/04/2019 05:21 PM

authorities.csv Magnifier (211 KB) Dan Gillean, 11/04/2019 05:21 PM

relations.csv Magnifier (9.34 KB) Dan Gillean, 11/04/2019 05:21 PM

History

#2 Updated by Mike Cantelon 9 months ago

  • Status changed from New to In progress
  • Assignee set to Mike Cantelon

#3 Updated by Mike Cantelon 9 months ago

  • Status changed from In progress to Code Review
  • Assignee deleted (Mike Cantelon)

#4 Updated by Steve Breker 9 months ago

  • Status changed from Code Review to Feedback
  • Assignee set to Mike Cantelon

CR complete - looks great!

#5 Updated by Mike Cantelon 9 months ago

  • Status changed from Feedback to QA/Review
  • Assignee deleted (Mike Cantelon)

Thanks Steve!

Merged into qa/2.6.x.

#6 Updated by Dan Gillean 7 months ago

As far as I can tell, the export is working. However, I really wanted to test roundtripping them. I noticed for example that relations appear in the CSV twice - once for each direction - and wanted to confirm this wouldn't lead to duplicate entries on re-import. Unfortunately, my import consistently fails.

I'm attaching the files I was using for testing. The command I have run to import them was:

php symfony csv:authority-import --alias-file=/vagrant/aliases.csv --relation-file=/vagrant/relations.csv /vagrant/authorities.csv

The import error I get is:

Could not find "Family" in type of entity terms array.

I've confirmed that the term exists in AtoM in both the actor entity type, and the actor relation type taxonomies. I even created new (duplicate) terms just in case, but the error persists - not sure why.

It's separate from this issue, but if an entity type is missing, AtoM should ideally make a note in the console log and then create the term and continue with the import.

In the meantime, can anyone see a reason why I can't roundtrip these records? They were exported using the new task described here.

#7 Updated by Dan Gillean 7 months ago

Ok, ignore the above - there were apparently Finnish rows in the master authority record CSV, and that was causing the import error. If I remove them, everything imports as expected.

However, my theory appears to be correct - right now, on import each relationship is being created twice, because in the export CSV, the relation file lists each relation two times - one for each entity.

#8 Updated by Mike Cantelon 5 months ago

  • Status changed from Feedback to Code Review
  • Assignee deleted (Mike Cantelon)

#9 Updated by Mike Cantelon 5 months ago

  • Status changed from Code Review to QA/Review

Fix to duplication issue merged into qa/2.6.x.

#10 Updated by Dan Gillean 4 months ago

  • Status changed from QA/Review to Verified

Looks good! Roundtripping worked without duplication. Thanks Mike!

For reference, the bulk export command:

php symfony csv:authority-export /path/to/export/directory

#11 Updated by Dan Gillean about 1 month ago

  • % Done changed from 0 to 100
  • Requires documentation deleted (Yes)
  • Tested version 2.6 added

Also available in: Atom PDF