Bug #11448

AtoM attempts to copy missing digital object and fails on csv:import

Added by Mike Gale almost 2 years ago. Updated 4 months ago.

Status:VerifiedStart date:08/22/2017
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:CSV import
Target version:Release 2.5.0
Google Code Legacy ID: Tested version:2.3, 2.4
Sponsored:No Requires documentation:

Description

To reproduce:
1. Attempt to run csv:import task on a CSV that contains a digitalObjectPath column
2. digitalObjectPath values refer to file paths that don't exist

Result: Various "File write to /home/user/workspace/atom/uploads/r/TEST/////TEST.jpg failed. See setting directory and file permissions documentation." errors indicating we're trying to copy an asset (and its blank checksum hence the ///// issue) that doesn't exist to our uploads folder.

Note that in the cases I tested despite the bug, it didn't actually affect the end results of the csv import (just didn't link the missing digital objects). It still shouldn't be attempting to copy non-existent files though; there could be another code path where this breaks things worse.

Expected result: A warning about the digital object path not existing, and to ignore the column for that row.

wpg.csv Magnifier (8.1 KB) Mike Gale, 08/25/2017 03:17 PM

fake-DO-path-test.csv Magnifier (4.01 KB) Dan Gillean, 01/25/2018 02:06 PM

History

#1 Updated by Mike Gale almost 2 years ago

  • File wpg.csv added

Attaching test CSV that has this issue.

Ran with:

php symfony csv:import --index wpg.csv

#2 Updated by Mike Gale almost 2 years ago

  • File deleted (wpg.csv)

#3 Updated by Mike Gale almost 2 years ago

#4 Updated by Mike Cantelon almost 2 years ago

  • Status changed from New to Code Review
  • Assignee changed from Mike Gale to Mike Cantelon

This looks like it was done as there's a pull request for it that isn't marked as "work in progress". I'll code review.

#5 Updated by Mike Cantelon almost 2 years ago

  • Status changed from Code Review to Feedback
  • Assignee changed from Mike Cantelon to Mike Gale

Looks good to me!

#6 Updated by Mike Gale almost 2 years ago

  • Status changed from Feedback to QA/Review
  • Assignee changed from Mike Gale to Dan Gillean

#7 Updated by Dan Gillean over 1 year ago

  • Target version set to Release 2.5.0

#8 Updated by Dan Gillean over 1 year ago

  • File fake-DO-path-test.csvMagnifier added
  • Status changed from QA/Review to Feedback
  • Assignee changed from Dan Gillean to Mike Gale

My import worked, but there was nothing in the job output giving a warning about the fake DO path I used. Attaching my test CSV. I imported via the user interface, not the CLI, but expected the job details page to include any warning that would have been shown in the console. Here's the output:

[info] [2018-01-25 14:01:06] Job 2003034 "arFileImportJob": Job started.
[info] [2018-01-25 14:01:06] Job 2003034 "arFileImportJob": Importing CSV file: fake-DO-path-test.csv.
[info] [2018-01-25 14:01:06] Job 2003034 "arFileImportJob": Indexing imported records.
[info] [2018-01-25 14:01:06] Job 2003034 "arFileImportJob": Update type: import-as-new
[info] [2018-01-25 14:01:09] Job 2003034 "arFileImportJob": php '/usr/share/nginx/atom/symfony' 'csv:import' --index     --quiet --source-name='fake-DO-path-test.csv' '/usr/share/nginx/atom/uploads/tmp/TMP41dacf2a'
[info] [2018-01-25 14:01:09] Job 2003034 "arFileImportJob": Import complete.
[info] [2018-01-25 14:01:09] Job 2003034 "arFileImportJob": Job finished.

Should I consider this the correct behavior, or should there in fact have been a warning?

#9 Updated by Dan Gillean 4 months ago

  • Status changed from Feedback to Verified
  • Assignee deleted (Mike Gale)

Never got an answer on this one, but there was a fix merged, so will mark it verified for now. if an issue arises again, we can open a new ticket, and link it to this one for reference.

Also available in: Atom PDF