Generate archival description reports asynchronously
|Category:||Reports||Estimated time:||70.00 hours|
|Target version:||Release 2.4.0|
|Google Code Legacy ID:||Tested version:|
AtoM includes several basic report options available from the archival description view page: file report, item report, CSV box labels, and a storage locations report. These reports are currently being generated synchronously on-demand, via the web browser. Because of this, they can often time out during larger requests (e.g. when the descriptive hierarchy is larger).
This feature will move the generation of these reports to the job scheduler, so they can be performed asynchronously in the background, thereby avoiding browser timeouts. We will follow a similar approach to how the Finding aid generation is handled in the user interface, making a link available to users on the current page after the report is generated. This way, public users will still have access to the public reports (file and item reports) without having to gain access to the Jobs page.
For download, users will have the option of generating the report in CSV or HTML. for the HTML version, users will still download the file, and view it locally with a browser.
#3 Updated by Mike Gale over 3 years ago
NOTE: there's a new setting on the admin settings' global form, whether to generate reports as public user or not. This determines if "Retrieval information" will show up in item/file list reports. Box label and physical storage location reports should be hidden from non-authenticated users in the new 'Existing reports' section of the reports page for archival descriptions regardless of settings (same as old behavior).
#12 Updated by Dan Gillean about 3 years ago
- Requires documentation changed from No to Yes
These are the pages that need to be updated for docs on this feature:
There are updates to the settings page as well - see Mike's comment 8 above.Some of the changes that need to be captured include:
- Reports can be generated in either CSV or HTML to be downloaded now (previously it was HTML served in the browser only)
- Once a report is generated, it remains available - public users access pre-generated file/item reports, but only authenticated users can generate them (if I recall correctly)
- The intro on the page should mention that this is being handled by the job scheduler, meaning users must ensure that the atom-worker is running for report generation to work. See for example: https://github.com/artefactual/atom-docs/blob/2.4/user-manual/import-export/import-xml.rst
#14 Updated by José Raddaoui Marín about 3 years ago
- Status changed from In progress to Code Review
- Assignee changed from José Raddaoui Marín to Nick Wilkinson
Kelly Stewart wrote:
As anonymous user, generated an item-level report of the Ada Lovelace fonds, which has 1 item in quotation marks. Quotation marks in the item list report become â€œ
It should be fixed in https://github.com/artefactual/atom/pull/582
#18 Updated by Dan Gillean almost 3 years ago
- Status changed from QA/Review to Feedback
- Assignee set to José Raddaoui Marín
Issue 1: if dates of creation are removed or changed to dates of accumulation, they are not removed from the resulting report, even if you regenerate the reports
I'm working in the most recent qa/2.4.x branch in a local Vagrant box, with the AtoM demo data set loaded - using the "Irving Steinberg Sudbury Slide Collection" for testing. To reproduce:
- Navigate to an item record and enter edit mode
- Change the date type to accumulation and save
- In a different item-level record, delete the current date of creation and add a date of accumulation instead, and save
- Generate the reports in both HTML and CSV formats
- Both report formats still display the outdated dates of creation information
- I tried clearing the cache, restarting memcached and php-fpm, and repeating the steps above, but got the same report
- Either the original reports are not being replaced by updated ones, or the report generation is somehow using out-of-date information.
As far as I know, the item and file reports are only configured to display dates of creation - the display date and the startDate are included. Accumulation dates normally don't show up. I would therefore expect that:
- Where there is only a date of accumulation, then no date is included in the updated report generated
- Where there is both a date of accumulation and a date of creation on the item, then only the date of creation will be displayed
The file and item list reports should use the same event columns used by the CSV import/export - so that multiple dates, and different event types, can be included in the output. Columns should include: eventTypes, eventDates, eventStartDates, and eventEndDates.
Issue 2: Digital object thumbnails are broken in the HTML reports generated. Testing was done on Chrome 59.0.3071.115.
#21 Updated by Mike Gale almost 3 years ago
Issue 1 in Dan's comment #18 was actually due to something unexpected. It seems the code, if it can't find a creation date for an item level description, will go up the hierarchy until it finds a parent with a creation date, and fill that in as the item's creation date as well. In Dan's case the parents' creation date was the same as the items' previous creation dates, hence why he thought there was outdated info in there.
I discussed with Dan and we're going to remove this functionality and have the items display blank for creation dates if they don't have one.
#26 Updated by Dan Gillean almost 3 years ago
- Status changed from Feedback to Verified
- Assignee deleted (
Ok, that was my bad.
For the thumbs, the code for the reports builds the URLs to the thumbs by using the Base URL available in Admin > Settings > Site information.
In my case, I'm using a copy of the demo data as my test corpus, so it was set to demo.accesstomemory.org - though my local vagrant box is actually at 10.10.10.10. When I updated the base URL setting, the problem was resolved - thumbnails now appear in the HTML as expected.
Thanks Mike, and sorry for the red herring!
#27 Updated by Dan Gillean almost 3 years ago
- Requires documentation deleted (
Documentation added to 2.4 in: https://github.com/artefactual/atom-docs/commit/9f0afaa2c8835c75d455e6770503408e92985dc3