Bug #13025

OAI requests do not convert local date/time information into UTC, leading to unexpected results

Added by Dan Gillean about 3 years ago. Updated 7 months ago.

Status:FeedbackStart date:05/14/2019
Priority:MediumDue date:
Assignee:-% Done:

100%

Category:OAI-PMH
Target version:Release 2.6.0
Google Code Legacy ID: Tested version:2.4, 2.5, 2.7
Sponsored:No Requires documentation:

Description

First reported by a community user via GitHub - see: https://github.com/artefactual/atom/issues/893

From the user:

I recently came to discover that the arOaiPlugin does not convert between UTC and local time. When one uses the from and until url parameters with oai, the result will be requested from the database exactly as it was sent to Atom, where it should be converted to local time first.

Example of current behaviour (omitting dates for clarity);

I modify an object at 20:00 AEST (+10).
I make an oai request with from=09:30&until=10:30
No results are returned
I make an oai request with from=19:30&until=20:30.
OAI returns the object with the a timestamp of 10:00

Since the OAI standard dictates that all date/times are to be in UTC, the above results are incorrect and should look more like below:

I modify an object at 20:00 AEST.
I make an oai request with from=09:30&until=10:30
OAI returns the object with a timestamp of 10:00.

A solution I have put in place was to modify the setUpdateParametersFromRequest within arOaiPluginComponent.class.php. I simply run strtotime on $request->from and $request->until and this converts the date/time to my local timezone, ready to be looked up in the database.


Related issues

Related to Access to Memory (AtoM) - Bug #13574: OAI-PMH resumptionToken forgets from & until arguments Verified 10/08/2021

History

#1 Updated by Dan Gillean about 3 years ago

  • Status changed from New to QA/Review
  • Target version set to Release 2.6.0

Fix for this issue submitted via community pull request from rferris: https://github.com/artefactual/atom/pull/896

Merged into qa/2.6.x and ready for QA testing.

#2 Updated by Dan Gillean almost 3 years ago

  • Status changed from QA/Review to Verified

Works!

#3 Updated by Dan Gillean almost 3 years ago

The ListIdentifiers verb in the documentation includes an example of using from and until time-based parameters in a request. We should add a note there, with an example, about using local time and having that convert to UTC.

See: https://www.accesstomemory.org/en/docs/2.5/user-manual/import-export/oai-pmh/#list-identifiers

#4 Updated by Dan Gillean about 2 years ago

  • % Done changed from 0 to 100
  • Requires documentation deleted (Yes)

#5 Updated by David Juhasz 8 months ago

This is not working as I would expect in qa/2.x (target release 2.7.0).

I created an new archival description and published it at 16:25:56 Vancouver time (UTC: 2021-09-21T23:25:56Z).

When I query the "ListIdentifiers" endpoint with the parameter "from=16:00" I see the new description:

<?xml version="1.0" encoding="UTF-8" ?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2021-09-21T23:52:53Z</responseDate>
  <request verb="ListIdentifiers" from="16:00" metadataPrefix="oai_dc">http://localhost:63001/;oai</request>
  <ListIdentifiers>
    <header>
      <identifier>oai:localhost:63001:djjuhca_28079</identifier>
      <datestamp>2021-09-21T23:25:56Z</datestamp>
      <setSpec>oai:localhost:63001:djjuhca_28079</setSpec>
    </header>
    </ListIdentifiers>
</OAI-PMH>

But if I search for "from=17:00", I get no results:

<?xml version="1.0" encoding="UTF-8" ?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2021-09-21T23:57:25Z</responseDate>
  <request verb="ListIdentifiers" from="17:00" metadataPrefix="oai_dc">http://localhost:63001/;oai</request>
  <error code="noRecordsMatch">The combination of the values of the from, until, set and metadataPrefix arguments results in an empty list.</error>
</OAI-PMH>

Because the description's UTC timestamp is 23:57 I would expect both queries to return the same result.

Notes:
  • I'm testing with an AtoM Docker development environment, and the Percona Server and AtoM container system timezones are both UTC.
  • My AtoM apps/qubit/config/settings.yml timezone setting is default_timezone: America/Vancouver
  • The MySQL timestamps are PDT (UTC -8)
    mysql> select * from object where id=2004225;
    +------------------------+---------------------+---------------------+---------+---------------+
    | class_name             | created_at          | updated_at          | id      | serial_number |
    +------------------------+---------------------+---------------------+---------+---------------+
    | QubitInformationObject | 2021-09-21 16:10:40 | 2021-09-21 16:25:56 | 2004225 |             0 |
    +------------------------+---------------------+---------------------+---------+---------------+
    1 row in set (0.00 sec)
    
  • My local system timezone is PDT (UTC -8)

#6 Updated by David Juhasz 8 months ago

  • Status changed from Verified to Feedback
  • Tested version 2.7 added

#7 Updated by David Juhasz 7 months ago

  • Related to Bug #13574: OAI-PMH resumptionToken forgets from & until arguments added

#8 Updated by David Juhasz 7 months ago

Important note: the OAI-PMH dates specification requires all dates with times to use the ISO8601 datetime format with the "Z" UTC timezone indicator (YYYY-MM-DDThh:mm:ssZ). This requirement includes datetimes used for the "from" and "until" list arguments, as well as datetimes included in OAI-PMH responses. Passing times such as "from=09:30&until=10:30" is not supported by the protocol.

Also available in: Atom PDF