Bug #4654

Add date range search in the advance search

Added by Jesús García Crespo over 9 years ago. Updated over 8 years ago.

Status:VerifiedStart date:02/10/2013
Priority:MediumDue date:
Assignee:José Raddaoui Marín% Done:

100%

Category:Search / BrowseEstimated time:16.00 hours
Target version:Release 1.4.0
Google Code Legacy ID: Tested version:
Sponsored:Yes Requires documentation:

Related issues

Related to Access to Memory (AtoM) - Feature #4666: For advanced search by dates allow more flexible date for... Verified 02/13/2013
Related to Access to Memory (AtoM) - Bug #6218: startDate and endDate fields missing from the search inde... Verified 01/16/2014
Related to Access to Memory (AtoM) - Feature #8648: Advanced search date range selection Verified 06/25/2015

History

#1 Updated by Jesús García Crespo over 9 years ago

Test scenario with six different cases suggested by Glenn Dingwall.
A test case has been added. See commit:71579eb9fe743a95ac66a6cb6f402cbc6eb26733.

The date of creation for a particular descriptive record is bounded by the interval (R1,R2), where R1 is the earliest date and R2 is the latest date. [...] We expect to retrieve all records where there is an intersection between (R1,R2) and (T1,T2).

The date range of the descriptive record R=(R1,R2) may interact with T=(T1,T2) in six possible ways:
  1. R2<T1 i.e., the range R lies entirely before the interval T)
  2. R1≥T1 AND T2≤R2 i.e., R lies entirely within T
  3. R1>T2 i.e., R lies entirely after T
  4. R1<T1 AND T1<R2≤T2 i.e., R overlaps the start of T
  5. T1<R1≤T2 AND R2>T2 i.e., R overlaps the end of T
  6. R1<T1 AND R2>T2 i.e., R overlaps all of T

See graph:

          >--------------- T1 ---------------- T2 --------------->
                           '                   '
Query Interval             [-------- T --------]
                           '                   '
Case 1            [---R---)'                   '
                           '                   '
                           '                   '
                           '                   '
Case 2                     [----R----]         '
                           '   [-----R------]  '
                           '           [---R---]
                           '                   '
                           '                   '
                           '                   '
Case 3                     '                   '(----R----]
                           '                   '
                           '                   '
                           '                   '
Case 4            [-----R-----]                '
                           '                   '
                           '                   '
Case 5               [---------------R----------------]
                           '                   '
                           '                   '
Case 6                     '               [-----R------]

A date search on ICA-AtoM should resolve as follows for each case:

        Expected behaviour
Case    Query should return this case
  1     No
  2     Yes
  3     No
  4     Yes
  5     Yes
  6     Yes

#2 Updated by Jesús García Crespo over 9 years ago

  • Status changed from New to QA/Review
  • % Done changed from 0 to 100

Applied in changeset commit:db1c6f0707c6f918a5e11fdd59303760cbee1585.

#3 Updated by Jesús García Crespo over 9 years ago

I've updated the code and tests to support searches by only one date field (start or end).
See commit:ded5c9d2a68eb39a8d64bc08ef8476b71a6ba29b.

          >--------------- T1 ---------------- T2 --------------->
                           '                   '
Query Interval             [-------- T --------]
                           '                   '
Case 1            [---R---)'                   '
                           '                   '
                           '                   '
                           '                   '
Case 2                     [----R----]         '
                           '   [-----R------]  '
                           '           [---R---]
                           '                   '
                           '                   '
                           '                   '
Case 3                     '                   '(----R----]
                           '                   '
                           '                   '
                           '                   '
Case 4            [-----R-----]                '
                           '                   '
                           '                   '
Case 5               [---------------R----------------]
                           '                   '
                           '                   '
Case 6                     '               [-----R------]

          >--------------- T1 ----------------------------------->
                           |
Query Interval             *
                           |
Case 7           [---R---] |
                           |
Case 8               [---R---]
                           |
Case 9                     | [---R---]

          >----------------------------------- T2 --------------->
                                               |
Query Interval                                 *
                                               |
Case 10                              [---R---] |
                                               |
Case 11                                  [---R---]
                                               |
Case 12                                        | [---R---]

A date search on ICA-AtoM should resolve as follows for each case:

Expected behaviour
Case Query should return this case
1 No
2 Yes
3 No
4 Yes
5 Yes
6 Yes
7 No
8 Yes
9 Yes
10 Yes
11 Yes
12 No

#4 Updated by Jesús García Crespo over 9 years ago

In ZSL, if you get the following error in the advanced search:

Terms per query limit is reached.

Increase the following limit as needed in plugins/sfLucenePlugin/lib/vendor/Zend/Search/Lucene.php

private static $_termsPerQueryLimit = 1024;

#5 Updated by Justin Simpson over 9 years ago

I updated the functional test and the code in advancedAction.class.php, but I put the wrong bug numer in my commit message, so it is not linked here.

commit:a5c3f2b36ae849aa990a9faf2b7e645b64cbe5fa

Advanced Search Date Range filter now works when the user supplies both a start date and an end date, or just one of those.

If the user gives just a start date, it is interpreted to mean 'filter to records that were active after this date', in other words 'end date of record > user supplied start date'

If the user gives just an end date, it means 'filter to records that were active before this date', in other words 'start date of record < user supplied end date'.

The test still does not include cases for when the information object does not have an enddate set.

#6 Updated by Dan Gillean about 9 years ago

  • Status changed from QA/Review to Feedback

We can discuss this in office perhaps, but I have received inconsistent results in testing.

First, I created a fonds with the dates 2005 - 2020

series A: 2010-1-1 to 2010-12-31
series B: 2010-2015
series C: 2015 (only display date entered, start and end both auto-populate to 2015)
series D: 2015-2020 (start 2015-03-03, end 2020)

Then I have attempted some queries, in combination with other fonds as well.

search for 20120000 - 20200000 = no hit on this fonds (by case 4 this is a fail)
search for 20060000 Start date = no hit on fonds, hit on Series A (series B/C/D should hit - fail on case 9)

search for 20120101 Start date = no hits. (fail case 8 and 9 for series B-D)

I will leave it there for now, pending further review and discussion, to be sure I've understood the test cases. I have a theory that the above may be because the dates are not properly formatted into ISO 8601 until exported - but I can't test it, because currently, until another issue is resolved, we are dropping dates on export.

#7 Updated by Anonymous about 9 years ago

  • Status changed from Feedback to QA/Review

#8 Updated by Jesús García Crespo about 9 years ago

  • Status changed from QA/Review to Feedback

#9 Updated by Jesús García Crespo almost 9 years ago

  • Estimated time set to 16.00

#10 Updated by Jesús García Crespo almost 9 years ago

  • Assignee changed from Jesús García Crespo to José Raddaoui Marín

#11 Updated by José Raddaoui Marín almost 9 years ago

  • Status changed from Feedback to QA/Review

Applied in changeset atom|commit:7893e8941d7fdbecb100fa7075211ea5c1108283.

#12 Updated by José Raddaoui Marín almost 9 years ago

  • Status changed from QA/Review to Feedback

It was a problem when searching dates in the future. It should be fixed now.

#13 Updated by José Raddaoui Marín almost 9 years ago

  • Status changed from Feedback to QA/Review

#14 Updated by Jessica Bushey almost 9 years ago

If I test the date searches using the date ranges provided in this example, but using the search parameters outlined here: [[https://www.ica-atom.org/doc/Search_fields#Searching_date_ranges]] then AtoM works.

If I use the GUI for date ranges in Advanced Search, I get results, but it isn't entirely clear how to restrict searches using just the start date or just the end date when using the GUI as opposed to the general search box, in which you can use startDate:[ ] and endDate:{ }

#15 Updated by Dan Gillean over 8 years ago

  • Status changed from QA/Review to Verified

Now that the datepicker has been added, I think this works fine. I will add extensive documentation to the user manual at the appropriate part.

#16 Updated by Dan Gillean over 8 years ago

Hi, is it possible for us to merge this into 2.x, so it can be included in 2.0.1?

#17 Updated by Dan Gillean almost 7 years ago

  • Related to Feature #8648: Advanced search date range selection added

Also available in: Atom PDF