Feature #8618

Update date format in ES documents

Added by Jesús García Crespo over 6 years ago. Updated over 4 years ago.

Status:NewStart date:06/29/2015
Priority:LowDue date:
Assignee:-% Done:

0%

Category:Search / Browse
Target version:-
Google Code Legacy ID: Tested version:
Sponsored:Yes Requires documentation:Yes

Description

We are going to use ES Date Format. Of the predefined list of dates, we are going to use "date": A formatter for a full date as four digit year, two digit month of year, and two digit day of month (yyyy-MM-dd).

This is going to require documentation updates as the date format used during searches will probably have to change too.

Reminder: if the date format is changed in the mapping, the new format has to be used when the document is populated in Elasticsearch, but the new format will be automatically returned when querying Elasticsearch without postprocessing needed. See the following example:

<?php
namespace Elastica\Test;

use Elastica\Document;
use Elastica\Test\Base as BaseTest;
use Elastica\Type\Mapping;

class IssueTest extends BaseTest
{
    private static
        $mapping =  array(
            'birth' => array('type' => 'date', 'format' => 'yyyy-MM-dd'),
            'death' => array('type' => 'date', 'format' => 'yyyyMMdd'),
        );

    /**
     * @group functional
     */
    public function testDateFormatWithExpectedInput()
    {
        $client = $this->_getClient();

        $index = $client->getIndex('test');
        $index->create(array(), true);

        $type = $index->getType('test');
        $type->setMapping(new Mapping($type, self::$mapping));

        $type->addDocument(new Document(1, array(
            'birth' => '1930-05-11',
            'death' => '20020806')));

        $index->refresh();

        $doc = $type->getDocument(1);
        $data = $doc->getData();

        $this->assertEquals($data['birth'], '1930-05-11');
        $this->assertEquals($data['death'], '20020806');
    }

    /**
     * @group functional
     */
    public function testDateFormatWithUnexpectedInput()
    {
        $client = $this->_getClient();

        $index = $client->getIndex('test');
        $index->create(array(), true);

        $type = $index->getType('test');
        $type->setMapping(new Mapping($type, self::$mapping));

        $this->setExpectedException('Elastica\Exception\ResponseException', 'MapperParsingException[failed to parse [birth]]; nested: MapperParsingException[failed to parse date field [1930-May-11], tried both date format [yyyy-MM-dd], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "1930-May-11" is malformed at "May-11"];');

        $type->addDocument(new Document(1, array(
            'birth' => '1930-May-11',
            'death' => '2002-Aug-06')));

        $index->refresh();

        $doc = $type->getDocument(1);
        $data = $doc->getData();

        $this->assertEquals($data['birth'], '1930-05-11');
        $this->assertEquals($data['death'], '20020806');
    }
}

issue-8618-inventory.png (30.2 KB) Jesús García Crespo, 06/29/2015 06:13 PM

issue-8618-search-result.png (32.7 KB) Jesús García Crespo, 06/29/2015 06:13 PM

History

#2 Updated by Jesús García Crespo over 6 years ago

  • Description updated (diff)
  • Estimated time deleted (8.00)

#3 Updated by Jesús García Crespo over 6 years ago

  • Description updated (diff)

#4 Updated by Jesús García Crespo over 6 years ago

  • Status changed from New to Code Review
  • Assignee changed from Jesús García Crespo to David Juhasz

#8 Updated by Jesús García Crespo over 6 years ago

  • Status changed from Code Review to Verified

#9 Updated by Jesús García Crespo over 6 years ago

  • Status changed from Verified to In progress
  • Assignee changed from David Juhasz to Jesús García Crespo
  • Priority changed from Medium to Low

#10 Updated by Jesús García Crespo almost 6 years ago

  • Status changed from In progress to New

#11 Updated by Jesús García Crespo almost 6 years ago

  • Assignee deleted (Jesús García Crespo)

#12 Updated by Redmine Admin over 5 years ago

  • Target version changed from Release 2.3.0 to Release 2.4.0

Bumping to 2.4. Also, about what's been said above, I can't remember the details but I remember that the proposal will require rethinking.

#13 Updated by Jesús García Crespo about 5 years ago

For one of our clients we tried a new approach: feed Elasticsearch with dates.startDate and dates.endDate in epoch format as integers. Elasticsearch will consistently return the dates in the same format that they were given. We convert from epoch to local format using the DateHelper during view rendering, making localization of dates possible. Timestamp was the choice because it's already the internal format in Elasticsearch (in longs) and it feels simpler.

#14 Updated by Dan Gillean over 4 years ago

  • Target version deleted (Release 2.4.0)

Also available in: Atom PDF