Task #10847

Upgrade Elasticsearch to version 5.0

Added by Nick Wilkinson over 2 years ago. Updated 29 minutes ago.

Status:VerifiedStart date:02/01/2017
Priority:MediumDue date:
Assignee:-% Done:

0%

Category:Search / Browse
Target version:Release 2.5.0
Google Code Legacy ID: Tested version:
Sponsored:No Requires documentation:

10847-ESPlugin-changes.diff Magnifier (9.37 KB) Steve Breker, 06/23/2017 12:56 PM

History

#1 Updated by Nick Wilkinson over 2 years ago

  • Assignee set to Steve Breker

#3 Updated by David Juhasz about 2 years ago

  • Subject changed from Upgrade ES to 5.0 to Upgrade Elasticsearch to version 5.0

#4 Updated by Steve Breker almost 2 years ago

All committed work-to-date has been pushed to this branch:

https://github.com/artefactual/atom/commits/dev/issue-10847-update-elasticsearch

This has been updated to use the latest version of Elastica as of about a week and a half ago.

#5 Updated by Steve Breker almost 2 years ago

This attached file 10847-ESPlugin-changes.diff contains my work in progress changes as I was attempting to get the ES plugin up and running. This is very incomplete and is just a start.

#6 Updated by Steve Breker almost 2 years ago

  • Assignee changed from Steve Breker to José Raddaoui Marín

Our AtoM installation docs will need to be updated for installing ES 5.2. I have made notes in the GDoc below based on our existing instructions. This document can be followed to get ES 5.2 up and running on your dev box.

An additional consideration is that ES 5.2 is configured by default to consume 2GB of RAM. I have included notes regarding this in the doc. The AtoM install doc will also need to be updated to include instructions on how to configure this memory setting. Notes I had seen indicate to allocate 50% of the system's mem to ES. On my 2GB VM I allocated 1GB to ES and it seemed to work just fine.

https://docs.google.com/a/artefactual.com/document/d/1vQCjZOI-sKJv3k818M6dj3mDd_ObM0sNDCL-jJVcgls/edit?usp=sharing

#7 Updated by Nick Wilkinson almost 2 years ago

Just a reminder that this work should not be merged into qa/2.4.x.

#8 Updated by José Raddaoui Marín almost 2 years ago

  • Status changed from New to In progress

#9 Updated by José Raddaoui Marín almost 2 years ago

  • Status changed from In progress to Code Review
  • Assignee changed from José Raddaoui Marín to Nick Wilkinson

#10 Updated by José Raddaoui Marín almost 2 years ago

Thanks for the doc, update and work done Steve! It was easy to get it all running and with the index populated ;)

#11 Updated by Nick Wilkinson almost 2 years ago

  • Assignee changed from Nick Wilkinson to Mike Gale

Hi Mike, assigning to you for CR.

#12 Updated by Mike Gale almost 2 years ago

  • Assignee changed from Mike Gale to José Raddaoui Marín

Wow, that's a huge PR. I didn't see anything too out of the ordinary, I had a question or two that's it. I think it'll be best to merge this (I guess in 2.5) and re-test all the search functionality! It's hard to say if every single query is correctly changed from the shear volume of it.

#13 Updated by Nick Wilkinson almost 2 years ago

  • Status changed from Code Review to In progress

#14 Updated by José Raddaoui Marín over 1 year ago

Steve Breker wrote:

Our AtoM installation docs will need to be updated for installing ES 5.2. I have made notes in the GDoc below based on our existing instructions. This document can be followed to get ES 5.2 up and running on your dev box.

An additional consideration is that ES 5.2 is configured by default to consume 2GB of RAM. I have included notes regarding this in the doc. The AtoM install doc will also need to be updated to include instructions on how to configure this memory setting. Notes I had seen indicate to allocate 50% of the system's mem to ES. On my 2GB VM I allocated 1GB to ES and it seemed to work just fine.

https://docs.google.com/a/artefactual.com/document/d/1vQCjZOI-sKJv3k818M6dj3mDd_ObM0sNDCL-jJVcgls/edit?usp=sharing

One thing I forgot to mention, is that I didn't have the memory issue using their deb packages to switch ES versions.

#15 Updated by José Raddaoui Marín over 1 year ago

  • Category set to Search / Browse
  • Status changed from In progress to QA/Review
  • Assignee changed from José Raddaoui Marín to Dan Gillean
  • Target version set to Release 2.5.0

This has been merged in qa/2.5.x and I think it should be tested over the time while we move forward with 2.5

#16 Updated by Dan Gillean over 1 year ago

  • Status changed from QA/Review to Feedback
  • Assignee changed from Dan Gillean to José Raddaoui Marín

User reported issues searching for identifiers in the Vagrant box here: https://groups.google.com/d/msg/ica-atom-users/rkoDk6pcOzY/Aa1Z9-R8CgAJ

Reproduced in current 2.5.0.0 vagrant box - no issue with same test data in 2.4.0.1 vagrant box.

In some cases, the issue seems to be with _all - you can target the field directly using the advanced search panel and get results, but nothing in the global search, despite the fact that both identifier and referenceCodeWithoutCountryAndRepo are supposed to be included in all (see: https://projects.artefactual.com/issues/11684#note-4).

I used the public demo data set in my boxes, and searched for U245. Shows 7 results in 2.4:

Zero results are returned in the 2.5 Vagrant box with the same data, freshly indexed with all caches cleared etc etc.

#17 Updated by José Raddaoui Marín over 1 year ago

  • Status changed from Feedback to Code Review
  • Assignee changed from José Raddaoui Marín to Nick Wilkinson

#18 Updated by Nick Wilkinson over 1 year ago

  • Assignee changed from Nick Wilkinson to Steve Breker

Hi Steve, assigning to you for CR.

#19 Updated by Steve Breker over 1 year ago

  • Status changed from Code Review to Feedback
  • Assignee changed from Steve Breker to José Raddaoui Marín

CR Looks good. Ready to merge.

#20 Updated by José Raddaoui Marín over 1 year ago

  • Status changed from Feedback to QA/Review
  • Assignee changed from José Raddaoui Marín to Dan Gillean

Fix merged in qa/2.5.x

#21 Updated by Dan Gillean over 1 year ago

This particular issue is resolved - thanks Radda! Going to leave this ticket open for now, so we can continue to test for further regressions.

#22 Updated by Dan Gillean over 1 year ago

  • Status changed from QA/Review to Verified
  • Assignee deleted (Dan Gillean)

Going to mark this verified for now - if we discover new regressions related to this, we should file individual tickets at this point. the general upgrade has been proven over time to be working.

#23 Updated by Corinne Rogers about 1 month ago

  • Requires documentation set to Yes

Has documentation for this been updated?

#24 Updated by José Raddaoui Marín about 1 month ago

Hi Corinne,

Yes, documentation was added in https://github.com/artefactual/atom-docs/pull/40.

#25 Updated by Dan Gillean 29 minutes ago

  • Requires documentation deleted (Yes)

Also available in: Atom PDF