Bug #12282

AtoM qa/2.5 and Elasticsearch 5 - Error when rebuilding index: "Limit of total fields [2000] in index [atom] has been exceeded"

Added by Miguel Angel Medinilla Luque almost 4 years ago. Updated almost 4 years ago.

Status:VerifiedStart date:06/20/2018
Priority:HighDue date:
Assignee:Nick Wilkinson% Done:

0%

Category:Search / Browse
Target version:Release 2.5.0
Google Code Legacy ID: Tested version:2.5
Sponsored:No Requires documentation:

Description

Working on https://projects.artefactual.com/issues/12221 we have found that the default seeting for "index.mapping.total_fields.limit" (2000) on the ansible-atom role is inssuficient.

Counting the fields in an imprecise way, it seems that this value would be higher than 2000 and lower than 3000:

artefactual@bis-test:~$ curl -s -XGET localhost:9200/atom/_mapping?pretty | grep type | wc -l
2894

So this default value I think should be changed to a higher value:

https://github.com/artefactual-labs/ansible-atom/blob/master/defaults/main.yml#L125

History

#1 Updated by Miguel Angel Medinilla Luque almost 4 years ago

  • Assignee set to Nick Wilkinson

#3 Updated by Nick Wilkinson almost 4 years ago

  • Assignee changed from Nick Wilkinson to José Raddaoui Marín

#5 Updated by Dan Gillean almost 4 years ago

  • Category set to Search / Browse
  • Priority changed from Medium to High
  • Target version set to Release 2.5.0

Ran into this in my Vagrant box today - can't reindex!

#6 Updated by José Raddaoui Marín almost 4 years ago

  • Status changed from New to In progress

I think this issue comes from:

https://github.com/artefactual/atom/commit/f1cba271170e1471c971d995e60b9ff6075e7371

And maybe another ones where we're adding new fields to the Elasticsearch index. I'll increase the limit to 3000 for now, but we should look for a better solution where you don't have to be that careful each time you make changes to the index, maybe looking for the exact fields on index creation and updating that value.

#7 Updated by José Raddaoui Marín almost 4 years ago

AtoM PR in here

I'll follow with PRs for the Ansible role and deploy-pub repositories.

#8 Updated by José Raddaoui Marín almost 4 years ago

  • Status changed from In progress to Code Review
  • Assignee changed from José Raddaoui Marín to Nick Wilkinson

ansible-atom PR: https://github.com/artefactual-labs/ansible-atom/pull/25

No PR is needed in deploy-pub as it's using the default from ansible-atom.

#9 Updated by Nick Wilkinson almost 4 years ago

  • Assignee changed from Nick Wilkinson to Steve Breker

Hi Steve, passing to you for CR.

#10 Updated by Steve Breker almost 4 years ago

  • Status changed from Code Review to Feedback
  • Assignee changed from Steve Breker to José Raddaoui Marín

Code review complete - looks good!

#11 Updated by José Raddaoui Marín almost 4 years ago

  • Status changed from Feedback to QA/Review
  • Assignee changed from José Raddaoui Marín to Dan Gillean

Merged PRs in atom/qa/2.5.x and ansible/atom/master.

#12 Updated by Michelle Curran almost 4 years ago

  • Status changed from QA/Review to Verified
  • Assignee changed from Dan Gillean to Nick Wilkinson

Not running into this problem any longer after updating vagrant and rebuilding index. Fix verified.

Also available in: Atom PDF