Spring Data Elasticsearch doesn't implicitly create mapping from annotations

Question

I'm trying to work with Spring Data Elasticsearch 4.0.1 and trying to figure out how and when the Elasticsearch mappings are being created by Spring or if they are being created by Spring.

If I have an entity like so:

@Document(indexName = "companies")
public class CompanyEntity {

  @Id
  private final String id;

  @MultiField(
      mainField = @Field(type = Text),
      otherFields = {
          @InnerField(suffix = "raw", type = Keyword)
      }
  )
  private final String companyName;

  @PersistenceConstructor
  public CompanyEntity(String id, String companyName) {
    this.id = id;
    this.companyName = companyName;
  }

  public String getId() {
    return id;
  }

  public String getCompanyName() {
    return companyName;
  }
}

I was under the impression that Spring would implicitly create the mapping for this index but I seem to be mistaken. Elasticsearch still creates the mapping for this index.

{
    "companies": {
        "mappings": {
            "properties": {
                "_class": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "companyName": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "id": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}

Clearly from the above, the suffix in the InnerField annotation is not used and the ignore_above value is not used as well since it defaults to -1, and Elasticsearch would remove this field altogether if trying to set ignore_above to -1.

The only way I was able to get the mapping for the above annotations was to explicitly set the mapping myself.

@Autowired private ElasticsearchOperations operations;

Document mapping = operations.indexOps(CompanyEntity.class).createMapping();
operations.indexOps(CompanyEntity.class).putMapping(mapping);

Which yields the expected mapping:

{
    "companies": {
        "mappings": {
            "properties": {
                "_class": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "companyName": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    }
                },
                "id": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}

Which is fine, but feels a bit odd to me as I could not find any details of this approach in the official Spring Data Elasticsearch docs. And the JavaDocs are kinda void of any details.

Is this the right approach to installing mappings into Elasticsearch from Spring Data?

1. Created a sample project with sprint-data-elastic (3.2.9.RELEASE) and defined the document mapping with a multi-field (text, keyword) and ran the project. The mapping generated is what I expect. Can you please share the version of spring-data-elastic & elastic-search which you are using. — Sahil Gupta
– Sahil Gupta, Commented Aug 21, 2020 at 4:41
2. You can set a debug point on ElasticsearchRestTemplate -> "putMapping(String indexName, String type, Object mapping)" to see what mapping is generated. — Sahil Gupta
– Sahil Gupta, Commented Aug 21, 2020 at 4:42
3. Is this the right approach to installing mappings into Elasticsearch from Spring Data? - Es is not like Mysql where-in you would get an exception if the field doesn't exist in the store, rather it creates a new field if not found. Hence it is alright to install mappings via spring data as long as you define it correctly and the generated mapping is what you intend to. Am i able to answer all your questions ? — Sahil Gupta
– Sahil Gupta, Commented Aug 21, 2020 at 4:45
@SahilGupta, I the version I am using is 4.0.1. And the expected mapping is the last mapping I detail in my post. The first mapping I detail is the mapping that Elasticsearch creates automatically. And I will try to do a bit of debugging on Spring's side to see if any mappings are being created. Regarding your last point, I understand that I might have to create the mapping manually, but this does break expectations of using Spring-Data. I'm trying to understand if I need to always create these mappings manually as this detail is not in the official spring docs. — FlatPenguin
– FlatPenguin, Commented Aug 21, 2020 at 11:36

Abacus · Accepted Answer · 2020-08-21 06:05:16Z

4

If you are using the ElasticsearchOperations to work with, then this is the right way to create the mapping. In version 4.1 there will be an additional method

boolean putMapping(Class<?> clazz)

which combines the two steps of creating and writing the mapping.

ElasticsearchOperations and IndexOperations - or better the implementations - are the more low level access to Elasticsearch in Spring Data Elasticsearch - create an index, put the mapping, write an entity etc. These are elemantary operations. You have the full control which operations are executed, but it's your respsonsibility to execute them.

Building on top of that is the repository support. If you define a repository interface

interface CompanyRepository extends ElasticsearchRepository<CompanyEntity, String>{}

which you inject in one of your classes

@Autowired CompanyRepository repository;

then on application startup Spring Data Elasticsearch will create an implementation of this interface and will check if the index defined by the entity's @Document annotation exists. If not, it will create the index and write the mapping - The @Document annotation has an argument createIndex which by default is true.

So for autocreation you must use the repository, the repository support also makes use of the ElasticsearchOperations to offer things like query derivation from method names, paging support and more.

edited Aug 21, 2020 at 6:05

answered Aug 21, 2020 at 5:32

Abacus

19.6k7 gold badges54 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

FlatPenguin Over a year ago

Hey @P.J.Meisch, my expectation was that Spring would automatically create this mapping for me as you mentioned in your second to last paragraph. But I find this is not the case. You're right, the index is created by Spring, but Elasticsearch is creating the mapping for the index and not Spring. The only way I can get the annotations on the Entity class to take effect is if I manually putMapping on the index itself, which is kinda not what I'm expecting. I hate to ask this, but would you have a sample repo that demos the mapping being created by Spring w/o manually doing so?

FlatPenguin Over a year ago

Hey @P.J.Meisch, I was able to sort the issue. I had a configuration problem in my Elasticsearch configuration class so my configuration was not being picked up by the Spring context. I resolved that, and then after adding the Repository I was able to see the expected mapping. Is this detailed in the official documentation somewhere, that the Repository is mandatory for mapping creation? Outside of the trivial example I detailed, I have a far more complex problem and cannot use a typical Repository to solve my query needs. I create the queries manually and execute with NativeSearchQuery.

Abacus Over a year ago

The documentation and javadoc for the @Document annotation state that the index creation is done on repository bootstrapping. I think it would be good for the next release to add a section about index management, as 4.1 adds functions like templates and improved alias handling as well. Made me an issue for that.

TEXHIK Over a year ago

Hey @P.J.Meisch, I'm using spring-data 4.1.3, operating with elastic via repository (as in your example) but the index, created on application startap has an empty mapping. It's only filled when saving documents, and it's filled by elastic. Any ideas?

Abacus Over a year ago

Do you have the properties of your entity annotated with @Field? And do these properties have getter and setter methods?

|

Collectives™ on Stack Overflow

Spring Data Elasticsearch doesn't implicitly create mapping from annotations

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related