Bulk insert to ElasticSearch with NEST

Question

I try to add 100k products to elasticsearch, but when i try i get: {"Validation Failed: 1: no requests added;"}

My code:

        var Node = new Uri("......");
        var ConnectionPool = new SniffingConnectionPool(new[] { Node });
        var Config = new ConnectionConfiguration(ConnectionPool)
                    .SniffOnConnectionFault(false)
                    .SniffOnStartup(false)
                    .SniffLifeSpan(TimeSpan.FromMinutes(10));
        var Client = new ElasticsearchClient(Config);

        var AllProducts = Product.GetProducts();
        var SPl = AllProducts.Split(100); // Split into 100 collections/requests

        var COll = new List<ElasticsearchResponse<DynamicDictionary>>();

        foreach (var I in SPl)
        {
            var Descriptor = new BulkDescriptor();

            foreach (var Product in I)
            {
                Descriptor.Index<Product>(op => op.Document(Product));
            }

            COll.Add(Client.Bulk(Descriptor));
        }

AllProducts contains a list of this object:

public class Product
{
 public int AffiliateNr { get; set; }
 public string AffiliateProductId { get; set; }
 public int BrandNr { get; set; }
 public string Currency { get; set; }
 public string IndexId { get; set; }
 public decimal Price { get; set; }
 public long ProductNr { get; set; }
 public string Title { get; set; }
}

So,

Where can i set the name of the index?
Why did i get, Validation Failed: 1: no requests added;?
IndexId is my id for the product. How do i tell Elasticsearch to use this id? Or must i specify a id?

thanx. I now have: SPl.Select(I => Client.IndexMany(I, "products")) And it works. But: IndexId is my id for the product. How do i tell Elasticsearch to use this id? Or must i specify a id? — mrcode
– mrcode, Commented Jun 5, 2015 at 7:03
Rename the IndexId field to Id and elasticsearch will use this by convention as the id of the document. — Manolis
– Manolis, Commented Jun 5, 2015 at 7:06

Shivang MIttal · Accepted Answer · 2018-01-03 09:12:32Z

7

Refering to your previous problem, you can use IndexMany for indexing the data. Now as per your question in comment, you can specify the id which elastic search will use. see the below example.

  ElasticType(IdProperty = "<fieldName>")]
    public class ClassName
    {

if you dont want to specify any Id to elastic search, create a dummy field dummyId(nullable) and put it in "IdProperty". Elastic search will auto assign the value to it if it null.

Edit : From 2.3 onwards, Its

[ElasticsearchType(IdProperty = "<fieldName>")]

edited Jan 3, 2018 at 9:12

answered Jun 5, 2015 at 10:39

Shivang MIttal

1,0011 gold badge17 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mrcode Over a year ago

Perfect. Is there a easy way to mark wich properties i dont want to index in elasticsearch? :)

Shivang MIttal Over a year ago

@mrcode : yes , please go through the answer stackoverflow.com/questions/23063839/…. mark the above answer if it is correct

Collectives™ on Stack Overflow

Bulk insert to ElasticSearch with NEST

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related