Elasticsearch and .NET

Question

We're considering to switch from Solr/Solr.net to Elasticsearch. We started with NEST. We have only 4 documents in search index.

private static void Main(string[] args)
{
    var node = new Uri("http://localhost:9200");
    var settings = new ConnectionSettings(
        node, "my-application");

    var client = new ElasticClient(settings);

    var stopwatch = Stopwatch.StartNew();
    var sr = client.Get<Movie>(1);

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
}

The code above takes approx. 250ms, while the same code with HttpClient and JsonSerializer takes 30-45ms. 250ms is too much time for just 4 documents.

Can NEST be used on high-traffic news website, or do you recommend HttpClient + JsonSerializer combo? The search page was the most visited page on our website in 2013.

Thanks in advance.

Try doing more than one request, I suppose NEST must do some kind of caching after first request to dramatically improve performance, otherwise nobody would be using it. — ryudice
– ryudice, Commented Jun 2, 2014 at 22:08
You're right. First client.Get or client.Search request is slow, but all following requests are much faster. — šljaker
– šljaker, Commented Jun 2, 2014 at 22:33
This is true, NEST only needs to cache strongly typed property accesses i.e p=>p.Name and type/index inference. The big delaying factor is JSON.NET's serialization caches that need to be warmed up. — Martijn Laarman
– Martijn Laarman, Commented Jun 5, 2014 at 11:23

C Tierney · Accepted Answer · 2014-06-17 20:03:56Z

4

There are two things that have to happen in order for NEST to make the first request.

The Json Serializer (Json.net) in this case has to cache the type so that it knows how to serialize and deserialize the object you are sending back and forth.
Nest has its own fluent language for queries that must be translated from the intial types that represent the fluent query language and the delivered as JSON to elastic search. These document types also must be learned by the Json Serializer.
The HTTP client has to be spun up to make the request.

I currently have over 4M documents in a single index that I use with NEST and my searches from server all the way to client over the internet are taking 50-70 ms using NEST. Like you, however, after a cold start the first request is slow.

answered Jun 17, 2014 at 20:03

C Tierney

1,1211 gold badge8 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Basic Over a year ago

Just to add a data point, I have over 100 million in a single sharded across multiple servers and NEST adds little overhead to the raw elastic response times

Henry H. · Accepted Answer · 2015-05-11 08:29:22Z

I suggest you to use https://github.com/ServiceStack/ServiceStack.Text, the fastest Json serializer for C#.

For the driver, use the low-level one, http://nest.azurewebsites.net/elasticsearch-net/quick-start.html

below the code I started to write to log my apps in detail and analyze them. Still have to work, but can be a good beginning.

using System;
using System.Configuration;
using Elasticsearch.Net;
using Elasticsearch;
using Elasticsearch.Net.Connection;
using Elasticsearch.Net.ConnectionPool;

namespace Common {

    /// <summary>
    /// Elastic search. Singletone, open connection and thread safe to be open for all the time
    /// the app is running, so we send ours logs to ealsticsearch to be analyzed, assychronly
    /// See the fourth version;
    /// http://csharpindepth.com/articles/general/singleton.aspx
    /// </summary>
    public sealed class ElasticSearch {
        // our instance of ourself as a singleton
        private static readonly ElasticSearch instance = new ElasticSearch();

        ElasticsearchClient client;
        string connectionString = ConfigurationManager.ConnectionStrings["Elasticsearch"].ConnectionString;

        /// <summary>
        /// Initializes a new instance of the <see cref="Common.ElasticSearch"/> class.
        /// Follow this: http://nest.azurewebsites.net/elasticsearch-net/connecting.html
        /// We use a ConnectionPool to make the connection fail-over, that means, if the 
        /// connection breaks, it reconnects automatically
        /// </summary>
        private ElasticSearch() {
            var node = new Uri(connectionString);
            var connectionPool = new SniffingConnectionPool(new[] { node });
            var config = new ConnectionConfiguration(connectionPool);
            client = new ElasticsearchClient(config);   // exposed in this class
        }

        static ElasticSearch() {
        }

        /// <summary>
        /// Gets the instance of our singleton class
        /// </summary>
        /// <value>The instance.</value>
        public static ElasticSearch Instance {
            get {
                return instance;
            }
        }

        /// <summary>
        /// Log the specified module, id and json.
        /// </summary>
        /// <param name="type">Here the entity you want to save your log,
        /// let's use it based on classes and StateMachines</param>
        /// <param name="id">Identifier. alwayes the next</param>
        /// <param name="json">Json.</param>
        public void Log(string type, string id, string json) {
            client.Index("mta_log", type, id, json);
        }

    }
}

Collectives™ on Stack Overflow

Elasticsearch and .NET

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related