Elasticsearch @timestamp field returned as current date time instead of from log message

Question

This is my logstash.conf file:

filter {
  grok {
    match => {

      "message" => '%{IPV4:client_ip} - - \[%{HTTPDATE:log_timestamp}\] "%{WORD:method} %{URIPATHPARAM:uri} HTTP/%{NUMBER:http_version}" %{NUMBER:response_code} (?:%{NUMBER:response_size}|-) "%{DATA:referrer}" "%{DATA:user_agent}"(?: %{GREEDYDATA:extra_fields})?'
    }
  }
date {
  match => ["log_timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
  target => "@timestamp"
  }
}
output {
  stdout {
    codec => rubydebug
  }
}

This is one sample output after running logstash:

"log_timestamp" => "20/May/2020:03:47:15 +0000", "client_ip" => "{IP_ADDRESS}", "@timestamp" => 2020-05-20T03:47:15.000Z, "user_agent" => "example", "type" => "s3-log", "message" => "[20/May/2020:03:47:15 +0000] \"GET /hello/ HTTP/1.1\" 200 5632 \"-\" \"example\"", "@version" => "1", "method" => "GET", "http_version" => "1.1", "event" => { "original" => "[20/May/2020:03:47:15 +0000] \"GET /hello/ HTTP/1.1\" 200 5632 \"-\" \"example\"" }, "response_code" => "200", "uri" => "/hello/", "response_size" => "5632", "referrer" => "-"

As you can see the value of log_timestamp and timestamp are similar.

This is my index mapping:

{
  "my-index-name": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "client_ip": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "event": {
          "properties": {
            "original": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "extra_fields": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "http_version": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "log_timestamp": {
          "type": "date"
        },
        "message": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "method": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "referrer": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "response_code": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "response_size": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "tags": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "type": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "uri": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "user_agent": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }
}

But in my kibana data view @timestamp is returned as current time instead of the time shown in logstash output (ie time take from log)

in dev tools on search its comming correctly

/my-index-name/_search

"log_timestamp": "20/May/2020:03:42:57 +0000", "@timestamp": "2020-05-20T03:42:57.000Z",

In Kibana, did you set up an index pattern, and did you select @timestamp as the time field? — Stephan Pieterse
– Stephan Pieterse, Commented May 14 at 11:52
Also, have you checked the values in documents in elasticsearch? Maybe via the devtools instead — Paulo
– Paulo, Commented May 14 at 20:38
The vales seems to be correct when I search using dev tools. Have added it the result. Elastic search version 9.0.0 — Origin
– Origin, Commented May 15 at 3:39

Murat K. · Accepted Answer · 2025-06-05 12:55:07Z

0

Most likely the the timestamp value is not set when you created the data view.

You can go to Management > Kibana > Data Views. Edit your data view and set the timestamp.

answered Jun 5 at 12:55

Murat K.

9791 gold badge10 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch @timestamp field returned as current date time instead of from log message

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related