Logstash use JSON field date instead of @timestamp

I'm facing this issue.
I'm trying to use custom JSON log date as my "usable" date instead of the @timestamp date field.

My JSON file to be processed by Logstash (comming from filebeat):

{
"start": {
"timestamp": {
"time": "Wed, 04 Apr 2018 09:36:39 GMT",
"timesecs": 1522834599
}
}
}


My logstash.yml file :

input {
beats {
port => 1337
codec => "json_lines"
}
}

filter {
date {
match => [ "time", "EEE, dd MM yyyy hh:mm:ss ZZZ" ]
}
}

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => "localhost:9200"
index => "testing"
}
}


Also tried to :

match => [ "[start][timestamp][time]", "EEE, dd MM yyyy hh:mm:ss ZZZ" ]


Still no luck.

Any help would be welcome.

Cheers,

2 Answers
2

Got to the solution like this :

mutate {
add_field => {
"mytime" => ""
}
}

date {
match => [ "[start][timestamp][time]", "EEE, dd MMM yyyy HH:mm:ss z" ]
target => "mytime"
locale => "en"
}


you don't need mutate filter. please read my answer
– Sufiyan Ghori
2 days ago

mutate

When you match the date using date filter, it stores the matching timestamp into the given target field. If target not provided, it will simply update the @timestamp field of the event with the new matching time.

match

date

target

@timestamp

time

target

Store the matching timestamp into the given target field. If not
provided, default to updating the @timestamp field of the event.

You don't even need to create a field with mutate filter, target will automatically create a field if it doesn't exists. Besides, add_field is a common option and available for date filter as well.

mutate

target

date

So following code is enough,

date {
match => [ "[start][timestamp][time]", "EEE, dd MMM yyyy HH:mm:ss z" ]
target => "newTimeField"
locale => "en"
remove_field => [ "[start][timestamp][time]" ]
}


remove_field above is another common option available for date filter. It is used to delete old time field once its stored in a new field.

remove_field

date

time

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.