LATEST POSTS





4/08/2016

Logstash Read Object Values Printed in the Log





Logstash is a tool for reading logs and send it to the elastic search.Lets say we have object is printed in the log file and we can't write a grok pattern to entire log file. We need to extract certain information from that object. I have log file as below


171.741.544.442 - kurt [28/Jun/2015:01:48:10 -0700] "POST User[name:raja:age:25 Location[city:blr;state:ka;country:india]]]

 Above log i gave simple example of object but many case object would be very big and its not possible to write the grok pattern for entire log. So in order to avoid this scenario we need to  use macro called GREEDYDATA

 %{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{GREEDYDATA}state:%{WORD:state} 

 I have wrote the grok till http method type post "171.741.544.442 - kurt [28/Jun/2015:01:48:10 -0700] "POST" rest of the part going to greedydata. Which means entire object goes to greedy data. Once greedy data is generated we are applying filter on top of it to get the state.Above example solve the problem of reading the logs in between in logstash  and reading objects printed in the log file using logstash.

0 comments:

Post a Comment

 
Design by Free WordPress Themes | Bloggerized by Lasantha - Premium Blogger Themes | coupon codes