How to drilldown to the root cause data from Alert notification?

Modified on Fri, 16 Mar, 2018 at 8:23 AM

You can empower the Users to drilldown to the root cause data from an Alert notification.  To provide this capability to users, embed the drill down Elasticsearch query and  Kibana query for the root cause data in alert notification


Please follow the below steps to get notified with Elasticsearch query and Kibana query in notification


1.If you have a custom application that can handle Elasticsearch query, then append the drill down Elasticsearch query in Notification

Add the merge parameter ${drilldownESQuery} to your custom application URL in notification message as shown below.  





    Below is a sample notification

Hi,


Alert has been triggered for alert Sample Alert on 02-06-2017 19:51:00


http://testurl.com?index=.net*&type=connection&query={"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":10}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte": "2016-08-01T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte": "2016-06-02T09:42:00.000Z","lte": "2017-06-02T09:43:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{}}


Thanks


The following parameters values are included in the notification:


${AlertName} - Sample Alert

${Timestamp} - 02-06-2017 19:51:00

${drilldownESQuery}  - {"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":10}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte": "2016-08-01T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte": "2016-06-02T09:42:00.000Z","lte": "2017-06-02T09:43:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{}}



If your Alert includes a compare condition, follow the below steps

Add the merge parameter ${drilldownESQuery1} (for alert rule of timewindow) and ${drilldownESQuery2} (for alert rule of compare timewindow)  in notification message as shown below,



    Below is a sample notification

Hi,


Alert has been triggered for alert Sample Alert on 02-06-2017 19:51:00


http://testurl.com?index=.net*&type=connection&query={"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":40}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte":"2016-05-18T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte":"2016-06-05T13:45:00.000Z","lte":"2017-06-05T13:45:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{"count":{"value_count":{"field":"_index"}}}}


http://testurl.com?index=.net*&type=connection&query={"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":40}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte":"2016-05-18T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte":"2017-04-05T13:45:00.000Z","lte":"2017-05-05T13:45:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{"count":{"value_count":{"field":"_index"}}}}


Thanks


The following parameters values are included in the notification:


${AlertName} - Sample Alert

${Timestamp} - 02-06-2017 19:51:00

${drilldownESQuery1}  - {"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":40}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte":"2016-05-18T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte":"2016-06-05T13:45:00.000Z","lte":"2017-06-05T13:45:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{"count":{"value_count":{"field":"_index"}}}}

${drilldownESQuery2} - {"size":10,"sort":[{"Timestamp":{"order":"desc"}}],"query":{"bool":{"must":[{"range":{"age":{"gt":40}}},{"term":{"dstCity":"Friesenmouth"}},{"range":{"Timestamp":{"format":"date_time","gte":"2016-05-18T00:00:00.000Z"}}},{"range":{"Timestamp":{"gte":"2017-04-05T13:45:00.000Z","lte":"2017-05-05T13:45:00.000Z","format":"date_time"}}},{"query_string":{"query":"*","analyze_wildcard":true}}],"should":[],"must_not":[]}},"aggs":{"count":{"value_count":{"field":"_index"}}}}


2.If you would like users to drilldown using a kibana dashboard that you have configured append the drill down kibana query in notification

With drill down Kibana query, you will be able to visualize the root cause data in Kibana by adding the particular Kibana dashboard /search URL till "?" as shown below


 Sample Kibana Dashboard URL -  http://localhost:5601/#/dashboard/Sample-Dashboard?

 Sample Kibana Search URL -http://localhost:5601/app/kibana#/discover?


Add the merge parameter ${drilldownKibanaQuery}  in notification message as shown below,



     Below is a sample notification

Hi,


Alert has been triggered for alert Sample Alert on 02-06-2017 19:51:00


http://localhost:5601/#/dashboard/Sample-Dashboard?_g=(time:(from:'2016-06-02T09:42:00.000Z',mode:absolute,to:'2017-06-02T09:43:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(Timestamp:(format:date_time,gte:'2016-08-01T00:00:00.000Z')))))))),query:(query_string:(analyze_wildcard:!t,query:'age:>40 AND dstCity:"Friesenmouth"')))


Thanks


The following parameters values are included in the notification:


${AlertName} - Sample Alert

${Timestamp} - 02-06-2017 19:51:00

${drilldownKibanaQuery} - _g=(time:(from:'2016-06-02T09:42:00.000Z',mode:absolute,to:'2017-06-02T09:43:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(Timestamp:(format:date_time,gte:'2016-08-01T00:00:00.000Z')))))))),query:(query_string:(analyze_wildcard:!t,query:'age:>40 AND dstCity:"Friesenmouth"')))



If your Alert includes a compare condition, follow the below steps

Add the merge parameter ${drilldownKibanaQuery1} (for alert rule of time window) and ${drilldownKibanaQuery2} (for alert rule of compare time window)  in notification message as shown below,



     Below is a sample notification

     Hi,


Alert has been triggered for alert Sample Alert on 02-06-2017 19:51:00


http://localhost:5601/app/Kibana#/dashboard/Sample-Dashboard?_g=(time:(from:'2016-06-05T13:45:00.000Z',mode:absolute,to:'2017-06-05T13:45:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(age:(gt:40))),(term:(dstCity:'Friesenmouth')),(range:(Timestamp:(format:'date_time',gte:'2016-05-18T00:00:00.000Z'))),(range:(Timestamp:(gte:'2016-06-05T13:45:00.000Z',lte:'2017-06-05T13:45:00.000Z',format:'date_time'))),(query_string:(query:'*',analyze_wildcard:!t))),should:!(),must_not:!())))),query:(query_string:(analyze_wildcard:!t,query:'*')))


http://localhost:5601/app/Kibana#/discover?_g=(time:(from:'2017-04-05T13:45:00.000Z',mode:absolute,to:'2017-05-05T13:45:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(age:(gt:40))),(term:(dstCity:'Friesenmouth')),(range:(Timestamp:(format:'date_time',gte:'2016-05-18T00:00:00.000Z'))),(range:(Timestamp:(gte:'2017-04-05T13:45:00.000Z',lte:'2017-05-05T13:45:00.000Z',format:'date_time'))),(query_string:(query:'*',analyze_wildcard:!t))),should:!(),must_not:!())))),query:(query_string:(analyze_wildcard:!t,query:'*')))


Thanks



The following parameters values are included in the notification:


${AlertName} - Sample Alert

${Timestamp} - 02-06-2017 19:51:00

${drilldownKibanaQuery1} - _g=(time:(from:'2016-06-05T13:45:00.000Z',mode:absolute,to:'2017-06-05T13:45:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(age:(gt:40))),(term:(dstCity:'Friesenmouth')),(range:(Timestamp:(format:'date_time',gte:'2016-05-18T00:00:00.000Z'))),(range:(Timestamp:(gte:'2016-06-05T13:45:00.000Z',lte:'2017-06-05T13:45:00.000Z',format:'date_time'))),(query_string:(query:'*',analyze_wildcard:!t))),should:!(),must_not:!())))),query:(query_string:(analyze_wildcard:!t,query:'*')))

${drilldownKibanaQuery2} - _g=(time:(from:'2017-04-05T13:45:00.000Z',mode:absolute,to:'2017-05-05T13:45:00.000Z'))&_a=(filters:!((query:(bool:(must:!((range:(age:(gt:40))),(term:(dstCity:'Friesenmouth')),(range:(Timestamp:(format:'date_time',gte:'2016-05-18T00:00:00.000Z'))),(range:(Timestamp:(gte:'2017-04-05T13:45:00.000Z',lte:'2017-05-05T13:45:00.000Z',format:'date_time'))),(query_string:(query:'*',analyze_wildcard:!t))),should:!(),must_not:!())))),query:(query_string:(analyze_wildcard:!t,query:'*')))

   

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article