Identifying newly occurred source IP addresses for too many open connections

Usecase

Compare current time window with previous time window and get the list of source IP addresses of too many connections with count which was not seen in the previous time window . 


Notification Types:  Email


User Inputs

Please find the below image for setting up alerts in Skedler-Alerts 


Notification received via Email

Hi,

Alert has been triggered for alert "Newly occured srcIp" on Sat Dec 16,2017 20:07:45 IST


Alert Condition:

Rule Type: New value, Keyword Filter: * - select * from .data* where timeWindow between Sat Dec 16,2017 19:07:45 IST - Sat Dec 16,2017 20:07:45 IST


TimeWindow:

Sat Dec 16,2017 19:07:45 IST - Sat Dec 16,2017 20:07:45 IST


Elasticsearch Query:


{\"size\":10000,\"sort\":[{\"Timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"date\"}}],\"query\":{\"bool\":{\"must\":[{\"range\":{\"Timestamp\":{\"gte\":\"2017-12-16T13:37:45.000Z\",\"lte\":\"2017-12-16T14:37:45.000Z\",\"format\":\"date_time\"}}},{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}}],\"should\":[],\"must_not\":[]}},\"aggs\":{\"count\":{\"value_count\":{\"field\":\"_index\"}}}}


{\"size\":10000,\"sort\":[{\"Timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"date\"}}],\"query\":{\"bool\":{\"must\":[{\"range\":{\"Timestamp\":{\"gte\":\"2017-12-16T12:37:45.000Z\",\"lte\":\"2017-12-16T13:37:45.000Z\",\"format\":\"date_time\"}}},{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}}],\"should\":[],\"must_not\":[]}},\"aggs\":{\"count\":{\"value_count\":{\"field\":\"_index\"}}}}


Matching Records:


New value for data generated from Sat Dec 16,2017 19:07:45 IST to Sat Dec 16,2017 20:07:45 IST not seen in previous Sat Dec 16,2017 19:07:45 IST to Sat Dec 16,2017 18:07:45 IST


Field
Value
count
srcIp
105.3.152.219
2
srcIp
129.89.1.70
1


http://localhost:3001/skedler-alerts/alertdetails?alertId=1auyTmABJG5Y3vs9VWIF&alertname=%22newly%22occured%22srcIp%22&triggered_time=2017-12-16T14:37:45.000Z


Thanks


Explanation

Parameters configured will be replaced as follows:

  1. ${AlertName}       -   Newly occured srcIp
  2. ${TimeStamp}      -  Sat Dec 16,2017 20:07:45 IST
  3. ${TimeWindow} -   Sat Dec 16,2017 19:07:45 IST - Sat Dec 16,2017 20:07:45 IST
  4. ${drilldownESQuery1} -

    {\"size\":10000,\"sort\":[{\"Timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"date\"}}],\"query\":{\"bool\":{\"must\":[{\"term\":{\"srcCountry\":\"Malaysia\"}},{\"range\":{\"Timestamp\":{\"gte\":\"2017-12-16T14:22:45.000Z\",\"lte\":\"2017-12-16T14:37:45.000Z\",\"format\":\"date_time\"}}},{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}}],\"should\":[],\"must_not\":[]}},\"aggs\":{\"count\":{\"value_count\":{\"field\":\"_index\"}}}}

  5. ${drilldownESQuery2} - 

    {\"size\":10000,\"sort\":[{\"Timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"date\"}}],\"query\":{\"bool\":{\"must\":[{\"term\":{\"srcCountry\":\"Malaysia\"}},{\"range\":{\"Timestamp\":{\"gte\":\"2017-12-16T14:07:45.000Z\",\"lte\":\"2017-12-16T14:22:45.000Z\",\"format\":\"date_time\"}}},{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}}],\"should\":[],\"must_not\":[]}},\"aggs\":{\"count\":{\"value_count\":{\"field\":\"_index\"}}}}

Note - For additional parameters, refer How to setup merge parameters for alert action?

         - To test the newly occurred value in current time window, add the parameters ${drilldownESQuery1} and ${drilldownESQuery2} in message, so will be able to get the IP addresses which is not in previous time window


Conclusion

 Below table shows the list of new source IP addresses which passes the count  > 0 (i.e occurred at least 1) for current time window which was not seen in previous time window


Field
Value
count
srcIp
105.3.152.219
2
srcIp
129.89.1.70
1