Introduction

Skedler – Alerts is an Elasticsearch companion tool with user-friendly UI which helps in alerting on the occurrence of inconsistent data.

If you have real-time data that has been written on Elasticsearch with the matching conditions given in Skedler – Alerts then the user will be alerted through email or webhook.


Skedler-Alerts concepts


S.No
Name

Mandatory / Optional

Description

1.
Alert Name

Mandatory

A name for scheduling an Alert

2.
Index Pattern (or) Name

Mandatory

List the available index-pattern configured in index-pattern settings.

3.
Index Type

Optional

List the available types for the given enclosed Elasticsearch index

4.

Time Field

Mandatory

Timestamp field is used to search or query the Elasticsearch indices against particular time range.

5.
Alert Conditions

Mandatory

Query to check for events to be alerted. Query can be based on rule keyword search or compare condition or aggregate condition or any of the above combination

6.
Schedule

Mandatory

Time interval to run the given alert conditions and check for inconsistency. Schedule can be in Seconds, minutes or hourly, daily, weekly

7.
Alert Action - Alias
Mandatory

Webhook alias to which the notifications will be sent when an alert event occurs

8.
Alert Action - Webhook template
Mandatory
Default message template for normal and slack webhook
9.
Alert Action - Webhook Parameters

Optional

You can insert a set of dynamic field values in additional data or Messages by adding the placeholder parameters.
An example is you can select the parameter name Alert Name and add the parameter by clicking merge in the required place. The actual alert name will be inserted

10.
Alert Action - Webhook
Include Result

Optional

Selected event data is sent in the notification

11.
Alert Action - Email To

Mandatory

emails ids (comma separated list) to which alert notifications are sent.

12.
Alert Action - Email CC

Optional

emails ids (comma separated list) to which alerts notifications are sent.

13.
Alert Action - Email Parameters

Optional

You can insert a set of dynamic field values in Subject and Messages by adding the placeholder parameters.
An example is you can select the parameter name Alert Name and add the parameter by clicking merge in the required place. The actual alert name will be inserted

14.
Alert Action - Email Subject

Mandatory

Subject for the email.

15.
Alert Action - Email Message

Mandatory

Notification message for alert conditions to be sent in email

16.
Alert Action - Email Include JSON

Optional

Events which caused the alert will be sent as a JSON attachment

17.
Alert Action - Email Select Fields

Optional
Particular fields occurred in the event to send in notification
18.
Alert Action - Index Pattern

Mandatory

Events which caused the alert
will be sent to given Elasticsearch

19.
Alert Action - Index Pattern
Select Field for Notification
Optional
Particular fields occurred in events to be pushed in given Elasticsearch


Access Skedler-Alerts

After installation, Skedler-Alerts can be accessed from the following URL if the Skedler-Alerts is using the port 3001 http://<yourserver>:3001


Schedule Skedler-Alerts

1. Alerts can be scheduled by filling the appropriate values

  • Alert Details
    • Classification will allow identifying the importance of Alerts ie. Critical / Warning / Information
    • Alert Name of your choice.
    • Fill the Index Name, Skedler Alerts will provide the available indices from Elasticsearch.
    • Optionally, you can select the Index Type for the selected Elasticsearch index.
    • Select the Time Field for the index.
    • Optionally, enter the Alert Tags which helps you to search the alert based on the tags.

  • Alert Conditions
    • Different type of alerts can be configured via Rule Type parameter. Below are the details on supported Rule Type and the corresponding settings.

      Rule type – monitoring pattern for a rule

      • Threshold – Match on any event matching a given filter
      • Spike – Match when the rate of events increases or decreases
      • New value – Match when a never before seen value appears in a field
      • Repeated value – Match when a repeated value appears in a field
      • Flatline – when event threshold attains dead state i.e threshold < 1
  • Threshold :
    Trigger for any event matching a given filter


  • Keyword Filter – Alerts will search the entire index for the matching keyword. Keyword filter functionality looks more similar to Elasticsearch Query String Query pattern.
    Note: For details on Elasticsearch Query String Query, please refer to the links below:
  • Aggregation Filter
    • Can perform aggregation operations like count, avg, min, max, sum based on the selection of field.
    • Select aggregation type, field, condition (like greater than, lesser than, equal to etc.) and value to apply the condition

  • Query Filter
    • Select field, query condition (Must be, Must not be, Should be), condition (like greater than, lesser than, equal to etc.) and value to apply the condition.
    • Can add multiple conditions by clicking the add icon
  • Group By – Group the result in buckets based on nested aggregation.
  • Order – Sort events in the bucket (Ascending or Descending)
  • Number of Documents – Number of bucket event counts to retrieve.
    For example – Input ‘5’ will retrieve only 5 bucket values matching condition
  • Time window – To generate an alert for a specific time range. For example – last two hrs, last 5 days.
  • With prior condition (optional)
    • Keyword Filter – Keyword filter for the prior time window condition
    • Aggregation Filter –  Aggregation filter for the prior time window condition
    • Query Filter –  Query filter for the prior time window condition

          Note: With the prior condition, an alert will trigger if the current time window and previous time window satisfies the condition


  • Spike :

    Trigger when the rate of events increases or decreases in the time window


  • Keyword Filter – Alerts will search the entire index for the matching keyword. Keyword filter functionality looks more similar to Elasticsearch Query String Query pattern.
  • Aggregation Filter
    • Can perform aggregation operations like count, avg, min, max, sum based on the selection of field.
    • Select aggregation type, field, condition (like greater than, lesser than, equal to etc.) and value to apply the condition
  • Query Filter
    • Select field, query condition (Must be, Must not be, Should be), condition (like greater than, lesser than, equal to etc.) and value to apply the condition.
    • Can add multiple conditions by clicking the add icon
  • Group By – Group the result in buckets based on nested aggregation.
  • Order – Sort events in the bucket (Ascending or Descending)
  • Number of Documents – Number of bucket event counts to retrieve.
    For example – Input ‘5’ will retrieve only 5 bucket values matching condition
  • Time window – To generate an alert for a specific time range. For example – last two hrs, last 5 days.
  • Compare to – Compare the current time window to some other time window.
    For example – comparing the data for last 5 days to previous 5 days
  • Value (times)- Compare ‘x’ times of data for the current time window to previous time window.
  • Operation – Compare the data of current time window condition (More than, Less than, More than equal to, Less than equal to) to the previous time window.
  • Relative Time Window (Last & to) – Compare the alert for the specific time range to the time window.
  • Includes (or) Excludes Time Window – Include option will include the current “Time window” in “Previous Time Window”, Exclude option will exclude the current “Time window” in “Previous Time Window”
  • Test Query – will test the given filters with request and response.

Refer the following article How to set alerts with rule type "Spike" - sample use case


  • New Value:
    Trigger values that were not seen in previous time window but seen in the current time window
  • Keyword Filter – Alerts will search the entire index for the matching keyword. Keyword filter functionality looks more similar to Elasticsearch Query String Query pattern.
  • Aggregation Filter
    • Can perform aggregation operations like count, avg, min, max, sum based on the selection of field.
    • Select aggregation type, field, condition (like greater than, lesser than, equal to etc.) and value to apply the condition
  • Query Filter
    • Select field, query condition (Must be, Must not be, Should be), condition (like greater than, lesser than, equal to etc.) and value to apply the condition.
    • Can add multiple conditions by clicking the add icon
  • Group By – Group the result in buckets based on nested aggregation.
  • Order – Sort events in the bucket (Ascending or Descending)
  • Number of Documents – Number of bucket event counts to retrieve.
    For example – Input ‘5’ will retrieve only 5 bucket values matching condition
  • Time window – To generate an alert for a specific time range. For example – last two hrs, last 5 days.
  • Field values – select fields to evaluate a new term appears which is not in previous time window
  • Relative Time Window (Last & to) – Compare the alert for the specific time range to the time window.
  • Includes (or) Excludes Time Window – Include option will include the current “Time window” in “Previous Time Window”, Exclude option will exclude the current “Time window” in “Previous Time Window”
  • Test Query – will test the given filters with request and response.

Refer the following article How to set alerts with rule type "New value" - sample use case


  • Repeated value:

    Trigger when values that were repeatedly seen in the current time window to the previous time window

  • Keyword Filter – Alerts will search the entire index for the matching keyword. Keyword filter functionality looks more similar to Elasticsearch Query String Query pattern.
  • Aggregation Filter
    • Can perform aggregation operations like count, avg, min, max, sum based on the selection of field.
    • Select aggregation type, field, condition (like greater than, lesser than, equal to etc.) and value to apply the condition
  • Query Filter
    • Select field, query condition (Must be, Must not be, Should be), condition (like greater than, lesser than, equal to etc.) and value to apply the condition.
    • Can add multiple conditions by clicking the add icon
  • Group By – Group the result in buckets based on nested aggregation.
  • Order – Sort events in the bucket (Ascending or Descending)
  • Number of Documents – Number of bucket event counts to retrieve.
    For example – Input ‘5’ will retrieve only 5 bucket values matching condition
  • Time window – To generate an alert for a specific time range. For example – last two hrs, last 5 days.
  • Field values – select fields to evaluate a repeated term appears which is in the previous time window
  • Relative Time Window (Last & to) – Compare the alert for the specific time range to the time window.
  • Includes (or) Excludes Time Window – Include option will include the current “Time window” in “Previous Time Window”, Exclude option will exclude the current “Time window” in “Previous Time Window”
  • Test Query – will test the given filters with request and response.

Refer the following article How to set alerts with rule type "Repeated value" - sample use case


  • Flatline

Trigger when event threshold attains dead state i.e threshold < 1

  • Keyword Filter – Alerts will search the entire index for the matching keyword. Keyword filter functionality looks more similar to Elasticsearch Query String Query pattern.
  • Query Filter
    • Select field, query condition (Must be, Must not be, Should be), condition (like greater than, lesser than, equal to etc.) and value to apply the condition.
    • Can add multiple of conditions by clicking the add icon
  • Group By – Group the result in buckets based on nested aggregation.
  • Order – Sort events in the bucket (Ascending or Descending)
  • Number of Documents – Number of bucket event counts to retrieve.
    For example – Input ‘5’ will retrieve only 5 bucket values matching condition
  • Time window – To generate an alert for a specific time range. For example – last two hrs, last 5 days.
  • Test Query – will test the given filters with request and response.

Refer the following article How to set alerts with rule type "Flatline" - sample use case



  • Schedule Details


If you would like to schedule the alerts enable the Schedule section check box and specify the following details, If you don’t want your alert to be generated uncheck the Schedule section checkbox.
1. Set up the schedule for the alert by selecting the Frequency Type as “Hourly”, “Daily”, “Weekly”, “Monthly”, “Yearly” , or  “Custom”

2. Enter the scheduled frequency time in the Schedule Frequency Time section to schedule alert generation.
3. Enter the start time to generate the alert from the Start Time field

  • Alert Actions

Alert can be notified by selecting the Alert action checkbox option. Unselecting the checkbox will disable alert action and triggered alert will not be notified.

                The alert action is of three types.

  1. Send alerts to Email.
  2. Send alerts to a Webhook.
  3. Send alerts to Elasticsearch index

Note: Multiple alert actions can be set for a single alert.

  • Email
    • You can also schedule your alert to be sent an email.
    • Fill the mandatory fields Subject, To, CC, Message for alert Email
    • Check "Include Json" and select fields to be sent from event occurred as JSON attachment.
    • To send a copy of the alert instantly, click the Mail Now button 
    • Click on save button to save the alert.
    • you can also send the other information like alert details, alert condition, alert time window etc. with mail using merge parameters along with email. Follow the below article on How to setup merge parameters for alert action?

 

  • Webhook
    • Select the Webhook ALIAS and template to be pushed to the webhook URL.
    • Selecting predefined template selection will provide messages to be sent to normal webhook URL and slack webhook URL
    • Based on template selection, we can send the notification to normal Webhook URL / Slack Webhook URL
    • Additional data - you can also send other additional data in the form of key/value pairs to the given webhook URL.
    • Include Result - you can send the selected event fields to the given webhook URL.
    • you can also send the other information like alert details, alert condition, alert time window etc. with mail using merge parameters along with webhook. Follow the below article on How to setup merge parameters for alert action? 
    • To send a copy of the alert instantly, click the Webhook Now button.
    • Click a save button to save the alert.

  • Elasticsearch Index
    • Give the Index Pattern in which the events matching the condition will be pushed.
    • Select fields for notification - Allow the specific fields to push into Elasticsearch 
    • Advantage of pushing the events matching the condition has been explained in detail in use case as below, How to visualize alerts in Kibana


 

2. Once the alert is scheduled, you can see the scheduled alert details on the home page as follows, 


3. To edit the scheduled alert. Click “Edit” icon.



4. To delete the scheduled alert, select the alert in the grid and click “Delete” button.



5. To clone the existing alert. Click “Clone” icon and change the alert name and click “clone” button


 


6. To snooze a scheduled alert. Click “snooze” icon and select the time interval for the alert to be snoozed. When you snooze a scheduled alert, Alert will be running but the notifications will no longer be sent via webhook/email until resumed.




7. To resume a snoozed alert, click “Un-snooze” icon


8. To view the alert history for particular alert click "time" icon


9. To drill down the details on last triggered alert click "down arrow" icon 



9. Refer the below article on How to set up alerts for different use cases