Technical preview

After you run an anomaly detection job and view the results, you might find that you need to alter the job configuration or settings.

To further tune your job, you can:

• Create calendars that contain a list of scheduled events for which you do not want to generate anomalies, such as planned system outages or public holidays.
• Create job rules that instruct anomaly detectors to change their behavior based on domain-specific knowledge that you provide. Your job rules can use filter lists, which contain values that you can use to include or exclude events from the machine learning analysis.
• Define custom URLs to make dashboards and other resources readily available when viewing job results.

For more information about tuning your job, refer to the how-to guides in the machine learning documentation. Note that the machine learning documentation may contain details that are not valid when using a fully-managed Elastic project.

Create calendars

Sometimes there are periods when you expect unusual activity to take place, such as bank holidays, "Black Friday", or planned system outages. If you identify these events in advance, no anomalies are generated during that period. The machine learning model is not ill-affected, and you do not receive spurious results.

To create a calendar and add scheduled events:

1. In your Observability project, go to AIOps → Anomaly detection.

2. Click Settings.

3. Under Calendars, click Create.

4. Enter an ID and description for the calendar.

5. Select the jobs you want to apply the calendar to, or turn on Apply calendar to all jobs.

6. Under Events, click New event or click Import events to import events from an iCalendar (ICS) file:

   

   — Click to open this image in fullscreen mode

   A scheduled event must have a start time, end time, and calendar ID. In general, scheduled events are short in duration (typically lasting from a few hours to a day) and occur infrequently. If you have regularly occurring events, such as weekly maintenance periods, you do not need to create scheduled events for these circumstances; they are already handled by the machine learning analytics. If your ICS file contains recurring events, only the first occurrence is imported.

7. When you're done adding events, save your calendar.

You must identify scheduled events before your anomaly detection job analyzes the data for that time period. Machine learning results are not updated retroactively. Bucket results are generated during scheduled events, but they have an anomaly score of zero.

Create job rules and filters

By default, anomaly detection is unsupervised, and the machine learning models have no awareness of the domain of your data. As a result, anomaly detection jobs might identify events that are statistically significant but are uninteresting when you know the larger context.

You can customize anomaly detection by creating custom job rules. Job rules instruct anomaly detectors to change their behavior based on domain-specific knowledge that you provide. When you create a rule, you can specify conditions, scope, and actions. When the conditions of a rule are satisfied, its actions are triggered.

Filters contain a list of values that you can use to include or exclude events from the machine learning analysis. You can use the same filter in multiple anomaly detection jobs.

To create a job rule, first create any filter lists you want to use in the rule, then configure the rule:

1. In your Observability project, go to AIOps → Anomaly detection.

2. (Optional) Create one or more filter lists:

   1. Click Settings.

   2. Under Filter lists, click Create.

   3. Enter the filter list ID. This is the ID you will select when you want to use the filter list in a job rule.

   4. Click Add item and enter one item per line.

   5. Click Add then save the filter list:

      

      — Click to open this image in fullscreen mode

3. Open the job results in the Single Metric Viewer or Anomaly Explorer.

4. From the Actions menu in the Anomalies table, select Configure job rules.

   

   — Click to open this image in fullscreen mode

5. Choose which actions to take when the job rule matches the anomaly: Skip result, Skip model update, or both.

6. Under Conditions, add one or more conditions that must be met for the action to be triggered.

7. Under Scope (if available), add one or more filter lists to limit where the job rule applies.

8. Save the job rule. Note that changes to job rules take effect for new results only. To apply these changes to existing results, you must clone and rerun the job.

Define custom URLs

You can optionally attach one or more custom URLs to your anomaly detection jobs. Links for these URLs will appear in the Actions menu of the anomalies table when viewing job results in the Single Metric Viewer or Anomaly Explorer. Custom URLs can point to dashboards, the Discover app, or external websites. For example, you can define a custom URL that enables users to drill down to the source data from the results set.

To add a custom URL to the Actions menu:

1. In your Observability project, go to AIOps → Anomaly detection.
2. From the Actions menu in the job list, select Edit job.
3. Select the Custom URLs tab, then click Add custom URL.
4. Enter the label to use for the link text.
5. Choose the type of resource you want to link to:

   If you select...	Do this...
   Kibana dashboard	Select the dashboard you want to link to.
   Discover	Select the data view to use.
   Other	Specify the URL for the external website.

6. Click Test to test your link.
7. Click Add, then save your changes.

Now when you view job results in Single Metric Viewer or Anomaly Explorer, the Actions menu includes the custom link:

— Click to open this image in fullscreen mode