Why monitor metric change?
Monitoring and measuring changes in features / raw inputs metrics allows for early detection of basic problems or changes in the model's input data.
For example - we can monitor and detect a deviation of more than 20% from the average of the feature 'age' from the average the monitor was trained with.
For this monitor type, you can select the following detection methods:
- Absolute Values - The metric value is lower or higher than a specific value.
- Anomaly Detection - Detects anomalies in the value of the metric in the inspected data and its value in a time period before the data was collected.
- Change In Percentage - Detects change in the ratio between the metric value of the inspected data and its value in a time period before the data was collected.
- Compared To Segment - Detects change in the ratio between the metric value of the inspected data and to its value in a different data segment.
- Compared To Training - Detects change in the ratio between the metric value of the inspected data and its value calculated from the reported training set.
Start from choosing the features / raw inputs you'd like to monitor. You can select as many as you want :-)
Next, choose the metric you'd like to monitor from the following options:
- Missing Count
- Standard Deviation
Note that the monitor configuration may vary between the detection method you choose.