Why monitor model activity?
In many cases, the number of predictions of a model is within a predictable range. Identifying deviations from the range can indicate on underlying problems, anomalous events, or an ongoing trend that is worth noting.
Causes of change in the number of predictions include:
- Natural increase in model invocations
- Serving environment fault
- Malicious attempt to analyse model behaviour
For this monitor type, you can select the following detection methods:
- Absolute Values - The prediction count is lower or higher than a specific value.
- Anomaly Detection - Detects anomalies in the prediction count of the inspected data, compared to the prediction count in a time period before the data was collected.
- Change In Percentage - Detects change in the ratio between the prediction count of the inspected data and the prediction count in a time period before the data was collected.
Note that the monitor configuration may vary between the detection method you choose.