In case the monitoring metrics provided by Aporia are insufficient for your use-case, you can define a metric based on custom logic.
For this monitor type, you can select the following detection methods:
- Absolute Values - The metric value is lower or higher than a specific value.
- Anomaly Detection - Detects anomalies in the value of the metric in the inspected data and its value in a time period before the data was collected.
- Change In Percentage - Detects change in the ratio between the metric value of the inspected data and its value in a time period before the data was collected.
Choose a custom metric that you defined previously, or create a new one
Custom Metric Definition Language
Custom metrics are defined using a syntax that is similar to Python code.
A custom metric is a single expression, which can be any of the following:
- A constant, numeric value (e.g.
- A function call - the currently available functions are
sumfunction must receive a single parameter, which must be a valid field expression
- The field expression must specify a field with a numeric value
- The function returns a sum aggregation on the specified field
prediction_countfunction can receive up to one parameter, which must be a valid segment filter
- If no parameters were passed, the function returns the total number of predictions reported by the model
- If a segment filter was passed, the function returns the number of predictions that fall within the data segment described by the filter.
- A binary operation (
**), in which the operands are constants or function calls (e.g.
sum(features.Annual_Premium) / 100)
A field expression describes a single field (feature, raw_input, prediction value or actual value), and allows operations on the values of that field across all reported predictions, or within a specific data segment
Field expressions have the following format:
- field_category is one of
- The available field categories and fields are based on the schema you defined when creating a model version.
actualscategory uses the fields defined for the
- The segment filter is optional
Data Segment Filters
A data segment filter is an expression that restricts the results of an action to a specific data segment.
Segment filters are boolean expressions - each boolean condition in a segment filter is a comparison between a field and a constant value, and conditions can be combined using
The following comparisons are possible:
- All fields can be checked for missing values using
is not None- for example:
features.Annual_Premium is not None
- Categorical, Boolean and String features can only be compared to constants using the
- Numeric fields can be compared to constants using the
- Datetime Fields are currently not supported
sum(features.Annual_Premium[features.Driving_License == True]) / prediction_count()
prediction_count(raw_inputs.Age <= 35 and raw_inputs.Region_Code == 28) * 3
prediction_count(features.Age > 27) / (sum(features.Annual_Premium) + sum(features.Vintage))