Base Evaluator

class BaseEvaluator(method_name: str, description: str)

Bases: ABC

Base class to enforce a common interface for all evaluators.

Provides a common interface and shared functionality for all evaluators in the Brisk framework. Includes access to services, color management, and metadata generation capabilities.

Parameters:

method_namestr

The name of the evaluator

descriptionstr

The description of the evaluator output

Attributes:

method_namestr

The name of the evaluator

descriptionstr

The description of the evaluator output

servicesServiceBundle or None

The global services bundle (set via set_services)

metric_configMetricManager or None

The metric configuration manager (set via set_metric_config)

primary_colorstr

Primary color for plots and visualizations (default: “#1175D5”)

secondary_colorstr

Secondary color for plots and visualizations (default: “#00A878”)

accent_colorstr

Accent color for plots and visualizations (default: “#DE6B48”)

Notes

This abstract base class provides the foundation for all evaluators in the Brisk framework. It ensures consistent access to services and provides common functionality for color management and metadata generation.

Subclasses must implement their specific evaluation logic while following the interface defined by this base class.

Examples

Create a custom evaluator:

>>> class CustomEvaluator(BaseEvaluator):
...     def __init__(self):
...         super().__init__("custom", "Custom evaluation method")
...     
...     def evaluate(self, model, X, y):
...         # Custom evaluation logic
...         pass

property io

Access to the I/O service.

Returns:

IOService

The I/O service for file operations

Raises:

RuntimeError

If services have not been set via set_services()

Notes

This property provides access to the I/O service, which handles file reading, writing, and other I/O operations.

property logger

Access to the logging service.

Returns:

LoggingService

The logging service for logging operations

Raises:

RuntimeError

If services have not been set via set_services()

Notes

This property provides access to the logging service, which handles all logging operations throughout the framework.

property metadata

Access to the metadata service.

Returns:

MetadataService

The metadata service for generating model metadata

Raises:

RuntimeError

If services have not been set via set_services()

Notes

This property provides access to the metadata service, which is used for generating metadata about models and evaluation results.

property reporting

Access to the reporting service.

Returns:

ReportingService

The reporting service for report generation

Raises:

RuntimeError

If services have not been set via set_services()

Notes

This property provides access to the reporting service, which handles report generation and formatting.

set_colors(colors: dict) → None

Set colors from PlotSettings.

Updates the evaluator’s color scheme with values from the plot settings. This allows for consistent theming across all evaluators.

Parameters:

colorsdict

Dictionary with ‘primary_color’, ‘secondary_color’, ‘accent_color’ keys. Missing keys will use current values.

Returns:

None

Notes

This method is typically called during evaluator initialization to apply the global color theme. Only provided color keys are updated; missing keys retain their current values.

set_metric_config(metric_config) → None

Set the metric configuration for this evaluator.

Configures the evaluator with access to the metric configuration manager, which provides access to evaluation metrics.

Parameters:

metric_configMetricManager

The metric configuration manager

Returns:

None

Notes

This method must be called before the evaluator can access metrics. The metric configuration manager provides access to all available evaluation metrics and their configurations.

set_services(services) → None

Set the services bundle for this evaluator.

Configures the evaluator with access to the global services bundle, which provides access to metadata, I/O, utility, logging, and reporting services.

Parameters:

servicesServiceBundle

The global services bundle

Returns:

None

Notes

This method must be called before using any service properties (metadata, io, utility, logger, reporting). The services bundle provides access to all shared functionality in the Brisk framework.

property utility

Access to the utility service.

Returns:

UtilityService

The utility service for common operations

Raises:

RuntimeError

If services have not been set via set_services()

Notes

This property provides access to the utility service, which provides common utility functions and operations used across the framework.