| | |
- fetch_all_system_predefined_metrics(ai_core_client: ai_core_sdk.ai_core_v2_client.AICoreV2Client, resource_group: str, error_collector: gen_ai_hub.evaluations.helpers.collector.ValidationCollector)
- Fetches all system-predefined metrics from the GenAI metrics server.
:param ai_core_client: AI Core client instance for API access.
:type ai_core_client: AICoreV2Client
:param resource_group: The resource group name.
:type resource_group: str
:param error_collector: ValidationCollector instance for collecting validation errors.
:type error_collector: ValidationCollector
:return: List of system-predefined metric templates.
:rtype: list
:raises RuntimeError: If fetching system predefined metrics fails.
- get_custom_metric_by_id(metric_id: str, ai_core_client: ai_core_sdk.ai_core_v2_client.AICoreV2Client, resource_group: str, error_collector: gen_ai_hub.evaluations.helpers.collector.ValidationCollector) -> dict | None
- Fetches a specific custom metric by its ID from the GenAI metrics server.
Note: Not using rest_client.get() because it converts camelCase to snake_case,
but the Metric Management Service requires exact camelCase field names like
'additionalProperties' (would become 'additional_properties' if using rest_client).
:param metric_id: The unique ID of the metric to retrieve.
:type metric_id: str
:param ai_core_client: AI Core client instance for API access.
:type ai_core_client: AICoreV2Client
:param resource_group: The resource group name.
:type resource_group: str
:param error_collector: ValidationCollector instance for collecting validation errors.
:type error_collector: ValidationCollector
:return: Parsed JSON response as a dictionary, or None if not found.
:rtype: dict | None
- get_metric_template_info_from_server(metric: str, ai_core_client: ai_core_sdk.ai_core_v2_client.AICoreV2Client, resource_group: str, error_collector: gen_ai_hub.evaluations.helpers.collector.ValidationCollector)
- Retrieves metric template information from the server by metric name.
:param metric: The name of the metric to retrieve.
:type metric: str
:param ai_core_client: AI Core client instance for API access.
:type ai_core_client: AICoreV2Client
:param resource_group: The resource group name.
:type resource_group: str
:param error_collector: ValidationCollector instance for collecting validation errors.
:type error_collector: ValidationCollector
:return: Metric information dictionary, or None if not found.
:rtype: dict | None
- get_metric_version_history(scenario: str, metric_id: str, version: str, ai_core_client: ai_core_sdk.ai_core_v2_client.AICoreV2Client, resource_group: str, error_collector: gen_ai_hub.evaluations.helpers.collector.ValidationCollector) -> dict | None
- Fetches the version history for a specific evaluation metric in a scenario.
Note: Not using rest_client.get() because it converts camelCase to snake_case,
but the Metric Management Service requires exact camelCase field names.
:param scenario: The name of the scenario.
:type scenario: str
:param metric_id: The unique ID of the evaluation metric.
:type metric_id: str
:param version: The version of the metric.
:type version: str
:param ai_core_client: AI Core client instance for API access.
:type ai_core_client: AICoreV2Client
:param resource_group: The resource group name.
:type resource_group: str
:param error_collector: ValidationCollector instance for collecting validation errors.
:type error_collector: ValidationCollector
:return: Parsed JSON response as a dictionary, or None if not found.
:rtype: dict | None
|