Data Models Reference

Complete reference for all data models, request/response classes, and enums.

Core Models

This section documents all data models used throughout the HoneyHive SDK.

Generated Models

Request and response models re-exported from honeyhive.models.

HoneyHive Models - Re-exported from auto-generated Pydantic models.

Usage:

from honeyhive.models import CreateConfigurationRequest, CreateDatasetRequest, EventType

class honeyhive.models.AddDatapointsResponse(*, inserted=None, datapoint_ids=None)[source]

Bases: BaseModel

AddDatapointsResponse model

Parameters:

  • inserted (bool | None)

  • datapoint_ids (List[str] | None)

inserted: bool | None
datapoint_ids: List[str] | None
class honeyhive.models.AddDatapointsToDatasetRequest(*, project, data, mapping)[source]

Bases: BaseModel

AddDatapointsToDatasetRequest model

Parameters:
project: str
data: List[Dict[str, Any]]
mapping: DatapointMapping
class honeyhive.models.Configuration(*, _id=None, project, name, env=None, provider, parameters, type=None, user_properties=None, org_id=None, project_id=None, tenant=None, created_at=None, updated_at=None)[source]

Bases: BaseModel

Configuration model

Parameters:

  • _id (str | None)

  • project (str)

  • name (str)

  • env (List[str] | None)

  • provider (str)

  • parameters (ConfigurationParameters)

  • type (str | None)

  • user_properties (Dict[str, Any] | None)

  • org_id (str | None)

  • project_id (str | None)

  • tenant (str | None)

  • created_at (str | None)

  • updated_at (str | None)

id: str | None
project: str
name: str
env: List[str] | None
provider: str
parameters: ConfigurationParameters
type: str | None
user_properties: Dict[str, Any] | None
org_id: str | None
project_id: str | None
tenant: str | None
created_at: str | None
updated_at: str | None
class honeyhive.models.CreateConfigurationRequest(*, project, name, provider, parameters, env=None, user_properties=None)[source]

Bases: BaseModel

CreateConfigurationRequest model

Parameters:

  • project (str)

  • name (str)

  • provider (str)

  • parameters (ConfigurationParameters)

  • env (List[str] | None)

  • user_properties (Dict[str, Any] | None)

project: str
name: str
provider: str
parameters: ConfigurationParameters
env: List[str] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.CreateDatapointRequest(*, project, inputs, history=None, ground_truth=None, linked_event=None, linked_datasets=None, metadata=None)[source]

Bases: BaseModel

CreateDatapointRequest model

Parameters:
project: str
inputs: Dict[str, Any]
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_event: str | None
linked_datasets: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.CreateDatapointResponse(*, result=None)[source]

Bases: BaseModel

CreateDatapointResponse model

Parameters:

result (InsertResult | None)

result: InsertResult | None
class honeyhive.models.CreateDatasetRequest(*, project, name, description=None, type=None, datapoints=None, linked_evals=None, saved=None, pipeline_type=None, metadata=None)[source]

Bases: BaseModel

CreateDatasetRequest model

Parameters:
project: str
name: str
description: str | None
type: str | None
datapoints: List[str] | None
linked_evals: List[str] | None
saved: bool | None
pipeline_type: str | None
metadata: Dict[str, Any] | None
class honeyhive.models.CreateDatasetResponse(*, inserted=None, result=None)[source]

Bases: BaseModel

CreateDatasetResponse model

Parameters:

  • inserted (bool | None)

  • result (InsertResult | None)

inserted: bool | None
result: InsertResult | None
class honeyhive.models.CreateEventBatchRequest(*, events, is_single_session=None, session_properties=None)[source]

Bases: BaseModel

CreateEventBatchRequest model

Parameters:
events: List[PostEventRequest]
is_single_session: bool | None
session_properties: SessionPropertiesBatch | None
class honeyhive.models.PostEventBatchResponse(*, event_ids=None, session_id=None, success=None)[source]

Bases: BaseModel

PostEventBatchResponse model

Parameters:

  • event_ids (List[str] | None)

  • session_id (str | None)

  • success (bool | None)

event_ids: List[str] | None
session_id: str | None
success: bool | None
class honeyhive.models.PostEventResponse(*, event_id=None, success=None)[source]

Bases: BaseModel

PostEventResponse model

Parameters:

  • event_id (str | None)

  • success (bool | None)

event_id: str | None
success: bool | None
class honeyhive.models.CreateModelEvent(*, project, model, provider, messages, response, duration, usage, cost=None, error=None, source=None, event_name=None, hyperparameters=None, template=None, template_inputs=None, tools=None, tool_choice=None, response_format=None)[source]

Bases: BaseModel

CreateModelEvent model

Parameters:
project: str
model: str
provider: str
messages: List[Dict[str, Any]]
response: Dict[str, Any]
duration: float
usage: Dict[str, Any]
cost: float | None
error: str | None
source: str | None
event_name: str | None
hyperparameters: Dict[str, Any] | None
template: List[Dict[str, Any]] | None
template_inputs: Dict[str, Any] | None
tools: List[Dict[str, Any]] | None
tool_choice: str | None
response_format: Dict[str, Any] | None
class honeyhive.models.CreateModelEventBatchRequest(*, model_events, is_single_session=None, session_properties=None)[source]

Bases: BaseModel

CreateModelEventBatchRequest model

Parameters:
model_events: List[CreateModelEvent]
is_single_session: bool | None
session_properties: SessionPropertiesBatch | None
class honeyhive.models.CreateModelEventBatchResponse(*, event_ids=None, success=None)[source]

Bases: BaseModel

CreateModelEventBatchResponse model

Parameters:
event_ids: List[str] | None
success: bool | None
class honeyhive.models.CreateModelEventRequestBody(*, model_event=None)[source]

Bases: BaseModel

CreateModelEventRequestBody model

Parameters:

model_event (CreateModelEvent | None)

model_event: CreateModelEvent | None
class honeyhive.models.CreateProjectRequest(*, name, description=None)[source]

Bases: BaseModel

CreateProjectRequest model

Parameters:

  • name (str)

  • description (str | None)

name: str
description: str | None
class honeyhive.models.PostEventRequestBody(*, event=None)[source]

Bases: BaseModel

PostEventRequestBody model

Parameters:

event (PostEventRequest | None)

event: PostEventRequest | None
class honeyhive.models.CreateToolRequest(*, task=None, project_name=None, name, description=None, parameters, type=None, tool_type=None)[source]

Bases: BaseModel

CreateToolRequest model

Parameters:

  • task (str | None)

  • project_name (str | None)

  • name (str)

  • description (str | None)

  • parameters (Dict[str, Any])

  • type (str | None)

  • tool_type (str | None)

task: str | None
project_name: str | None
name: str
description: str | None
parameters: Dict[str, Any]
type: str | None
tool_type: str | None
class honeyhive.models.CreateToolResponse(*, result=None)[source]

Bases: BaseModel

CreateToolResponse model

Parameters:

result (Dict[str, Any] | None)

result: Dict[str, Any] | None
class honeyhive.models.Datapoint(*, _id=None, tenant=None, project_id=None, created_at=None, updated_at=None, inputs=None, history=None, ground_truth=None, linked_event=None, linked_evals=None, linked_datasets=None, saved=None, type=None, metadata=None)[source]

Bases: BaseModel

Datapoint model

Parameters:
id: str | None
tenant: str | None
project_id: str | None
created_at: str | None
updated_at: str | None
inputs: Dict[str, Any] | None
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_event: str | None
linked_evals: List[str] | None
linked_datasets: List[str] | None
saved: bool | None
type: str | None
metadata: Dict[str, Any] | None
class honeyhive.models.Dataset(*, _id=None, dataset_id=None, org_id=None, project_id=None, tenant=None, project=None, name=None, description=None, type=None, datapoints=None, num_points=None, linked_evals=None, saved=None, pipeline_type=None, created_at=None, updated_at=None, metadata=None)[source]

Bases: BaseModel

Dataset model

Parameters:

  • _id (str | None)

  • dataset_id (str | None)

  • org_id (str | None)

  • project_id (str | None)

  • tenant (str | None)

  • project (str | None)

  • name (str | None)

  • description (str | None)

  • type (str | None)

  • datapoints (List[str] | None)

  • num_points (int | None)

  • linked_evals (List[str] | None)

  • saved (bool | None)

  • pipeline_type (str | None)

  • created_at (str | None)

  • updated_at (str | None)

  • metadata (Dict[str, Any] | None)

id: str | None
dataset_id: str | None
org_id: str | None
project_id: str | None
tenant: str | None
project: str | None
name: str | None
description: str | None
type: str | None
datapoints: List[str] | None
num_points: int | None
linked_evals: List[str] | None
saved: bool | None
pipeline_type: str | None
created_at: str | None
updated_at: str | None
metadata: Dict[str, Any] | None
class honeyhive.models.DeleteDatapointResponse(*, deleted=None)[source]

Bases: BaseModel

DeleteDatapointResponse model

Parameters:

deleted (bool | None)

deleted: bool | None
class honeyhive.models.DeleteExperimentRunResponse(*, id=None, deleted=None)[source]

Bases: BaseModel

DeleteExperimentRunResponse model

Parameters:

  • id (str | None)

  • deleted (bool | None)

id: str | None
deleted: bool | None
class honeyhive.models.EvaluationRun(*, run_id=None, project=None, created_at=None, event_ids=None, dataset_id=None, datapoint_ids=None, results=None, configuration=None, metadata=None, status=None, name=None)[source]

Bases: BaseModel

EvaluationRun model

Parameters:
run_id: str | None
project: str | None
created_at: str | None
event_ids: List[str] | None
dataset_id: str | None
datapoint_ids: List[str] | None
results: Dict[str, Any] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
status: str | None
name: str | None
class honeyhive.models.Event(*, project_id=None, source=None, event_name=None, event_type=None, event_id=None, session_id=None, parent_id=None, children_ids=None, config=None, inputs=None, outputs=None, error=None, start_time=None, end_time=None, duration=None, metadata=None, feedback=None, metrics=None, user_properties=None)[source]

Bases: BaseModel

Event model

Parameters:
project_id: str | None
source: str | None
event_name: str | None
event_type: str | None
event_id: str | None
session_id: str | None
parent_id: str | None
children_ids: List[str] | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
start_time: float | None
end_time: int | None
duration: float | None
metadata: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metrics: Dict[str, Any] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.EventFilter(*, field=None, value=None, operator=None, type=None)[source]

Bases: BaseModel

EventFilter model

Parameters:

  • field (str | None)

  • value (str | None)

  • operator (str | None)

  • type (str | None)

field: str | None
value: str | None
operator: str | None
type: str | None
class honeyhive.models.ExperimentComparisonResponse(*, metrics=None, commonDatapoints=None, event_details=None, old_run=None, new_run=None)[source]

Bases: BaseModel

ExperimentComparisonResponse model

Parameters:
metrics: List[Dict[str, Any]] | None
commonDatapoints: List[str] | None
event_details: List[Dict[str, Any]] | None
old_run: Dict[str, Any] | None
new_run: Dict[str, Any] | None
class honeyhive.models.ExperimentResultResponse(*, status=None, success=None, passed=None, failed=None, metrics=None, datapoints=None)[source]

Bases: BaseModel

ExperimentResultResponse model

Parameters:
status: str | None
success: bool | None
passed: List[str] | None
failed: List[str] | None
metrics: Dict[str, Any] | None
datapoints: List[Dict[str, Any]] | None
class honeyhive.models.GetDatapointResponse(*, datapoint=None)[source]

Bases: BaseModel

GetDatapointResponse model

Parameters:

datapoint (List[Datapoint | None] | None)

datapoint: List[Datapoint | None] | None
class honeyhive.models.GetDatapointsResponse(*, datapoints=None)[source]

Bases: BaseModel

GetDatapointsResponse model

Parameters:

datapoints (List[Datapoint | None] | None)

datapoints: List[Datapoint | None] | None
class honeyhive.models.GetDatasetsResponse(*, testcases=None)[source]

Bases: BaseModel

GetDatasetsResponse model

Parameters:

testcases (List[Dataset | None] | None)

testcases: List[Dataset | None] | None
class honeyhive.models.GetEventsRequest(*, project, filters, dateRange=None, projections=None, limit=None, page=None)[source]

Bases: BaseModel

GetEventsRequest model

Parameters:
project: str
filters: List[EventFilter]
dateRange: Dict[str, Any] | None
projections: List[str] | None
limit: float | None
page: float | None
class honeyhive.models.GetEventsResponse(*, events=None, totalEvents=None)[source]

Bases: BaseModel

GetEventsResponse model

Parameters:
events: List[Event | None] | None
totalEvents: float | None
class honeyhive.models.GetExperimentRunResponse(*, evaluation=None)[source]

Bases: BaseModel

GetExperimentRunResponse model

Parameters:

evaluation (EvaluationRun | None)

evaluation: EvaluationRun | None
class honeyhive.models.GetRunsResponse(*, evaluations=None)[source]

Bases: BaseModel

GetRunsResponse model

Parameters:

evaluations (List[EvaluationRun | None] | None)

evaluations: List[EvaluationRun | None] | None
class honeyhive.models.Metric(*, _id=None, name, criteria, code_snippet=None, prompt=None, task=None, type, description=None, enabled_in_prod=None, needs_ground_truth=None, return_type=None, threshold=None, pass_when=None, event_name=None, event_type=None, model_provider=None, model_name=None, child_metrics=None, sampling_percentage=None, filters=None, created_at=None, updated_at=None)[source]

Bases: BaseModel

Metric model

Parameters:

  • _id (str | None)

  • name (str)

  • criteria (str)

  • code_snippet (str | None)

  • prompt (str | None)

  • task (str | None)

  • type (str)

  • description (str | None)

  • enabled_in_prod (bool | None)

  • needs_ground_truth (bool | None)

  • return_type (str | None)

  • threshold (Dict[str, Any] | None)

  • pass_when (bool | None)

  • event_name (str | None)

  • event_type (str | None)

  • model_provider (str | None)

  • model_name (str | None)

  • child_metrics (List[Dict[str, Any]] | None)

  • sampling_percentage (float | None)

  • filters (Dict[str, Any] | None)

  • created_at (str | None)

  • updated_at (str | None)

id: str | None
name: str
criteria: str
code_snippet: str | None
prompt: str | None
task: str | None
type: str
description: str | None
enabled_in_prod: bool | None
needs_ground_truth: bool | None
return_type: str | None
threshold: Dict[str, Any] | None
pass_when: bool | None
event_name: str | None
event_type: str | None
model_provider: str | None
model_name: str | None
child_metrics: List[Dict[str, Any]] | None
sampling_percentage: float | None
filters: Dict[str, Any] | None
created_at: str | None
updated_at: str | None
class honeyhive.models.PostEventRequest(*, project=None, source=None, event_name=None, event_type, event_id=None, session_id=None, parent_id=None, children_ids=None, config=None, inputs=None, outputs=None, error=None, start_time=None, end_time=None, duration=None, metadata=None, feedback=None, metrics=None, user_properties=None)[source]

Bases: BaseModel

PostEventRequest model

Parameters:
project: str | None
source: str | None
event_name: str | None
event_type: str
event_id: str | None
session_id: str | None
parent_id: str | None
children_ids: List[str] | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
start_time: float | None
end_time: int | None
duration: float | None
metadata: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metrics: Dict[str, Any] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.PostExperimentRunRequest(*, project, name, event_ids=None, dataset_id=None, datapoint_ids=None, configuration=None, metadata=None, status=None)[source]

Bases: BaseModel

PostExperimentRunRequest model

Parameters:
project: str
name: str
event_ids: List[str] | None
dataset_id: str | None
datapoint_ids: List[str] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
status: str | None
class honeyhive.models.PostExperimentRunResponse(*, evaluation=None, run_id=None)[source]

Bases: BaseModel

PostExperimentRunResponse model

Parameters:
evaluation: EvaluationRun | None
run_id: str | None
class honeyhive.models.Project(*, id=None, name, description=None, org_id=None, is_active=None, created_at=None, updated_at=None)[source]

Bases: BaseModel

Project model

Parameters:

  • id (str | None)

  • name (str)

  • description (str | None)

  • org_id (str | None)

  • is_active (bool | None)

  • created_at (str | None)

  • updated_at (str | None)

id: str | None
name: str
description: str | None
org_id: str | None
is_active: bool | None
created_at: str | None
updated_at: str | None
class honeyhive.models.PutExperimentRunRequest(*, event_ids=None, dataset_id=None, datapoint_ids=None, configuration=None, metadata=None, name=None, status=None)[source]

Bases: BaseModel

PutExperimentRunRequest model

Parameters:
event_ids: List[str] | None
dataset_id: str | None
datapoint_ids: List[str] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
name: str | None
status: str | None
class honeyhive.models.PutExperimentRunResponse(*, evaluation=None, warning=None)[source]

Bases: BaseModel

PutExperimentRunResponse model

Parameters:
evaluation: EvaluationRun | None
warning: str | None
class honeyhive.models.SessionPropertiesBatch(*, session_name=None, source=None, session_id=None, config=None, inputs=None, outputs=None, error=None, user_properties=None, metrics=None, feedback=None, metadata=None)[source]

Bases: BaseModel

SessionPropertiesBatch model

Parameters:
session_name: str | None
source: str | None
session_id: str | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
user_properties: Dict[str, Any] | None
metrics: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metadata: Dict[str, Any] | None
class honeyhive.models.SessionStartRequest(*, project, session_name, source, session_id=None, children_ids=None, config=None, inputs=None, outputs=None, error=None, duration=None, user_properties=None, metrics=None, feedback=None, metadata=None, start_time=None, end_time=None)[source]

Bases: BaseModel

SessionStartRequest model

Parameters:
project: str
session_name: str
source: str
session_id: str | None
children_ids: List[str] | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
duration: float | None
user_properties: Dict[str, Any] | None
metrics: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metadata: Dict[str, Any] | None
start_time: float | None
end_time: int | None
class honeyhive.models.StartSessionRequestBody(*, session=None)[source]

Bases: BaseModel

StartSessionRequestBody model

Parameters:

session (SessionStartRequest | None)

session: SessionStartRequest | None
class honeyhive.models.StartSessionResponse(*, session_id=None)[source]

Bases: BaseModel

StartSessionResponse model

Parameters:

session_id (str | None)

session_id: str | None
class honeyhive.models.Tool(*, _id=None, task=None, name=None, description=None, parameters=None, tool_type=None, type=None, org_id=None, project_id=None, tenant=None, created_at=None, updated_at=None)[source]

Bases: BaseModel

Tool model

Parameters:

  • _id (str | None)

  • task (str | None)

  • name (str | None)

  • description (str | None)

  • parameters (Dict[str, Any] | None)

  • tool_type (str | None)

  • type (str | None)

  • org_id (str | None)

  • project_id (str | None)

  • tenant (str | None)

  • created_at (str | None)

  • updated_at (str | None)

id: str | None
task: str | None
name: str | None
description: str | None
parameters: Dict[str, Any] | None
tool_type: str | None
type: str | None
org_id: str | None
project_id: str | None
tenant: str | None
created_at: str | None
updated_at: str | None
class honeyhive.models.UpdateConfigurationRequest(*, project, name, provider, parameters, env=None, type=None, user_properties=None)[source]

Bases: BaseModel

UpdateConfigurationRequest model

Parameters:

  • project (str)

  • name (str)

  • provider (str)

  • parameters (ConfigurationParameters)

  • env (List[str] | None)

  • type (str | None)

  • user_properties (Dict[str, Any] | None)

project: str
name: str
provider: str
parameters: ConfigurationParameters
env: List[str] | None
type: str | None
user_properties: Dict[str, Any] | None
class honeyhive.models.UpdateDatapointRequest(*, inputs=None, history=None, ground_truth=None, linked_evals=None, linked_datasets=None, metadata=None)[source]

Bases: BaseModel

UpdateDatapointRequest model

Parameters:
inputs: Dict[str, Any] | None
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_evals: List[str] | None
linked_datasets: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.UpdateDatasetRequest(*, dataset_id, name=None, description=None, datapoints=None, linked_evals=None, metadata=None)[source]

Bases: BaseModel

UpdateDatasetRequest model

Parameters:
dataset_id: str
name: str | None
description: str | None
datapoints: List[str] | None
linked_evals: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.UpdateEventRequest(*, event_id, metadata=None, feedback=None, metrics=None, outputs=None, config=None, user_properties=None, duration=None, error=None)[source]

Bases: BaseModel

UpdateEventRequest model

Parameters:
event_id: str
metadata: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metrics: Dict[str, Any] | None
outputs: Dict[str, Any] | None
config: Dict[str, Any] | None
user_properties: Dict[str, Any] | None
duration: float | None
error: str | None
class honeyhive.models.UpdateMetricRequest(*, metric_id, criteria=None, name=None, description=None, code_snippet=None, prompt=None, type=None, enabled_in_prod=None, needs_ground_truth=None, return_type=None, threshold=None, pass_when=None, event_name=None, event_type=None, model_provider=None, model_name=None, child_metrics=None)[source]

Bases: BaseModel

UpdateMetricRequest model

Parameters:

  • metric_id (str)

  • criteria (str | None)

  • name (str | None)

  • description (str | None)

  • code_snippet (str | None)

  • prompt (str | None)

  • type (str | None)

  • enabled_in_prod (bool | None)

  • needs_ground_truth (bool | None)

  • return_type (str | None)

  • threshold (Dict[str, Any] | None)

  • pass_when (bool | None)

  • event_name (str | None)

  • event_type (str | None)

  • model_provider (str | None)

  • model_name (str | None)

  • child_metrics (List[Dict[str, Any]] | None)

metric_id: str
criteria: str | None
name: str | None
description: str | None
code_snippet: str | None
prompt: str | None
type: str | None
enabled_in_prod: bool | None
needs_ground_truth: bool | None
return_type: str | None
threshold: Dict[str, Any] | None
pass_when: bool | None
event_name: str | None
event_type: str | None
model_provider: str | None
model_name: str | None
child_metrics: List[Dict[str, Any]] | None
class honeyhive.models.UpdateProjectRequest(*, project_id, name=None, description=None)[source]

Bases: BaseModel

UpdateProjectRequest model

Parameters:

  • project_id (str)

  • name (str | None)

  • description (str | None)

project_id: str
name: str | None
description: str | None
class honeyhive.models.UpdateToolRequest(*, id, name, description=None, parameters)[source]

Bases: BaseModel

UpdateToolRequest model

Parameters:
id: str
name: str
description: str | None
parameters: Dict[str, Any]
honeyhive.models.CreateMetricRequest

alias of Metric

class honeyhive.models.EventType(*values)[source]

Bases: str, Enum

Event types for tracing decorators.

Example:

from honeyhive import trace
from honeyhive.models import EventType

@trace(event_type=EventType.tool)
def my_function():
    pass
model = 'model'
tool = 'tool'
chain = 'chain'
session = 'session'
generic = 'generic'

Note

Key Models Included:

Request Models: - PostExperimentRunRequest - Create experiment runs - CreateDatasetRequest - Create datasets - CreateProjectRequest - Create projects - CreateToolRequest - Create tools - PutExperimentRunRequest, UpdateProjectRequest, UpdateToolRequest - Update operations

Response Models: - PostExperimentRunResponse - Run creation response - Dataset - Dataset information - DeleteExperimentRunResponse - Deletion confirmation - GetExperimentRunResponse, GetRunsResponse - Run retrieval - ExperimentComparisonResponse - Run comparison payload

Supporting Models: - SessionStartRequest, SessionPropertiesBatch - Session management - ExperimentComparisonResponse, ExperimentResultResponse - Experiment results - Configuration - Saved configuration payloads - Metric / CreateMetricRequest - Metrics - EventType - Tracing decorator event type enum

Enums: - EventType - Event categories for traced operations

Configuration Models

ServerURLMixin

class honeyhive.config.models.base.ServerURLMixin[source]

Bases: object

Mixin for server URL configuration with HH_API_URL environment variable support.

This mixin provides the server_url field with proper environment variable loading for classes that need to support custom HoneyHive server URLs. It can be used by both APIClientConfig and TracerConfig to avoid field duplication.

Environment Variables:

HH_API_URL: Custom HoneyHive server URL

Examples

>>> class MyConfig(BaseHoneyHiveConfig, ServerURLMixin):
...     pass
>>> config = MyConfig()  # Loads from HH_API_URL if set
server_url: str = FieldInfo(annotation=NoneType, required=False, default='https://api.honeyhive.ai', alias_priority=2, validation_alias=AliasChoices(choices=['HH_API_URL', 'server_url']), description='Custom HoneyHive server URL', examples=['https://api.honeyhive.ai', 'https://custom.honeyhive.com'])
classmethod validate_server_url(v)[source]

Validate server URL format with graceful degradation.

Parameters:

v (Any) – The server URL to validate

Returns:

The validated and normalized server URL, or default if invalid

Return type:

str

Experiment Models

ExperimentRunStatus

class honeyhive.experiments.models.ExperimentRunStatus(*values)[source]

Bases: str, Enum

Extended status enum with all backend values.

The generated Status enum only includes ‘pending’ and ‘completed’, but the backend supports additional states.

PENDING = 'pending'
COMPLETED = 'completed'
RUNNING = 'running'
FAILED = 'failed'
CANCELLED = 'cancelled'

RunComparisonResult

class honeyhive.experiments.models.RunComparisonResult(*, new_run_id, old_run_id, common_datapoints, new_only_datapoints=0, old_only_datapoints=0, metric_deltas=<factory>)[source]

Bases: BaseModel

Comparison between two experiment runs.

This model represents the delta analysis between a new run and an old run, including metric changes and datapoint differences.

Retrieved from: GET /runs/:new_run_id/compare-with/:old_run_id

Parameters:

  • new_run_id (str)

  • old_run_id (str)

  • common_datapoints (int)

  • new_only_datapoints (int)

  • old_only_datapoints (int)

  • metric_deltas (Dict[str, Any])

new_run_id: str
old_run_id: str
common_datapoints: int
new_only_datapoints: int
old_only_datapoints: int
metric_deltas: Dict[str, Any]
get_metric_delta(metric_name)[source]

Get delta information for a specific metric.

Parameters:

metric_name (str) – Name of the metric

Returns:

Delta information including new_value, old_value, delta, percent_change

Return type:

Dict[str, Any] | None

Example

>>> comparison.get_metric_delta("accuracy")
{
    'new_value': 0.85,
    'old_value': 0.80,
    'delta': 0.05,
    'percent_change': 6.25
}
list_improved_metrics()[source]

List metrics that improved in the new run.

Returns:

List of metric names where improved_count > 0

Return type:

List[str]

list_degraded_metrics()[source]

List metrics that degraded in the new run.

Returns:

List of metric names where degraded_count > 0

Return type:

List[str]

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

ExperimentContext

class honeyhive.experiments.core.ExperimentContext(run_id, dataset_id, project, *, run_name=None, source='evaluation', metadata=None)[source]

Bases: object

Lightweight experiment context for metadata linking.

NOTE: This is NOT a replacement for tracer config. This is just a convenience class for organizing experiment metadata that gets passed to the tracer.

The tracer handles actual metadata propagation when is_evaluation=True.

Parameters:
run_id

Experiment run identifier

dataset_id

Dataset identifier (may have EXT- prefix)

project

Project identifier

source

Source identifier (default: “evaluation”)

metadata

Additional metadata dictionary

Example

>>> context = ExperimentContext(
...     run_id="run-123",
...     dataset_id="EXT-abc",
...     project="my-project"
... )
>>> tracer_config = context.to_tracer_config("dp-1")
>>> tracer_config["is_evaluation"]
True
to_tracer_config(datapoint_id)[source]

Convert to tracer initialization config.

This returns kwargs for HoneyHiveTracer(…) initialization. The tracer will automatically propagate all metadata to spans when is_evaluation=True.

Parameters:

datapoint_id (str) – Datapoint identifier for this execution

Returns:

Dictionary of tracer initialization kwargs

Return type:

Dict[str, Any]

Example

>>> config = context.to_tracer_config("dp-1")
>>> config
{
    'project': 'my-project',
    'is_evaluation': True,
    'run_id': 'run-123',
    'dataset_id': 'EXT-abc',
    'datapoint_id': 'dp-1',
    'source': 'evaluation'
}

See Also