mlos_bench.storage.sql.trial module

Saving and updating benchmark data using SQLAlchemy backend.

class mlos_bench.storage.sql.trial.Trial(*, engine: Engine, schema: DbSchema, tunables: TunableGroups, experiment_id: str, trial_id: int, config_id: int, opt_targets: Dict[str, Literal['min', 'max']], config: Dict[str, Any] | None = None)

Bases: Trial

Store the results of a single run of the experiment in SQL database.

Attributes:

opt_targets

Get the Trial’s optimization targets and directions.

status

Get the status of the current trial.

trial_id

ID of the current trial.

tunable_config_id

ID of the current trial (tunable) configuration.

tunables

Tunable parameters of the current trial.

Methods

config([global_config])	Produce a copy of the global configuration updated with the parameters of the current trial.
update(status, timestamp[, metrics])	Update the storage with the results of the experiment.
update_telemetry(status, timestamp, metrics)	Save the experiment's telemetry data and intermediate status.

update(status: Status, timestamp: datetime, metrics: Dict[str, Any] | None = None) → Dict[str, Any] | None

Update the storage with the results of the experiment.

Parameters:

statusStatus

Status of the experiment run.

timestamp: datetime

Timestamp of the status and metrics.

metricsOptional[Dict[str, Any]]

One or several metrics of the experiment run. Must contain the (float) optimization target if the status is SUCCEEDED.

Returns:

metricsOptional[Dict[str, Any]]

Same as metrics, but always in the dict format.

update_telemetry(status: Status, timestamp: datetime, metrics: List[Tuple[datetime, str, Any]]) → None

Save the experiment’s telemetry data and intermediate status.

Parameters:

statusStatus

Current status of the trial.

timestamp: datetime

Timestamp of the status (but not the metrics).

metricsList[Tuple[datetime, str, Any]]

Telemetry data.