mlos_bench.storage.sql.storage.SqlStorage

class mlos_bench.storage.sql.storage.SqlStorage(config: dict, global_config: dict | None = None, service: Service | None = None)

An implementation of the Storage interface using SQLAlchemy backend.

Attributes:

experiments

Retrieve the experiments’ data from the storage.

Methods

Experiment(*, tunables, experiment_id, ...)	Base interface for storing the results of the experiment.
Trial(*, tunables, experiment_id, trial_id, ...)	Base interface for storing the results of a single run of the experiment.
experiment(*, experiment_id, trial_id, ...)	Create a new experiment in the storage.

__init__(config: dict, global_config: dict | None = None, service: Service | None = None)

Create a new storage object.

Parameters:

configdict

Free-format key/value pairs of configuration parameters.

experiment(*, experiment_id: str, trial_id: int, root_env_config: str, description: str, tunables: TunableGroups, opt_targets: Dict[str, Literal['min', 'max']]) → Experiment

Create a new experiment in the storage.

We need the opt_target parameter here to know what metric to retrieve when we load the data from previous trials. Later we will replace it with full metadata about the optimization direction, multiple objectives, etc.

Parameters:

experiment_idstr

Unique identifier of the experiment.

trial_idint

Starting number of the trial.

root_env_configstr

A path to the root JSON configuration file of the benchmarking environment.

descriptionstr

Human-readable description of the experiment.

tunablesTunableGroups

opt_targetsDict[str, Literal[“min”, “max”]]

Names of metrics we’re optimizing for and the optimization direction {min, max}.

Returns:

experimentStorage.Experiment

An object that allows to update the storage with the results of the experiment and related data.

property experiments: Dict[str, ExperimentData]

Retrieve the experiments’ data from the storage.

Returns:

experimentsDict[str, ExperimentData]

A dictionary of the experiments’ data, keyed by experiment id.