mlos_bench.storage.base_trial_data module¶
Base interface for accessing the stored benchmark trial data.
- class mlos_bench.storage.base_trial_data.TrialData(*, experiment_id: str, trial_id: int, tunable_config_id: int, ts_start: datetime, ts_end: datetime | None, status: Status)¶
Bases:
object
Base interface for accessing the stored experiment benchmark trial data.
A trial is a single run of an experiment with a given configuration (e.g., set of tunable parameters).
- Attributes:
experiment_id
ID of the experiment this trial belongs to.
metadata_df
Retrieve the trials’ metadata parameters as a dataframe.
metadata_dict
Retrieve the trials’ metadata parameters as a dict.
results_df
Retrieve the trials’ results from the storage.
results_dict
Retrieve the trials’ results from the storage as a dict.
status
Status of the trial.
telemetry_df
Retrieve the trials’ telemetry from the storage as a dataframe.
trial_id
ID of the trial.
ts_end
End timestamp of the trial (UTC).
ts_start
Start timestamp of the trial (UTC).
tunable_config
Retrieve the trials’ tunable configuration data from the storage.
tunable_config_id
ID of the (tunable) configuration of the trial.
tunable_config_trial_group
Retrieve the trial’s (tunable) config trial group data from the storage.
- property experiment_id: str¶
ID of the experiment this trial belongs to.
- abstract property metadata_df: DataFrame¶
Retrieve the trials’ metadata parameters as a dataframe.
Note: this corresponds to the Trial object’s “config” property.
- Returns:
- metadatapandas.DataFrame
An optional dataframe with the metadata associated with the trial. It has two str columns, “parameter” and “value”. Returns an empty dataframe if there is no metadata.
- property metadata_dict: dict¶
Retrieve the trials’ metadata parameters as a dict.
Note: this corresponds to the Trial object’s “config” property.
- Returns:
- metadatadict
- abstract property results_df: DataFrame¶
Retrieve the trials’ results from the storage.
- Returns:
- resultspandas.DataFrame
A dataframe with the trial results. It has two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, the dataframe is empty.
- property results_dict: Dict[str, int | float | str | None]¶
Retrieve the trials’ results from the storage as a dict.
- Returns:
- resultsdict
- abstract property telemetry_df: DataFrame¶
Retrieve the trials’ telemetry from the storage as a dataframe.
- Returns:
- configpandas.DataFrame
A dataframe with the trial telemetry, if there is any. It has one datetime column, “ts”, and two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, or there is no telemetry data, the dataframe is empty.
- property trial_id: int¶
ID of the trial.
- property ts_end: datetime | None¶
End timestamp of the trial (UTC).
- property ts_start: datetime¶
Start timestamp of the trial (UTC).
- abstract property tunable_config: TunableConfigData¶
Retrieve the trials’ tunable configuration data from the storage.
Note: this corresponds to the Trial object’s “tunables” property.
- Returns:
- tunable_configTunableConfigData
A TunableConfigData object.
- property tunable_config_id: int¶
ID of the (tunable) configuration of the trial.
- abstract property tunable_config_trial_group: TunableConfigTrialGroupData¶
Retrieve the trial’s (tunable) config trial group data from the storage.