mlos_bench.environments.composite_env module¶
Composite benchmark environment.
- class mlos_bench.environments.composite_env.CompositeEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)¶
Bases:
Environment
Composite benchmark environment.
- Attributes:
children
Return the list of child environments.
parameters
Key/value pairs of all environment parameters (i.e., const_args and tunable_params).
tunable_params
Get the configuration space of the given environment.
Methods
new
(*, env_name, class_name, config[, ...])Factory method for a new environment with a given config.
pprint
([indent, level])Pretty-print the environment and its children.
run
()Submit a new experiment to the environment.
setup
(tunables[, global_config])Set up the children environments.
status
()Check the status of the benchmark environment.
teardown
()Tear down the children environments.
- property children: List[Environment]¶
Return the list of child environments.
- pprint(indent: int = 4, level: int = 0) str ¶
Pretty-print the environment and its children.
- Parameters:
- indentint
Number of spaces to indent the output at each level. Default is 4.
- levelint
Current level of indentation. Default is 0.
- Returns:
- prettystr
Pretty-printed environment configuration.
- run() Tuple[Status, datetime, Dict[str, int | float | str | None] | None] ¶
Submit a new experiment to the environment. Return the result of the last child environment if successful, or the status of the last failed environment otherwise.
- Returns:
- (status, timestamp, output)(Status, datetime, dict)
3-tuple of (Status, timestamp, output) values, where output is a dict with the results or None if the status is not COMPLETED. If run script is a benchmark, then the score is usually expected to be in the score field.
- setup(tunables: TunableGroups, global_config: dict | None = None) bool ¶
Set up the children environments.
- Parameters:
- tunablesTunableGroups
A collection of tunable parameters along with their values.
- global_configdict
Free-format dictionary of global parameters of the environment that are not used in the optimization process.
- Returns:
- is_successbool
True if all children setup() operations are successful, false otherwise.
- status() Tuple[Status, datetime, List[Tuple[datetime, str, Any]]] ¶
Check the status of the benchmark environment.
- Returns:
- (benchmark_status, timestamp, telemetry)(Status, datetime, list)
3-tuple of (benchmark status, timestamp, telemetry) values. timestamp is UTC time stamp of the status; it’s current time by default. telemetry is a list (maybe empty) of (timestamp, metric, value) triplets.
- teardown() None ¶
Tear down the children environments.
This method is idempotent, i.e., calling it several times is equivalent to a single call. The environments are being torn down in the reverse order.