mlos_bench.environments.remote package

Remote Tunable Environments for mlos_bench.

class mlos_bench.environments.remote.HostEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: Environment

Remote host environment.

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Execute the run script for this environment.
setup(tunables[, global_config])	Check if host is ready.
status()	Check the status of the benchmark environment.
teardown()	Shut down the Host and release it.

setup(tunables: TunableGroups, global_config: dict | None = None) → bool

Check if host is ready. (Re)provision and start it, if necessary.

Parameters:

tunablesTunableGroups

A collection of groups of tunable parameters along with the parameters’ values. HostEnv tunables are variable parameters that, together with the HostEnv configuration, are sufficient to provision and start a Host.

global_configdict

Free-format dictionary of global parameters of the environment that are not used in the optimization process.

Returns:

is_successbool

True if operation is successful, false otherwise.

teardown() → None

Shut down the Host and release it.

class mlos_bench.environments.remote.NetworkEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: Environment

Network Environment.

Used to model creating a virtual network (and network security group), but no real tuning is expected for it … yet.

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Execute the run script for this environment.
setup(tunables[, global_config])	Check if network is ready.
status()	Check the status of the benchmark environment.
teardown()	Shut down the Network and releases it.

setup(tunables: TunableGroups, global_config: dict | None = None) → bool

Check if network is ready. Provision, if necessary.

Parameters:

tunablesTunableGroups

A collection of groups of tunable parameters along with the parameters’ values. NetworkEnv tunables are variable parameters that, together with the NetworkEnv configuration, are sufficient to provision and start a set of network resources (e.g., virtual network and network security group).

global_configdict

Free-format dictionary of global parameters of the environment that are not used in the optimization process.

Returns:

is_successbool

True if operation is successful, false otherwise.

teardown() → None

Shut down the Network and releases it.

class mlos_bench.environments.remote.OSEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: Environment

OS Level Environment for a host.

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Execute the run script for this environment.
setup(tunables[, global_config])	Check if the host is up and running; boot it, if necessary.
status()	Check the status of the benchmark environment.
teardown()	Clean up and shut down the host without deprovisioning it.

setup(tunables: TunableGroups, global_config: dict | None = None) → bool

Check if the host is up and running; boot it, if necessary.

Parameters:

tunablesTunableGroups

A collection of groups of tunable parameters along with the parameters’ values. HostEnv tunables are variable parameters that, together with the HostEnv configuration, are sufficient to provision and start a host.

global_configdict

Free-format dictionary of global parameters of the environment that are not used in the optimization process.

Returns:

is_successbool

True if operation is successful, false otherwise.

teardown() → None

Clean up and shut down the host without deprovisioning it.

class mlos_bench.environments.remote.RemoteEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: ScriptEnv

Environment to run benchmarks and scripts on a remote host OS.

e.g. Application Environment

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Runs the run script on the remote environment.
setup(tunables[, global_config])	Check if the environment is ready and set up the application and benchmarks on a remote host.
status()	Check the status of the benchmark environment.
teardown()	Clean up and shut down the remote environment.

run() → Tuple[Status, datetime, Dict[str, int | float | str | None] | None]

Runs the run script on the remote environment.

This can be used to, for instance, submit a new experiment to the remote application environment by (re)configuring an application and launching the benchmark, or run a script that collects the results.

Returns:

(status, timestamp, output)(Status, datetime, dict)

3-tuple of (Status, timestamp, output) values, where output is a dict with the results or None if the status is not COMPLETED. If run script is a benchmark, then the score is usually expected to be in the score field.

setup(tunables: TunableGroups, global_config: dict | None = None) → bool

Check if the environment is ready and set up the application and benchmarks on a remote host.

Parameters:

tunablesTunableGroups

A collection of tunable OS and application parameters along with their values. Setting these parameters should not require an OS reboot.

global_configdict

Free-format dictionary of global parameters of the environment that are not used in the optimization process.

Returns:

is_successbool

True if operation is successful, false otherwise.

teardown() → None

Clean up and shut down the remote environment.

class mlos_bench.environments.remote.SaaSEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: Environment

Cloud-based (configurable) SaaS environment.

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Execute the run script for this environment.
setup(tunables[, global_config])	Update the configuration of a remote SaaS instance.
status()	Check the status of the benchmark environment.
teardown()	Tear down the benchmark environment.

setup(tunables: TunableGroups, global_config: dict | None = None) → bool

Update the configuration of a remote SaaS instance.

Parameters:

tunablesTunableGroups

A collection of groups of tunable parameters along with the parameters’ values.

global_configdict

Free-format dictionary of global parameters of the environment that are not used in the optimization process.

Returns:

is_successbool

True if operation is successful, false otherwise.

class mlos_bench.environments.remote.VMEnv(*, name: str, config: dict, global_config: dict | None = None, tunables: TunableGroups | None = None, service: Service | None = None)

Bases: HostEnv

Remote VM/host environment.

Note: this is just a class alias for HostEnv for historical purposes.

Attributes:

parameters

Key/value pairs of all environment parameters (i.e., const_args and tunable_params).

tunable_params

Get the configuration space of the given environment.

Methods

new(*, env_name, class_name, config[, ...])	Factory method for a new environment with a given config.
pprint([indent, level])	Pretty-print the environment configuration.
run()	Execute the run script for this environment.
setup(tunables[, global_config])	Check if host is ready.
status()	Check the status of the benchmark environment.
teardown()	Shut down the Host and release it.

Submodules

• mlos_bench.environments.remote.host_env module
  • HostEnv
    • HostEnv.setup()
    • HostEnv.teardown()
• mlos_bench.environments.remote.network_env module
  • NetworkEnv
    • NetworkEnv.setup()
    • NetworkEnv.teardown()
• mlos_bench.environments.remote.os_env module
  • OSEnv
    • OSEnv.setup()
    • OSEnv.teardown()
• mlos_bench.environments.remote.remote_env module
  • RemoteEnv
    • RemoteEnv.run()
    • RemoteEnv.setup()
    • RemoteEnv.teardown()
• mlos_bench.environments.remote.saas_env module
  • SaaSEnv
    • SaaSEnv.setup()
• mlos_bench.environments.remote.vm_env module
  • VMEnv