mlos_bench.services.types package

Service types for implementing declaring Service behavior for Environments to use in mlos_bench.

class mlos_bench.services.types.SupportsAuth(*args, **kwargs)

Bases: Protocol[T_co]

Protocol interface for authentication for the cloud services.

Methods

get_access_token()	Get the access token for cloud services.
get_auth_headers()	Get the authorization part of HTTP headers for REST API calls.
get_credential()	Get the credential object for cloud services.

get_access_token() → str

Get the access token for cloud services.

Returns:

access_tokenstr

Access token.

get_auth_headers() → dict

Get the authorization part of HTTP headers for REST API calls.

Returns:

access_headerdict

HTTP header containing the access token.

get_credential() → T_co

Get the credential object for cloud services.

Returns:

credentialT

Cloud-specific credential object.

class mlos_bench.services.types.SupportsConfigLoading(*args, **kwargs)

Bases: Protocol

Protocol interface for helper functions to lookup and load configs.

Methods

build_environment(config, tunables[, ...])	Factory method for a new environment with a given config.
load_config(json_file_name, schema_type)	Load JSON config file.
load_environment_list(json_file_name, tunables)	Load and build a list of environments from the config file.
load_services(json_file_names[, ...])	Read the configuration files and bundle all service methods from those configs into a single Service object.
resolve_path(file_path[, extra_paths])	Prepend the suitable _config_path to path if the latter is not absolute.

build_environment(config: dict, tunables: TunableGroups, global_config: dict | None = None, parent_args: Dict[str, int | float | str | None] | None = None, service: Service | None = None) → Environment

Factory method for a new environment with a given config.

Parameters:

configdict

A dictionary with three mandatory fields:

“name”: Human-readable string describing the environment; “class”: FQN of a Python class to instantiate; “config”: Free-format dictionary to pass to the constructor.

tunablesTunableGroups

A (possibly empty) collection of groups of tunable parameters for all environments.

global_configOptional[dict]

Global parameters to add to the environment config.

parent_argsOptional[Dict[str, TunableValue]]

An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

service: Optional[Service]

An optional service object (e.g., providing methods to deploy or reboot a VM, etc.).

Returns:

envEnvironment

An instance of the Environment class initialized with config.

load_config(json_file_name: str, schema_type: ConfigSchema | None) → dict | List[dict]

Load JSON config file. Search for a file relative to _config_path if the input path is not absolute. This method is exported to be used as a service.

Parameters:

json_file_namestr

Path to the input config file.

schema_typeOptional[ConfigSchema]

The schema type to validate the config against.

Returns:

configUnion[dict, List[dict]]

Free-format dictionary that contains the configuration.

load_environment_list(json_file_name: str, tunables: TunableGroups, global_config: dict | None = None, parent_args: Dict[str, int | float | str | None] | None = None, service: Service | None = None) → List[Environment]

Load and build a list of environments from the config file.

Parameters:

json_file_namestr

The environment JSON configuration file. Can contain either one environment or a list of environments.

tunablesTunableGroups

A (possibly empty) collection of tunables to add to the environment.

global_configOptional[dict]

Global parameters to add to the environment config.

parent_argsOptional[Dict[str, TunableValue]]

An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

serviceOptional[Service]

An optional reference of the parent service to mix in.

Returns:

envList[Environment]

A list of new benchmarking environments.

load_services(json_file_names: Iterable[str], global_config: Dict[str, Any] | None = None, parent: Service | None = None) → Service

Read the configuration files and bundle all service methods from those configs into a single Service object.

Parameters:

json_file_nameslist of str

A list of service JSON configuration files.

global_configdict

Global parameters to add to the service config.

parentService

An optional reference of the parent service to mix in.

Returns:

serviceService

A collection of service methods.

resolve_path(file_path: str, extra_paths: Iterable[str] | None = None) → str

Prepend the suitable _config_path to path if the latter is not absolute. If _config_path is None or path is absolute, return path as is.

Parameters:

file_pathstr

Path to the input config file.

extra_pathsIterable[str]

Additional directories to prepend to the list of search paths.

Returns:

pathstr

An actual path to the config or script.

class mlos_bench.services.types.SupportsFileShareOps(*args, **kwargs)

Bases: Protocol

Protocol interface for file share operations.

Methods

download(params, remote_path, local_path[, ...])	Downloads contents from a remote share path to a local path.
upload(params, local_path, remote_path[, ...])	Uploads contents from a local path to remote share path.

download(params: dict, remote_path: str, local_path: str, recursive: bool = True) → None

Downloads contents from a remote share path to a local path.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of (optional) connection details.

remote_pathstr

Path to download from the remote file share, a file if recursive=False or a directory if recursive=True.

local_pathstr

Path to store the downloaded content to.

recursivebool

If False, ignore the subdirectories; if True (the default), download the entire directory tree.

upload(params: dict, local_path: str, remote_path: str, recursive: bool = True) → None

Uploads contents from a local path to remote share path.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of (optional) connection details.

local_pathstr

Path to the local directory to upload contents from.

remote_pathstr

Path in the remote file share to store the uploaded content to.

recursivebool

If False, ignore the subdirectories; if True (the default), upload the entire directory tree.

class mlos_bench.services.types.SupportsHostProvisioning(*args, **kwargs)

Bases: Protocol

Protocol interface for Host/VM provisioning operations.

Methods

deallocate_host(params)	Deallocates the Host/VM by shutting it down then releasing the compute resources.
deprovision_host(params)	Deprovisions the Host/VM by deleting it.
provision_host(params)	Check if Host/VM is ready.
wait_host_deployment(params, *, is_setup)	Waits for a pending operation on a Host/VM to resolve to SUCCEEDED or FAILED.

deallocate_host(params: dict) → Tuple[Status, dict]

Deallocates the Host/VM by shutting it down then releasing the compute resources.

Note: This can cause the VM to arrive on a new host node when its restarted, which may have different performance characteristics.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters.

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

deprovision_host(params: dict) → Tuple[Status, dict]

Deprovisions the Host/VM by deleting it.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters.

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

provision_host(params: dict) → Tuple[Status, dict]

Check if Host/VM is ready. Deploy a new Host/VM, if necessary.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters. VMEnv tunables are variable parameters that, together with the VMEnv configuration, are sufficient to provision a VM.

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

wait_host_deployment(params: dict, *, is_setup: bool) → Tuple[Status, dict]

Waits for a pending operation on a Host/VM to resolve to SUCCEEDED or FAILED. Return TIMED_OUT when timing out.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters.

is_setupbool

If True, wait for Host/VM being deployed; otherwise, wait for successful deprovisioning.

Returns:

result(Status, dict)

A pair of Status and result. Status is one of {PENDING, SUCCEEDED, FAILED, TIMED_OUT} Result is info on the operation runtime if SUCCEEDED, otherwise {}.

class mlos_bench.services.types.SupportsLocalExec(*args, **kwargs)

Bases: Protocol

Protocol interface for a collection of methods to run scripts and commands in an external process on the node acting as the scheduler.

Can be useful for data processing due to reduced dependency management complications vs the target environment. Used in LocalEnv and provided by LocalExecService.

Methods

local_exec(script_lines[, env, cwd])	Execute the script lines from script_lines in a local process.
temp_dir_context([path])	Create a temp directory or use the provided path.

local_exec(script_lines: Iterable[str], env: Mapping[str, int | float | str | None] | None = None, cwd: str | None = None) → Tuple[int, str, str]

Execute the script lines from script_lines in a local process.

Parameters:

script_linesIterable[str]

Lines of the script to run locally. Treat every line as a separate command to run.

envMapping[str, Union[int, float, str]]

Environment variables (optional).

cwdstr

Work directory to run the script at. If omitted, use temp_dir or create a temporary dir.

Returns:

(return_code, stdout, stderr)(int, str, str)

A 3-tuple of return code, stdout, and stderr of the script process.

temp_dir_context(path: str | None = None) → TemporaryDirectory | nullcontext

Create a temp directory or use the provided path.

Parameters:

pathstr

A path to the temporary directory. Create a new one if None.

Returns:

temp_dir_contextTemporaryDirectory

Temporary directory context to use in the with clause.

class mlos_bench.services.types.SupportsNetworkProvisioning(*args, **kwargs)

Bases: Protocol

Protocol interface for Network provisioning operations.

Methods

deprovision_network(params[, ignore_errors])	Deprovisions the Network by deleting it.
provision_network(params)	Check if Network is ready.
wait_network_deployment(params, *, is_setup)	Waits for a pending operation on a Network to resolve to SUCCEEDED or FAILED.

deprovision_network(params: dict, ignore_errors: bool = True) → Tuple[Status, dict]

Deprovisions the Network by deleting it.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters.

ignore_errorsboolean

Whether to ignore errors (default) encountered during the operation (e.g., due to dependent resources still in use).

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

provision_network(params: dict) → Tuple[Status, dict]

Check if Network is ready. Deploy a new Network, if necessary.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters. NetworkEnv tunables are variable parameters that, together with the NetworkEnv configuration, are sufficient to provision a NetworkEnv.

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

wait_network_deployment(params: dict, *, is_setup: bool) → Tuple[Status, dict]

Waits for a pending operation on a Network to resolve to SUCCEEDED or FAILED. Return TIMED_OUT when timing out.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of tunable parameters.

is_setupbool

If True, wait for Network being deployed; otherwise, wait for successful deprovisioning.

Returns:

result(Status, dict)

A pair of Status and result. Status is one of {PENDING, SUCCEEDED, FAILED, TIMED_OUT} Result is info on the operation runtime if SUCCEEDED, otherwise {}.

class mlos_bench.services.types.SupportsRemoteConfig(*args, **kwargs)

Bases: Protocol

Protocol interface for configuring cloud services.

Methods

configure(config, params)	Update the parameters of a SaaS service in the cloud.
is_config_pending(config)	Check if the configuration of a service requires reboot or restart.

configure(config: Dict[str, Any], params: Dict[str, Any]) → Tuple[Status, dict]

Update the parameters of a SaaS service in the cloud.

Parameters:

configDict[str, Any]

Key/value pairs of configuration parameters (e.g., vmName).

paramsDict[str, Any]

Key/value pairs of the service parameters to update.

Returns:

result(Status, dict={})

A pair of Status and result. The result is always {}. Status is one of {PENDING, SUCCEEDED, FAILED}

is_config_pending(config: Dict[str, Any]) → Tuple[Status, dict]

Check if the configuration of a service requires reboot or restart.

Parameters:

configDict[str, Any]

Key/value pairs of configuration parameters (e.g., vmName).

Returns:

result(Status, dict)

A pair of Status and result. A Boolean field “isConfigPendingRestart” indicates whether the service restart is required. If “isConfigPendingReboot” is set to True, rebooting a VM is necessary. Status is one of {PENDING, TIMED_OUT, SUCCEEDED, FAILED}

class mlos_bench.services.types.SupportsRemoteExec(*args, **kwargs)

Bases: Protocol

Protocol interface for Service types that provide helper functions to run scripts on a remote host OS.

Methods

get_remote_exec_results(config)	Get the results of the asynchronously running command.
remote_exec(script, config, env_params)	Run a command on remote host OS.

get_remote_exec_results(config: dict) → Tuple[Status, dict]

Get the results of the asynchronously running command.

Parameters:

configdict

Flat dictionary of (key, value) pairs of tunable parameters. Must have the “asyncResultsUrl” key to get the results. If the key is not present, return Status.PENDING.

Returns:

result(Status, dict)

A pair of Status and result. Status is one of {PENDING, SUCCEEDED, FAILED, TIMED_OUT}

remote_exec(script: Iterable[str], config: dict, env_params: dict) → Tuple[Status, dict]

Run a command on remote host OS.

Parameters:

scriptIterable[str]

A list of lines to execute as a script on a remote VM.

configdict

Flat dictionary of (key, value) pairs of parameters. They usually come from const_args and tunable_params properties of the Environment.

env_paramsdict

Parameters to pass as shell environment variables into the script. This is usually a subset of config with some possible conversions.

Returns:

result(Status, dict)

A pair of Status and result. Status is one of {PENDING, SUCCEEDED, FAILED}

Submodules

• mlos_bench.services.types.authenticator_type module
  • SupportsAuth
    • SupportsAuth.get_access_token()
    • SupportsAuth.get_auth_headers()
    • SupportsAuth.get_credential()
• mlos_bench.services.types.config_loader_type module
  • SupportsConfigLoading
    • SupportsConfigLoading.build_environment()
    • SupportsConfigLoading.load_config()
    • SupportsConfigLoading.load_environment_list()
    • SupportsConfigLoading.load_services()
    • SupportsConfigLoading.resolve_path()
• mlos_bench.services.types.fileshare_type module
  • SupportsFileShareOps
    • SupportsFileShareOps.download()
    • SupportsFileShareOps.upload()
• mlos_bench.services.types.host_ops_type module
  • SupportsHostOps
    • SupportsHostOps.restart_host()
    • SupportsHostOps.start_host()
    • SupportsHostOps.stop_host()
    • SupportsHostOps.wait_host_operation()
• mlos_bench.services.types.host_provisioner_type module
  • SupportsHostProvisioning
    • SupportsHostProvisioning.deallocate_host()
    • SupportsHostProvisioning.deprovision_host()
    • SupportsHostProvisioning.provision_host()
    • SupportsHostProvisioning.wait_host_deployment()
• mlos_bench.services.types.local_exec_type module
  • SupportsLocalExec
    • SupportsLocalExec.local_exec()
    • SupportsLocalExec.temp_dir_context()
• mlos_bench.services.types.network_provisioner_type module
  • SupportsNetworkProvisioning
    • SupportsNetworkProvisioning.deprovision_network()
    • SupportsNetworkProvisioning.provision_network()
    • SupportsNetworkProvisioning.wait_network_deployment()
• mlos_bench.services.types.os_ops_type module
  • SupportsOSOps
    • SupportsOSOps.reboot()
    • SupportsOSOps.shutdown()
    • SupportsOSOps.wait_os_operation()
• mlos_bench.services.types.remote_config_type module
  • SupportsRemoteConfig
    • SupportsRemoteConfig.configure()
    • SupportsRemoteConfig.is_config_pending()
• mlos_bench.services.types.remote_exec_type module
  • SupportsRemoteExec
    • SupportsRemoteExec.get_remote_exec_results()
    • SupportsRemoteExec.remote_exec()
• mlos_bench.services.types.vm_provisioner_type module
  • SupportsVMOps
    • SupportsVMOps.vm_deprovision()
    • SupportsVMOps.vm_provision()
    • SupportsVMOps.vm_restart()
    • SupportsVMOps.vm_start()
    • SupportsVMOps.vm_stop()
    • SupportsVMOps.wait_vm_deployment()
    • SupportsVMOps.wait_vm_operation()