mlos_bench.services package

Services for implementing Environments for mlos_bench.

class mlos_bench.services.FileShareService(config: Dict[str, Any] | None = None, global_config: Dict[str, Any] | None = None, parent: Service | None = None, methods: Dict[str, Callable] | List[Callable] | None = None)

Bases: Service, SupportsFileShareOps

An abstract base of all file shares.

Attributes:

config_loader_service

Return a config loader service.

Methods

download(params, remote_path, local_path[, ...])	Downloads contents from a remote share path to a local path.
export()	Return a dictionary of functions available in this service.
merge_methods(ext_methods, local_methods)	Merge methods from the external caller with the local ones.
new(class_name[, config, global_config, parent])	Factory method for a new service with a given config.
pprint()	Produce a human-readable string listing all public methods of the service.
register(services)	Register new mix-in services.
upload(params, local_path, remote_path[, ...])	Uploads contents from a local path to remote share path.

abstract download(params: dict, remote_path: str, local_path: str, recursive: bool = True) → None

Downloads contents from a remote share path to a local path.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of (optional) connection details.

remote_pathstr

Path to download from the remote file share, a file if recursive=False or a directory if recursive=True.

local_pathstr

Path to store the downloaded content to.

recursivebool

If False, ignore the subdirectories; if True (the default), download the entire directory tree.

abstract upload(params: dict, local_path: str, remote_path: str, recursive: bool = True) → None

Uploads contents from a local path to remote share path.

Parameters:

paramsdict

Flat dictionary of (key, value) pairs of (optional) connection details.

local_pathstr

Path to the local directory to upload contents from.

remote_pathstr

Path in the remote file share to store the uploaded content to.

recursivebool

If False, ignore the subdirectories; if True (the default), upload the entire directory tree.

class mlos_bench.services.LocalExecService(config: Dict[str, Any] | None = None, global_config: Dict[str, Any] | None = None, parent: Service | None = None, methods: Dict[str, Callable] | List[Callable] | None = None)

Bases: TempDirContextService, SupportsLocalExec

Collection of methods to run scripts and commands in an external process on the node acting as the scheduler.

Can be useful for data processing due to reduced dependency management complications vs the target environment.

Attributes:

config_loader_service

Return a config loader service.

Methods

export()	Return a dictionary of functions available in this service.
local_exec(script_lines[, env, cwd])	Execute the script lines from script_lines in a local process.
merge_methods(ext_methods, local_methods)	Merge methods from the external caller with the local ones.
new(class_name[, config, global_config, parent])	Factory method for a new service with a given config.
pprint()	Produce a human-readable string listing all public methods of the service.
register(services)	Register new mix-in services.
temp_dir_context([path])	Create a temp directory or use the provided path.

local_exec(script_lines: Iterable[str], env: Mapping[str, TunableValue] | None = None, cwd: str | None = None) → Tuple[int, str, str]

Execute the script lines from script_lines in a local process.

Parameters:

script_linesIterable[str]

Lines of the script to run locally. Treat every line as a separate command to run.

envMapping[str, Union[int, float, str]]

Environment variables (optional).

cwdstr

Work directory to run the script at. If omitted, use temp_dir or create a temporary dir.

Returns:

(return_code, stdout, stderr)(int, str, str)

A 3-tuple of return code, stdout, and stderr of the script process.

class mlos_bench.services.Service(config: Dict[str, Any] | None = None, global_config: Dict[str, Any] | None = None, parent: Service | None = None, methods: Dict[str, Callable] | List[Callable] | None = None)

Bases: object

An abstract base of all Environment Services and used to build up mix-ins.

Attributes:

config_loader_service

Return a config loader service.

Methods

export()	Return a dictionary of functions available in this service.
merge_methods(ext_methods, local_methods)	Merge methods from the external caller with the local ones.
new(class_name[, config, global_config, parent])	Factory method for a new service with a given config.
pprint()	Produce a human-readable string listing all public methods of the service.
register(services)	Register new mix-in services.

property config_loader_service: SupportsConfigLoading

Return a config loader service.

Returns:

config_loader_serviceSupportsConfigLoading

A config loader service.

export() → Dict[str, Callable]

Return a dictionary of functions available in this service.

Returns:

servicesdict

A dictionary of string -> function pairs.

static merge_methods(ext_methods: Dict[str, Callable] | List[Callable] | None, local_methods: Dict[str, Callable] | List[Callable]) → Dict[str, Callable]

Merge methods from the external caller with the local ones.

This function is usually called by the derived class constructor just before invoking the constructor of the base class.

classmethod new(class_name: str, config: Dict[str, Any] | None = None, global_config: Dict[str, Any] | None = None, parent: Service | None = None) → Service

Factory method for a new service with a given config.

Parameters:

class_name: str

FQN of a Python class to instantiate, e.g., “mlos_bench.services.remote.azure.AzureVMService”. Must be derived from the Service class.

configdict

Free-format dictionary that contains the service configuration. It will be passed as a constructor parameter of the class specified by class_name.

global_configdict

Free-format dictionary of global parameters.

parentService

A parent service that can provide mixin functions.

Returns:

svcService

An instance of the Service class initialized with config.

pprint() → str

Produce a human-readable string listing all public methods of the service.

register(services: Dict[str, Callable] | List[Callable]) → None

Register new mix-in services.

Parameters:

servicesdict or list

A dictionary of string -> function pairs.

Subpackages

• mlos_bench.services.local package
  • LocalExecService
    • LocalExecService.local_exec()
  • Submodules
    • mlos_bench.services.local.local_exec module
      • LocalExecService
      • split_cmdline()
    • mlos_bench.services.local.temp_dir_context module
      • TempDirContextService
• mlos_bench.services.remote package
  • Subpackages
    • mlos_bench.services.remote.azure package
      • AzureAuthService
      • AzureFileShareService
      • AzureNetworkService
      • AzureSaaSConfigService
      • AzureVMService
      • Submodules
    • mlos_bench.services.remote.ssh package
      • SshFileShareService
      • SshHostService
      • Submodules
• mlos_bench.services.types package
  • SupportsAuth
    • SupportsAuth.get_access_token()
    • SupportsAuth.get_auth_headers()
    • SupportsAuth.get_credential()
  • SupportsConfigLoading
    • SupportsConfigLoading.build_environment()
    • SupportsConfigLoading.load_config()
    • SupportsConfigLoading.load_environment_list()
    • SupportsConfigLoading.load_services()
    • SupportsConfigLoading.resolve_path()
  • SupportsFileShareOps
    • SupportsFileShareOps.download()
    • SupportsFileShareOps.upload()
  • SupportsHostProvisioning
    • SupportsHostProvisioning.deallocate_host()
    • SupportsHostProvisioning.deprovision_host()
    • SupportsHostProvisioning.provision_host()
    • SupportsHostProvisioning.wait_host_deployment()
  • SupportsLocalExec
    • SupportsLocalExec.local_exec()
    • SupportsLocalExec.temp_dir_context()
  • SupportsNetworkProvisioning
    • SupportsNetworkProvisioning.deprovision_network()
    • SupportsNetworkProvisioning.provision_network()
    • SupportsNetworkProvisioning.wait_network_deployment()
  • SupportsRemoteConfig
    • SupportsRemoteConfig.configure()
    • SupportsRemoteConfig.is_config_pending()
  • SupportsRemoteExec
    • SupportsRemoteExec.get_remote_exec_results()
    • SupportsRemoteExec.remote_exec()
  • Submodules
    • mlos_bench.services.types.authenticator_type module
      • SupportsAuth
    • mlos_bench.services.types.config_loader_type module
      • SupportsConfigLoading
    • mlos_bench.services.types.fileshare_type module
      • SupportsFileShareOps
    • mlos_bench.services.types.host_ops_type module
      • SupportsHostOps
    • mlos_bench.services.types.host_provisioner_type module
      • SupportsHostProvisioning
    • mlos_bench.services.types.local_exec_type module
      • SupportsLocalExec
    • mlos_bench.services.types.network_provisioner_type module
      • SupportsNetworkProvisioning
    • mlos_bench.services.types.os_ops_type module
      • SupportsOSOps
    • mlos_bench.services.types.remote_config_type module
      • SupportsRemoteConfig
    • mlos_bench.services.types.remote_exec_type module
      • SupportsRemoteExec
    • mlos_bench.services.types.vm_provisioner_type module
      • SupportsVMOps

Submodules

• mlos_bench.services.base_fileshare module
  • FileShareService
    • FileShareService.download()
    • FileShareService.upload()
• mlos_bench.services.base_service module
  • Service
    • Service.config_loader_service
    • Service.export()
    • Service.merge_methods()
    • Service.new()
    • Service.pprint()
    • Service.register()
• mlos_bench.services.config_persistence module
  • ConfigPersistenceService
    • ConfigPersistenceService.BUILTIN_CONFIG_PATH
    • ConfigPersistenceService.build_environment()
    • ConfigPersistenceService.build_optimizer()
    • ConfigPersistenceService.build_scheduler()
    • ConfigPersistenceService.build_service()
    • ConfigPersistenceService.build_storage()
    • ConfigPersistenceService.config_paths
    • ConfigPersistenceService.load_config()
    • ConfigPersistenceService.load_environment()
    • ConfigPersistenceService.load_environment_list()
    • ConfigPersistenceService.load_services()
    • ConfigPersistenceService.prepare_class_load()
    • ConfigPersistenceService.resolve_path()