mlos_bench.services.config_persistence

Helper functions to load, instantiate, and serialize Python objects that encapsulate a benchmark Environment, tunables, Service functions, etc from JSON configuration files and strings.

Classes

ConfigPersistenceService

Collection of methods to deserialize the Environment, Service, and TunableGroups

Module Contents

class mlos_bench.services.config_persistence.ConfigPersistenceService(config: dict[str, Any] | None = None, global_config: dict[str, Any] | None = None, parent: mlos_bench.services.base_service.Service | None = None, methods: dict[str, collections.abc.Callable] | list[collections.abc.Callable] | None = None)

Bases: mlos_bench.services.base_service.Service, mlos_bench.services.types.config_loader_type.SupportsConfigLoading

Collection of methods to deserialize the Environment, Service, and TunableGroups objects.

Create a new instance of config persistence service.

Parameters:

• config (dict) – Free-format dictionary that contains parameters for the service. (E.g., root path for config files, etc.)

• global_config (dict) – Free-format dictionary of global parameters.

• parent (Service) – An optional parent service that can provide mixin functions.

• methods (dict[str, Callable] | list[Callable] | None) – New methods to register with the service.

build_environment(config: dict[str, Any], tunables: mlos_bench.tunables.tunable_groups.TunableGroups, global_config: dict[str, Any] | None = None, parent_args: dict[str, mlos_bench.tunables.tunable.TunableValue] | None = None, service: mlos_bench.services.base_service.Service | None = None) → mlos_bench.environments.base_environment.Environment

Factory method for a new Environment with a given config.

Parameters:

• config (dict) –

  A dictionary with three mandatory fields:

  ”name”: Human-readable string describing the environment; “class”: FQN of a Python class to instantiate; “config”: Free-format dictionary to pass to the constructor.

• tunables (TunableGroups) – A (possibly empty) collection of groups of tunable parameters for all environments.

• global_config (dict) – Global parameters to add to the environment config.

• parent_args (dict[str, TunableValue]) – An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

• service (Service) – An optional service object (e.g., providing methods to deploy or reboot a VM, etc.).

Returns:

env – An instance of the Environment class initialized with config.

Return type:

Environment

build_optimizer(*, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, service: mlos_bench.services.base_service.Service, config: dict[str, Any], global_config: dict[str, Any] | None = None) → mlos_bench.optimizers.base_optimizer.Optimizer

Instantiation of mlos_bench Optimizer that depend on Service and TunableGroups.

Parameters:

• tunables (TunableGroups) – Tunable parameters of the environment. We need them to validate the configurations of merged-in experiments and restored/pending trials.

• service (Service) – An optional service object (e.g., providing methods to load config files, etc.)

• config (dict) – Configuration of the class to instantiate, as loaded from JSON.

• global_config (dict) – Global configuration parameters (optional).

Returns:

inst – A new instance of the Optimizer class.

Return type:

Optimizer

build_scheduler(*, config: dict[str, Any], global_config: dict[str, Any], trial_runners: list[mlos_bench.schedulers.trial_runner.TrialRunner], optimizer: mlos_bench.optimizers.base_optimizer.Optimizer, storage: mlos_bench.storage.base_storage.Storage, root_env_config: str) → mlos_bench.schedulers.base_scheduler.Scheduler

Instantiation of mlos_bench Scheduler.

Parameters:

• config (dict) – Configuration of the class to instantiate, as loaded from JSON.

• global_config (dict) – Global configuration parameters.

• trial_runners (List[TrialRunner]) – The TrialRunners (Environments) to use.

• optimizer (Optimizer) – The optimizer to use.

• storage (Storage) – The storage to use.

• root_env_config (str) – Path to the root environment configuration.

Returns:

inst – A new instance of the Scheduler.

Return type:

Scheduler

build_service(config: dict[str, Any], global_config: dict[str, Any] | None = None, parent: mlos_bench.services.base_service.Service | None = None) → mlos_bench.services.base_service.Service

Factory method for a new service with a given config.

Parameters:

• config (dict) –

  A dictionary with 2 mandatory fields:

  ”class”: FQN of a Python class to instantiate; “config”: Free-format dictionary to pass to the constructor.

• global_config (dict) – Global parameters to add to the service config.

• parent (Service) – An optional reference of the parent service to mix in.

Returns:

svc – An instance of the Service class that is a combination of all services from the list plus the parent mix-in.

Return type:

Service

build_storage(*, service: mlos_bench.services.base_service.Service, config: dict[str, Any], global_config: dict[str, Any] | None = None) → mlos_bench.storage.base_storage.Storage

Instantiation of mlos_bench Storage objects.

Parameters:

• service (Service) – An optional service object (e.g., providing methods to load config files, etc.)

• config (dict) – Configuration of the class to instantiate, as loaded from JSON.

• global_config (dict) – Global configuration parameters (optional).

Returns:

inst – A new instance of the Storage class.

Return type:

Storage

get_config_paths() → list[str]

Gets the list of config paths this service will search for config files.

Return type:

list[str]

load_config(json: str, schema_type: mlos_bench.config.schemas.config_schemas.ConfigSchema | None) → dict[str, Any]

Load JSON config file or JSON string. Search for a file relative to config_paths if the input path is not absolute. This method is exported to be used as a SupportsConfigLoading type Service.

Parameters:

• json (str) – Path to the input config file or a JSON string.

• schema_type (ConfigSchema | None) – The schema type to validate the config against.

Returns:

config – Free-format dictionary that contains the configuration.

Return type:

dict | list[dict]

load_environment(json: str, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, global_config: dict[str, Any] | None = None, parent_args: dict[str, mlos_bench.tunables.tunable.TunableValue] | None = None, service: mlos_bench.services.base_service.Service | None = None) → mlos_bench.environments.base_environment.Environment

Load and build new Environment from the config file or JSON string.

Parameters:

• json (str) – The environment JSON configuration file or JSON string.

• tunables (TunableGroups) – A (possibly empty) collection of tunables to add to the environment.

• global_config (dict) – Global parameters to add to the environment config.

• parent_args (dict[str, TunableValue]) – An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

• service (Service) – An optional reference of the parent service to mix in.

Returns:

env – A new benchmarking environment.

Return type:

Environment

load_environment_list(json: str, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, global_config: dict[str, Any] | None = None, parent_args: dict[str, mlos_bench.tunables.tunable.TunableValue] | None = None, service: mlos_bench.services.base_service.Service | None = None) → list[mlos_bench.environments.base_environment.Environment]

Load and build a list of Environments from the config file or JSON string.

Parameters:

• json (str) – The environment JSON configuration file or a JSON string. Can contain either one environment or a list of environments.

• tunables (TunableGroups) – An (possibly empty) collection of tunables to add to the environment.

• global_config (dict) – Global parameters to add to the environment config.

• service (Service) – An optional reference of the parent service to mix in.

• parent_args (dict[str, TunableValue]) – An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

Returns:

env – A list of new benchmarking environments.

Return type:

list[Environment]

load_services(jsons: collections.abc.Iterable[str], global_config: dict[str, Any] | None = None, parent: mlos_bench.services.base_service.Service | None = None) → mlos_bench.services.base_service.Service

Read the configuration files or JSON strings and bundle all Service methods from those configs into a single Service object.

Notes

Order of the services in the list matters. If multiple Services export the same method, the last one in the list will be used.

Parameters:

• jsons (list of str) – A list of service JSON configuration files or JSON strings.

• global_config (dict) – Global parameters to add to the service config.

• parent (Service) – An optional reference of the parent service to mix in.

Returns:

service – A collection of service methods.

Return type:

Service

load_tunables(jsons: collections.abc.Iterable[str], parent: mlos_bench.tunables.tunable_groups.TunableGroups | None = None) → mlos_bench.tunables.tunable_groups.TunableGroups

Load a collection of tunable parameters from JSON files or strings into the parent TunableGroup.

This helps allow standalone environment configs to reference overlapping tunable groups configs but still allow combining them into a single instance that each environment can reference.

Parameters:

• jsons (list of str) – A list of JSON files or JSON strings to load.

• parent (TunableGroups) – A (possibly empty) collection of tunables to add to the new collection.

Returns:

tunables – The larger collection of tunable parameters.

Return type:

TunableGroups

prepare_class_load(config: dict[str, Any], global_config: dict[str, Any] | None = None, parent_args: dict[str, mlos_bench.tunables.tunable.TunableValue] | None = None) → tuple[str, dict[str, Any]]

Extract the class instantiation parameters from the configuration. Mix-in the global parameters and resolve the local file system paths, where it is required.

Parameters:

• config (dict) – Configuration of the optimizer.

• global_config (dict) – Global configuration parameters (optional).

• parent_args (dict[str, TunableValue]) – An optional reference of the parent CompositeEnv’s const_args used to expand dynamic config parameters from.

Returns:

(class_name, class_config) – Name of the class to instantiate and its configuration.

Return type:

(str, dict)

resolve_path(file_path: str, extra_paths: collections.abc.Iterable[str] | None = None) → str

Resolves and prepends the suitable config_paths to file_path if the latter is not absolute. If config_paths is None or file_path is absolute, return file_path as is.

Parameters:

• file_path (str) – Path to the input config file.

• extra_paths (Iterable[str]) – Additional directories to prepend to the list of config_paths search paths.

Returns:

path – An actual path to the config or script.

Return type:

str

BUILTIN_CONFIG_PATH = ''

A calculated path to the built-in configuration files shipped with the mlos_bench package.

property config_paths: list[str]

Gets the list of config paths this service will search for config files.

Return type:

list[str]