MLOS Package APIs Overview

This is a list of major functions and classes provided by the MLOS packages.

mlos-core API

This is a list of major functions and classes provided by mlos_core.

Optimizers

`OptimizerType`(value[, names, module, ...])	Enumerate supported MlosCore optimizers.
`OptimizerFactory`()	Simple factory class for creating BaseOptimizer-derived objects.
`OptimizerFactory.create`(*, parameter_space, ...)	Create a new optimizer instance, given the parameter space, optimizer type, and potential optimizer options.

BaseOptimizer(*, parameter_space, ...[, ...])

Optimizer abstract base class defining the basic interface.

RandomOptimizer(*, parameter_space, ...[, ...])

Optimizer class that produces random suggestions.

FlamlOptimizer(*, parameter_space, ...[, ...])

Wrapper class for FLAML Optimizer: A fast library for AutoML and tuning.

`BaseBayesianOptimizer`(*, parameter_space, ...)	Abstract base class defining the interface for Bayesian optimization.
`SmacOptimizer`(*, parameter_space, ...[, ...])	Wrapper class for SMAC based Bayesian optimization.

Spaces

Converters

configspace_to_flaml_space(config_space)

Converts a ConfigSpace.ConfigurationSpace to dict.

Space Adapters

`SpaceAdapterType`(value[, names, module, ...])	Enumerate supported MlosCore space adapters.
`SpaceAdapterFactory`()	Simple factory class for creating BaseSpaceAdapter-derived objects.
`SpaceAdapterFactory.create`(*, parameter_space)	Create a new space adapter instance, given the parameter space and potential space adapter options.

BaseSpaceAdapter(*, orig_parameter_space)

SpaceAdapter abstract class defining the basic interface.

IdentityAdapter(*, orig_parameter_space)

Identity (no-op) SpaceAdapter class.

LlamaTuneAdapter(*, orig_parameter_space[, ...])

Implementation of LlamaTune, a set of parameter space transformation techniques, aimed at improving the sample-efficiency of the underlying optimizer.

mlos-bench API

This is a list of major functions and classes provided by mlos_bench.

Main

run.py

The script to run the benchmarks or the optimization loop.

Also available as mlos_bench command line tool.

Note

The are json config examples and json schemas on the main source code repository site.

Benchmark Environments

`Status`(value[, names, module, qualname, ...])	Enum for the status of the benchmark/environment.
`Environment`(*, name, config[, ...])	An abstract base of all benchmark environments.
`CompositeEnv`(*, name, config[, ...])	Composite benchmark environment.
`MockEnv`(*, name, config[, global_config, ...])	Scheduler-side environment to mock the benchmark results.

Local Environments

`LocalEnv`(*, name, config[, global_config, ...])	Scheduler-side Environment that runs scripts locally.
`LocalFileShareEnv`(*, name, config[, ...])	Scheduler-side Environment that runs scripts locally and uploads/downloads data to the shared file storage.

Remote Environments

`RemoteEnv`(*, name, config[, global_config, ...])	Environment to run benchmarks and scripts on a remote host OS.
`OSEnv`(*, name, config[, global_config, ...])	OS Level Environment for a host.
`VMEnv`(*, name, config[, global_config, ...])	Remote VM/host environment.
`HostEnv`(*, name, config[, global_config, ...])	Remote host environment.

Tunable Parameters

`Tunable`(name, config)	A tunable parameter definition and its current value.
`TunableGroups`([config])	A collection of covariant groups of tunable parameters.

Service Mix-ins

`Service`([config, global_config, parent, methods])	An abstract base of all Environment Services and used to build up mix-ins.
`FileShareService`([config, global_config, ...])	An abstract base of all file shares.

ConfigPersistenceService([config, ...])

Collection of methods to deserialize the Environment, Service, and TunableGroups objects.

Local Services

LocalExecService([config, global_config, ...])

Collection of methods to run scripts and commands in an external process on the node acting as the scheduler.

Remote Azure Services

`AzureVMService`([config, global_config, ...])	Helper methods to manage VMs on Azure.
`AzureFileShareService`([config, ...])	Helper methods for interacting with Azure File Share.

Optimizer Adapters

`Optimizer`(tunables, config[, global_config, ...])	An abstract interface between the benchmarking framework and mlos_core optimizers.
`MockOptimizer`(tunables, config[, ...])	Mock optimizer to test the Environment API.
`MlosCoreOptimizer`(tunables, config[, ...])	A wrapper class for the mlos_core optimizers.

Storage

Base Runtime Backends

Storage(config[, global_config, service])

An abstract interface between the benchmarking framework and storage systems (e.g., SQLite or MLFLow).

from_config(config_file[, global_configs])

Create a new storage object from JSON5 config file.

SQL DB Storage Backend

SqlStorage(config[, global_config, service])

An implementation of the Storage interface using SQLAlchemy backend.

Analysis Client Access APIs

ExperimentData(*, experiment_id, ...)

Base interface for accessing the stored experiment benchmark data.

TrialData(*, experiment_id, trial_id, ...)

Base interface for accessing the stored experiment benchmark trial data.

TunableConfigData(*, tunable_config_id)

Base interface for accessing the stored experiment benchmark (tunable) config data.

TunableConfigTrialGroupData(*, ...[, ...])

Base interface for accessing the stored experiment benchmark tunable config trial group data.

mlos-viz API

This is a list of major functions and classes provided by mlos_viz.

MlosVizMethod(value[, names, module, ...])

What method to use for visualizing the experiment results.

plot([exp_data, results_df, objectives, ...])

Plots the results of the experiment.