cotengra.hyperoptimizers.hyper

Base hyper optimization functionality.

Attributes

_PATH_FNS
_OPTLIB_FNS
_OPTLIB_DEFAULTS
_HYPER_SEARCH_SPACE
_HYPER_CONSTANTS

Classes

HyperOptLib	Base class for hyper-optimization library interfaces.
TrialSetObjective
TrialConvertTree
TrialTreeMulti
SlicedTrialFn
SimulatedAnnealingTrialFn
ReconfTrialFn
SlicedReconfTrialFn
CompressedReconfTrial
ComputeScore	The final score wrapper, that performs some simple arithmetic on the
HyperOptimizer	A path optimizer that samples a series of contraction trees
ReusableHyperOptimizer	Like HyperOptimizer but it will re-instantiate the optimizer
HyperCompressedOptimizer	A compressed contraction path optimizer that samples a series of ordered
ReusableHyperCompressedOptimizer	Like HyperCompressedOptimizer but it will re-instantiate the
HyperMultiOptimizer	A path optimizer that samples a series of contraction trees

Functions

get_default_hq_methods()
get_default_optlib_eco()	Get the default optimizer favoring speed.
get_default_optlib()	Get the default optimizer balancing quality and speed.
get_hyper_space()
get_hyper_constants()
register_hyper_optlib(name, cls[, defaults])	Register a hyper-optimization library backend.
register_hyper_function(name, ssa_func, space[, constants])	Register a contraction path finder to be used by the hyper-optimizer.
list_hyper_functions()	Return a list of currently registered hyper contraction finders.
base_trial_fn(inputs, output, size_dict, method, **kwargs)
progress_description(best[, info])

Module Contents

cotengra.hyperoptimizers.hyper.get_default_hq_methods()

cotengra.hyperoptimizers.hyper.get_default_optlib_eco()

Get the default optimizer favoring speed.

cotengra.hyperoptimizers.hyper.get_default_optlib()

Get the default optimizer balancing quality and speed.

cotengra.hyperoptimizers.hyper._PATH_FNS

cotengra.hyperoptimizers.hyper._OPTLIB_FNS

cotengra.hyperoptimizers.hyper._OPTLIB_DEFAULTS

cotengra.hyperoptimizers.hyper._HYPER_SEARCH_SPACE

cotengra.hyperoptimizers.hyper._HYPER_CONSTANTS

cotengra.hyperoptimizers.hyper.get_hyper_space()

cotengra.hyperoptimizers.hyper.get_hyper_constants()

class cotengra.hyperoptimizers.hyper.HyperOptLib

Base class for hyper-optimization library interfaces.

Subclasses should implement setup, get_setting, and report_result.

abstractmethod setup(methods, space, optimizer=None, **kwargs)

Initialize the optimizer state.

Parameters:

• methods (list[str]) – The list of contraction methods to optimize over.

• space (dict[str, dict[str, dict]]) – The search space for each method.

• optimizer (HyperOptimizer, optional) – The parent HyperOptimizer instance, for accessing attributes like max_repeats.

• kwargs – Extra options specific to the optimizer library.

abstractmethod get_setting()

Suggest the next setting to trial.

Returns:

setting – Must contain at least {"method": str, "params": dict}. May also include tokens for reporting.

Return type:

dict

abstractmethod report_result(setting, trial, score)

Report the result of a trial.

Parameters:

• setting (dict) – The setting dict returned by get_setting.

• trial (dict) – The trial result dict.

• score (float) – The scalar score for this trial.

cleanup()

Clean up any resources (threads, connections, etc.).

Called at the end of each HyperOptimizer._search() run. The default implementation does nothing.

cotengra.hyperoptimizers.hyper.register_hyper_optlib(name, cls, defaults=None)

Register a hyper-optimization library backend.

Parameters:

• name (str) – The name of the backend.

• cls (type) – A HyperOptLib subclass.

cotengra.hyperoptimizers.hyper.register_hyper_function(name, ssa_func, space, constants=None)

Register a contraction path finder to be used by the hyper-optimizer.

Parameters:

• name (str) – The name to call the method.

• ssa_func (callable) – The raw function that returns a ‘ContractionTree’, with signature (inputs, output, size_dict, **kwargs).

• space (dict[str, dict]) – The space of hyper-parameters to search.

cotengra.hyperoptimizers.hyper.list_hyper_functions()

Return a list of currently registered hyper contraction finders.

cotengra.hyperoptimizers.hyper.base_trial_fn(inputs, output, size_dict, method, **kwargs)

class cotengra.hyperoptimizers.hyper.TrialSetObjective(trial_fn, objective)

trial_fn

objective

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.TrialConvertTree(trial_fn, cls)

trial_fn

cls

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.TrialTreeMulti(trial_fn, varmults, numconfigs)

trial_fn

varmults

numconfigs

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.SlicedTrialFn(trial_fn, **opts)

trial_fn

opts

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.SimulatedAnnealingTrialFn(trial_fn, **opts)

trial_fn

opts

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.ReconfTrialFn(trial_fn, forested=False, parallel=False, **opts)

trial_fn

forested = False

parallel = False

opts

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.SlicedReconfTrialFn(trial_fn, forested=False, parallel=False, **opts)

trial_fn

forested = False

parallel = False

opts

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.CompressedReconfTrial(trial_fn, minimize=None, **opts)

trial_fn

minimize = None

opts

__call__(*args, **kwargs)

class cotengra.hyperoptimizers.hyper.ComputeScore(fn, score_fn, score_compression=0.75, score_smudge=1e-06, on_trial_error='warn', seed=0)

The final score wrapper, that performs some simple arithmetic on the trial score to make it more suitable for hyper-optimization.

fn

score_fn

score_compression = 0.75

score_smudge = 1e-06

on_trial_error = 'warn'

rng

__call__(*args, **kwargs)

cotengra.hyperoptimizers.hyper.progress_description(best, info='concise')

class cotengra.hyperoptimizers.hyper.HyperOptimizer(methods=None, minimize='flops', max_repeats=128, max_time=None, parallel='auto', simulated_annealing_opts='auto', slicing_opts='auto', slicing_reconf_opts='auto', reconf_opts='auto', optlib=None, optlib_opts=None, space=None, score_compression=0.75, on_trial_error='warn', max_training_steps=None, constants=None, progbar=False, **kwargs)

Bases: cotengra.oe.PathOptimizer

A path optimizer that samples a series of contraction trees while optimizing the hyper parameters used to generate them. The drivers specified in methods are used to generate the trial contraction trees according to certain hyper-parameters, and the results are scored according to minimize and fed back to optlib to suggest new parameters.

If any of simulated_annealing_opts, slicing_opts, slicing_reconf_opts, or reconf_opts are supplied, then once a trial tree is generated, it will be modified by the corresponding options (and in the order above) and the flops and size of the trial will be updated to the modified version, before the score is reported.

Parameters:

• methods (None or sequence[str] or str, optional) – Which method(s) to use from list_hyper_functions().

• minimize (str, Objective or callable, optional) – How to score each trial, used to train the optimizer and rank the results. If a custom callable, it should take a trial dict as its argument and return a single float. It is also supplied by default to any relevant refinement stages, such as subtree reconfiguration.

• max_repeats (int, optional) – The maximum number of trial contraction trees to generate. Default: 128.

• max_time (None or float, optional) – The maximum amount of time to run for. Use None for no limit. You can also set an estimated execution ‘rate’ here like 'rate:1e9' that will terminate the search when the estimated FLOPs of the best contraction found divided by the rate is greater than the time spent searching, allowing quick termination on easy contractions.

• parallel ('auto', False, True, int, or distributed.Client) – Whether to parallelize the search.

• simulated_annealing_opts (dict, optional) –

  If supplied, once a trial contraction path is found, refine it using simulated annealing with the given options, and then update the flops and size of the trial with the refined version. Notable options and defaults:

  • tsteps=50: number of temperature steps,

  • target_size: simulteneously slice the tree to this size,

  • tfinal=0.05: final temperature,

  • tstart=2: initial temperature.

  See ContractionTree.simulated_anneal() for full details.

• slicing_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try slicing with the given options, and then update the flops and size of the trial with the sliced versions. Notable options:

  • target_size: slice until reaching this size,

  • target_slices: slice into this many slices.

  See ContractionTree.slice() for full details.

• slicing_reconf_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try slicing interleaved with subtree reconfiguation with the given options, and then update the flops and size of the trial with the sliced and reconfigured versions.

  • target_size: slice until reaching this size,

  • reconf_opts: options passed to the subtree reconfiguration stage, see below.

  See ContractionTree.slice_and_reconfigure() for full details.

• reconf_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try subtree reconfiguation with the given options, and then update the flops and size of the trial with the reconfigured versions. Notable options and defaults are:

  • subtree_size=6: size of subtree to optimally reconfigure,

  • maxiter=”auto”: maximum number of subtree reconfigurations, by default scales with size of contraction (up to 1024),

  • select=’max’: which subtrees to prioritize for reconfiguration.

  See ContractionTree.subtree_reconfigure() for full details.

• optlib ({'optuna', 'cmaes', 'nevergrad', 'sses', 'sbplx', ...}, optional) – Which optimizer to sample and train with.

• optlib_opts – Supplied to the hyper-optimizer library initialization.

• space (dict, optional) – The hyper space to search, see get_hyper_space for the default.

• score_compression (float, optional) – Raise scores to this power in order to compress or accentuate the differences. The lower this is, the more the selector will sample from various optimizers rather than quickly specializing.

• on_trial_error ({'warn', 'raise', 'ignore'}, optional) – What to do if a trial fails. If 'warn' (default), a warning will be printed and the trial will be given a score of inf. If 'raise' the error will be raised. If 'ignore' the trial will be given a score of inf silently.

• max_training_steps (int, optional) – The maximum number of trials to train the optimizer with. Setting this can be helpful when the optimizer itself becomes costly to train (e.g. for Gaussian Processes).

• constants (dict[dict], optional) – A dict mapping method name to a dict of constant parameters to pass to the trial function for that method. Any parameters specified here will override those in the search space.

• progbar (bool, optional) – Show live progress of the best contraction found so far.

• kwargs – Extra options to pass to the optimizer library initialization, on top of those in optlib_opts (which take precedence).

compressed = False

multicontraction = False

max_repeats = 128

_repeats_start = 0

max_time = None

property parallel

method_choices = []

param_choices = []

scores = []

times = []

costs_flops = []

costs_write = []

costs_size = []

simulated_annealing_opts = None

slicing_opts = None

reconf_opts = None

slicing_reconf_opts = None

property minimize

score_compression = 0.75

on_trial_error = 'warn'

best_score

max_training_steps = None

best

trials_since_best = 0

_optimizer

progbar = False

property tree

property path

setup(inputs, output, size_dict)

_maybe_cancel_futures()

_maybe_report_result(setting, trial)

_gen_results(repeats, trial_fn, trial_args)

_get_and_report_next_future()

_gen_results_parallel(repeats, trial_fn, trial_args)

_search(inputs, output, size_dict)

search(inputs, output, size_dict)

Run this optimizer and return the ContractionTree for the best path it finds.

get_tree()

Return the ContractionTree for the best path found.

__call__(inputs, output, size_dict, memory_limit=None)

opt_einsum interface, returns direct path.

get_trials(sort=None)

print_trials(sort=None)

to_df()

Create a single pandas.DataFrame with all trials and scores.

to_dfs_parametrized()

Create a pandas.DataFrame for each method, with all parameters and scores for each trial.

plot_trials

plot_trials_alt

plot_scatter

plot_scatter_alt

plot_parameters_parallel

class cotengra.hyperoptimizers.hyper.ReusableHyperOptimizer(*, directory=None, overwrite=False, hash_method='a', cache_only=False, directory_split='auto', **opt_kwargs)

Bases: cotengra.reusable.ReusableOptimizer

Like HyperOptimizer but it will re-instantiate the optimizer whenever a new contraction is detected, and also cache the paths (and sliced indices) found.

Parameters:

• directory (None, True, or str, optional) – If specified use this directory as a persistent cache. If True auto generate a directory in the current working directory based on the options which are most likely to affect the path (see ReusableHyperOptimizer._get_path_relevant_opts).

• overwrite (bool or 'improved', optional) – If True, the optimizer will always run, overwriting old results in the cache. This can be used to update paths without deleting the whole cache. If 'improved' then only overwrite if the new path is better.

• hash_method ({'a', 'b', ...}, optional) – The method used to hash the contraction tree. The default, 'a', is faster hashwise but doesn’t recognize when indices are permuted.

• cache_only (bool, optional) – If True, the optimizer will only use the cache, and will raise KeyError if a contraction is not found.

• directory_split ("auto" or bool, optional) – If specified, the hash will be split into two parts, the first part will be used as a subdirectory, and the second part will be used as the filename. This is useful for avoiding a very large flat diretory. If “auto” it will check the current cache if any and guess from that.

• opt_kwargs – Supplied to HyperOptimizer.

_get_path_relevant_opts()

Get a frozenset of the options that are most likely to affect the path. These are the options that we use when the directory name is not manually specified.

_get_suboptimizer()

_deconstruct_tree(opt, tree)

_reconstruct_tree(inputs, output, size_dict, con)

class cotengra.hyperoptimizers.hyper.HyperCompressedOptimizer(chi=None, methods=('greedy-compressed', 'greedy-span', 'kahypar-agglom'), minimize='peak-compressed', simulated_annealing_opts='auto', reconf_opts='auto', **kwargs)

Bases: HyperOptimizer

A compressed contraction path optimizer that samples a series of ordered contraction trees while optimizing the hyper parameters used to generate them.

Parameters:

• chi (None or int, optional) – The maximum bond dimension to compress to. If None then use the square of the largest existing dimension. If minimize is specified as a full scoring function, this is ignored.

• methods (None or sequence[str] or str, optional) – Which method(s) to use from list_hyper_functions().

• minimize (str, Objective or callable, optional) – How to score each trial, used to train the optimizer and rank the results. If a custom callable, it should take a trial dict as its argument and return a single float.

• max_repeats (int, optional) – The maximum number of trial contraction trees to generate. Default: 128.

• max_time (None or float, optional) – The maximum amount of time to run for. Use None for no limit. You can also set an estimated execution ‘rate’ here like 'rate:1e9' that will terminate the search when the estimated FLOPs of the best contraction found divided by the rate is greater than the time spent searching, allowing quick termination on easy contractions.

• parallel ('auto', False, True, int, or distributed.Client) – Whether to parallelize the search.

• slicing_opts (dict, optional) – If supplied, once a trial contraction path is found, try slicing with the given options, and then update the flops and size of the trial with the sliced versions.

• slicing_reconf_opts (dict, optional) – If supplied, once a trial contraction path is found, try slicing interleaved with subtree reconfiguation with the given options, and then update the flops and size of the trial with the sliced and reconfigured versions.

• reconf_opts (dict, optional) – If supplied, once a trial contraction path is found, try subtree reconfiguation with the given options, and then update the flops and size of the trial with the reconfigured versions.

• optlib ({'cmaes', 'optuna', 'nevergrad', 'skopt', ...}, optional) – Which optimizer to sample and train with.

• space (dict, optional) – The hyper space to search, see get_hyper_space for the default.

• score_compression (float, optional) – Raise scores to this power in order to compress or accentuate the differences. The lower this is, the more the selector will sample from various optimizers rather than quickly specializing.

• max_training_steps (int, optional) – The maximum number of trials to train the optimizer with. Setting this can be helpful when the optimizer itself becomes costly to train (e.g. for Gaussian Processes).

• progbar (bool, optional) – Show live progress of the best contraction found so far.

• optlib_opts – Supplied to the hyper-optimizer library initialization.

compressed = True

multicontraction = False

class cotengra.hyperoptimizers.hyper.ReusableHyperCompressedOptimizer(chi=None, methods=('greedy-compressed', 'greedy-span', 'kahypar-agglom'), minimize='peak-compressed', **kwargs)

Bases: ReusableHyperOptimizer

Like HyperCompressedOptimizer but it will re-instantiate the optimizer whenever a new contraction is detected, and also cache the paths found.

Parameters:

• chi (None or int, optional) – The maximum bond dimension to compress to. If None then use the square of the largest existing dimension. If minimize is specified as a full scoring function, this is ignored.

• directory (None, True, or str, optional) – If specified use this directory as a persistent cache. If True auto generate a directory in the current working directory based on the options which are most likely to affect the path (see ReusableHyperOptimizer._get_path_relevant_opts).

• overwrite (bool, optional) – If True, the optimizer will always run, overwriting old results in the cache. This can be used to update paths without deleting the whole cache.

• hash_method ({'a', 'b', ...}, optional) – The method used to hash the contraction tree. The default, 'a', is faster hashwise but doesn’t recognize when indices are permuted.

• cache_only (bool, optional) – If True, the optimizer will only use the cache, and will raise KeyError if a contraction is not found.

• opt_kwargs – Supplied to HyperCompressedOptimizer.

_get_suboptimizer()

_deconstruct_tree(opt, tree)

_reconstruct_tree(inputs, output, size_dict, con)

class cotengra.hyperoptimizers.hyper.HyperMultiOptimizer(methods=None, minimize='flops', max_repeats=128, max_time=None, parallel='auto', simulated_annealing_opts='auto', slicing_opts='auto', slicing_reconf_opts='auto', reconf_opts='auto', optlib=None, optlib_opts=None, space=None, score_compression=0.75, on_trial_error='warn', max_training_steps=None, constants=None, progbar=False, **kwargs)

Bases: HyperOptimizer

A path optimizer that samples a series of contraction trees while optimizing the hyper parameters used to generate them. The drivers specified in methods are used to generate the trial contraction trees according to certain hyper-parameters, and the results are scored according to minimize and fed back to optlib to suggest new parameters.

If any of simulated_annealing_opts, slicing_opts, slicing_reconf_opts, or reconf_opts are supplied, then once a trial tree is generated, it will be modified by the corresponding options (and in the order above) and the flops and size of the trial will be updated to the modified version, before the score is reported.

Parameters:

• methods (None or sequence[str] or str, optional) – Which method(s) to use from list_hyper_functions().

• minimize (str, Objective or callable, optional) – How to score each trial, used to train the optimizer and rank the results. If a custom callable, it should take a trial dict as its argument and return a single float. It is also supplied by default to any relevant refinement stages, such as subtree reconfiguration.

• max_repeats (int, optional) – The maximum number of trial contraction trees to generate. Default: 128.

• max_time (None or float, optional) – The maximum amount of time to run for. Use None for no limit. You can also set an estimated execution ‘rate’ here like 'rate:1e9' that will terminate the search when the estimated FLOPs of the best contraction found divided by the rate is greater than the time spent searching, allowing quick termination on easy contractions.

• parallel ('auto', False, True, int, or distributed.Client) – Whether to parallelize the search.

• simulated_annealing_opts (dict, optional) –

  If supplied, once a trial contraction path is found, refine it using simulated annealing with the given options, and then update the flops and size of the trial with the refined version. Notable options and defaults:

  • tsteps=50: number of temperature steps,

  • target_size: simulteneously slice the tree to this size,

  • tfinal=0.05: final temperature,

  • tstart=2: initial temperature.

  See ContractionTree.simulated_anneal() for full details.

• slicing_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try slicing with the given options, and then update the flops and size of the trial with the sliced versions. Notable options:

  • target_size: slice until reaching this size,

  • target_slices: slice into this many slices.

  See ContractionTree.slice() for full details.

• slicing_reconf_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try slicing interleaved with subtree reconfiguation with the given options, and then update the flops and size of the trial with the sliced and reconfigured versions.

  • target_size: slice until reaching this size,

  • reconf_opts: options passed to the subtree reconfiguration stage, see below.

  See ContractionTree.slice_and_reconfigure() for full details.

• reconf_opts (dict, optional) –

  If supplied, once a trial contraction path is found, try subtree reconfiguation with the given options, and then update the flops and size of the trial with the reconfigured versions. Notable options and defaults are:

  • subtree_size=6: size of subtree to optimally reconfigure,

  • maxiter=”auto”: maximum number of subtree reconfigurations, by default scales with size of contraction (up to 1024),

  • select=’max’: which subtrees to prioritize for reconfiguration.

  See ContractionTree.subtree_reconfigure() for full details.

• optlib ({'optuna', 'cmaes', 'nevergrad', 'sses', 'sbplx', ...}, optional) – Which optimizer to sample and train with.

• optlib_opts – Supplied to the hyper-optimizer library initialization.

• space (dict, optional) – The hyper space to search, see get_hyper_space for the default.

• score_compression (float, optional) – Raise scores to this power in order to compress or accentuate the differences. The lower this is, the more the selector will sample from various optimizers rather than quickly specializing.

• on_trial_error ({'warn', 'raise', 'ignore'}, optional) – What to do if a trial fails. If 'warn' (default), a warning will be printed and the trial will be given a score of inf. If 'raise' the error will be raised. If 'ignore' the trial will be given a score of inf silently.

• max_training_steps (int, optional) – The maximum number of trials to train the optimizer with. Setting this can be helpful when the optimizer itself becomes costly to train (e.g. for Gaussian Processes).

• constants (dict[dict], optional) – A dict mapping method name to a dict of constant parameters to pass to the trial function for that method. Any parameters specified here will override those in the search space.

• progbar (bool, optional) – Show live progress of the best contraction found so far.

• kwargs – Extra options to pass to the optimizer library initialization, on top of those in optlib_opts (which take precedence).

compressed = False

multicontraction = True