cotengra.hyperoptimizers.hyper_neldermead

Hyper optimization using the Nelder-Mead simplex method.

Attributes

_INIT
_REFLECT
_EXPAND
_CONTRACT
_SHRINK

Classes

_NMCore	Minimal Nelder-Mead simplex state machine on raw vectors. Manages n+1
HyperNelderMeadSampler	Nelder-Mead simplex optimizer in raw [-1, 1] space.
NelderMeadOptLib	Hyper-optimization backend using the Nelder-Mead simplex method.

Functions

_clip(x[, lo, hi])	Clip scalar x to the interval [lo, hi].
clamp(xs[, lo, hi])	Clip each element of the list xs to the interval [lo, hi].

Module Contents

cotengra.hyperoptimizers.hyper_neldermead._INIT = 'init'

cotengra.hyperoptimizers.hyper_neldermead._REFLECT = 'reflect'

cotengra.hyperoptimizers.hyper_neldermead._EXPAND = 'expand'

cotengra.hyperoptimizers.hyper_neldermead._CONTRACT = 'contract'

cotengra.hyperoptimizers.hyper_neldermead._SHRINK = 'shrink'

cotengra.hyperoptimizers.hyper_neldermead._clip(x, lo=-1.0, hi=1.0)

Clip scalar x to the interval [lo, hi].

cotengra.hyperoptimizers.hyper_neldermead.clamp(xs, lo=-1.0, hi=1.0)

Clip each element of the list xs to the interval [lo, hi].

class cotengra.hyperoptimizers.hyper_neldermead._NMCore(ndim, center, scales, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, convergence_tol=0.01, psi=None, inject_diameter_fraction=1.5, inject_restart_fraction=0.6)

Minimal Nelder-Mead simplex state machine on raw vectors. Manages n+1 vertices and iteratively improves via reflection, expansion, contraction, and shrink. When the simplex diameter drops below convergence_tol the converged flag is set and no further points are issued.

Parameters:

• ndim (int) – Number of raw dimensions.

• center (list[float]) – Center point for the initial simplex.

• scales (list[float]) – Per-dimension scale for axis-aligned perturbations used to construct the initial simplex around center.

• adaptive (bool) – Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If True then alpha, gamma, rho, and sigma are ignored.

• alpha (float) – Reflection coefficient (standard NM default: 1). If adaptive is True then this is ignored and set to 1.0.

• gamma (float) – Expansion coefficient (standard NM default: 2). If adaptive is True then this is ignored and set to 1 + 2 / ndim.

• rho (float) – Contraction coefficient (standard NM default: 0.5). If adaptive is True then this is ignored and set to 0.75 - 1 / (2 * ndim).

• sigma (float) – Shrink coefficient (standard NM default: 0.5). If adaptive is True then this is ignored and set to 1 - 1 / ndim.

• convergence_tol (float) – When the Chebyshev diameter of the simplex falls below this value the core signals convergence.

• psi (float, optional) – Relative simplex reduction target. When set, the core also converges once the simplex diameter has been reduced below psi times its initialized diameter. This mirrors the internal convergence mode used by Sbplx in NLopt.

• inject_diameter_fraction (float) – Maximum allowed simplex diameter inflation when injecting an external vertex. The candidate’s max distance to any non-worst vertex must be at most inject_diameter_fraction * current_diameter. Use 1.0 (default) to prevent any inflation, > 1.0 to allow some growth, or float('inf') to disable the gate entirely.

• inject_restart_fraction (float) – When injecting an external vertex that is far from the current simplex (beyond the diameter-based gate above), if its score is better than this fraction of the current best score, flag convergence to trigger an early restart focused on the better region. Set to 0.0 to disable this behavior.

ndim

convergence_tol = 0.01

psi = None

inject_diameter_fraction = 1.5

inject_restart_fraction = 0.6

_token_counter = 0

_tell_count = 0

_best_vertex = None

_best_score

_converged = False

_initial_simplex_diameter = None

_vertices = []

_scores = []

_ask_queue = []

_token_map

_results

_state = 'init'

_centroid = None

_reflect_x = None

_reflect_score = None

_contract_inside = False

_pending_injection = None

property converged

True when simplex diameter < convergence_tol.

property best_vertex

property best_score

_centroid_of(vertices)

_reflect(centroid, worst)

_expand(centroid, reflected)

_contract_outside_pt(centroid, reflected)

_contract_inside_pt(centroid, worst)

_shrink_vertex(best, vertex)

_sort_simplex()

_simplex_diameter()

Chebyshev (L-inf) diameter of the current simplex.

_diameter_converged()

_initialize_simplex(center, scales)

_enqueue(x, role)

_try_advance()

_try_advance_init()

inject_vertex(x, score)

Defer replacement of the worst vertex with an external point. The replacement is applied at the next _begin_reflect call so that in-progress NM operations are not disrupted. Only accepted when the simplex is initialized and score improves on the current worst vertex.

Parameters:

• x (list[float]) – Raw coordinate vector (same dimensionality as the simplex).

• score (float) – Objective value at x.

Returns:

accepted – Whether the injection was accepted (deferred).

Return type:

bool

_begin_reflect()

_try_advance_reflect()

_try_advance_expand()

_try_advance_contract()

_begin_shrink()

_try_advance_shrink()

ask()

Return (token, raw_x) or None if blocked.

tell(token, score)

Report the objective value for a previously issued token. Unknown tokens (e.g. from a replaced core) are silently ignored so that late-arriving results from a previous core instance do not cause errors.

class cotengra.hyperoptimizers.hyper_neldermead.HyperNelderMeadSampler(space, seed=None, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, initial_scale=0.6, restart_tol=0.01, restart_scale=0.5, filler_scale=0.3, n_initial=None, explore_prob=0.05, inject_diameter_fraction=1.5, inject_restart_fraction=0.6, exponential_param_power=None)

Nelder-Mead simplex optimizer in raw [-1, 1] space.

Wraps a _NMCore state machine and adds parameter mapping, random filler point generation while the core is blocked, and automatic restarts when the simplex converges.

Parameters:

• space (dict[str, dict]) – The search space for a single contraction method.

• seed (None or int or random.Random, optional) – Random seed.

• adaptive (bool, optional) – Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If True then alpha, gamma, rho, and sigma are ignored.

• alpha (float, optional) – Reflection coefficient.

• gamma (float, optional) – Expansion coefficient.

• rho (float, optional) – Contraction coefficient.

• sigma (float, optional) – Shrink coefficient.

• initial_scale (float, optional) – Scale of the initial simplex around the origin.

• restart_tol (float, optional) – When the simplex diameter falls below this, restart.

• restart_scale (float, optional) – Scale of the restarted simplex around the best point.

• filler_scale (float, optional) – Standard deviation of the gaussian noise used for filler points issued while the core Nelder-Mead routine is blocked.

• n_initial (int or None, optional) – Number of Latin Hypercube Sampled (LHS) warm-up points to evaluate before starting the simplex. The best result seeds the initial simplex center. Default None means 2 * ndim. Set to 0 to disable.

• explore_prob (float, optional) – Probability of issuing a uniform random point instead of the NM-directed point during normal operation. Maintains diversity throughout the search.

• inject_diameter_fraction (float, optional) – Passed to _NMCore — controls the maximum allowed simplex diameter inflation when injecting an external vertex.

• exponential_param_power (float, optional) – Passed through to the shared parameter mapping.

rng

params = []

ndim

adaptive = False

alpha = 1.0

gamma = 2.0

rho = 0.5

sigma = 0.5

initial_scale = 0.6

restart_tol = 0.01

restart_scale = 0.5

filler_scale = 0.3

explore_prob = 0.05

inject_diameter_fraction = 1.5

inject_restart_fraction = 0.6

n_initial = None

_trial_counter = 0

_restart_count = 0

_best_x = None

_best_score

_trial_map

_make_core(center, scales)

_ask_filler()

ask()

Return the next candidate setting. During the LHS warm-up phase, pre-generated Latin Hypercube points are issued one at a time. Once all warm-up results have been collected, the NM simplex is initialized centered on the best warm-up point. During normal operation, with probability explore_prob a uniform random point is returned to maintain diversity. If the NM state machine is blocked waiting for results, a filler point is returned instead.

tell(trial_number, score)

Record a completed trial result.

class cotengra.hyperoptimizers.hyper_neldermead.NelderMeadOptLib

Bases: cotengra.hyperoptimizers.hyper.HyperOptLib

Hyper-optimization backend using the Nelder-Mead simplex method.

setup(methods, space, optimizer=None, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, initial_scale=0.6, restart_tol=0.01, restart_scale=0.5, filler_scale=0.3, n_initial=None, explore_prob=0.05, inject_diameter_fraction=1.5, inject_restart_fraction=0.6, method_exploration=1.0, method_temperature=1.0, exponential_param_power=None, seed=None, **kwargs)

Initialize Nelder-Mead optimizers for each method.

Parameters:

• methods (list[str]) – The contraction methods to optimize over.

• space (dict[str, dict[str, dict]]) – The per-method hyperparameter search space.

• optimizer (HyperOptimizer, optional) – The parent optimizer.

• adaptive (bool, optional) – Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If True then alpha, gamma, rho, and sigma are ignored.

• alpha (float, optional) – Reflection coefficient.

• gamma (float, optional) – Expansion coefficient.

• rho (float, optional) – Contraction coefficient.

• sigma (float, optional) – Shrink coefficient.

• initial_scale (float, optional) – Scale of the initial simplex.

• restart_tol (float, optional) – Simplex diameter threshold for restart.

• restart_scale (float, optional) – Scale of the restarted simplex.

• filler_scale (float, optional) – Gaussian noise scale for filler points.

• n_initial (int or None, optional) – Number of LHS warm-up points per method. Default None means 2 * ndim. Set to 0 to disable.

• explore_prob (float, optional) – Probability of issuing a uniform random exploration point during normal operation.

• inject_diameter_fraction (float, optional) – Passed to _NMCore — controls the maximum allowed simplex diameter inflation when injecting an external vertex.

• method_exploration (float, optional) – Exploration strength for the LCB method chooser.

• method_temperature (float, optional) – Noise temperature for the LCB method chooser.

• exponential_param_power (float, optional) – Passed to the shared parameter mapping.

• seed (None or int, optional) – Random seed.

get_setting()

Choose a method, then request its next setting.

report_result(setting, trial, score)

Report a completed trial back to the method chooser and the method specific Nelder-Mead sampler.