cotengra.hyperoptimizers.hyper_neldermead ========================================= .. py:module:: cotengra.hyperoptimizers.hyper_neldermead .. autoapi-nested-parse:: Hyper optimization using the Nelder-Mead simplex method. Attributes ---------- .. autoapisummary:: cotengra.hyperoptimizers.hyper_neldermead._INIT cotengra.hyperoptimizers.hyper_neldermead._REFLECT cotengra.hyperoptimizers.hyper_neldermead._EXPAND cotengra.hyperoptimizers.hyper_neldermead._CONTRACT cotengra.hyperoptimizers.hyper_neldermead._SHRINK Classes ------- .. autoapisummary:: cotengra.hyperoptimizers.hyper_neldermead._NMCore cotengra.hyperoptimizers.hyper_neldermead.HyperNelderMeadSampler cotengra.hyperoptimizers.hyper_neldermead.NelderMeadOptLib Functions --------- .. autoapisummary:: cotengra.hyperoptimizers.hyper_neldermead._clip cotengra.hyperoptimizers.hyper_neldermead.clamp Module Contents --------------- .. py:data:: _INIT :value: 'init' .. py:data:: _REFLECT :value: 'reflect' .. py:data:: _EXPAND :value: 'expand' .. py:data:: _CONTRACT :value: 'contract' .. py:data:: _SHRINK :value: 'shrink' .. py:function:: _clip(x, lo=-1.0, hi=1.0) Clip scalar ``x`` to the interval ``[lo, hi]``. .. py:function:: clamp(xs, lo=-1.0, hi=1.0) Clip each element of the list ``xs`` to the interval ``[lo, hi]``. .. py:class:: _NMCore(ndim, center, scales, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, convergence_tol=0.01, psi=None, inject_diameter_fraction=1.5, inject_restart_fraction=0.6) Minimal Nelder-Mead simplex state machine on raw vectors. Manages n+1 vertices and iteratively improves via reflection, expansion, contraction, and shrink. When the simplex diameter drops below ``convergence_tol`` the ``converged`` flag is set and no further points are issued. :param ndim: Number of raw dimensions. :type ndim: int :param center: Center point for the initial simplex. :type center: list[float] :param scales: Per-dimension scale for axis-aligned perturbations used to construct the initial simplex around ``center``. :type scales: list[float] :param adaptive: Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If `True` then `alpha`, `gamma`, `rho`, and `sigma` are ignored. :type adaptive: bool :param alpha: Reflection coefficient (standard NM default: 1). If ``adaptive`` is `True` then this is ignored and set to 1.0. :type alpha: float :param gamma: Expansion coefficient (standard NM default: 2). If ``adaptive`` is `True` then this is ignored and set to ``1 + 2 / ndim``. :type gamma: float :param rho: Contraction coefficient (standard NM default: 0.5). If ``adaptive`` is `True` then this is ignored and set to ``0.75 - 1 / (2 * ndim)``. :type rho: float :param sigma: Shrink coefficient (standard NM default: 0.5). If ``adaptive`` is `True` then this is ignored and set to ``1 - 1 / ndim``. :type sigma: float :param convergence_tol: When the Chebyshev diameter of the simplex falls below this value the core signals convergence. :type convergence_tol: float :param psi: Relative simplex reduction target. When set, the core also converges once the simplex diameter has been reduced below ``psi`` times its initialized diameter. This mirrors the internal convergence mode used by Sbplx in NLopt. :type psi: float, optional :param inject_diameter_fraction: Maximum allowed simplex diameter inflation when injecting an external vertex. The candidate's max distance to any non-worst vertex must be at most ``inject_diameter_fraction * current_diameter``. Use ``1.0`` (default) to prevent any inflation, ``> 1.0`` to allow some growth, or ``float('inf')`` to disable the gate entirely. :type inject_diameter_fraction: float :param inject_restart_fraction: When injecting an external vertex that is far from the current simplex (beyond the diameter-based gate above), if its score is better than this fraction of the current best score, flag convergence to trigger an early restart focused on the better region. Set to ``0.0`` to disable this behavior. :type inject_restart_fraction: float .. py:attribute:: ndim .. py:attribute:: convergence_tol :value: 0.01 .. py:attribute:: psi :value: None .. py:attribute:: inject_diameter_fraction :value: 1.5 .. py:attribute:: inject_restart_fraction :value: 0.6 .. py:attribute:: _token_counter :value: 0 .. py:attribute:: _tell_count :value: 0 .. py:attribute:: _best_vertex :value: None .. py:attribute:: _best_score .. py:attribute:: _converged :value: False .. py:attribute:: _initial_simplex_diameter :value: None .. py:attribute:: _vertices :value: [] .. py:attribute:: _scores :value: [] .. py:attribute:: _ask_queue :value: [] .. py:attribute:: _token_map .. py:attribute:: _results .. py:attribute:: _state :value: 'init' .. py:attribute:: _centroid :value: None .. py:attribute:: _reflect_x :value: None .. py:attribute:: _reflect_score :value: None .. py:attribute:: _contract_inside :value: False .. py:attribute:: _pending_injection :value: None .. py:property:: converged True when simplex diameter < convergence_tol. .. py:property:: best_vertex .. py:property:: best_score .. py:method:: _centroid_of(vertices) .. py:method:: _reflect(centroid, worst) .. py:method:: _expand(centroid, reflected) .. py:method:: _contract_outside_pt(centroid, reflected) .. py:method:: _contract_inside_pt(centroid, worst) .. py:method:: _shrink_vertex(best, vertex) .. py:method:: _sort_simplex() .. py:method:: _simplex_diameter() Chebyshev (L-inf) diameter of the current simplex. .. py:method:: _diameter_converged() .. py:method:: _initialize_simplex(center, scales) .. py:method:: _enqueue(x, role) .. py:method:: _try_advance() .. py:method:: _try_advance_init() .. py:method:: inject_vertex(x, score) Defer replacement of the worst vertex with an external point. The replacement is applied at the next ``_begin_reflect`` call so that in-progress NM operations are not disrupted. Only accepted when the simplex is initialized and ``score`` improves on the current worst vertex. :param x: Raw coordinate vector (same dimensionality as the simplex). :type x: list[float] :param score: Objective value at ``x``. :type score: float :returns: **accepted** -- Whether the injection was accepted (deferred). :rtype: bool .. py:method:: _begin_reflect() .. py:method:: _try_advance_reflect() .. py:method:: _try_advance_expand() .. py:method:: _try_advance_contract() .. py:method:: _begin_shrink() .. py:method:: _try_advance_shrink() .. py:method:: ask() Return ``(token, raw_x)`` or ``None`` if blocked. .. py:method:: tell(token, score) Report the objective value for a previously issued token. Unknown tokens (e.g. from a replaced core) are silently ignored so that late-arriving results from a previous core instance do not cause errors. .. py:class:: HyperNelderMeadSampler(space, seed=None, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, initial_scale=0.6, restart_tol=0.01, restart_scale=0.5, filler_scale=0.3, n_initial=None, explore_prob=0.05, inject_diameter_fraction=1.5, inject_restart_fraction=0.6, exponential_param_power=None) Nelder-Mead simplex optimizer in raw ``[-1, 1]`` space. Wraps a ``_NMCore`` state machine and adds parameter mapping, random filler point generation while the core is blocked, and automatic restarts when the simplex converges. :param space: The search space for a single contraction method. :type space: dict[str, dict] :param seed: Random seed. :type seed: None or int or random.Random, optional :param adaptive: Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If `True` then `alpha`, `gamma`, `rho`, and `sigma` are ignored. :type adaptive: bool, optional :param alpha: Reflection coefficient. :type alpha: float, optional :param gamma: Expansion coefficient. :type gamma: float, optional :param rho: Contraction coefficient. :type rho: float, optional :param sigma: Shrink coefficient. :type sigma: float, optional :param initial_scale: Scale of the initial simplex around the origin. :type initial_scale: float, optional :param restart_tol: When the simplex diameter falls below this, restart. :type restart_tol: float, optional :param restart_scale: Scale of the restarted simplex around the best point. :type restart_scale: float, optional :param filler_scale: Standard deviation of the gaussian noise used for filler points issued while the core Nelder-Mead routine is blocked. :type filler_scale: float, optional :param n_initial: Number of Latin Hypercube Sampled (LHS) warm-up points to evaluate before starting the simplex. The best result seeds the initial simplex center. Default ``None`` means ``2 * ndim``. Set to ``0`` to disable. :type n_initial: int or None, optional :param explore_prob: Probability of issuing a uniform random point instead of the NM-directed point during normal operation. Maintains diversity throughout the search. :type explore_prob: float, optional :param inject_diameter_fraction: Passed to ``_NMCore`` — controls the maximum allowed simplex diameter inflation when injecting an external vertex. :type inject_diameter_fraction: float, optional :param exponential_param_power: Passed through to the shared parameter mapping. :type exponential_param_power: float, optional .. py:attribute:: rng .. py:attribute:: params :value: [] .. py:attribute:: ndim .. py:attribute:: adaptive :value: False .. py:attribute:: alpha :value: 1.0 .. py:attribute:: gamma :value: 2.0 .. py:attribute:: rho :value: 0.5 .. py:attribute:: sigma :value: 0.5 .. py:attribute:: initial_scale :value: 0.6 .. py:attribute:: restart_tol :value: 0.01 .. py:attribute:: restart_scale :value: 0.5 .. py:attribute:: filler_scale :value: 0.3 .. py:attribute:: explore_prob :value: 0.05 .. py:attribute:: inject_diameter_fraction :value: 1.5 .. py:attribute:: inject_restart_fraction :value: 0.6 .. py:attribute:: n_initial :value: None .. py:attribute:: _trial_counter :value: 0 .. py:attribute:: _restart_count :value: 0 .. py:attribute:: _best_x :value: None .. py:attribute:: _best_score .. py:attribute:: _trial_map .. py:method:: _make_core(center, scales) .. py:method:: _ask_filler() .. py:method:: ask() Return the next candidate setting. During the LHS warm-up phase, pre-generated Latin Hypercube points are issued one at a time. Once all warm-up results have been collected, the NM simplex is initialized centered on the best warm-up point. During normal operation, with probability ``explore_prob`` a uniform random point is returned to maintain diversity. If the NM state machine is blocked waiting for results, a filler point is returned instead. .. py:method:: tell(trial_number, score) Record a completed trial result. .. py:class:: NelderMeadOptLib Bases: :py:obj:`cotengra.hyperoptimizers.hyper.HyperOptLib` Hyper-optimization backend using the Nelder-Mead simplex method. .. py:method:: setup(methods, space, optimizer=None, adaptive=False, alpha=1.0, gamma=2.0, rho=0.5, sigma=0.5, initial_scale=0.6, restart_tol=0.01, restart_scale=0.5, filler_scale=0.3, n_initial=None, explore_prob=0.05, inject_diameter_fraction=1.5, inject_restart_fraction=0.6, method_exploration=1.0, method_temperature=1.0, exponential_param_power=None, seed=None, **kwargs) Initialize Nelder-Mead optimizers for each method. :param methods: The contraction methods to optimize over. :type methods: list[str] :param space: The per-method hyperparameter search space. :type space: dict[str, dict[str, dict]] :param optimizer: The parent optimizer. :type optimizer: HyperOptimizer, optional :param adaptive: Whether to use the adaptive NM coefficients recommended by Gao and Han (2010), which scale with problem dimension. If `True` then `alpha`, `gamma`, `rho`, and `sigma` are ignored. :type adaptive: bool, optional :param alpha: Reflection coefficient. :type alpha: float, optional :param gamma: Expansion coefficient. :type gamma: float, optional :param rho: Contraction coefficient. :type rho: float, optional :param sigma: Shrink coefficient. :type sigma: float, optional :param initial_scale: Scale of the initial simplex. :type initial_scale: float, optional :param restart_tol: Simplex diameter threshold for restart. :type restart_tol: float, optional :param restart_scale: Scale of the restarted simplex. :type restart_scale: float, optional :param filler_scale: Gaussian noise scale for filler points. :type filler_scale: float, optional :param n_initial: Number of LHS warm-up points per method. Default ``None`` means ``2 * ndim``. Set to ``0`` to disable. :type n_initial: int or None, optional :param explore_prob: Probability of issuing a uniform random exploration point during normal operation. :type explore_prob: float, optional :param inject_diameter_fraction: Passed to ``_NMCore`` — controls the maximum allowed simplex diameter inflation when injecting an external vertex. :type inject_diameter_fraction: float, optional :param method_exploration: Exploration strength for the LCB method chooser. :type method_exploration: float, optional :param method_temperature: Noise temperature for the LCB method chooser. :type method_temperature: float, optional :param exponential_param_power: Passed to the shared parameter mapping. :type exponential_param_power: float, optional :param seed: Random seed. :type seed: None or int, optional .. py:method:: get_setting() Choose a method, then request its next setting. .. py:method:: report_result(setting, trial, score) Report a completed trial back to the method chooser and the method specific Nelder-Mead sampler.