<div id="readability-page-1" class="page"><div>
<p>I couldn’t find an explanation on how one can run optuna in multiprocessing settings to avoid GIL caused performance degradation. So here’s sample code.</p>


<div><pre title="">from concurrent.futures import ProcessPoolExecutor

import optuna

STORAGE = "sqlite:///optuna_parallel.db"
N_TRIALS_PER_PROCESS = 200
N_PROCESSES = 5


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2


def optimize(idx):
    # Using different seeds per process helps optuna try different parameters
    # esp. during initial runs
    sampler = optuna.samplers.TPESampler(seed=42 + idx)
    study = optuna.load_study(
        study_name="optuna_parallel", storage=STORAGE, sampler=sampler
    )
    study.optimize(objective, n_trials=N_TRIALS_PER_PROCESS)


if __name__ == "__main__":
    try:
        optuna.delete_study(study_name="optuna_parallel", storage=STORAGE)
        print("Previous study deleted")
    except KeyError:
        print("Previous study not found")
    print(
        f"Starting optimization of ({N_PROCESSES=}) * ({N_TRIALS_PER_PROCESS=}) = {N_PROCESSES * N_TRIALS_PER_PROCESS:,} trials"
    )
    optuna.create_study(study_name="optuna_parallel", storage=STORAGE)
    with ProcessPoolExecutor(max_workers=N_PROCESSES) as executor:
        for i in range(N_PROCESSES):
            executor.submit(optimize, i)

    study = optuna.load_study(study_name="optuna_parallel", storage=STORAGE)
    print(f"Best value: {study.best_value}, Best params: {study.best_params}")
</pre></div></div></div>

Passion is like genius; a miracle.

I couldn’t find an explanation on how one can run optuna in multiprocessing settings to avoid GIL caused performance degradation. So here’s sample code.

Optuna in multiprocessing

댓글