# CMA-ES

Lightweight Covariance Matrix Adaptation Evolution Strategy (CMA-ES)[1] implementation.

![visualize-six-hemp-camel](./visualizer/six-hemp-camel.gif)

<details>
<summary>Himmelblau function.</summary>

![visualize-himmelblau](./visualizer/himmelblau.gif)

</details>

<details>
<summary>Rosenbrock function.</summary>

![visualize-rosenbrock](./visualizer/rosenbrock.gif)

</details>

<details>
<summary>Quadratic function.</summary>

![visualize-quadratic](./visualizer/quadratic.gif)

</details>

These GIF animations are generated by [visualizer.py](./visualizer/visualizer.py).


## Installation

Supported Python versions are 3.6 or later.

```
$ pip install cmaes
```

## Usage

This library provides two interfaces that an Optuna's sampler interface and a low-level interface.
I strongly recommend you to use this library via Optuna.

### Optuna's sampler interface

[Optuna](https://github.com/optuna/optuna) [2] is an automatic hyperparameter optimization framework.
Optuna officially implements [a sampler based on pycma](https://optuna.readthedocs.io/en/latest/reference/integration.html#optuna.integration.CmaEsSampler).
It achieves almost the same performance. But this library is faster and simple.

```python
import optuna
from cmaes.sampler import CMASampler

def objective(trial: optuna.Trial):
    x1 = trial.suggest_uniform("x1", -4, 4)
    x2 = trial.suggest_uniform("x2", -4, 4)
    return (x1 - 3) ** 2 + (10 * (x2 + 2)) ** 2

def main():
    sampler = CMASampler()
    study = optuna.create_study(sampler=sampler)
    study.optimize(objective, n_trials=250)

if __name__ == "__main__":
    main()
```

Note that CMASampler doesn't support categorical distributions.
Although pycma's sampler supports categorical distributions, it also has a problem (especially on high-cardinality categorical distribution).
If your search space contains a categorical distribution, please use [TPESampler](https://optuna.readthedocs.io/en/latest/reference/samplers.html#optuna.samplers.TPESampler).


### Low-level interface

```python
import numpy as np
from cmaes.cma import CMA

def quadratic(x1: float, x2: float):
    return (x1 - 3) ** 2 + (10 * (x2 + 2)) ** 2

def main():
    cma_es = CMA(mean=np.zeros(2), sigma=1.3)

    best_value = float("inf")
    best_param = None

    for generation in range(50):
        solutions = []
        for _ in range(cma_es.population_size):
            z, x = cma_es.ask()
            evaluation = quadratic(x[0], x[1])

            if evaluation < best_value:
                best_value = evaluation
                best_param = x

            solutions.append((z, evaluation))

        cma_es.tell(solutions)
        print(f"#{generation}: {best_value} (x1={best_param[0]}, x2 = {best_param[1]})")

    print(f"RESULT: {best_value} (x1={best_param[0]}, x2 = {best_param[1]})")

if __name__ == "__main__":
    main()
```

## Benchmark results

| [Rosenbrock function](https://www.sfu.ca/~ssurjano/rosen.html) | [Six-Hemp Camel function](https://www.sfu.ca/~ssurjano/camel6.html) |
| ------------------- | ----------------------- |
| ![rosenbrock](./benchmark/rosenbrock.png) | ![six-hemp-camel](./benchmark/six_hemp_camel.png) |

This implementation (green) stands comparison with [pycma](https://github.com/CMA-ES/pycma) (blue).
See [benchmark](./benchmark) for details.

Links
-----

**Other libraries:**

I respect all libraries involved in CMA-ES.

* [pycma](https://github.com/CMA-ES/pycma) : Most famous CMA-ES implementation by Nikolaus Hansen.
* [cma-es](https://github.com/srom/cma-es) : A Tensorflow v2 implementation.

**References:**

* [1] [N. Hansen, The CMA Evolution Strategy: A Tutorial. arXiv:1604.00772, 2016.](https://arxiv.org/abs/1604.00772)
* [2] [Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’19), August 4–8, 2019.](https://dl.acm.org/citation.cfm?id=3330701)
