Adaptive Evolutionary Algorithm

Adaptive evolutionary algorithm for creating a deep-learning architecture on the example of human activity recognition (HAR).

This work builds on the evolutionary architecture search I wrote about before. During those runs, one observation kept coming back: in different phases of the optimization, different evolutionary techniques seemed to have an effect. Big changes to the model helped early, while fine-tuning only paid off once a promising candidate existed.

The adaptive algorithm turns that observation into the mechanism. It goes through three optimization stages, each with a different set of techniques: the first stage uses techniques that make big changes to the model, while the last stage focuses on fine-tuning.

The switching is adaptive. If the model’s accuracy reaches a certain threshold, the algorithm moves to a stage with more fine-tuning techniques. If it cannot find a new best model within a certain time frame, it goes back to a stage with techniques that make bigger changes to the model.

This adaptive process helps the algorithm find the right balance between exploration and exploitation, resulting in a more efficient search for the best deep-learning architecture.

In an example of human activity recognition, the algorithm went through a fascinating process of finding an architecture to optimize the model’s accuracy.

The code is available on GitHub.