Genetic Algorithms Explained: A Python Implementation Guide

In 2006, NASA launched a satellite known as Space Technology 5 (ST5). Bolted to its hull was a small, irregularly bent piece of wire—an antenna whose appearance suggested a crumpled paper clip rather than the product of a JPL design lab. No human engineer designed the antenna. It was evolved. Starting from a population of random wire shapes, a genetic algorithm bred better performers over thousands of generations, and the final design outperformed every antenna the human engineers had proposed. It was the first artificial object in space to result from a computational evolutionary process, and its performance in orbit confirmed the approach.

This case illustrates the appeal of genetic algorithms. The form of the answer need not be known in advance. Derivatives, closed-form models, and analytical insights are not required. The only requirements are a method for scoring candidate solutions and sufficient compute to let simulated evolution proceed. The remainder of this post examines how a genetic algorithm operates, develops one from scratch in Python, and identifies the cases in which GAs are most and least effective.

The Central Idea: Evolution as a Search Algorithm

Most optimization techniques presented in introductory courses assume a smooth, well-behaved function. The derivative is taken, set to zero, and solved. The approach works elegantly for convex problems such as linear regression and logistic regression. It fails as soon as the landscape becomes rugged: non-differentiable, discontinuous, combinatorial, or riddled with local optima. One cannot take the derivative of “which twelve cities should a truck visit, and in what order.” One cannot apply gradient descent to a Boolean satisfiability problem.

Nature faced a similar problem. The fitness landscape of biological organisms is exceptionally complex, high-dimensional, non-differentiable, and deceptive, yet evolution navigated it without recourse to calculus. It uses a population rather than a single candidate, measures fitness empirically rather than analytically, reproduces with variation, and over many generations converges on remarkable designs. Genetic algorithms, introduced formally by John Holland in his 1975 monograph Adaptation in Natural and Artificial Systems, constitute the computational transcription of this idea.

The Darwinian analogy maps cleanly onto code. A population is a set of candidate solutions. Each candidate is a chromosome, a data structure encoding one possible answer. A fitness function scores the quality of each candidate. Selection identifies the fittest individuals as parents. Crossover combines two parents into offspring. Mutation introduces random variation so that the population does not stagnate. The process repeats until a satisfactory solution emerges.

When GAs Are Most Effective

Genetic algorithms are the appropriate tool when several of the following conditions hold: no gradient is available, the search space is combinatorial (permutations, subsets, graphs), the problem is NP-hard and a good solution rather than a provably optimal one is required, the goal is exploration of a design space with diverse candidates, or multiple competing objectives demand a Pareto frontier rather than a single answer.

Documented applications include the design of jet-engine components, optimization of investment portfolios, scheduling of airline crews, evolution of game-playing AI, tuning of hyperparameters for neural networks, image compression, and routing of delivery vehicles. Boeing has used evolutionary methods for wing-shape refinement. Waste-management companies have evolved garbage-collection routes. Researchers have applied GAs to the 85,900-city “pla85900” Traveling Salesman instance and obtained solutions within a fraction of one percent of the proven optimum.

When GAs Are Not Appropriate

GAs are also easy to misuse. For a convex and differentiable problem, gradient descent identifies the optimum in a fraction of the time. When the search space is small enough to enumerate, brute force is simpler and exact. When a specialized solver exists, such as integer linear programming, SAT solving, mixed-integer programming, or dynamic programming, it should be preferred. GAs are a tool of last resort for problems where nothing else works well, not a default optimizer.

GA Mechanics, Step by Step

A GA is defined by five design decisions: how to represent a solution, how to score it, how to select parents, how to combine them, and how to mutate offspring. Correct choices produce convergence. Incorrect choices lead to populations that drift without progress for substantial periods of compute.

Chromosome Representation

The chromosome is the encoding of a candidate solution as data. The representation profoundly affects everything that follows: which crossover and mutation operators are valid, how difficult it is to generate valid solutions, and how smoothly the fitness landscape maps onto the genotype.

• Binary strings: the classical Holland-style encoding. A candidate might be [1,0,1,1,0,0,1,0]. Works naturally for feature selection, knapsack problems, and anywhere the decisions are on/off.
• Real-valued vectors: a list of floats. Natural for continuous optimization like tuning a physical parameter or minimizing a mathematical function. Most modern GAs use this.
• Permutations: an ordering of items, like the sequence of cities in a TSP tour. Requires specialized operators that preserve the permutation property.
• Trees: used in Genetic Programming, where the chromosome is an expression tree representing an actual program. This is how Koza’s famous GP work evolved symbolic regression formulas.

The Fitness Function: The Most Important Decision

If there is one place where GAs fail, it is here. The fitness function defines what “better” means, and the algorithm will optimize it relentlessly. Any loophole in the fitness function will be discovered. The AI-safety community describes this phenomenon as “specification gaming,” and it appears regularly in evolutionary systems. A well-known example concerned a GA tasked with evolving fast simulated creatures: it evolved very tall, thin creatures that fell over rapidly and “moved” by converting height into forward momentum—technically correct, yet entirely useless.

A good fitness function is cheap to evaluate (it will be called millions of times), smooth enough to provide gradient information (nearby solutions should have similar fitness), and resistant to loopholes. For constrained problems, penalty terms for constraint violations are preferable to discarding invalid chromosomes outright.

Selection Methods

Selection identifies the parents that will produce the next generation. There is a fundamental tension between exploitation (favoring the current best) and exploration (preserving diversity). Excessive exploitation produces premature convergence to a local optimum; excessive exploration reduces the algorithm to random search.

In practice, most modern GA implementations use tournament selection with k = 3 combined with modest elitism (the top 1 to 5 percent). Tournament selection is simple, scale-invariant, and easy to parallelize. It also degrades gracefully: when two candidates have nearly equal fitness, the competition becomes approximately a coin flip, which helps preserve diversity.

Crossover (Recombination)

Crossover is the engine of innovation. It takes two parent chromosomes and combines them to produce offspring, recombining existing useful building blocks into new configurations. The expectation, formalized by Holland’s schema theorem, is that short, high-fitness sub-patterns propagate through the population even as whole chromosomes change.

Mutation

Mutation injects randomness. Without it, the gene pool can only reshuffle existing alleles; once a position has converged across the population (every chromosome shares the same value at that locus), crossover cannot restore diversity. Mutation rates are typically small, between 0.5 and 5 percent per gene, because excessive mutation reduces the GA to random search. A useful heuristic is mutation rate ≈ 1/L, where L is the chromosome length, so that on average one gene mutates per offspring.

Termination Criteria

Stopping criteria vary. Common choices include a fixed number of generations (the simplest), a wall-clock time budget, a target fitness threshold, or detection of a fitness plateau (no improvement in the best or average fitness for N generations). In competitions and time-constrained production settings, a time budget is typical. For research, a fixed generation count ensures reproducibility.

A Full Python Implementation from Scratch

The following implementation builds a complete GA that minimizes the Rastrigin function, a classic non-convex optimization benchmark defined as f(x) = 10n + ∑ [xi2 − 10 cos(2πxi)]. It has a single global minimum at the origin and dozens of local minima nearby, which makes it well suited to illustrating both the difficulty for gradient descent and the value of population-based search.

import numpy as np
import random
from dataclasses import dataclass, field
from typing import Callable, List, Optional, Tuple


@dataclass
class GAConfig:
    """Configuration for the genetic algorithm."""
    pop_size: int = 100
    gene_count: int = 10
    gene_low: float = -5.12
    gene_high: float = 5.12
    crossover_rate: float = 0.8
    mutation_rate: float = 0.1          # per-gene probability
    mutation_sigma: float = 0.3         # std dev of Gaussian noise
    tournament_k: int = 3
    elitism: int = 2
    generations: int = 300
    seed: Optional[int] = 42


class GeneticAlgorithm:
    """A real-valued genetic algorithm for continuous optimization.

    Minimizes fitness_fn. If you have a maximization problem, negate it.
    """

    def __init__(self, fitness_fn: Callable[[np.ndarray], float], config: GAConfig):
        self.fitness_fn = fitness_fn
        self.cfg = config
        if config.seed is not None:
            random.seed(config.seed)
            np.random.seed(config.seed)

        self.population: np.ndarray = self._init_population()
        self.fitness: np.ndarray = self._evaluate(self.population)
        self.history: List[dict] = []

    # -------- Initialization --------
    def _init_population(self) -> np.ndarray:
        c = self.cfg
        return np.random.uniform(c.gene_low, c.gene_high, size=(c.pop_size, c.gene_count))

    def _evaluate(self, pop: np.ndarray) -> np.ndarray:
        return np.array([self.fitness_fn(ind) for ind in pop])

    # -------- Selection --------
    def _tournament(self) -> np.ndarray:
        """Tournament selection: pick k at random, return the best."""
        idx = np.random.randint(0, self.cfg.pop_size, self.cfg.tournament_k)
        best = idx[np.argmin(self.fitness[idx])]
        return self.population[best].copy()

    # -------- Crossover --------
    def _crossover(self, p1: np.ndarray, p2: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
        """Blend crossover for real values: child = alpha*p1 + (1-alpha)*p2."""
        if random.random() > self.cfg.crossover_rate:
            return p1.copy(), p2.copy()
        alpha = np.random.uniform(-0.25, 1.25, size=p1.shape)  # BLX-alpha style
        c1 = alpha * p1 + (1 - alpha) * p2
        c2 = alpha * p2 + (1 - alpha) * p1
        return self._clip(c1), self._clip(c2)

    def _clip(self, x: np.ndarray) -> np.ndarray:
        return np.clip(x, self.cfg.gene_low, self.cfg.gene_high)

    # -------- Mutation --------
    def _mutate(self, ind: np.ndarray) -> np.ndarray:
        mask = np.random.random(ind.shape) < self.cfg.mutation_rate
        noise = np.random.normal(0.0, self.cfg.mutation_sigma, size=ind.shape)
        ind = ind + mask * noise
        return self._clip(ind)

    # -------- Evolution loop --------
    def run(self) -> Tuple[np.ndarray, float]:
        c = self.cfg
        for gen in range(c.generations):
            # Sort by fitness (ascending — we minimize)
            order = np.argsort(self.fitness)
            self.population = self.population[order]
            self.fitness = self.fitness[order]

            # Elitism: keep top N unchanged
            new_pop = [self.population[i].copy() for i in range(c.elitism)]

            # Fill the rest via selection + crossover + mutation
            while len(new_pop) < c.pop_size:
                p1 = self._tournament()
                p2 = self._tournament()
                c1, c2 = self._crossover(p1, p2)
                new_pop.append(self._mutate(c1))
                if len(new_pop) < c.pop_size:
                    new_pop.append(self._mutate(c2))

            self.population = np.array(new_pop)
            self.fitness = self._evaluate(self.population)

            best_idx = int(np.argmin(self.fitness))
            self.history.append({
                "generation": gen,
                "best_fitness": float(self.fitness[best_idx]),
                "mean_fitness": float(self.fitness.mean()),
                "best_chromosome": self.population[best_idx].copy(),
            })

            if gen % 20 == 0:
                print(f"Gen {gen:4d} | best={self.fitness[best_idx]:.6f} | mean={self.fitness.mean():.4f}")

        best_idx = int(np.argmin(self.fitness))
        return self.population[best_idx], float(self.fitness[best_idx])


# -------- Example: Rastrigin function --------
def rastrigin(x: np.ndarray) -> float:
    A = 10.0
    return A * len(x) + np.sum(x * x - A * np.cos(2 * np.pi * x))


if __name__ == "__main__":
    cfg = GAConfig(pop_size=120, gene_count=10, generations=300)
    ga = GeneticAlgorithm(rastrigin, cfg)
    best_x, best_f = ga.run()
    print(f"\nBest solution: {best_x}")
    print(f"Best fitness:  {best_f:.6f}  (true minimum = 0.0 at x = 0)")

When this is run, the best fitness drops from approximately 80–100 (random initialization on a ten-dimensional Rastrigin) to values near zero within a few hundred generations. The population converges visibly: printing self.population.std(axis=0) shows the spread contracting generation by generation.

A Second Example: Traveling Salesman

The Rastrigin example uses real-valued chromosomes with blend crossover. TSP requires permutation chromosomes and a specialized order crossover (OX) that preserves the permutation property. A compact implementation follows.

import numpy as np
import random


def tour_length(tour: list, dist: np.ndarray) -> float:
    return sum(dist[tour[i], tour[(i + 1) % len(tour)]] for i in range(len(tour)))


def order_crossover(p1: list, p2: list) -> list:
    """OX: copy a slice from p1, fill the rest from p2 in order, skipping duplicates."""
    n = len(p1)
    a, b = sorted(random.sample(range(n), 2))
    child = [None] * n
    child[a:b] = p1[a:b]
    fill = [g for g in p2 if g not in child[a:b]]
    j = 0
    for i in range(n):
        if child[i] is None:
            child[i] = fill[j]
            j += 1
    return child


def swap_mutation(tour: list, rate: float = 0.02) -> list:
    tour = tour[:]
    for i in range(len(tour)):
        if random.random() < rate:
            j = random.randrange(len(tour))
            tour[i], tour[j] = tour[j], tour[i]
    return tour


def tournament(pop, fitnesses, k=3):
    idx = random.sample(range(len(pop)), k)
    return pop[min(idx, key=lambda i: fitnesses[i])]


def ga_tsp(coords: np.ndarray, pop_size=200, generations=500, elite=4):
    n = len(coords)
    # Precompute distance matrix
    dist = np.linalg.norm(coords[:, None, :] - coords[None, :, :], axis=-1)

    population = [random.sample(range(n), n) for _ in range(pop_size)]
    fitnesses = [tour_length(t, dist) for t in population]

    for gen in range(generations):
        order = sorted(range(pop_size), key=lambda i: fitnesses[i])
        population = [population[i] for i in order]
        fitnesses = [fitnesses[i] for i in order]

        new_pop = population[:elite]
        while len(new_pop) < pop_size:
            p1 = tournament(population, fitnesses)
            p2 = tournament(population, fitnesses)
            child = order_crossover(p1, p2)
            child = swap_mutation(child, rate=0.02)
            new_pop.append(child)

        population = new_pop
        fitnesses = [tour_length(t, dist) for t in population]

        if gen % 50 == 0:
            print(f"Gen {gen:4d} | best tour length = {min(fitnesses):.2f}")

    best = min(range(pop_size), key=lambda i: fitnesses[i])
    return population[best], fitnesses[best]


if __name__ == "__main__":
    np.random.seed(0)
    random.seed(0)
    coords = np.random.rand(30, 2) * 100  # 30 random cities in a 100x100 square
    tour, length = ga_tsp(coords)
    print(f"\nBest tour length: {length:.2f}")

On thirty random cities, this implementation converges to near-optimal tours within roughly five hundred generations on a laptop. For serious TSP work, the GA is typically combined with a local-search step such as 2-opt after each generation, producing a memetic algorithm. This hybrid approach was used to solve the 85,900-city instance to within 0.04 percent of the optimum.

Real-World Applications

GAs are used wherever the search space is rugged and the objective is clear. The categories in which they have had the greatest impact are summarized below.

Engineering Design

NASA’s ST5 antenna is the canonical example. The evolved design met the mission’s bandwidth, gain, and radiation-pattern requirements simultaneously, an outcome that human antenna engineers had failed to achieve for that form factor. Boeing has used evolutionary methods for wing-shape refinement in computational fluid dynamics loops, where each fitness evaluation is an expensive CFD simulation. Automotive crashworthiness teams have evolved body-panel geometry to distribute impact energy. In each case, the search space is substantial, gradients are expensive or unavailable, and the form of the optimum is not known in advance.

Scheduling and Routing

University timetabling, airline crew scheduling, hospital shift rostering, and factory job-shop scheduling are highly constrained NP-hard problems involving thousands of interdependent decisions. GAs with domain-specific repair operators (which restore feasibility after crossover) are a standard tool in this space. Vehicle-routing problems for delivery logistics—variants of TSP with capacity, time-window, and driver-hour constraints—benefit similarly, and many commercial routing solvers combine GAs with local search.

Machine Learning

In machine learning, GAs appear in three principal contexts. First, hyperparameter optimization: evolving learning rates, batch sizes, and regularization strengths. This is competitive with Bayesian optimization when the search space contains integer or categorical dimensions. Second, feature selection: evolving binary masks over input features to identify the most predictive subset, which is relevant for small-data regimes and interpretable models. Third, neural architecture search via methods such as NEAT and NeuroEvolution, in which entire network topologies are evolved. OpenAI’s 2017 paper on “Evolution Strategies as a Scalable Alternative to Reinforcement Learning” demonstrated that evolution strategies could rival deep reinforcement learning on Atari and MuJoCo with substantially simpler and embarrassingly parallel code.

For workflows centered on time series, GAs are well suited to tuning forecasting model ensembles and to selecting detector thresholds in anomaly-detection pipelines, where the objective mixes precision, recall, and alert-fatigue constraints that no gradient cleanly expresses.

Finance

Portfolio optimization with non-convex constraints—integer position sizing, cardinality constraints (holding at most thirty of five hundred assets), transaction costs, and tax-lot accounting—defeats classical mean-variance optimization. GAs handle these cases cleanly because the fitness function can incorporate any computation expressible in Python.

Game AI and Design

Evolving game-playing strategies has a long history, from tic-tac-toe policies and checkers heuristics to StarCraft build orders. Procedural content generation in games (levels, creatures, weapons) sometimes uses GAs to produce items that satisfy designer-specified fitness functions while maintaining diversity.

Advanced Topics: NSGA-II, Genetic Programming, and Hybrids

Multi-Objective Optimization: NSGA-II

Real problems rarely involve a single objective. A portfolio is desired with high return and low risk. A car design is desired with high safety, low weight, and low cost. A neural architecture is desired with high accuracy and low latency. Classical optimization scalarizes via weights, which requires committing to trade-offs in advance. Multi-objective GAs instead identify the Pareto frontier: the set of solutions for which improving any one objective would worsen another.

NSGA-II (Deb et al., 2002) is the standard algorithm. Instead of a scalar fitness, each individual is assigned a vector of objective values, and the population is ranked by non-dominated sorting: front 1 contains all solutions not dominated by any other; front 2 contains solutions dominated only by front 1; and so on. Ties within a front are broken by crowding distance, which favors solutions in less-crowded regions to preserve diversity along the frontier. The result is a GA that returns an entire Pareto-optimal set rather than a single answer, enabling a human decision-maker to select the appropriate trade-off.

Genetic Programming

Ordinary GAs evolve fixed-length chromosomes. Genetic programming, developed by John Koza in the early 1990s, evolves expression trees: actual programs. A chromosome might be the parse tree for (x + 3) * sin(y). Crossover swaps random subtrees; mutation replaces a node with a new random subtree. GP has been used for symbolic regression (finding formulas that fit data), for evolving controllers for robots, and for automatic algorithm design. The result is a striking demonstration of computational evolution.

Hybrid and Parallel Methods

Pure GAs are often outperformed by memetic algorithms that combine a GA with a local-search step. In each generation, every offspring (or some fraction of them) is improved by hill-climbing or by a problem-specific heuristic such as 2-opt for TSP. The GA handles exploration while local search handles refinement. For the 85,900-city TSP instance mentioned earlier, the winning approach was a memetic algorithm using Lin-Kernighan local search.

Island-model GAs run several populations in parallel on different processes, with occasional migration of individuals between islands. This preserves diversity (each island can converge to a different basin) and maps cleanly to multi-core and distributed infrastructure. Orchestrating these experiments with tools such as Apache Airflow is a convenient way to manage long-running evolutionary campaigns with checkpointing.

Related Methods

GAs belong to a family of population-based or stochastic methods. Particle Swarm Optimization (PSO) uses swarming behavior without crossover. Differential Evolution (DE) is highly effective for continuous optimization and frequently outperforms GAs on real-valued problems. CMA-ES adapts a covariance matrix to the landscape and is the standard for smooth-but-difficult continuous optimization. Simulated Annealing uses a single candidate with a cooling temperature and is simple, effective, and often underestimated. On any given problem, one of these methods is likely to outperform GAs; it is worth benchmarking several.

Practical Tips for Making GAs Work

These values serve as starting points and should be tuned subsequently. Several rules of thumb tend to hold across problems.

• Elitism should always be used: the top 1 to 5 percent should be preserved. Without elitism, the current best can be lost to unfavorable crossover or mutation. With 100 percent elitism, premature convergence results.
• The mutation rate should be tuned by monitoring diversity. If the standard deviation of the population collapses too quickly, more mutation is required. If the best fitness oscillates widely, mutation is excessive.
• The initial population should be seeded intelligently where possible. Including a few hand-crafted known-good solutions among the random ones can accelerate convergence considerably.
• Convergence should be detected and the search restarted. If fitness plateaus for fifty generations, re-randomizing all but the top few individuals is often productive. A single run converging to a local optimum is luck; multiple restarts constitute a method.
• Fitness evaluation should be parallelized. Fitness is almost always the bottleneck. multiprocessing.Pool or Ray can be used because each individual’s fitness is independent and embarrassingly parallel.
• Code should be reproducible. RNGs should be seeded, each generation’s statistics logged, and checkpoints saved. GAs are stochastic, and debugging them without reproducibility is impractical. Following clean-code principles and keeping experiment configurations under version control is therefore important.

Python Libraries: DEAP, PyGAD, pymoo, inspyred

Custom implementations are not required for production work. Several mature Python libraries exist, each with a distinct design philosophy.

For most production work today, DEAP serves as the general-purpose toolkit and pymoo is the standard for multi-objective problems. PyGAD is the appropriate choice when a data scientist wishes to evolve hyperparameters or weights without configuring operators in detail. A minimal DEAP example is shown below.

from deap import base, creator, tools, algorithms
import random, numpy as np

creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
creator.create("Individual", list, fitness=creator.FitnessMin)

toolbox = base.Toolbox()
toolbox.register("gene", random.uniform, -5.12, 5.12)
toolbox.register("individual", tools.initRepeat, creator.Individual, toolbox.gene, 10)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)

def rastrigin(ind):
    x = np.array(ind)
    return (10 * len(x) + np.sum(x * x - 10 * np.cos(2 * np.pi * x))),

toolbox.register("evaluate", rastrigin)
toolbox.register("mate", tools.cxBlend, alpha=0.3)
toolbox.register("mutate", tools.mutGaussian, mu=0, sigma=0.3, indpb=0.1)
toolbox.register("select", tools.selTournament, tournsize=3)

pop = toolbox.population(n=120)
hof = tools.HallOfFame(1)
algorithms.eaSimple(pop, toolbox, cxpb=0.8, mutpb=0.2, ngen=300, halloffame=hof, verbose=False)
print("Best:", hof[0], "fitness:", hof[0].fitness.values)

Limitations and Pitfalls

GAs are powerful and genuinely useful, but they are heuristics rather than guaranteed methods. A candid account of their failure modes is warranted.

• No convergence guarantee. Unlike gradient descent on convex problems, no theorem states that running the GA long enough will identify the global optimum. The schema theorem and related results describe expected propagation of building blocks, not optimality.
• Tuning is an empirical exercise. Population size, mutation rate, crossover rate, selection pressure, and elitism all interact, and the appropriate settings are problem-dependent. Substantial tuning effort should be expected.
• Expensive fitness functions are a practical limitation. A GA with a population of 100 running for 300 generations performs 30,000 fitness evaluations. If each evaluation is a CFD simulation requiring ten minutes, the total is 208 CPU-days. Surrogate models (cheap approximations used inside the GA, with occasional true evaluations) mitigate this but add complexity.
• Premature convergence to local optima is the default failure mode. Excessive selection pressure, insufficient mutation, or inadequate diversity preservation produces a converged but suboptimal population. Population diversity (standard deviation of genes) should be monitored over time as a diagnostic.
• Fitness-function design is the most common point of failure. A flawed fitness function causes the GA to optimize the wrong objective with great efficiency. Evolution does not honor intent; it optimizes the stated objective.
• Performance is modest relative to specialized methods. On convex or near-convex continuous problems, well-implemented gradient methods or quasi-Newton methods typically outperform a GA by orders of magnitude.

None of this implies that GAs are inadequate. They are a tool for specific tasks: black-box, combinatorial, multi-objective, or design-space problems. Outside that niche, they tend to disappoint.

Frequently Asked Questions

When should I use a Genetic Algorithm instead of gradient descent?

Use gradient descent whenever the objective is differentiable and the search space is continuous—it will always be faster. Reach for a GA when you have a combinatorial search space (permutations, subsets, graphs), a non-differentiable objective, multiple competing objectives, a black-box simulator as your fitness function, or when you need to explore a design space rather than find a single best point.

Are Genetic Algorithms still relevant in the era of deep learning?

Yes, in specific niches. Deep learning dominates when you have gradients, data, and a smooth parameterization. GAs complement deep learning in hyperparameter optimization, neural architecture search (NEAT, regularized evolution), reinforcement learning (OpenAI ES rivals policy gradient on many tasks), and domain-specific design problems where the fitness function is an engineering simulation rather than a loss on labeled data. They are also widely used in non-ML engineering optimization where deep learning simply doesn’t apply.

How do I choose population size and mutation rate?

Start with population size 100–200 and mutation rate ≈ 1/L (where L is chromosome length). Then watch diagnostics: if the population diversity collapses fast, increase mutation or population size. If the best fitness jitters without improving, decrease mutation. Harder problems need larger populations; finer-grained search needs lower mutation. Always run several seeds and report averages—GAs are stochastic and a single run tells you little.

Can GAs train neural networks?

They can, but for supervised learning with large networks, backpropagation is vastly more efficient. Where evolutionary methods are competitive is in reinforcement learning (OpenAI’s Evolution Strategies paper), neural architecture search, and small-network tasks where gradients are noisy or unavailable. NEAT famously evolved both weights and topology simultaneously. For a typical image classification or language model, stick to backprop.

What’s the difference between a Genetic Algorithm and Genetic Programming?

A Genetic Algorithm evolves fixed-length chromosomes (bit strings, real vectors, permutations) representing parameters or choices. Genetic Programming evolves variable-size tree structures that represent actual programs or expressions, e.g., the formula sin(x) + 2y. GP is a specialization of GAs for the case where you want to evolve computation itself rather than parameter values.

References and Further Reading

• Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press. The original formulation of genetic algorithms.
• Hornby, G. S., Globus, A., Linden, D. S., & Lohn, J. D. (2006). “Automated Antenna Design with Evolutionary Algorithms.” AIAA Space. The NASA ST5 antenna paper.
• Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002). “A fast and elitist multiobjective genetic algorithm: NSGA-II.” IEEE Transactions on Evolutionary Computation. The canonical multi-objective reference.
• Koza, J. R. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press.
• Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017). “Evolution Strategies as a Scalable Alternative to Reinforcement Learning.” arXiv:1703.03864.
• DEAP documentation,distributed evolutionary algorithms in Python.
• pymoo documentation—multi-objective optimization in Python.
• PyGAD documentation—beginner-friendly GA library with ML integration.

Disclaimer: The financial and portfolio examples in this article are for informational purposes only and do not constitute investment advice. Evolutionary methods applied to financial data are particularly prone to overfitting; any strategy developed via GA should be rigorously validated out-of-sample and stress-tested before real-world use.