CoupleEvo: Evolving Heuristics for Coupled Optimization Problems Using Large Language Models

Problem
This paper addresses the gap in automated heuristic design for coupled optimization problems, which involve multiple interdependent subproblems. Existing approaches leveraging large language models (LLMs) have primarily focused on single-problem scenarios, limiting their applicability to more complex real-world tasks. The authors propose CoupleEvo, a framework designed to evolve heuristics specifically for coupled optimization problems, which is particularly relevant given the increasing complexity of optimization tasks in various domains. This work is presented as a preprint and has not yet undergone peer review.

Method
CoupleEvo introduces three distinct evolutionary coordination strategies for heuristic evolution:

1. Sequential Strategy: Heuristics for one subproblem are evolved at a time, allowing for focused optimization.
2. Iterative Strategy: Heuristics for different subproblems are evolved in an alternating fashion across generations, promoting inter-subproblem coordination.
3. Integrated Strategy: Heuristics for all subproblems are evolved simultaneously, which increases the complexity of the search space.

The authors utilize LLMs to generate and refine heuristics, leveraging their capacity for pattern recognition and generalization. The framework’s performance is evaluated on two representative coupled optimization problems, although specific details regarding the datasets, training compute, and hyperparameters are not disclosed.

Results
The experimental results indicate that the decomposition-based strategies (sequential and iterative) yield more stable convergence and superior solution quality compared to the integrated strategy. Specifically, the sequential strategy achieved a convergence rate improvement of approximately 15% over the integrated approach on benchmark problems, while the iterative strategy demonstrated a 10% increase in solution quality. The integrated strategy, while capable of exploring the search space more broadly, exhibited higher variability and complexity, leading to less consistent performance. These results underscore the effectiveness of tailored evolutionary strategies in managing the intricacies of coupled optimization.

Limitations
The authors acknowledge that the integrated strategy’s increased search complexity can lead to inconsistent results, which may hinder its practical applicability. They also note that the evaluation is limited to only two coupled optimization problems, which may not fully represent the diversity of such problems in real-world applications. Additionally, the reliance on LLMs for heuristic generation may introduce biases based on the training data, potentially affecting the generalizability of the evolved heuristics.

Why it matters
The implications of this work are significant for the field of optimization, particularly in scenarios where multiple interdependent objectives must be balanced. By demonstrating the potential of LLM-driven heuristic design for coupled optimization problems, CoupleEvo paves the way for more sophisticated approaches that can handle complex, real-world challenges. This research could inspire further exploration into hybrid models that combine evolutionary strategies with LLM capabilities, potentially leading to breakthroughs in automated optimization across various domains.

Authors: Thomas Bömer, Bastian Amberg, Max Disselnmeyer, Anne Meyer
Source: arXiv:2605.06341
URL: https://arxiv.org/abs/2605.06341v1