FunFuzz: An LLM-Powered Evolutionary Fuzzing Framework

Problem
This paper addresses the limitations of existing LLM-driven fuzzing techniques, which are often sensitive to prompt initialization and sampling variance. These issues can lead to inefficient exploration of the input space and redundancy in generated inputs. The authors propose FunFuzz, a novel multi-island evolutionary fuzzing framework designed to enhance the exploration capabilities of LLMs in generating structured inputs for fuzz testing. This work is presented as a preprint and has not yet undergone peer review.

Method
FunFuzz employs a multi-island evolutionary strategy, where multiple isolated searches (islands) run in parallel. Each island is initialized with topic-specific instructions derived from relevant documentation, allowing for tailored prompt generation. The framework continuously adapts these prompts based on feedback from the fuzzing process, specifically using a feedback-guided selection mechanism. During the fuzzing phase, candidate inputs are prioritized based on incremental compiler coverage metrics, which measure the extent of code exercised by the inputs. Additionally, the framework leverages compiler-internal failure signals to identify inputs that induce crashes. The architecture is designed to maintain diversity among candidate inputs through periodic migration of high-value candidates between islands.

Results
FunFuzz was evaluated through extensive 24-hour fuzzing campaigns on the GCC and Clang compilers. The results indicate that FunFuzz significantly outperforms previous LLM-driven fuzzing baselines in terms of compiler coverage and the discovery of unique failure-triggering inputs. Specifically, FunFuzz achieved a compiler coverage increase of approximately 15% over the best-performing baseline, and it discovered 30% more unique crashes compared to prior methods. These results demonstrate the effectiveness of the multi-island approach and the adaptive prompt generation in enhancing fuzzing performance.

Limitations
The authors acknowledge several limitations in their work. First, the performance of FunFuzz is contingent on the quality of the initial prompts derived from documentation, which may vary in comprehensiveness. Second, the framework’s reliance on compiler-internal signals may limit its applicability to other domains outside of compiler fuzzing. Additionally, the computational overhead associated with maintaining multiple islands and the migration process could be a concern in resource-constrained environments. The authors do not address the potential impact of prompt engineering on the overall effectiveness of the framework, nor do they explore the scalability of FunFuzz to larger or more complex software systems.

Why it matters
The implications of FunFuzz extend to the broader field of software testing and security. By improving the efficiency and effectiveness of fuzz testing through the integration of LLMs and evolutionary strategies, this framework has the potential to uncover vulnerabilities in software systems more effectively than traditional methods. The adaptive nature of FunFuzz could inspire future research into hybrid approaches that combine LLMs with other AI techniques for automated testing. Furthermore, the insights gained from this work may lead to advancements in the development of more robust and resilient software systems.

Authors: Mario Rodríguez Béjar, B. Romera-Paredes, Jose L. Hernández-Ramos
Source: arXiv:2605.02789
URL: https://arxiv.org/abs/2605.02789v1