Autofy: Automated Synthesis of Formally Verified Code
- Technical University of Munich
Autofy: Automated Synthesis of Formally Verified Code - Technical University of Munich
Autofy: Automated Synthesis of Formally Verified Code
-
Nils Purschke
-
, André Schamschurko
-
, Sven Kirchner
-
, Chengdong Wu
-
, Alois Knoll
-
Informatics 6 - Chair of Robotics, Artificial Intelligence and Real-time Systems
-
Technical University of Munich
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review
Abstract
Although large language models (LLMs) are becoming increasingly capable at coding, ensuring the formal correctness of their output remains a major challenge. We present Autofy, a system that orchestrates multiple LLM roles to synthesize SPARK/Ada programs that (i) compile, (ii) pass formal verification and (iii) satisfy functional tests. Autofy integrates Alire for project bootstrap, GNAT/GCC and gnatpp for style/syntax/semantics checks and compilation, GNATprove for formal verification, as well as functional tests to ensure guard-railed results. Two complementary flows are supported: a Multi-Version approach that cross-combines independent contract and implementation variants, and a Director-Feedback approach in which a Director LLM reads compiler/prover/test logs and emits targeted edits. In evaluations across multiple datasets and models, Autofy solved up to 31% (Multi-Version) / 71% (Director-Feedback) of SPARK coding tasks - substantially outperforming naive prompting (5%). We discuss system design, empirical results, and open challenges for combining LLMs with sound software engineering and formal methods.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2026 IEEE/ACM 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2026 |
| Publisher | Association for Computing Machinery, Inc |
| Pages | 126-137 |
| Number of pages | 12 |
| ISBN (Electronic) | 9798400724787 |
| DOIs | https://doi.org/10.1145/3793656.3793688 |
| State | Published - 10 Jul 2026 |
| Event | 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2024 - Rio de Janeiro, Brazil Duration: 12 Apr 2026 → 13 Apr 2026 |
Publication series
| Name | Proceedings - 2026 IEEE/ACM 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2026 |
|---|
Conference
| Conference | 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2024 |
|---|---|
| Country/Territory | Brazil |
| City | Rio de Janeiro |
| Period | 12/04/26 → 13/04/26 |
Access to Document
- 10.1145/3793656.3793688