Autofy: Automated Synthesis of Formally Verified Code

  -  Technical University of Munich

Autofy: Automated Synthesis of Formally Verified Code - Technical University of Munich

Autofy: Automated Synthesis of Formally Verified Code

  • Nils Purschke

  • , André Schamschurko

  • , Sven Kirchner

  • , Chengdong Wu

  • , Alois Knoll

  • Informatics 6 - Chair of Robotics, Artificial Intelligence and Real-time Systems

  • Technical University of Munich

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Although large language models (LLMs) are becoming increasingly capable at coding, ensuring the formal correctness of their output remains a major challenge. We present Autofy, a system that orchestrates multiple LLM roles to synthesize SPARK/Ada programs that (i) compile, (ii) pass formal verification and (iii) satisfy functional tests. Autofy integrates Alire for project bootstrap, GNAT/GCC and gnatpp for style/syntax/semantics checks and compilation, GNATprove for formal verification, as well as functional tests to ensure guard-railed results. Two complementary flows are supported: a Multi-Version approach that cross-combines independent contract and implementation variants, and a Director-Feedback approach in which a Director LLM reads compiler/prover/test logs and emits targeted edits. In evaluations across multiple datasets and models, Autofy solved up to 31% (Multi-Version) / 71% (Director-Feedback) of SPARK coding tasks - substantially outperforming naive prompting (5%). We discuss system design, empirical results, and open challenges for combining LLMs with sound software engineering and formal methods.

Original languageEnglish
Title of host publicationProceedings - 2026 IEEE/ACM 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2026
PublisherAssociation for Computing Machinery, Inc
Pages126-137
Number of pages12
ISBN (Electronic)9798400724787
DOIshttps://doi.org/10.1145/3793656.3793688
StatePublished - 10 Jul 2026
Event14th International Conference on Formal Methods in Software Engineering, FormaliSE 2024 - Rio de Janeiro, Brazil Duration: 12 Apr 2026 → 13 Apr 2026

Publication series

NameProceedings - 2026 IEEE/ACM 14th International Conference on Formal Methods in Software Engineering, FormaliSE 2026

Conference

Conference14th International Conference on Formal Methods in Software Engineering, FormaliSE 2024
Country/TerritoryBrazil
CityRio de Janeiro
Period12/04/26 → 13/04/26

Access to Document

  • 10.1145/3793656.3793688