A summary of how spec-driven development and formal proof tools like Lean4 can mathematically guarantee correctness of AI-generated code beyond what tests or human review can achieve.
Adapted from @CoreyGallon. @varun_pant_ makes the case that none of the ways teams currently check AI-generated code, LLM-as-judge, tests, human review, can actually prove it correct for every input. Formal verification can, and that's the subject of his talk, "Your Code Has Bugs. Lean4 Has Proofs: Formal Verification for Engineers," on @aiDotEngineer's YouTube. Varun builds AI products at AWS, currently working on neurosymbolic AI. He walks through what it takes to get from a written specification to a mathematically checked implementation, using real production systems as evidence. - Specs are upstream, code is downstream. In spec-driven development, humans write and validate the specification (formally in Lean, or in natural language that gets auto-formalized), the coding agent implements it, and a formal verification tool proves the implementation matches. - Lean is code and proof, in one language. There's no separate proof language and no translation layer. A small trusted kernel checks every proof, and that kernel can be independently reimplemented and verified in C++, Rust, or Lean itself. - zlib, translated and proven. AI converted the C compression library zlib into Lean, specification first, then implementation, then proof, producing about 32,000 lines of proof. - Cedar checks Rust against a Lean spec. AWS's Cedar authorization policy language has its specification written in Lean and its production code in Rust. About 100 million differential random tests run nightly to confirm both give the same answer, and no version ships until they do. - Verus and the Z3 solver. Rust code gets "requires" and "ensures" annotations as pre and postconditions, checked by the Z3 solver as a static check that's erased at runtime. - Strata: one verifier for any language. AWS's open source, work-in-progress project lets you define a "dialect" for a language, lower it into a shared Lean-based representation, and dispatch it to Lean's prover, SMT solvers, or model checkers. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!
