NVIDIA's Hephaestus framework saves pilot teams up to 10 weeks of development time with AI-generated tests
NVIDIA's DriveOS team built Hephaestus (HEPH), an internal generative AI framework that automates test-case creation from software architecture and interface control documents, saving pilot teams up to 10 weeks of development time per engagement.
Background
NVIDIA's DriveOS QNX team manages a large body of software requirements — functional, safety, and interface specifications — that must each be covered by test cases. Manually creating test specifications and C/C++ test implementations from these documents is time-consuming and labor-intensive, and at the scale of automotive software (industry sources cite 450,000 requirements for a well-equipped mid-sized vehicle), the manual approach is a significant bottleneck to release velocity.
What Was Implemented
- HEPH, an internal multi-agent generative AI framework built on LLMs, ingesting SWADs, ICDs, and requirement data from Jama
- Automated pipeline: requirement extraction → SWAD/ICD traceability via embedding database → test specification generation (positive and negative cases) → executable C/C++ test implementation
- Feedback loop: test coverage data fed back into the model to regenerate specs for missed cases
- Integration with Confluence and JIRA for documentation and ticket management
- Supports PDF, RST, RSTI, and HTML input formats
Results
In trials with multiple pilot teams, teams reported saving up to 10 weeks of development time per engagement. The framework compiles and executes generated tests and validates them for correctness, reducing the manual effort required for test creation from design through implementation. The team acknowledges future needs: modular support for custom test workflows and an interactive mode for human feedback at each generation step.
Lessons
- A multi-agent LLM pipeline — with distinct agents for document traceability, spec generation, and code generation — can automate the full test-creation workflow, not just individual steps
- Embedding-database indexing of architecture and interface documents is the critical infrastructure enabling context-aware test generation
- Feedback loops that re-run coverage analysis and regenerate missing test cases significantly improve output completeness
- Pilot deployments reveal the boundaries of automation: novel frameworks and non-standard workflows still require custom modules
- Publishing internal AI tooling case studies accelerates cross-industry learning even when the tool itself is not externally available