NTNU empirical study finds AI-assisted backlog grooming achieves 100% precision and 45% faster duplicate detection than manual review
In a controlled study of 5 participants and 30 Jira issues, a generative AI tool operated as a human-in-the-loop assistant achieved 100% precision on duplicate detection while reducing time per duplicate found by 45% versus manual grooming.
Background
Agile teams with large backlogs face a recurring challenge: duplicate or near-duplicate issues accumulate over time, inflating apparent scope and causing rework. Manual grooming is slow and inconsistent, particularly in large backlogs.
What Was Implemented
- Generative AI assistant integrated into a Jira-like backlog environment
- Human-in-the-loop mode: AI surfaces candidates; human confirms or rejects
- Evaluated in a controlled experiment with 5 participants, 30 issues, 41 known duplicate pairs
- Compared against pure manual grooming (same participants, same backlog)
Results
- 100% precision in human-in-the-loop mode (all false positives rejected by human reviewers) - 45% reduction in time per duplicate found (2 min 54 sec manual → 1 min 35 sec AI-assisted) - Participants rated the AI tool "far superior" to manual systems - Automated mode precision: ~81% (without human review)
Lessons
- Positioning AI as an assistant rather than a decision-maker is critical: human review eliminates false positives without sacrificing speed gains
- Controlled study results suggest strong potential but limited generalizability from a 5-participant, 30-issue design
- Time savings are measured per duplicate found, not as raw session time — framing matters for communicating value