NeFut Logo NeFut
中 Admin Login

[CS.AI] KG-AST for Cheaper CI/CD: Budgeted Test Selection and AI Bug Review

Published at: 2026-11-07 22:00 Last updated: 2026-11-08 12:04
#AI #Machine Learning #Open Source

This paper introduces KG‑AST, a technique that allocates CI and AI‑assisted review resources under a fixed budget. Conventional pipelines run the same heavy tests and AI reviews for every commit, even though most commits do not introduce bugs. KG‑AST first scores each commit’s risk using git history and code‑graph features, then runs heavy testing or AI review only on the riskiest commits, providing the AI reviewer with a graph slice around the change.\n\nOn 155 open‑source projects spanning 11 languages, a single cross‑language model trained on other projects ranks commits so that testing the top 20% risky commits captures 49%–64% of bug‑introducing commits (language‑dependent), compared to ~20% for random selection and 40%–55% for selecting the largest change first. Every language outperforms the largest‑change baseline by 3.8 to 13.2 points. For Java (20 projects) the gain at a 20% budget is 15.1 points (64.6% vs 49.5%, 95% interval +11.6 to +18.4), and the model wins in 19 of the 20 projects.\n\nOnly about twelve training projects are sufficient; adding richer features, hyper‑parameter tuning, or cleaner labels yields no measurable improvement. An AI reviewer that sees only the diff finds 26% of later‑fixed defects exactly, rising to 32% when graph context is added. About half of the findings that do not match a later fix are judged real by independent AI judges. A static call‑graph selector runs roughly 8% of a test suite while including a test written for the fix in 85% of 67 fix commits.\n\nThe Poincaré embedding used for retrieval adds no measurable benefit: commit‑risk AUC, bug‑review performance, and severity ranking remain unchanged or worsen. Labels are noisy and the budget metric assumes a heavy test catches the bug it covers; actual CI minutes were not measured.\n\nReview: KG‑AST demonstrates a practical way to concentrate limited CI resources on high‑risk commits, achieving a substantial increase in bug capture across languages. While label noise and the lack of real‑time cost measurement limit the analysis, the large‑scale open‑source evaluation validates its effectiveness.

Original Source: https://arxiv.org/abs/2611.06950

[h] Back to Home