A
argbe.tech - news
1min read

Meta’s JiTTesting Pitches PR-Scoped, Disposable Tests as a CI/CD Gate for Agentic QA

Meta outlined JiTTesting, a just-in-time approach that generates transient, change-specific tests for each pull request. The aim is to cut test maintenance and use the generated suite as a pull-request gate for agentic development.

Meta described JiTTesting (ArXiv:2601.22832) as a pull-request gate for agentic development, using LLMs to generate transient, change-specific test suites instead of maintaining permanent test code.

  • JiTTests are created on-the-fly for each PR and are not meant to live in the repository long-term.
  • A “Catching JiTTest” step uses mutation testing, intentionally inserting faults to check whether the generated tests actually detect failures.
  • The protocol relies on ensembles of rule-based and LLM-based assessors to filter noisy signals and prioritize true-positive failures over false positives.
  • Tests are designed to infer the intent of a change and probe for faults that are specific to that change, rather than re-running a generic suite.
  • By treating tests as disposable artifacts, the framework removes the need to code-review test files and pushes attention back to concrete bug reports.