The coding assistant says all tests pass, but no test log exists. Can I accept it?
Unverified generated engineering claimsRequire actual execution evidence from the authorized build environment. Separate generated test code from executed tests and record failures, skipped checks, and missing environments honestly. Review the change against requirements and security boundaries before accepting the assistant's summary.
What to do now
- Inspect the actual code changes
- Run the relevant tests in the real environment
- Capture results and unresolved failures
- Approve only evidence-supported claims
When to escalate
A consequential release depends on tests that were never executed.
The AI service owner and relevant authorized reviewer
Example
The assistant wrote a browser test but the reviewer records that it was not run until a test server is available.
Mistake to avoid
Accepting a confident success summary as a substitute for test output.
Prevent a repeat
Re-evaluate after changes to models, prompts, tools or source data; keep important decisions accountable to people.
Suggested ApprovedFlow setup
- Tasks
- Documents
- Approvals
Track evaluation and review tasks, and keep approved evidence with the work; do not claim the platform provides an unverified AI agent or automated diagnosis. See the platform
Related cases
- Can I use an AI estimate as the project commitment?Unvalidated model-generated estimate
- Our AI project plan looks complete but misses approvals. How do we validate it?Incomplete generated work breakdown
- Our test suite passes, but production failures continue. What should we inspect?Test relevance gap
- Our integration tests use unrealistic data. What should we improve?Unrepresentative integration evidence