Summary
Test evidence retention ensures that automated test results are captured, linked to specific code changes, and preserved for audit, providing verifiable proof that software was tested before deployment.
Description
Test evidence retention addresses the requirement to produce, on demand, machine-generated records that demonstrate testing occurred for any given code change or release. Without retained evidence, an organisation cannot prove to a regulator, auditor, or customer that its test suite operated as intended at the time a change was deployed. The mere existence of a test suite is insufficient — evidence must be tied to specific commits, cover the expected scope, and survive for the full regulatory retention period.
This control is distinct from test execution (SDLC-PREV-016), which governs whether required testing was performed and any failures appropriately reviewed. Test evidence retention concerns the preservation and retrievability of the artefacts that test execution produces. An organisation may execute testing correctly but still face an evidence failure if the resulting records are not retained in a durable, tamper-evident form linked to the change they attest to.
In regulated financial services environments, the inability to produce test evidence for a released change is treated by examiners as equivalent to the testing not having occurred. Coverage metrics, test run logs, pass/fail records, and the identity of the system that executed them form part of the SDLC governance trail required to demonstrate that software development risk is managed.
Requirements
- Automated test suites MUST be executed as part of the CI/CD pipeline for every pull request targeting a protected branch and every build that produces a deployable artefact
- For Automated test execution results MUST be captured in a structured, machine-readable format (e.g., JUnit XML, CTRF) and stored as persistent artefacts linked to the triggering commit SHA, branch, built artifact, pipeline run identifier, and timestamp or any other references to ensure traceability back to the intented release
- Manual test evidence must be associated with the intented release for traceability, downstream gating and reporting.
- Test evidence artefacts MUST be retained for a minimum period aligned with the organisation’s regulatory audit retention policy; this period MUST NOT be less than the longest applicable regulatory retention requirement
- Retained artefacts MUST be immutable after capture; they MUST NOT be modifiable, deletable, or re-runnable outside an approved and auditable exception process
- Each deployment record MUST include a reference to the test evidence artefact that corresponds to the artefact being deployed, enabling end-to-end traceability from deployment back to the test run
- A defined and documented exception process MUST exist for cases where test evidence cannot be produced before deployment; exceptions MUST be approved by an appropriate authority, time-bound, and automatically tracked to closure
Examples & Commentary
- Artefact Storage: Configure CI pipelines to publish test results and coverage reports as pipeline artefacts. Where the pipeline’s native retention is short-lived (e.g., 30 days), archive to long-term storage such as an object store with immutable object policies enabled (e.g., S3 Object Lock). Record the artefact location in the deployment record
- Traceability: Tag artefacts with the commit SHA or release tag so that the evidence can be addressible later by downstream gating or for reporting purposes.
- Immutability: Use object storage features such as write-once-read-many (WORM) policies or versioning with deletion protection to ensure artefacts cannot be altered after the pipeline writes them. Audit access logs periodically to detect any access pattern inconsistent with normal read-only retrieval