Proper versioning and integration of automated tests are critical to ensure that checks correspond to the current state of the project.
History of the issue
Initially, automated tests were often maintained separately from the main project, leading to incompatibilities and support issues. The evolution of multiple branches, frequent software and test releases created the need for a unified version control system.
Problem
Without versioning and coordinated integration, the following issues arise:
Solution
Modern approach:
# General approach: git checkout -b feature/new_login # Feature and tests are developed and tested together # After review, they are merged together into the main branch
Key features:
Can tests be stored in a separate repository (from the project code)?
Yes, but it becomes harder to maintain the relevance of the tests, manual synchronization is required, and there is a risk of "forgetting" to update something during a release or bug fix.
Should tests immediately cover all new functionality when creating a PR?
Ideally - yes, practically it often happens that they cover MVP/main scenarios in the first PR, and complex cases are handled as separate tasks. The main thing is that critical functionality is covered immediately.
Can only test changes be rolled back without rolling back code?
If tests and code are together in one branch - yes, revisions can be rolled back. But it is better to avoid "rolling back" tests without code: this worsens the quality of verification.
Project with a separate repository for automated tests. After the release, developers "forgot" to update the tests - tests failed, outdated checks passed, bugs were caught in production.
Pros:
Cons:
Tests and project code are maintained in one git branch: with every new pull request, automated tests are invariably updated for the added code. All changes undergo code review and automated checks.
Pros:
Cons: