Automating negative scenarios is an integral part of a comprehensive testing system. These are checks where the system's resilience to erroneous actions, incorrect data, service failures, and other abnormal situations is validated.
Background:
Previously, the main focus was on positive scenarios (happy paths) because they are easier to automate and maintain. However, quality requirements have increased, and more bugs are now arising at the boundaries with erroneous or not obviously incorrect conditions. Manual testing of these quickly becomes obsolete, whereas automation allows for tracking degradation.
Problems:
Solutions:
Key features:
Is it enough to only check what errors are returned during negative testing?
No. It is important to check not only the content of the error but also that there was no change in the system's state after the error (for example, that an incorrect entry was not added to the database).
Should all possible negative scenarios be automated?
No. There can be infinitely many; it's necessary to highlight the most likely and critical ones (Boundary Value, Null/Empty, incorrect type, SQL injections, etc.), while handling others as bugs arise.
Can the same handling be used for all negative cases?
No. Different negative scenarios require different checks, exception handling, and rollbacks; template asserts are insufficient.
The system only tests valid registration scenarios. Registration with an empty email is not automated. As a result, the problem that a user can register with an empty email is noticed only in production.
Pros:
Cons:
For each negative scenario (missing email, incorrect format, SQL injection), there is an automated test that explicitly checks for the absence of a new account and the content of the error message.
Pros:
Cons: