Automated Testing (IT)Automation QA Engineer

How to automate testing of email notifications: history, challenges, and solutions?

Pass interviews with Hintsage AI assistant

Answer.

Background:

The automation of email notification testing became relevant with the development of alert systems (for example, for registration, password changes, reminders). Initially, such tests were performed manually and checked through real email (which is extremely ineffective and unsafe in CI/CD). Then specialized tools emerged (MailHog, Mailtrap, fake SMTP servers) that made automation possible and safe.

Problem:

The main difficulties are related to the fact that email is sent "outward" from the system, so it is necessary to: a) intercept the sending, b) verify the content of the email, c) check the correctness of sending (header format, presence of attachments, links, i18n, correctness of the correspondence with events in the system). There are additional challenges related to delivery speed, clearing inboxes between tests, handling SMTP errors.

Solution:

  • Use specialized email intercepting services: Mailtrap, MailHog, or integrated fake SMTP servers that do not deliver emails to real mail but keep them in a test "basket".

  • Integrate with these tools via API frameworks or REST wrappers and implement waiting for emails, checking key email parameters, validating attachments, and ensuring the presence of correct links.

  • Generate unique email tricks for tests, isolate test data (for example, using a test domain), and clear inboxes before each test run.

  • Check not only the body of the email but also the operability of links within it, sending from different SMTPs, and correct display for different locales and formats.

Key features:

  • Use of "fake" email boxes, SMTP interceptors.
  • Checking the content of emails, links, headers, attachments.
  • Working with API for clearing/verifying the inbox before and after the test.

Trick Questions.

Can you test email notifications by sending emails to regular inboxes in gmail/yandex and checking them through IMAP?

No, this is often done, but this approach is unreliable (dependency on the operation of an external email service, spam filters, no isolation between tests, difficulty in clearing the inbox, delivery is not guaranteed).

Is it enough to just check that the email was received?

No, the body, attachments, presence of necessary links, correct language template, correspondence of headers (From/To/Subject), and correctness of mime structure must be validated.

Is it okay not to clear the fake inbox between scenarios in the test?

No! If you do not clear the inbox between tests, confusion will quickly arise, and catching bugs will become impossible. Scenarios will "see" each other's emails, and tests will become invalid.

Common Mistakes and Anti-Patterns

  • Using real inboxes that are not isolated in CI/CD.
  • Only checking the fact of receiving the email without checking content and attachments.
  • Failing to clear the inbox between tests.

Real-World Example

Negative Case

Tests on email were built through a test gmail, logging in via IMAP, only checking that "something arrived" in the inbox. Emails often ended up in spam, some emails did not reach, and after running parallel tests in one inbox, analyzing emails became impossible.

Pros:

  • Quickly launched notification checks without separate infrastructure.

Cons:

  • No isolation: other people's emails interfere with analysis.
  • Random test failures due to issues with external mail.
  • Attachments, links, templates are not validated.

Positive Case

Implemented Mailtrap, allocated unique inboxes for each test case, implemented automatic clearing, content and attachment validation, waiting period, and language checks.

Pros:

  • Reliable automated verification of all details of email notifications.
  • Fast, isolated tests without external dependencies.

Cons:

  • Time was required to implement the infrastructure (fake SMTP, API integration).