Notify teams of AB tests close to expiry

[cemms1]

What does this change?

Adds a scheduled lambda for the new AB testing framework to notify teams of upcoming AB test expiry dates or to let them know that tests have already expired.

• The lambda is scheduled to run at 5am every weekday for PROD but not for CODE, though in both environments it can be triggered manually
• It fetches active AB tests (those switched "ON" in the code) and filters by those within 2 days of expiry or which have already expired
• It then gets a unique list of AB test owners for those tests and creates three lists of tests for each owner:
  • Expired tests
  • Tests expiring today at midnight
  • Tests expiring tomorrow at midnight
• For each owner of an AB test close to expiry, it sends an email using SES with a breakdown of their expiring tests in tabular format

Example email format

Tested in CODE

Why?

We want to make sure expired tests are turned off and/or removed from the codebase if no longer active. Notifying teams of this helps to keep them aware of this.

How to test

Running locally

• Fetch AWS credentials for the frontend account
• Add a fake test to /ab-testing/config/abTests.ts that either has an expiry less than two days in the future or an expiry in the past. Ensure your email is marked as an owner of that test
• Run the lambda locally (run node src/run.ts from the notification-lambda directory)
• Check that you have received an email for the expiring test

Running in CODE

• Add a fake test to /ab-testing/config/abTests.ts that either has an expiry less than two days in the future or an expiry in the past. Ensure your email is marked as an owner of that test
• Deploy the latest build of this branch to CODE via RiffRaff (make sure no one else is using the CODE environment at the same time by flagging this in the Semaphore chat)
• Run the lambda from the AWS console or by using the AWS CLI
• Check that you have received an email for the expiring test

[github-actions]

Deploy build 367 of dotcom:ab-testing to CODE

All deployment options

• Deploy build 367 of dotcom:ab-testing to CODE
• Deploy parts of build 367 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions]

Deploy build 23526 of dotcom:rendering-all to CODE

All deployment options

• Deploy build 23526 of dotcom:rendering-all to CODE
• Deploy parts of build 23526 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions]

Hello 👋! When you're ready to run Chromatic, please apply the run_chromatic label to this PR.

You will need to reapply the label each time you want to run Chromatic.

Click here to see the Chromatic project.

[deedeeh]

Awesome work @cemms1 👏🏼👏🏼 You've covered everything from the main functionality to testing which I particularly love about this PR ✨

It took me a bit of time to understand few things but I just had to look up for and I am glad I did!

I left just a comment just to understand your thoughts on the implementation.

[Jakeii]

This is great! Should we also notify someone else if the email failed to send to the test owner? Say if someone mistyped an email address?

[cemms1]

This is great! Should we also notify someone else if the email failed to send to the test owner? Say if someone mistyped an email address?

I did wonder that! I have set up an SNS topic for lambda failures but I haven't set up a subscription for anyone to be notified yet... Who do you think would be best to add to that?

[Jakeii]

Will this catch prevent the SNS topic being notified of an error?

[cemms1]

Aha, maybe not... I'll fix this in a follow up PR and test failures properly!

[Jakeii]

I was more thinking of a fallback email of the expired tests, perhaps to [email protected] like the expiring feature switches

But an SNS topic alert also makes sense!

[gu-prout]

Seen on PROD (merged by @cemms1 7 minutes and 36 seconds ago) Please check your changes!