Flight 2022 Shared Staging Deployment Policy

Greetings!
As everyone knows, shared staging (in)stability has remained a challenging, if not frustrating, issue to address despite the improvement attempts we've made the last few years.
We are still pushing for local test practice and enabling comprehensive API/service test as the long term fix, however it may take a significant amount of time until we feel significant stability improvements.
Meanwhile, new features/products can't wait, and so can't the need to relieve some of the testing pain points in the interim.
To illustrate the point better: as per Transport QA’s feedback, QAs typically can only use shared staging reliably for only 2-4 hours in a day because of shared staging instability!
After reviewing reports made in staging incident tracker and discussing with some staging users, we learned that a significant portion of instability comes from dependency problems. Following this insight, we are going to start implementing new deployment window policy for shared staging environment.

Implementation plan

In-scope

Shared staging
All FPR BE services
Infra modifications that cause a downtime to any existing services. e.g.: upgrading/downgrading RDS instance type

Not in-scope

Flight-staging
Desktop & m-web deployments
Infra modifications that do NOT cause a downtime to any existing services. e.g.: creating new RDS that has not been accessed by any existing services.

Effective starting

22 Aug 2022

When to deploy to shared-staging

Weekdays:

Before 09:00 (UTC+7)
After 15:00 (UTC+7)

Weekends / Indonesia's public holidays:

Anytime, i.e. no deployment time window

Please read this document for a more detailed rationale, and let your lead and/or @chandrawibowo know should you have questions/suggestions pertaining to this new policy. Thanks!