TVLK-AST-STG Transition & Migration Plan
Please find the detailed doc here.
Introduction
We would like to initiate the migration of our dev environment from tvlk-dev to ast-stg.
There will be a transition period before we fully committed to ast-stg. During the transition, we encourage all of us to use ast-stg directly as dev environment; though tvlk-dev would still be available to use if there is any unexpected behavior or error exclusively found in ast-stg.
There are other challenges during the transition, one of them is related to our data store location. Because central team won't allow VPC peering between new-account and tvlk-dev, ast services won’t be able to connect to databases in tvlk-dev. So there will be some database copy and sync activities from tvlk-dev to ast-stg and vice versa.
Transition & Migration Plan
Please find the plan below, structured by day to day basis.
Friday, 1-Nov-2019
- Cutoff TERA v2.31 release branch, will start not later than 10 AM
- Deploy all services to both of tvlk-dev and ast-stg
- Replace mongod in ast-stg by using mongod images from tvlk-dev
- From now on, all engineers are expected to do double write for any requiredly new DDL and DML for their features into both datastore in tvlk-dev and ast-stg
- End to end testing in ast-stg by QA and Multi-Account (MA) team (and perhaps by other engineers)
- During this time, other engineers would still be able to develop and test their features in tvlk-dev, but there might be a case when this is not applicable, such as when they need to trigger the features from hb because hb at this time would pointing to astresv
- If any engineers doing testing in tvlk-dev (e.g. deploy from non 2.31 release branch), they should deploy back to 2.31 version after the test is finished
- If any engineers found bug related to their features, they are expected to double push their fix into 2.31 release branch and deploy to both tvlk-dev and ast-stg
- MA team will try to fix any anomalies exclusively found in ast-stg during end to end testing
- Hopefully all anomalies would be fixed at the end of the day
Monday, 4-Nov-2019
- If ast-stg turns out to be still unstable, we will use tvlk-dev back as our main dev environment
- MA team will work on to fix remaining anomalies in ast-stg
- After all anomalies has been fixed, MA team will post announcement to use ast-stg back again
- If ast-stg is stable, we will use ast-stg exclusively from now on
- MA team will announce to all domains that AXT/ AST resources in tvlk-dev are deprecated and is now taken into decommission plan (exact plan detail and date is TBD)
- MA team will gradually stop all of our resources in tvlk-dev
- MA team will fully decommission all of our resources in tvlk-dev
Tuesday, 5-Nov-2019
- If ast-stg still unstable, MA team will continue work on to fix any remaining anomalies in ast-stg
- Repeat the process until ast-stg is stable
For any questions and concerns, please reach me out directly @micky
Thank you.