New Process for Payment Backend Engineers
Hello, the awesome payment backend engineers,
There will be some new changes that we’d like to introduce as part of our engineering process, please keep in mind that all of these processes are iterative and always open to feedback and objections, we will also regularly review on how are we doing with these processes.
[1] Two-layers Review as a replacement for Payment Review Board
Please see Review Board Feedback Discussion for more detailed information
Background:
- Based on the collected feedback, the majority of the engineers are still willing to have a process to improve our code review
- There are some concerns related to our recent Payment Review Board which we weren’t expecting when we initially introduced this board e.g. discouraging peer reviews,
Objective:
- Improve the quality of code by having more pair of eyes when reviewing our code
- Eliminate exclusivity impression from the engineers
- Re-expose the importance of having peer reviews
- Improve review speed by having more reviewers
Details:
- There will be no more review board, or in essence, everyone is part of the board now ;)
- Each Pull Request/Code Review will require two approvals
- It is encouraged to get reviews from people who might know more context (domain or technical expertise) or might be more meticulous than you
- Sharing knowledge and experience that learned from Code Review or during Development. A confluence page has been created and everyone is welcome to contribute.
Effective Date: 2 June 2020
PIC: @dennis.dao
—
[2] Oncall Journal and Handover
Please see Oncall Journal & Handover for more detailed information
Background:
- We often miss or neglect any important events that usually found during on-call activities (reported bugs, repetitive issues, important changes)
Objective:
- Keep track of important stuff during the on-call period that usually overlooked e.g. reported bugs, repetitive issues, anomaly metrics or alerts and make sure all of them are properly addressed
Details:
- There will be on-call journal (a confluence page) which will be initiated by the respective on-call engineer during their on-call period
- Any important event will be tracked in this journal e.g. reported bugs or issues, announcement, incident
- This journal will be used as a handover to the next on-call engineer so the next on-call engineer will get the context on what was going on and what thing to take care
- Engineering leads will have a regular session (Monthly Tech-ops Review) to review on-call journal in the past 1 month to make sure we address all important items
Effective Date: 2 June 2020 (@kemal will be responsible for guiding the initial writing of the journal and use the first journal as a benchmark for the next following weeks)
PIC: @kemal
—
[3] Regression Test as Weekly Release SOP
Please see Regression Test as Weekly Release SOP for more detailed information
Background:
- We currently have no clear SOP on our regular release that reduces the possibility of regression to happen after doing production release
- Release managers are still overwhelmed with pressure when doing the release due to low confidence
Objective:
- Increase release manager confidence while doing a production release
- Reduce possible cases of rollback especially those we can detect early in the staging environment.
Details:
- The release manager will be responsible for making sure all API test scenarios are passed by the release candidates. This will become a condition to be fulfilled before releasing the services to production.
- In case there’s a failed test or error, the release manager is responsible for finding out if a) it is related to someone’s changes or b) it is because the staging environment is unstable and the release manager will act accordingly case by case.
- Additionally, since our API testing does not completely cover all test scenarios yet, release managers are encouraged to pick up some scenarios to implement during their on-call period.
Effective Date: 2 June 2020
PIC: @emily
—
For any inquiry about the above processes, you can reach out to me, the respective PIC or your lead, and again we really appreciate your input and feedback.
Thank you!