Kafka Decommission (1 March 2021)

Background

We have been double writing tracking events to both Kafka and Google Pub/Sub all this time. It is costing us a lot of money to have two infrastructures, not mentioning maintenance cost of the self-managed Kafka. Data team folks have migrated data ETLs out from Kafka downstream. So, we are ready to decommission Kafka to save cost.

We are going to turn off the Kafka servers on 1 March 2021 12PM (GMT+7)

What do I need to do?

Monorepo
We have turned off Kafka publishing from backend-monorepo (see this PR) by changing the config files. We need your help to release your services to include the commit c7b970f before 1 March 2021.

Multirepo
We need your help to change your tracking config files with simple change like this and release your services with the change before 1 March 2021.

Do I need to change/release my services?

Yes, if these two conditions are met:

What will happen if I don’t change/release my services before 1 March?

Decommission plan