Notify Data Team for Data Model or Database Changes

tl;dr

Since we depend on ingestion to retrieve data from production services database, we need your help to notify Data Team for any changes on Data Model (e.g. change data type or field name) or Database (e.g. move from mongo to postgres, or to memcached).

Notify Data Team by submit a ticket to each domain's JIRA Service Desk according to its mission. Service desk can be found here:

https://29022131.atlassian.net/servicedesk/customer/portal/25 go to the last option (Data model/DB changes)

Then, the Data Leads in the particular mission will assign the ticket to respective DE PIC within each of the mission. DE PIC within each the mission will then assess the changes impact towards business continuity and provide proper response/feedback to PEs. That way we could ensure reliability of insight delivered to you.

Ingestion: query data from hidden secondary / read replica and write it into S3 data lake.

Longer Story

There are two means to get data from production services: ingestion and tracking. Ingestion is querying data from hidden secondary / read replica and write it into S3 data lake periodically. Tracking is production service actively push data to our tracking service via tracking library.

Since ingestion query data to production database hidden secondary, it will be affected by changes on production database, be it changes on data model, or changes on database technology itself. In the long run, we are planning separate two use cases:

data that's need to be financially correct will be the only one that need to be ingested
other than that will go via tracking

Just like any other migration, it takes some preparation to achieve to that point. (We are currently doing pilot on doing tracking for ingested data with Hodftel canonical data).

In the mean time, we'd like to ask you to be involved in data change management by notifying Data Team for any changes on Data Model (e.g. change data type or field name) or Database (e.g. move from mongo to postgres). Notify Data Team by submit a ticket to notify Data Team by submit a ticket to each domain's JIRA Service Desk according to its mission. That way we could ensure reliability of insight delivered to you.

Thank you.
Data Team