fb-mongod is one of the oldest legacy, shared databased used in traveloka.
Currently the contents of fb-mongod are occupied primarily by booking data of products which are launched prior to 2018 (flight, accommodation, F+H, train, connectivity, experience, and airport transport).
Other than booking data, fb-mongod also contains a multitude of smaller collections (data tables) which is being utilized by various teams (see Data Distributions section below)
Reliability Risk
As with any shared infrastructure, if regression happens to the database, it will affect all of its clients since it won't be possible to do proper isolation for the impact. In the case of fb-mongod, since it currently serves the booking flow for almost all major traveloka products, failure to this database will translate as a sitewide impact to the business.
Capacity Risk
fb-mongod currently runs a 4.8 TB disk and per January 2021 the disk usage capacity has touched 92% usage. If no action is done, with current data growth rate, disk capacity is projected to be filled up around October 2021.
Scalability and Operational Risk
Due to the bloated state of the database, it’s very difficult to utilize / introduce new use case to the existing data stored there. A simple action of adding a new index to an existing collection have been known to introduce regression to the database performance.
Having multiple services owned by different teams connect directly to the database also makes it hard to implement proper operational / security practices for the database.
Outdated Technology
fb-mongod is currently running MongoDB 2.6 which has reached it’s EOL support on October 2016.
A lot of improvements and new features in MongoDB will not be able to be utilized on fb-mongod. Along with several undesirable features of older MongoDB which fb-mongod is forced to live with, e.g. MMAPV1 engine, database level concurrency locking, etc
Ineffective Cost Attribution and Spending
On 2019 fb-mongod cluster alone costs about ~$11,000 / month. Since March 2020 fb-mongod has been downgraded to follow the reduced traffic due to COVID-19 pandemic, so the cost is currently ~$5,000 / month. However as Traveloka’s traffic recovers, eventually the database might need to be upgraded again.
Also, since this is a shared database, it is very difficult to attribute the usage cost across the different teams that utilize it.
Blocker for Major Tech Initiatives
as a tech debt, fb-mongod makes it hard / slow down velocity in doing improvements to systems which still have coupling to this database. For example, this will be a blocker for full multi account migration for services that still have direct connection to fb-mongod.
Please identify ownership of your collections by filling the data distribution spreadsheet below by February 19th, 2021 at the latest. Access for orphan collections with no owner will be revoked on the following week.
Afterwards, you will need to migrate your collections data from fb-mongod into your own database before the shut down schedule on November 3rd, 2021.
* a separate announcement will follow specific for agent.booking collection (more info on the confluence post)
Collection distributions in fb-mongod can be seen in the following document : https://docs.google.com/spreadsheets/d/1ciVenXJvxmaOgnG3PGLlj6ToJsYbKMSQGFSQOOlHssY/edit#gid=391611078
Confluence post : https://29022131.atlassian.net/wiki/spaces/USER/pages/1592790322/fb-mongod+decommissioning