Some code checking notes

KYC and MF Public API

There are incidents that the disk on mfpapi get full so fast. One of the reason for this is because the log is a lot. One of the reason is because we log every requests spec and response. Some possible solutions:

Log only if there are failure (log is only useful for debugging normally, right?)

This can be done by creating API filter (ex: https://bitbucket.org/tv-plutus/plutus/pull-requests/2277)
Send the log directly to cloudwatch without touching disk (need infra support for this), (can add resource consumption because require network and threads).

Danamas Loan Purchase Request

Every time user make purchases, we add purchase detail to S3 to be processed later on using workflow, this has some possible problems:

add latency during purchase, need to add async task to upload to S3. Solution:

Make async

hard to query the data later on, the function of using S3 can be replaced by using database which is better for tracking and marking statuses. (example: notification db in workflow db). Suggestion:

Use DB instead

Possible Problems (Maybe Design)

Register User for KYC in Danamas

This process is submitting user id to SQS and there is a consumer that gonna consume it, I think this was for async purposes.

The consumer create kyc to danamas and then update the status in the excel sheet that existed on google drive.

There is a tiny chance of record lost because the code didn't check when updating file if the file is still the same (wasn't changed by other thread). Solution:

Add checksum check when trully want to update the file.

When doing this, the code call mfuser via RPC and the process takes on average 3s, this can trigger false monitoring, right now it is saved because other method latency are way2 lower, so the warning become small. Solution:

Put the service/logic under one running instances.

This code resides in helper, thus isn't legal to be exposed to RPC call, gonna hard to back fill later on. (P.S. I personally hate helper classes, so might be bias, helper is like a service without responsibility). Solution:

Make it/ combine it into a responsible service

Money source code (ex: danamas)

If you see, some of the code are in the ops, mgmt, and user services (and maybe KYC). This has possible several problems.

Maintanability

It's gonna be hard to track danamas related logic because it's not really in one place.
Also because it runs accross services, might trigger false problem on other service (ex: user rpc latency high, but unrelated to user code). Also right now, this cause mgmt to call real person, which are not supposed to be.
Possible solution:

Create ops-danamas-api/impl (like ops-cust-api/impl)
Create danamas-api/impl
Create mfdanamas/mfdnms

Extensibility

There is no abstraction for money source yet, it's gonna be quite hard for adding new one right now.
Possible solution:

Create abstraction

KYC code

Even thought the code module are separated, KYC code actually runs on mfpapi. This can cause problem in the future, such as when KYC has bug/problem, activated user can't repay loan, or see loan detail. Possible solution:

Create mfkyc

User Service

The problem is this service sooooo god damn huge.

It might cause scalability and operation problem in the near future.

Possible solution

Don't add anymore new domain to existing service (esp: UserInfoService), create a new one instead (ex: UserLocationService, DanamasService, KycService, UserLoanService).
Move out userCommunicationService and PdfRenderer probably.
Do some refactoring on this.