Import CloudWatch Log Groups

Background

We’ve seen current practice on create log group is on application level and using our internal building block (created by BEI). So, the instance role have permission logs:CreateLogGroup. Currently our building block is not support in create tags when the application create log group name.

There are no hard technical issue on updating the building block to support tagging when creating log group. The downside if we put the capability of create log group name in instance (application) level are:

We give wide policy to instances. Which is not less privilege.
When we are doing decommissioning, we need to manually delete the CWL using CLI or console. Since the CWL creation is not part of IaC.

Based on those reasons, we ask you guys to import it for cwl log group in tvlk-prod .
Also we have a plan to take out logs:CreateLogGroup policy from CommonEC2 in future (we will announce it together with BEI). Currently, we are still on stage of analyze it and creating safe strategy to roll-out in multi-account, since its also impact to multi-account. The issue can be see here.

What You Need To Do?

Every new stack, please create CWL log group name as part of IaC and tag it properly.

resource "aws_cloudwatch_log_group" "app_application_log" {
name = "/tvlk/app-java/abc123/application.log"
retention_in_days = "14"
tags = {
Environment = "production"
Service = "abc123"
ProductDomain = "abc"
ManagedBy = "terraform"
}
}

Log group naming follows site infra convention: CWL Logs Groups Naming.

Existing Stack

Import CWL logs and tag it properly. You can see from Terraform Offical Docs.
Do terraform plan and adjust/update it on the terraform config.

Example:
Import on flat terraform config
awsudo -u <profile> -- terraform import aws_cloudwatch_log_group.<resource_name> /tvlk/app-java/abc123/application.log

Import inside module (if you use this Terraform MongoDB Module for create log group)
General: awsudo -u <profile> – terraform import module.<your_terraform_config_module_name>.aws_cloudwatch_log_group.<resources_name_inside_module> <log group name>

awsudo -u <profile> – terraform import module.loggroup.aws_cloudwatch_log_group.mongodb_log /tvlk/mongod/abc123/mongod.log

If your existing log group haven’t following naming convention, please update it.
Reference: CWL Logs Groups Naming.

After you have done above actions, make sure your application configuration put the destination log in the right path/name.

Important Dates

We will decommission log group name in tvlk-prod which do not have ProductDomain Tag on:

Monday - June 18th, 2020T15:00:00 (GMT+7)

If you have any questions, feel free to ask in this thread or in #infra-mentorship channel.
Thank you :tada: