[Action Required] NEW Unit Test Code Coverage Quality Gate Roll-Out

What is this about?

We will terminate SonarCloud on 11 January 2025. One of the main SonarCloud features that we rely on is Unit Test Code Coverage Quality Gate, this is to ensure that New Code always have sufficient Unit Test Coverage. We have built a solution to replace this functionality. Thus, we are starting the roll-out process to ensure that by 11 January 2025, all repositories have implemented the new solution.

Who is this announcement for?

All backend engineers

What you need to do?

Please read and apply https://traveloka.roadie.so/docs/default/component/traveloka-backstage-docs/quality/testing-code-coverage/ to all Java repositories. Make sure to test the change by running ./gradlew test jacocoTestReport testCodeCoverageReport . The jacocoTestReport and testCodeCoverageReport tasks shouldn't report any errors.

Timeline

13 December 2024	BEI to update bei-springboot-template to use plugin 8.3.8 [DONE]
16 December 2024	BEI to update BFlow to execute diff-cover and Jacoco report aggregation if possible, in addition to running sonar command. Comments about new code coverage from godwitbot will start to appear on PRs [DONE]
~~6 January 2025~~ 8 January 2025	BEI to update BFlow to stop running any Sonar command for repositories that have applied new plugin [DONE]
10 January 2025	BEI to remove SonarCloud required status check from all repositories still having that status check. [DONE] BFlow to stop running any Sonar command [DONE]
11 January 2025	SonarCloud is terminated
13 January 2025	BEI to update BFlow to enforce New Code Coverage checks, at least 60% of new code in the PR must be covered by tests. The percentage will be increased slowly back to 80% [DONE 60%]

Questions/Concerns?

Should you have any questions or concerns, feel free to reply to this thread and/or reach out to @U0XL5NYUF and/or @U08G58KL3.

FAQ

Q: The Job summary for the CI starts reporting failure for sonar or testCodeCoverageReport. Is this ok?
A: Yes, until 6 January 2025, it's ok for one of the task to fail. Sonar will fail if all Sonar configuration is removed from the repository, while the testCodeCoverageReport will fail if the new plugin version hasn't been applied. What's not ok is for both of those tasks to fail in the same CI. So pic below is not an issue.

Q: The error below appears when trying to run any gradle command after doing all the changes needed. What is the cause?
A: The most common cause is the beiartf token being invalid, perhaps due to expiration. Rerun dev_login.sh and try running the gradle command again.

* What went wrong:
Plugin [id: 'com.traveloka.common.gradle.gradle-settings', version: '8.3.8'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Included Builds (No included builds contain this plugin)
- Plugin Repositories (could not resolve plugin artifact 'com.traveloka.common.gradle.gradle-settings:com.traveloka.common.gradle.gradle-settings.gradle.plugin:8.3.8')
  Searched in the following repositories:
    maven(https://org-codeartifact-domain-015110552125-242c507c3f-015110552125.d.codeartifact.ap-southeast-1.amazonaws.com/maven/org-backend_external_cache-maven-015110552125-4c9d05c06c701f074074/)
    maven2(s3://beiartf-maven-artifacts-015110552125-aeb600c7e0343453.s3.ap-southeast-1.amazonaws.com/release)
    MavenLocal(file:/Users/<your_username>/.m2/repository/)

Q: An error about cannot add a ReportSpec when applying plugin base-root
A: The most common fix is to remove `apply plugin: java` from root build.gradle

Where:
Build file '/Users/<your_username>/Project/acd-marketing-backend/build.gradle' line: 51

What went wrong:
A problem occurred evaluating root project 'acd-marketing-backend'.
> Failed to apply plugin 'com.traveloka.common.gradle.base-root'.
   > Cannot add a ReportSpec with name 'testCodeCoverageReport' as a ReportSpec with that name already exists.

Q: When running testCodeCoverageReport, there is an error can't add different class with same name
A: Since we are aggregating Jacoco report from all modules into one, this has some side effect in that the combination of package and class name must be unique across the whole repository, not just in the module itself. Either exclude one of the modules, or exclude the class.

Caused by: java.lang.IllegalStateException: Can't add different class with same name: com/traveloka/flight/test/Bar
    at org.jacoco.core.analysis.CoverageBuilder.visitCoverage(CoverageBuilder.java:106)
    at org.jacoco.core.analysis.Analyzer$1.visitEnd(Analyzer.java:100)
    at org.objectweb.asm.ClassVisitor.visitEnd(ClassVisitor.java:395)
    at org.jacoco.core.internal.flow.ClassProbesAdapter.visitEnd(ClassProbesAdapter.java:100)
    at org.objectweb.asm.ClassReader.accept(ClassReader.java:749)
    at org.objectweb.asm.ClassReader.accept(ClassReader.java:425)
    at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:117)
    at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:133)
    ... 152 more