Our 10-Point Technical Debt Assessment

This post is part of our historical archive. It represents the beliefs, actions, products, and services of Code Climate as of its publication date. Today, Code Climate focuses on providing enterprise leaders the software development data, context layer, and playbooks needed to build the AI-native software organization their enterprise needs.

Noah Davis
Oct 12, 2017
• 7 min read

In the six years since we first introduced radically simple metrics for code quality, we’ve found that clear, actionable metrics lead to better code and more productive teams. To further that, we recently revamped the way we measure and track quality to provide a new, clearer way to understand your projects.

Our new rating system is built on two pillars: maintainability (the opposite of technical debt) and test coverage. For test coverage, the calculations are simple. We take the covered lines of code compared to the total “coverable” lines of code as a percentage, and map it to a letter grade from A to F.

Technical debt, on the other hand, can be a challenge to measure. Static analysis can examine a codebase for potential structural issues, but different tools tend to focus on different (and often overlapping) problems. There has never been a single standard, and so we set out to create one.

Our goals for a standardized technical debt assessment were:

• Cross-language applicability
  A good standard should not feel strained when applied to a variety of languages, from Java to Python to JavaScript. Polyglot systems are the new normal and engineers and organizations today tend to work in an increasing number of programming languages.
• Easy to understand
  Ultimately, the goal of assessing technical debt with static analysis is to empower engineers to make better decisions.
• Customizable
  Different engineers and teams have differing preferences for how they structure and organize their code.
• DRY (Don’t Repeat Yourself)
  Certain static analysis checks produce highly correlated results. We sought to avoid a system of checks where a violation of one check was likely to be regularly accompanied by the violation of another.
• Balanced (or opposing)
  Tracking metrics that encourage only one behavior can create an undesirable overcorrection.

Ten technical debt checks

With these goals in mind, we ended up with ten technical debt checks to assess the maintainability of a file (or, when aggregated, an entire codebase):

1. Argument count
   Methods or functions defined with a high number of arguments
2. Complex boolean logic
   Boolean logic that may be hard to understand
3. File length
   Excessive lines of code within a single file
4. Identical blocks of code
   Duplicate code which is syntactically identical
5. Method count
   Classes defined with a high number of functions or methods
6. Method length
   Excessive lines of code within a single function or method
7. Nested control flow
   Deeply nested control structures like if or case
8. Return statements
   Functions or methods with a high number of return statements
9. Similar blocks of code
   Duplicate code which is not identical but shares the same structure
10. Method complexity
    Functions or methods that may be hard to understand

Check types

The ten checks break down into four main categories:

Size

Four of the checks simply look for the size or count of a unit within the codebase: method length, file length, argument count and method count.

Control flow

The return statements and nested control flow checks are intended to help catch pieces of code that may be reasonably sized but are hard to follow.

Complexity

The complex boolean logic check looks for conditionals laced together with many operators.

Copy/paste detection

Finally, the similar and identical blocks of code checks look for the especially nefarious case of copy and pasted code.

Rating system

Once we’ve identified all of the violations (or issues) of technical debt within a block of code, we do a little more work to make the results as easy to understand as possible.

File ratings

First, for each issue, we estimate the amount of time it may take an engineer to resolve the problem.

Once we have the total remediation time for a source code file, we simply map it onto a letter grade scale. Low remediation time is preferable and receives a higher rating. As the total remediation time of a file increases, the rating declines accordingly.

Repository ratings

Last, we created a system for grading the technical debt of an entire project. A low technical debt ratio is preferable.

Get a free technical debt assessment for your own codebase

If you’re interested in trying out our 10-point technical debt assessments on your codebase, give Code Climate Quality a try. It’s always free for open source, and we have a 14-day free trial for use with private projects.