How to Use DORA DevOps Metrics in Large Organizations

This post is part of our historical archive. It represents the beliefs, actions, products, and services of Code Climate as of its publication date. Today, Code Climate focuses on providing enterprise leaders the software development data, context layer, and playbooks needed to build the AI-native software organization their enterprise needs. Head to codeclimate.com to learn more.

Jan 18, 2024

Using DORA DevOps metrics to communicate with leadership

Understanding the performance of engineering teams at large companies is no easy feat. For many, this is due to the diversity of processes across teams and the resulting complexity of collecting consistent data. Companies need a standard way of measuring and understanding engineering performance and a common language to communicate it to company leaders and individual contributors. In this article, we’ll discuss how large organizations leverage DORA metrics to do just that.

In startups, engineering actions are often more directly linked to business goals, making it possible for leaders to understand what engineering is doing and communicate its impact. For example, if a startup is launching its flagship product, contributors from sales, marketing, and product management collaborate with engineering, often with executive support and oversight, to ensure the business goals are met.

Now scale that same workflow across dozens of teams launching and maintaining different products on varying timelines. While engineering will aim to align goals with business objectives, those goals may vary from team to team, and success will look different for each group. That’s why it’s crucial to establish which metrics are important to the company as a whole and create a framework to measure them.

Using a framework like DORA is a great place to start. The four DORA metrics, Deployment Frequency (DF), Mean Lead Time for Changes (MLTC), Mean Time to Recover (MTTR), and Change Failure Rate (CFR), can be communicated to leadership to give them a holistic view of how the engineering organization is performing.

Supporting developers with granular engineering metrics

Using DORA as the foundational framework across teams lets engineering leaders understand how a team is doing within the context of the broader organization and drill down into data from a specific team to learn more about the way it's working. DORA metrics can highlight areas worth attention, serving as a starting point from which managers and their teams can investigate the efficacy of their processes and make changes, then track the impact of those changes.

To do this, they can add context to the four DORA metrics and pair them with complementary metrics to get more insight into what’s happening with individual teams and what improvements might be useful. Common metrics pairings include:

• Change Failure Rate and Unreviewed Pull Requests. If a high CFR is correlated to a high percentage of unreviewed PRs, managers may consider adjusting the process to prevent unreviewed PRs from being merged and causing issues.
• Deployment Frequency and PR Size. If DF is low, managers can use PR size to investigate it. If large PRs are correlated with a low DF, they can coach team members to break work into smaller PRs to see if DF improves.
• Mean Time to Recovery and Revert Rate: A long MTTR could indicate a high Revert Rate, therefore disrupting production and lengthening the time it takes to recover from an unplanned outage or defect. If there’s a correlation, managers can drill down into each revert and see whether the issue is a defect or an undesirable change.
• Mean Lead Time for Changes and Cycle Time. If MLTC is high, it could indicate that Cycle Time is also high. Managers can view these metrics in tandem and dig deeper into related metrics like Time to Open, Time to Merge, and Time to First Review to find the root cause.

How DORA software creates a common language from ICs to the C-suite

Large companies can benefit from a Software Engineering Intelligence (SEI) platform to understand engineering performance at every level of the organization. It allows engineering managers to standardize measurement and reporting on the four DORA metrics to communicate performance to company leadership and ensure that the pace of work meets business needs. Managers can also combine DORA with other engineering metrics in their SEI platform to communicate with their teams to ensure they have what they need to be successful and roadblocks are quickly identified and removed.

Without a strong framework and a centralized platform to measure it, engineering data can become a tangled mess as the number of engineers at a company increases. Measuring DORA and complimentary engineering metrics in an SEI platform helps leaders make sense of their data to ensure that engineering work is optimized and aligned with business objectives.

To find out more about how an SEI platform can benefit leaders at large organizations, request a consultation.