How to Gradually Roll Out Velocity’s Engineering Metrics to Your Engineers

This post is part of our historical archive. It represents the beliefs, actions, products, and services of Code Climate as of its publication date. Today, Code Climate focuses on providing enterprise leaders the software development data, context layer, and playbooks needed to build the AI-native software organization their enterprise needs. Head to codeclimate.com to learn more.

When you discover the benefits of data-driven conversations, it’s likely you’ll want to invite your entire team to view all of their engineering metrics. But an abundance of data can make it difficult to hone in on the most important trends, and data that’s lacking context is more likely to be misused or misinterpreted. That’s why we recommend rolling out Velocity gradually. Then, as your team becomes comfortable with the engineering metrics Velocity reveals, sharing more can become increasingly valuable.

Jan 6, 2021 • 5 min read

When you discover the benefits of data-driven conversations, it’s likely you’ll want to invite your entire team to view all of their engineering metrics.

But an abundance of data can make it difficult to hone in on the most important trends, and data that’s lacking context is more likely to be misused or misinterpreted. That’s why we recommend rolling out Velocity gradually. Then, as your team becomes comfortable with the engineering metrics Velocity reveals, sharing more can become increasingly valuable.

What does a rollout look like?

Velocity is Software Engineering Intelligence platform, built to improve how teams make decisions. To that end, a “rollout” can be as simple as exposing your engineers to the metrics and dashboards through curated slides, or as involved as giving each team member access to the application.

Here are the different levels of exposure you can consider:

• Contextual exposure: Share what data you’re tracking and present relevant reports in meetings with teams and individuals.
• Scoped exposure: Share the Velocity reports you’d like and limit the data scope by team or by individual so that engineers only see the metrics that are most relevant to them.
• Full access: Provide engineers with access to all their own and their team member’s data.

Different teams have varying levels of trust and familiarity with engineering performance metrics, so the right exposure will differ team-to-team. Use the guidelines below to gut-check when and how to introduce each type of rollout.

Introducing the idea of data-driven engineering with contextual exposure

The Challenge: Engineers are naturally skeptical and concerned about engineering metrics, so it’ll be up to you to advocate for the data and engender a positive culture around analytics.

The Preparation: We recommend that you work with your CSM to concretely define problem statements and get to know the dashboards and metrics that will support your objectives before sharing the tool with your team.

For instance, your objective may be to decrease batch sizes in an effort to improve the efficiency of your Code Review process. In this case, your supporting metrics may be Pull Request Size, Time to Open, and Review Speed.

Move on to rollout if you can answer yes to all of these questions:

• Can you articulate the problems that Velocity data will help you better understand?
• Do you know how the metrics you’re tracking connect to the problems you’ve defined?
• Will you be able to defend and explain these metrics to your team?
• Will your team feel confident that you’ll be using these metrics in a way that will benefit (not hurt) the team?

The Rollout: Introduce your problem statements and supporting metrics to your team. Have concrete examples of how you’re already working through a previously discussed problem using data.

Empowering engineers to own their success with scoped exposure

The Challenge: It’s one thing to periodically show org-wide metrics during team and individual meetings, but another to give engineers access to their own metrics. Even engineers who are interested in becoming more data-driven in theory can be wary of metrics when they’re frequently confronting their own dashboard.

The Preparation: To earn buy-in for limited use of Velocity, you’ll want to demonstrate how using data can help make 1:1s and retros more actionable. Use one of these meetings as an opportunity to find a concrete unit of work that represents an engineer’s experience or frustration. It can be helpful to involve a senior engineer earlier on, so they can pair what they know from experience with what is being surfaced in the data. Begin using Velocity to start conversations with real-world examples, so engineers can understand how data can be used to distinguish the signal through the noise.

Move on to rollout if you can answer yes to all of these questions:

• Can you correlate the experience of developers to specific PRs that Velocity is highlighting?
• Do you have a senior engineer who can advocate for the positive intention with which Velocity will be used?
• Do you have a sense of which metrics will be improved should you work through the identified challenges?

The Rollout: Work with your CSM to set up data scopes to display the appropriate reports and metrics for each engineer. At the end of your next retrospective, demo the Velocity product for your team. Encourage them to periodically view their dashboards and check in on key metrics at each retrospective so your team achieves the desired outcome.

Creating an autonomous team with full access

The Challenge: Introducing engineers to all of Velocity’s metrics means that not only does leadership have to interpret the metrics responsibly, but so does every single individual on the team. Exposing all of your metrics requires an extremely safe and trusting team environment.

The Preparation: First, you’ll want to show how setting quantifiable goals can benefit developers. Use a retrospective meeting to identify one significant problem they’re experiencing and work together to identify key metrics that will help work through the stated problem.

Move on to rollout if you can answer yes to all of these questions:

• Does your team have an identified problem that Velocity metrics will help work through?
• Have you identified both leading (health) metrics, and lagging (success) metrics for your stated problem?
• Can each team member explain why each metric was chosen in association with the stated problem?
• Do you have a hypothesis about what success looks like with these metrics?

The Rollout: Work with your CSM to set up data scopes to display all the reports and metrics for each engineer’s team. At the end of your next retrospective, demo the Velocity product for your team. Encourage them to periodically view their dashboards and check in on key metrics at each retrospective until the desired outcome is achieved.

Reach out to get started with Velocity, and find out how engineering metrics can help your team meet its goals.