Blog Category: Case Studies | Code Climate

OpenTable Leverages Code Climate To Predictably Roll Out Platform Rewrite

Jan 25, 2024

OpenTable is a renowned leader in restaurant tech, with products that anticipate the ever-evolving needs of both restaurants and diners. Its website, opentable.com — which helps restaurants fill more than 1.6 billion seats a year — is supported by globally distributed teams of engineers.

In 2019, the site was rearchitected, and OpenTable was faced with the challenge of bringing together many teams as the commercial site was rebuilt as a monolith. During this project, Code Climate supported engineering leaders in gaining further visibility into their engineering processes, which enabled them to bring consistency and reliability to development — ultimately empowering their teams to collaborate more effectively.

Reliable Engineering Management with Code Climate

John Nolan, Senior Director of Engineering at OpenTable, said their approach to rearchitecting was essential for consolidating ideas and ensuring consistency across many globally distributed teams. Code Climate helped them to better measure effectiveness and more easily identify bottlenecks that might hold up progress.

After learning about the benefits of Software Engineering Intelligence platforms at a Code Climate event, Nolan knew Code Climate could provide the insight OpenTable needed to more formally measure engineering impact.

Code Climate delivered insight into engineering processes so that the business could better understand if their changes were effective. For instance, he was able to see the changes over time in Code Climate for metrics like Pull Request (PR) Throughput and Cycle Time.

The goal was to work with Code Climate and identify opportunities to help the teams work together more effectively. Ultimately, they would support improvements in efficiency and productivity as they improved communication and collaboration. In the first year, PR Throughput per Contributor increased by 40%. With more people working more productively, the team realized gains in output.

“Code Climate gave us valuable data to see that the things we were doing were solving the problems that we needed to,” Nolan said. “We saw metrics get better because we were building the right things in the right way.”

Critical Insight for Engineering Leadership

Nolan is in a new role and another step away from the code, and he finds the insight from Code Climate valuable in understanding the complexities of the engineering organization at OpenTable. Both standard and custom reports have proven helpful in identifying activity patterns and maintaining development speed across their widely distributed teams. He said that, while individual engineering managers often know about their own teams’ activities, the visibility Code Climate provides is useful in identifying trends and understanding team-by-team activity.

OpenTable has expanded Code Climate use and has now deployed the platform across all product engineering teams throughout its global engineering organization. Because Code Climate built its solution with enterprises in mind, with a unique ability to ingest and normalize data with reliability and speed at scale, supporting the engineering team at OpenTable to better visualize metrics and understand engineering impact and team health.

“It's invaluable to have a glance at teams' health and have the data necessary to understand where we need to address any issues,” he said.

After OpenTable again expanded their partnership with Code Climate, it has further increased visibility and supported leaders in identifying more opportunities to improve throughput and move faster as an organization. With that information, it was possible for the organization to make changes that kept PR Throughput high while decreasing Cycle Time by 57% and Time to Merge by 23%.

Code Climate supports OpenTable with the data they need. With Code Climate, they have the visibility and understanding necessary to maintain consistency across teams, ensure efficiency, and cultivate a culture of engineering excellence throughout the organization.

Key Highlights

Velocity Drives Data-Driven Decision-Making at Alloy

Dec 6, 2023

Alloy, a leading provider of financial risk solutions, was founded on a simple idea — the more data you have to make a decision, the more confident you’ll be in that decision. So, it's no surprise that Code Climate’s Velocity, a Software Engineering Intelligence (SEI) platform that provides rich insights on team performance and impact, has been a critical resource for their organization since 2021.

Velocity connects to the tools their teams use, then extracts and interprets data with added context to provide deep insights, analysis, and visualizations that make it easy to see and understand team health and performance. Metrics like PR Throughput, Time to Merge, and Cycle Time allow them to keep a pulse on team efficiency and identify any bottlenecks that might be impacting their ability to reliably deliver software.

“Of course, the data is out there, but using Velocity is so much faster and more convenient. The quality of presentation and reporting enables us to more easily consume the information, surface trends, and put our findings into action. And that is extremely valuable to us,” one engineering leader said.

Data from Velocity helps Alloy understand the impact of their teams and showcase improvements in execution. Since they started using the platform, they’ve seen improvements in metrics across the board. Some highlights include: reducing their Cycle Time by 57%, improving both Time to Open and Time to Merge by 55%, and, most notably, increasing PR Throughput per contributor by an incredible 388%.

Fostering Team Health and Talent

The ability to drill down to the contributor level gives leaders at Alloy meaningful insights to help with coaching and goal setting for individual engineers. There are many examples of managers using Velocity to identify high performers who might be looking for their next challenge, or team members who might not meet benchmarks and could benefit from additional coaching. Alloy team leaders often use data from the platform to reveal areas of strength and weakness to help guide engineering talent on paths of growth and success.

Another engineering leader explained: “It helps to have data and benchmarks to show both individuals and teams for coaching. We can set realistic targets and then showcase improvement and growth. Velocity is important to creating paths for success for our engineers and for fostering a healthy engineering culture.”

Key Highlights

How Faire Uses Velocity for Critical Visibility During Rapid Expansion

Jul 13, 2023

Faire is a fast-growing online wholesale marketplace connecting independent retailers and emerging brands worldwide. Since launching in 2017, the company now supports 100,000 brands and 700,000 retailers across 50,000 cities. Reaching this point of scale in just a few years required aggressive yet mindful growth within engineering. During this period, Faire leadership relied on Code Climate Velocity to provide critical visibility into team performance to ensure they could predictably deliver on objectives while continually onboarding new engineers.

The power of engineering data

Paul Poppert, head of engineering at Faire, is a huge believer in the power of data.

“If you want to understand the evolution of anything — from personal finances to health and fitness to organizational excellence — you need a way to collect data around the activity and find the signals within that data,” he said.

Visibility into engineering processes — and the ability to capture signals from engineering data — has been critical to the evolution of Faire’s engineering team, which has grown 18x since Poppert assumed his role in 2019. Before selecting Code Climate Velocity, he and his team tried using a few competitive and home-grown tools, but these proved difficult to integrate, time-consuming to manage, and lacking in capabilities. It was a massive distraction from high-value work.

“I was trying to onboard new managers that didn't understand how you use these tools, and we were just too busy for that. And so that's when we decided we needed a more advanced platform and wanted to partner with a company where it would be a healthy business relationship. We investigated multiple companies, and Code Climate was the best fit.”

Faire onboarded with Code Climate when it was on the cusp of massive company growth. In the two years since, the team has grown from 100 to 360 engineers, and visibility into performance has enabled Poppert to keep teams working predictably toward business outcomes. For example, as Faire was onboarding large cohorts, he used Velocity to understand how this impacted overall output and execution, so he could prepare for onboarding future groups. They could adjust workloads and commitments, and reallocate time to getting new team members up to speed, knowing it would benefit their output in the long run.

“We were able to invest a ton in bringing new engineers up to speed, nurturing them, and pairing them with more senior engineers,” Poppert explained. “We could do this with the visibility we had from Velocity because we could set realistic expectations, and knew we would see the rebound in execution when these investments started to pay off. And we absolutely did.”

Though Poppert was already happy with the teams’ output, now his teams are hitting record output levels per person because they were able to manage onboarding mindfully. And with Velocity, he has the data to show company leadership and peers his team’s value stream. Within the last year, PR Throughput has increased by 60% and Push Count has increased by 65%.

Velocity powers engineering excellence

When it comes to metrics, Poppert said he focuses less on goal setting and more on using metrics to understand what is happening within his engineering organization, which is made up of five teams — four product and one platform team. He can look at metrics like PR per contributor, PR Throughput, Cycle Time, or Time to First Review and compare between teams or over time to identify changes that might signal problems to be addressed. He noted, however, that the platform team’s work differs significantly from a product team’s. So, naturally, metrics do as well, with much lower active coding days yet higher PRs per contributor based on the type of work they are responsible for.

Engineering leaders at Faire also use Velocity as a way to foster a healthy culture. For instance, the Team360 view offers insight into how and when teams work, which is a valuable tool for managers to spot early warning signs of burnout and adjust workloads accordingly. The calendar view can show when teams may have more unplanned work or when engineers work at night and on weekends, which can be discussed in retros with the team.

Velocity also reveals important insights about performance indicators, which help managers to guide individual and team growth. The platform can help leaders identify high performers who may be prolific contributors, or those who can quickly and easily adapt to changes in project requirements. It can also identify the team players doing frequent Code Reviews and mentoring other engineers, and even shine a light on team members who may be struggling so managers can get them support before a problem occurs.

“If you have a large team, you cannot always trust personal perceptions or status reports, which can be subjective,” Poppert said. “You need real visibility into what is happening in your engineering organization. Code Climate Velocity gives us that.”

How HealthSherpa Got Value on Day One with Code Climate Velocity

Mar 13, 2023

HealthSherpa is the largest marketplace for private health insurance in the U.S., with millions of customers enrolled each year. HealthSherpa offers affordable coverage provided by 40-plus insurance carriers and more than 45,000 agents nationwide. Their online platform is designed to simplify the insurance enrollment process for consumers.

As a software company, the engineering department at HealthSherpa plays a critical role in achieving business outcomes. HealthSherpa's VP of Engineering, Joseph Gefroh, believes data-driven processes are the key to healthy, high-performing teams. When he joined HealthSherpa in 2021, the company was still operating as a startup — innovative and agile, but not actively evaluating its practices to ensure repeatable success.

In working to mature the company’s DevOps practices, Gefroh said his initial assessments were intuition-based. He drew from more than a decade of experience in software engineering to implement best practices to streamline processes and align his teams with business goals. However, he quickly moved on to tackling more significant opportunities for improvement, but gathering data to perform proper evaluation proved cumbersome and time-consuming.

“I was building lists and spreadsheets using the minimal resources I had, but this consumed dozens of hours each month,” Gefroh said. “And I was still only gathering basic metrics like Cycle Time. I couldn’t even get data like Time to Merge, which would help us actually improve some of the bottlenecks in our process.”

Actionable Insights From Day One

HealthSherpa needed a better way to get accurate, actionable insights to drive improvements to their DevOps processes, and Code Climate’s Velocity platform delivered immediate value.

“Other tools for measuring engineering performance are complicated. Many of them use their own scoring system for what it means to be productive with a lot of data that isn’t relevant for my team and not based on industry standards,” said Gefroh. “Velocity has a user-friendly interface. It provides the metrics I actually want, and it’s easy to make correlations immediately.”

With access to accurate metrics, HealthSherpa was able to identify bottlenecks and surface trends that opened up opportunities for improvement. Within the first year, they were able to decrease Cycle Time by 31% and Time to Merge by 41%. As a result, PR Throughput increased 48%, Commit Volume went up 54%, and Pushes increased by 94%. They also saw a 62% improvement year over year in Time to First Review.

Gefroh said Time to Merge is a key metric, as it helps it identify issues that slow down development processes. HealthSherpa saw this data trending upward, which led them to investigate further. By viewing correlations between metrics within Velocity, they were able to find that their new, more rigid code review process had impacted their Time to Merge. Ultimately, that the team was more carefully considering code quality and non-functional requirements during reviews was a necessary, beneficial change, even if it meant an increased Time to Merge.

Comprehensive visibility has been essential to planning for the seasonal changes that are inherent to the health insurance industry. Seeing the immediate impact as seasons shift — and correlating that data with historical trends — enables them to intentionally plan for changes in the risk profile and increased demand, and intervene more quickly when issues arise.

HealthSherpa has also gained value from Code Climate’s continuous innovation efforts, particularly the recent introduction of DORA metrics into the Velocity Analytics module.

“Code Climate’s Velocity platform already played a critical role in improving the health of our engineering organization. With the insights provided, we could remove bottlenecks and reduce our Cycle Time from 120 hours to less than 48 hours,” said Joseph Gefroh, Vice President of Engineering at HealthSherpa. “Now, we can view DORA metrics in the context of the other Velocity data, which saves substantial time and provides a clear picture of our team's performance.”