Improving developer productivity is a common goal for companies, yet it's a minefield to navigate.
How do you improve something so notoriously difficult to quantify? And how do you gauge the effectiveness of your efforts when there's no clear benchmark to follow?
Recently, there's been a surge in discussions surrounding different frameworks for measuring developer productivity, such as DORA and SPACE. Some of these frameworks have faced criticism for their overreliance on measuring developer output. Metrics like lines of code written, commits merged or tickets closed are a poor proxy for productivity. Even worse, tracking these measures incentivizes gaming the system and can be demotivating for top performers—in many cases writing less code or even deleting code can be the most productive action!
So where do we go from here?
Reframing the Problem: A Focus on the Developer Experience
It begins with reframing the problem. Instead of fixating on developer output, we should focus on identifying the fundamental components of a productive development team and work environment.
An engineering leader I spoke with recently put it simply: "You want to create an environment that not only attracts the best people but also empowers them to perform at their best."
Solving this requires a different approach—a focus on optimizing the developer experience. This approach aims to understand the day-to-day experience of developers and the various friction points they may encounter at work.
The good news is that measuring the developer experience is a far more manageable challenge. How? You can simply ask people what it's like working on their team and whether they feel they are able to be fully productive. Then work to understand the underlying drivers of their feedback.
Measuring Developer Experience: The Three Core Dimensions
Earlier this year, Nicole Forgensen and others released some fantastic research into measuring the developer experience and the impact it has on both developer engagement and productivity.
This framework distills the developer experience to its three core dimensions: feedback loops, cognitive load, and flow state.
Diagram source: https://queue.acm.org/detail.cfm?id=3595878
In the rest of this post, we'll delve deeper into these dimensions, exploring how they can be more directly measured and leveraged to improve both engagement and productivity on development teams.
1. Flow State
Solving complex technical problems requires deep focus or being in a flow state for extended periods. Unfortunately, in many organizations, finding the time for flow is challenging. Interruptions from meetings, Slack/Teams, and email drive context switching and break flow. Luckily, measuring these interruptions is a tractable problem.
Below is an illustration of the day in the life of a typical software developer and the interruptions to flow:
And if we combine this with data on self-reported developer productivity, we can see that the amount of time you have for focus/flow does appear to be a significant driver.
An interesting related finding is that bringing developers back to the office tends to increase disruption and lower focus on those days. The result is that developers often report lower productivity after the implementation of RTO programs.
2. Cognitive Load
Cognitive load refers to the amount of mental processing required for a developer to perform their work. For instance, having to navigate complex approval processes or face significant friction around decision-making.
We track a variety of metrics which are proxies for cognitive load faced by developers. Collaborator count is one metric we often see drive down perceived productivity. It provides insight into how many people you need to work with in order to get things done? If developers need to collaborate with more than 7 or 8 people to complete their work, they tend to report struggling to be productive.
Relatedly, collaborative demands from cross-functional stakeholders also contribute to cognitive load. If engineers are frequently being asked to work with colleagues outside of their department, it can be a drag on their velocity.
To make this actionable, it’s helpful to understand what teams are taking up developer time. By looking at what percent of their week is spent on cross-functional collaboration, you can pinpoint where additional cognitive load is coming from in order to shield developers from over-collaboration.
3. Feedback Loops
Feedback loops refer to the speed and quality of responses to development work performed. Picture a typical day in the life of a developer—it's a tapestry of iterative tasks intricately woven with feedback from diverse sources. Developers receive specifications from product teams, seek clarifications, and then find themselves in a holding pattern, eagerly awaiting feedback. They might submit their code for review, only to pause, awaiting approval or change requests that temporarily halt their progress. And perhaps most crucially, there's the loop that connects them to the heartbeat of their work: customer feedback. How swiftly can an idea evolve into a tangible feature in the customer's hands? Research underscores that shortening these feedback loops is pivotal for driving developer productivity.
The good news again is that many of these feedback loops can be quantified, directly or indirectly.
As an example, to gauge the efficiency of feedback loops between product and engineering, one can look at how quickly the product team responds to questions posted on JIRA tickets. When these inquiries receive speedy replies, it's a strong indicator of efficient communication and collaboration between teams. Our findings reveal that faster feedback loops of this kind consistently correlate with higher perceived productivity among developers.
Code review and feedback cycles are equally measurable and can provide helpful insight into friction in the development process.
In the realm of developer productivity, swift and effective feedback loops serve as the catalysts that propel ideas into action, ensuring that the software evolves at the pace of innovation.
Measuring the Developer Experience
This focus on the developer experience represents a shift towards a more developer-centric approach to boosting productivity. By prioritizing enhancements to the daily lives of developers, organizations are better positioned to secure buy-in for their improvement initiatives. This approach not only resonates with developers but also offers the advantage of being readily quantifiable.
To delve deeper into this framework and explore a comprehensive array of metrics that we employ to measure the developer experience, we invite you to check out our full report. We eagerly anticipate hearing your thoughts and insights, as together, we continue to refine the path toward optimized developer productivity.
