Excuse me, did someone say developer productivity?

Buckle up 🙂

There’s been a fair share of controversy across the technology space given recent conversations around a McKinsey article on developer productivity that was published back in August of 2023. The article sparked heated debates across tech publications and social media platforms like LinkedIn and proved to be divisive among the software development community. McKinsey takes the stance that “measuring developer productivity is difficult,” yet entirely possible, and points out that “software development is perennially under-measured” compared to functions like sales or customer operations before providing suggestions on what to measure, what frameworks to use, and what missteps to avoid. The perspective from the large consulting firm got people thinking differently and provided an outlet for detractors to question the validity of the question itself: “can you measure developer productivity?” It also gave supporters an outlet to express their opinions around how you can both measure developer productivity effectively and avoid it being a black box.

The debate shouldn’t center around whether it’s possible – it’s already happening. We need to be talking about why we arrived here, how we move forward in a positive direction, and what we can do to get there.

I hear a strawman argument from those against measuring developer productivity all the time that goes something like this: “picking a set of metrics and then measuring it doesn’t give me the information I want! Measuring lines of code will give me the exact lines of code, so I go and quantify commits or PRs, etc. and get exactly what I want.” No one metric can paint a full picture, though, and even a group of several individual metrics won’t paint a holistic picture. 

It’s ironic that the same people who complain about the idea of measuring developer productivity are often the same ones that recommend using a single metric, or a single set of metrics, like number of commits, number of code reviews, or average commit size. However, these kinds of metrics lack important context when viewed through the paradigm of the business and unintentionally create a dependency on a single metric of failure that can negatively impact success.

If your engineering teams rally around single metrics of failure, there’s more bad than good. You create a zero-sum game that doesn’t look at productivity holistically, weighing too heavily on developers’ individual activity, can encourage and reward those who try to game the system, and confines engineering success upstream without considering how that aligns to the needs of the business. It doesn’t have to be this way.

We’re not at IBM in the 90s anymore, Toto

The reality is that we live in a different world today. “Tell me how many lines of code you wrote” is an antiquated argument and we’re not at IBM in the 90s anymore. Judging quality based on quantity or solely on activity metrics alone will have a negative impact on culture and a negative consequence on the business, as well as the perception of what goes on internally from outside perspectives. Some of the most common metrics to measure developer productivity and success make the least amount of sense when used in isolation:

• Hours worked
• Lines of code
• Bugs fixed
• Tasks completed
• Number of commits
• Velocity points
• Pull request count
• Features shipped
• Activity metrics

We too often try to measure productivity in our space the same way a factory would measure output.  Developer experience is the ability to add to the whole picture the human element because after all, engineers are not machines nor do they "push a button." We know that developer happiness and developer productivity are linked, so there’s no good way to tell the whole story when looking at metrics based solely on outputs above without incorporating developer experience and developer happiness.

We have to use legitimate indicators – and we’re not going to boomerang back from this new world. Therefore, we have to make it successful and we can do that by using both leading and lagging indicators that tell us how we’re progressing to achieve our goals. Developers shouldn’t fear that the only reason they’re being measured is because Big Brother needs to track their activity in order to calculate their worth, or potentially have their activity used against them in a way that’s nefarious. They deserve to understand how their productivity translates to business value, and, inversely, the business should invest in understanding how tension at the individual, team, and company level impacts each other. 

For example, some metrics exist with intention for one another - like cohort metrics that look across teams within the context of business priorities. Maybe there’s a need for you to reduce churn, which can be improved by reducing defects, and that’s accomplished by putting more care into the code written and encouraging developers to embrace TDD. The tradeoff here may be quantity of features shipped, but the ROI when quality improves negates any concerns around volume. It’s important to account for both developer inputs and developer outputs, not just one or the other. Inputs allude to investments made by developers and where they spend their time and effort. It’s also important to consider differences in developer workflows, deployment methodologies, team dynamics, and overall software delivery processes.

When it comes to metrics, not everything is interrelated and not everything is mutually exclusive. Finding out where things are related, however, can be helpful. To go back to the connection between developer joy and productivity, optimizing developer satisfaction, in turn, improves developer productivity. Subsequently, this can increase both the quality and frequency of value delivered and make an impact on the business in meaningful ways like increasing revenue, customer satisfaction, and customer retention.

What you can do

The easiest way to shape how you measure developer productivity, and measure it the right way, is by actually doing it - start measuring things. It doesn’t have to be perfect, but it does need to account for some of the common pitfalls so that you don’t put all of your eggs in the basket of solely relying on developer activity metrics and outputs. 

There are tons of great frameworks (DORA, SPACE, Flow) that can help and you don’t have to begin by measuring everything under the sun. Pick a few metrics that are important first and then grow from there until you can confidently measure lots of engineering metrics in different ways, and be intentional about what you want to get out of it from the beginning. Understand your goals in terms of the business instead of isolating engineering success on its own and condensing metrics down to a single metric or a single set of activity metrics. Regularly revisit how your product and customer experience gets impacted and evaluate success based on the overall value delivered and a balanced set of metrics. 

Again, don’t base success off of individual developer productivity when the benchmark measurement revolves around things like the number of code commits or average time spent on code reviews and then wonder why there’s a disconnect between quantity, quality, and value at the team and organizational level. Think about metrics that portray an accurate picture of productivity across an entire development team if a developer works late, outputs are low quality due to the nature of the work, or a given output doesn’t add explicit customer value but is valuable work to internal lines of business.

Four good areas to begin measuring and creating metrics around are velocity, flow, quality, and organizational effectiveness. These are integral parts of many popular frameworks, like the DORA’s of the world, and are continuously cited as effective ways to start measuring engineering by subject matter experts in our industry, in analyst research, and by companies experiencing success with high performing development teams.

How to make it real

Here’s my advice:

Define clear goals

Start by defining clear goals aligned with your business objectives. Understand what success looks like for your team and how developer productivity contributes to the overall value delivered to the business and its customers.

Choose a balanced set of metrics

Select a balanced set of metrics that provide a comprehensive view of productivity. Consider a mix of leading indicators (predictive metrics) and lagging indicators (outcome metrics). This could include metrics related to code quality, delivery speed, collaboration, and customer satisfaction. 

Avoid dependencies on outputs alone

Acknowledge that metrics solely based on outputs, such as lines of code or number of commits, lack the necessary context to measure true productivity. Instead, focus on inputs and outputs, taking into account the efforts invested by developers, variations in workflows, and the overall software delivery process. Don’t fall into the trap of single metrics of failure because even a set of activity based metrics can lack the necessary context to be effective.

Take developer experience seriously

Recognize the importance of developer experience and happiness. Metrics should not only measure outputs but also reflect the satisfaction and well-being of the development team. Happy developers are often more productive, leading to better outcomes.

Evaluate interconnected metrics

Understand that not all metrics are mutually exclusive, and some are interconnected. For instance, optimizing developer satisfaction can positively impact productivity, leading to improved code quality and delivery speed. Evaluate how different metrics influence each other and contribute to overall success.

Iterate and evolve

Embrace an iterative approach to metrics and measurement. Regularly revisit and reassess your metrics, adjusting them as the team evolves and as business priorities shift. Be flexible and always consider the impact that metrics have at the individual, team, and company level.

For more details on effective metrics and how to best use them, see my next blog post about avoiding single metrics of failure.

Interested in seeing what Allstacks can do for you and your teams? Book a demo.