If you can measure it, you can improve it.
When you can measure something, you’ll be able to track the information that’s needed to better understand what should improve. And once you know this, you can make those improvements to meet your desired goals. Though it may sound simple, it's not so when it comes to software engineering KPIs (key performance indicators).As it happens with any creative or technical work, you just can’t focus on quantity alone when you want to measure the efficiency of software engineering teams. Yet, you’ll often come across KPIs that focus on counting the lines of code, the number of bugs reported, or the hours worked. But using traditional KPIs could do more harm than good. Imagine a scenario where zero bugs is one of the software engineering KPIs. But as humans, we are bound to make mistakes and bugs – big or small or something in-between, are sure to happen. So, the KPI could encourage people not to report bugs at all. Thus, if the emphasis is on measuring the number of bugs quantitatively, people may tend to avoid registering them. Or let’s take the case where the KPI focuses on introducing less than 10 bugs each month. The software development team could decide not to launch any new features and thus, game the KPI. It’s perhaps due to such scenarios that Goodhart's Law stated below feels pretty apt:
When a measure becomes a target, it ceases to be a good measure.
Software engineering isn’t just about codes and hours worked. When you take into account the developing and iterative nature of Agile, you’ll realize that you need to rethink how to set KPIs while broadening your horizons to ensure the fluidity and progress of software engineering is measured truly. The ultimate goal of software engineering KPIs is to encourage constant improvement at a predictable and acceptable pace throughout the product’s lifespan. A “good metric” is something that encircles the below characteristics.
- Comparative: The metric should help in a comparative evaluation of various parameters.
- Discoverable: The metric should be easy and fast to compute reliably. One that requires a massive group of people or hours of work will ultimately fail to serve its purpose.
- Understandable and Actionable: The metric should make it easy to understand what needs to be addressed. At the same time, it should be able to let you set actionable goals to motivate the team to make improvements to their processes that will boost motivation performance, and productivity.
- Consequential: Your chosen metric must be linked to a business outcome.
Here are a few software engineering KPIs that would let you better understand and evaluate your software teams’ efficacy and progress.
Related: Hire Cloud Engineers: The Essential Guide
1. Time from the first commit to deployment
When you want to measure the entire development flow, the easiest way is to find the time from a software engineer’s first commit to when it reaches the stage of production. After you have created multiple commits, or in case you have cloned a repository that has an existing commit history, perhaps you’ll want to take stock of things by looking back to notice what has happened. A powerful tool to get it done is by using the git log command. The git log lists the commits you have made in that repository in reverse chronological order, which means you’ll see the most recent commits being displayed on the top at first. If you want to restrict the number of log entries shown, you could use the option of -p that displays the difference (the patch output) introduced in every single commit. Thus, by using -2, you’ll get to see just the last two commits. Since you can use git’s commits history to automate the entire process, finding the time taken from the first commit to when it reaches production would be easy. You may get tempted to use the User Story Lead Time where you track the time spent from a card being added to the Kanban system’s “backlog” column to the time it changes status and finally moves to the “done” column. But there’s a problem with this system. Just because a card gets added to the “backlog” column won’t mean it is all set for development. For instance, it could still miss layout specifications, instructions for testing, or other necessary requirements. Thus, using it as a KPI would simply give you an overview of the product’s flow. But what you should actually focus upon is how to improve the development flow that begins with "doing” a thing (say, building an application) and ends in it being “done” (that is when the app is deployed in production).By measuring the commit-to-deployment time, you would be able to do a few things such as:
- Decrease the time to market
- Bring down the instances of rework and waste
- Optimize engineering efficiency
- Reduce the cost associated with delay
2. Deployment frequency
As the stage of deployment concludes a job, it’s related to productivity. The deployment frequency matters as it’s a strong indicator of the extent of value your team is able to get into your customers’ hands. Thus, having a low deployment frequency could indicate you’re unable to deliver enough value, even if you’ve got an efficient pipeline. What KPI you would choose to evaluate productivity should be decided carefully though. For instance, deployment frequency in terms of the number of features delivered may not be related precisely with your business goals. Your quarterly business goals are likely to be more abstract, say where you want to decrease the time to market when launching new features. Thus, instead of using deployment frequency for evaluating your team’s productivity, you could rather use it to measure other relevant goals such as:
- Improving the team’s responsiveness to failures triggered by bugs
- Reducing the time to market
- Boosting the speed of detecting and fixing security vulnerabilities
3. Team velocity
This agile metric measures the amount of work your team completes during a sprint. Hours worked or story points are generally used to measure the amount of work done. Though it isn’t a comparison or prediction tool, it does give teams an idea about the extent of work they can get done in the next sprint. For every team, the velocity index is unique. Thus, it needs to be set in such a way that helps in the evaluation of how realistic the commitment is. This means if the project backlog has, for instance, 150 story points and the team completes - on average - 30 story points per sprint, it would need about 5 sprints to complete the project. Velocity index helps the product owners calculate how fast a team can work through the backlog. But you should refrain from comparing velocity across teams. Say, for an experienced team, the velocity has probably achieved a plateau of predictable flow and anticipated performance. But when a team has just adopted the agile methodology, you’re likely to find the velocity estimates to be erratic for the first few sprints. However, this would change as the team gains experience. Again, if the velocity index indicates a decrease in consistent flow, there’re likely to be some problems in the development that call for a change. Thus, team velocity can be very useful for the following:
- Planning future sprints
- Forecasting
- Understanding if the process changes are effective or the team is choking on them
For example, let's say you want to complete 300 story points in the backlog. Now, on average, your development team is capable of completing about 20 story points per iteration. Armed with this information, you can foretell that the team will need 15 iterations to complete the required work.
4. Cumulative workflow
The cumulative flow metric visualizes the status of different tasks over a period of time. The easiest way to measure it is via charts where the x-axis shows the dates and the y-axis indicates the number of story points (or work items). Such a chart easily lets you visualize how various tasks are distributed at different stages. There are different color bands in the charts (say, waiting in line, ready for peer review, in progress, follow up, etc.) that indicate the work-in-progress (WIP).A stable throughput is indicated when the bands are progressing in parallel. Such a chart indicates the number of new tasks entering the workflow is equal to the number of tasks leaving it. A rapidly narrowing band shows your capacity is more than your requirements and thus, points to the need of reallocating the capacity for optimized workflow. A rapidly widening band does just the opposite and shows higher number of tasks entering the workflow that what are leaving it. With cumulative workflow, you can do the following:
- Spot areas of sudden bottlenecks and inconsistent workflows, which can then be smoothed out
- Identify backlog growth, which could point to unresolved tasks that are either obsolete or too low priority to deal with at the moment
- Visualize the average cycle time based on the width of each band
- Identify if there’s a considerable widening of the “in progress” area as it could indicate the team may not be able to finish the entire project on time
Wrapping Up
If you want to break free off the shackles of code failures, missed deadlines, lengthy time to market, and low-quality products, start implementing the software engineering KPIs mentioned above. Doing this will greatly help you to take the guesswork out of the development lifecycle. Though these metrics will provide useful insight into your team’s performance, processes, and overall satisfaction, avoid fixating on them. Rather, make sure that you initiate regular discussions on the applicable KPIs, adjust them as needed, welcome your team’s feedback on them, and encourage a supportive setting of healthy competition and planning to achieve common goals.
Related Pages:
.webp){kind=small}
