Accurately Predicting Software Delivery (Part 1 of 2)

Note that the count of alerts is heavily skewed towards issues with lower story point estimates. Obviously, this is because most issues in a development cycle will have low story point estimates, and therefore the alerts that we generate should follow that distribution. However, you’d expect the distribution to be a bit less skewed than shown above. In theory, common story point issues should be “easier” and therefore encounter fewer problems. The data we’ve collected thus far indicates that is not the case, which highlights that one problem with a high 13 story point issue is likely going to be less impactful than multiple problems with a few of your lower story point issues.

Additionally, the following chart depicts the correlation between completion time (for issues that did not encounter any alerts) and story point estimates:

The X-axis represents story point cohorts. Bars represent completion times for issues that did not encounter alerts.

We show this correlation chart because agile purist will argue that story point estimates do not correlate to time. This logic is flawed, and the data we’ve collected indicates that there definitely is a correlation. The more complex that an issue is, the longer it should take to complete. Teams see this bear out in planning when looking at past velocity to predict how many story points can be completed in a specific time frame. While there isn’t a magic conversion (1 story point is equivalent to 1 day), teams establish a pattern of predictability used for planning, which ultimately results in a crude measure of time from story points.

Therefore, the fact that there definitely is a correlation between story points and completion time lends further credibility to the forecasted risk-adjusted completion times that Allstacks ultimately provides to engineering teams.

Now that some context has been established, the following couple of sections will show baseline estimates for the impacts that different alerts have on completion times, ultimately driving our risk-adjusted forecasts.

Data Process

To begin establishing baseline results, we pull together a dataset that looks something like the following table:

The X-axis represents story point cohorts. Bars represent completion times for different types of alerts.

A couple of quick examples of how to interpret the graph:

• For items with story point estimates of 2 to 2.5:
• The median completion time for items that don’t encounter alerts is 12.42 days.
• The median completion time for items with high comment activity is 27.35 days.

Takeaways

In short, there are a few key takeaways from the baseline results:

• In general, items that encounter alerts take a longer time to complete than items that don’t. The exception here is added items to a sprint late, which seem to be completed a bit quicker.
• The presence of multiple alerts further delays the item completion time.
• The most significant percentage difference between non-alert and alert items is in the “null” story point cohort.
  • Items without alerts are completed very quickly.
  • Items that encounter alerts are delayed very significantly.

However, these results establish baseline performance. In the next few posts in this series, we’ll explore how Allstacks uses additional data sources to develop more accurate completion times for different items and completion time differences between non-alert and alert items.

Nonetheless, it’s important to point out that all of the data presented here and in future posts is aggregate data. If you are interested in determining how these risks impact your specific team or organization, contact us at Allstacks!

Check out Part 2 here!