Forecasting software delivery. Engineering's new best friend.

This type of development generated countless horror stories, especially in the software boom of the 1980s. Business leaders were only just beginning to learning the world of software, market tensions were high, and customers expected to consume technology the same way they bought durable goods. This philosophy brought many innovations in the field, but this tumultuous period also brought many more late nights and 100-hour workweeks. These "Death Marches," as they are often referred to in project management, left their mark on many engineers. So it was no surprise when engineers declared independence from this old world of software development in 2001 with "The Agile Manifesto." With this, we moved into the next generation of software engineering.

I LOVE a hard deadline!...said no one ever.

Building software was unlike anything humans had tried before. When we first started looking at software as projects with the same scale and manpower as a skyscraper, it made sense to build it.

But, unlike building a building, where if you decide that you want to add a few more floors after you break ground, you’re liable to get thrown out of the room, in software, it’s a relatively easy change. This new flexibility (along with the ability to easily update software) meant that organizations could start building software without knowing the whole end state. We could define a bit, then build a bit, then ship more quickly. Tighter iterations between the business and engineering created a lot of inclusion, a great feedback cycle, and scope that changed over the project’s lifetime. Delivery dates started to grow fuzzy.

The agile ideals are sound, but the implementation often starts with great intentions and ends with extra meetings and a marathon of sprints. Occasionally, an even more dysfunctional cycle ensues. Many companies still wanted all the benefits of agile but couldn’t let go of having a single delivery date for the final product. Attempting to build requirements an iteration at a time, but still expecting a firm commitment for the full (and unknown) scope, was a recipe for a conflict.

As a veteran engineering leader once described to me,

"Agile became a revolt against organizations that imposed deadlines first and created requirements second."

Some engineering teams successfully used the agile manifesto to plan just one sprint at a time, never committing to anything beyond a week or two of effort. While technically Agile, this only furthered the divide between customer value, business expectations, and outcomes, and engineering output. This left businesses without any ability to plan operations, forecast when the software would be delivered, forecast revenue, or make long-term commitments to customers and partners – all in the name of Agile. The pendulum swung too far in the other direction.

Meeting in the middle to build allies between engineers the business

Software companies need to make commitments to customers and plan reliable roadmaps to be successful. At the same time, developers must be involved in the planning and product delivery process if expected to set realistic expectations. Critically, each side of the organization needs to incorporate and communicate discoveries. This ability to communicate and collaboratively iterate is at the heart of building healthy relationships across the business. 

Both ends of the Agile-to-Waterfall spectrum leave everyone disappointed and at odds. From the business side, an inability to plan around larger engineering efforts that drive sales and marketing feels short-sighted and at the mercy of engineering. From engineering’s perspective, being held to an arbitrary date at all costs feels unnecessarily punitive. 

Thankfully, there’s a language that can help both parties advocate for themselves while fostering collaboration instead of tension: Delivery Forecasts. 

How to make forecasting your friend

Forecasted Delivery Dates differ in one key way from Due Dates: they are sourced from the forecasted Delivery Dates differ from Due Dates because they are sourced from the engineering team and based on historical work patterns. They represent a more accurate and realistic best-effort prediction of when work will complete using the info we know at any given time about that project’s status. They are dynamic, just like the work itself, and it’s a data-based way to generate approximate delivery dates and align expectations and efforts across teams.

With this approach, the business can track and plan around reliable dates and defined increments. The engineering team can communicate the impact of scope changes and risks in real-time, and the company can visualize, understand, and answer. It’s a happy marriage of both the business and engineering values of Agile methodology.

Admittedly, there is both an art and a science to developing these forecast dates. It involves collaboration around defining the minimum set of work needed to accomplish a goal – not just single user stories, but all valuable effort to the business. Sometimes you may need to forecast epics or complete user stories; other times, you may need to forecast when a particular feature will be delivered to support a customer so you can communicate and set expectations effectively. 

Using language and focus around customer value helps align stakeholders throughout the business. You can become more reliable and consistent in your delivery and better manage expectations tied to key business outcomes. Both the effort to develop the question and the effort to develop the answer require well-oiled cross-team collaboration.

Luckily, Allstacks is building something to help with this. We’ve found that unifying engineering data in a platform that creates a single source of truth for value stream intelligence across teams and projects is a fantastic place to start getting the whole enterprise on the same page and speaking the same language – literally. With accessible reports and dashboards that interpret technical data into critical business insights, teams gain the visibility they need to plan, eliminate surprises, and make strategic decisions around customer value.

Learn more about how we help product stakeholders get better visibility into engineering effort in order to drive better business outcomes here.