COVID-19 and the impact on engineering productivity

Insight: Higher Organizational Performance Despite Initial Dip in Throughput

Based on our research and in working alongside our customers, most organizations switched from being in the office to at-home, remote work the first week of March.

There was a dip in throughput in the first 2 weeks of March as individuals figured out how to exist in the new world where everything takes place at home. Organizations were able to quickly regroup, which see can see in the rebound during the week of March 16 and beyond; software development activity went back to being very stable. The data shows that more work was focused on stories and denotes higher organizational performance overall, despite the initial dip.

Hypothesis: Throughput decreased as a result of the sudden need to change processes, planning, and communication methods. Organizations had to establish the fundamentals first, such as getting individuals the right equipment and access (i.e. VPNs), to do their jobs effectively and securely. Once software teams settled into working from home, they focused on stories, versus another type of work, which indicates that they executed on planned work versus addressing unexpected, new work.

Insight: 25% Increase in weekend work

Work patterns remained steady day-by-day with the exceptions of Monday, Saturday and Sunday. Work activity went up by 10% on Monday. We also see a dramatic increase in work activity during the weekend with a 25% increase in work on Saturday and an 8% increase on Sundays.

Hypothesis: Developers are juggling many responsibilities and blurring the lines between work and home. People had to adapt to a new environment where they are simultaneously prioritizing work and home, including taking care of their children, tending to homeschool, and ensuring the well-being of their elderly family members. The increase in work on Mondays is likely attributed to the fact that there are fewer status meetings scheduled, freeing up time for employees to perform the work. It’s common for organizations to schedule meetings early on in the week to ensure alignment for the rest of the workweek.

Considerations: When employees don’t have clear boundaries between home and work, it can easily lead to burnout. As a leader, you can encourage your team(s) to rest, take care of their mental health, and better separate their work and home life to prevent individuals from feeling like they are always working. While flexible work schedules offer many benefits, unclear work schedules can result in unpredictable outcomes. Keep an eye on work patterns and potential issues to prevent surprises.

Insight: The work day is starting earlier for remote employees with quieter afternoon hours.

When we look at the work activity on an hourly basis, we can clearly see that people are starting their day earlier, with most individuals beginning at 7am; the data also indicates that more work is performed during lunchtime and that workdays are ending a bit earlier. Afternoons are now quieter, and there’s an uptick in activity around 7pm.

Hypothesis: We see a drastic increase in morning work during what would normally be commute time. Individuals have reshaped their days to create routines that best fit their ‘work, live, and play" lifestyle in the same environment. As previously mentioned, people had to figure out the best times to take care of their family responsibilities along with their work duties. Oftentimes, this means parents are taking turns juggling childcare. Without commute times, people are able to get straight to work which accounts for earlier start times. Additionally, without the constraints of a traditional 9-5 office setup, individuals settled into a more natural work pattern for their chronotype, also known as an “internal clock.” They were able to create their ideal daily routine based on their biological predisposition.

Considerations: Encourage your team to work when they can be most productive. As a leader, focus on how your team is tracking towards outcomes versus worrying about when the work is getting done. It’s your job to find out how you can help them deliver work in a consistent, predictable way.

Insight: Work Patterns and Collaboration Remain Steady

The data shows normal fluctuations in work patterns. Cycle times remained steady across the board with a few variants; there were slight upticks in March and April following a stable February. Pull request (PR) cycle times pre and post COVID-19 had no material difference from the time a PR ticket gets opened to when the pull request gets merged into the codebase. Individuals and teams were able to meet commitments.

Hypothesis: Through the transition, people have been able to remain collaborative and drive their outcomes forward.

Insights: Cycle Time Remained Steady

Cycle time remained fairly steady and trended downwards from January to May; this is encouraging because it means individuals and teams were able to work predictably and make progress towards goals.

Insight: More Process Visibility & Consistency

Our study shows that organizations are leaning into processes and being more diligent in making updates within their tracking tools; in this case, we see developers documenting each “transition,” as work status progresses from one stage to next in the lifecycle. The adherence to process updates adds transparency and clarity to overall project status.

Hypothesis: Prior to the remote work environment, it was common for developers to start working on tasks and forget to update their issues in their tracking tools until the end, once the work was completed. With fewer in-person meetings and face-to-face interaction during the workday, employees are more likely to track work on a granular level.

Insight: Increased Developer Collaboration in Delivery Tools

Developers have increased collaboration within their issue tracking tools, like Jira, and are adding more comments within the cards. Individuals and teams are responsible for tracking what they are working on and how it’s going. More comments and discussions within the thread of the issue itself offer better context and clarity, versus using chat or posting comments elsewhere to get clarification.

Hypothesis: Reducing face-to-face communication means more communication needs to take place in the context of the specific issue or work task. It’s much easier to keep all of the information they need in one place, so anyone can follow the thread. Adhering to this process keeps everyone aligned.

Insight: Commit Frequency is Steady but Commits are Smaller

Commit frequency has remained consistent but commit size has changed. See below.

The number of large commits has decreased while the number of small commits has increased. As previously mentioned, the frequency of commits has remained the same, so the increase in small commits is overall a positive change. Small commits are easier to understand when reviewing a pull request and are easier to manage. Big commits are harder to revert; you may only want to revert a portion of the changes versus the entire commit.

Hypothesis: The increase in small commits is likely due to the fact that there are more distractions working remotely.

Insight: Remote Work Schedules Fluctuate vs. In-Office Work Schedules

As discussed in the first part of this series in “People,” work schedules fluctuate and are more volatile compared to in-office work patterns. As a result, cycle time from first commit to opened PR has also has become sporadic.

Hypothesis: This behavior is likely due to the increase in smaller commits done throughout the day.

Insight: More Developer Collaboration During Code Review Than Comments in the Code

The data points to an increase in the number of developer comments in the merged PRs, similar to how we saw an increase in comments at the issue and story level within tracking tools. There’s a lot more discussion in code review and commentary in the code itself, which demonstrates teams’ ability to collaborate efficiently where it makes the most sense for them.

Insight: Increased Bouncebacks as Developers Adjusted

Allstacks tracks alerts and risks to find anomalies in how your organization typically works. For this report, we studied our population sets to find out how the new circumstances have affected work in progress. Are stories and issues getting stuck? Are they bouncing back to in progress? Are these behaviors any different than what you were experiencing pre-WFH?

Initially, the data shows a decline in the number of bouncebacks, but they increased as we got later into the WFH cycle. There was a corresponding increase in the number of items that got stuck. At the onset of COVID-19, from March 2 to April 6, teams focused on newer items over existing items; eventually, teams saw the need to revisit old work, and there was a shift back to address the backlog. From May onward, balance was restored, as we see work normalize with developers fluctuating back and forth between new and existing backlog work.

Hypothesis:  The volatility in engineering bouncebacks from March to April can be explained by the need to revisit requirements for old work; as you can imagine, after a certain period of time, developers need a refresher on what the work item was about and what exactly needs to be done.

Insight: Fewer Distractions Contributed to Higher Focus on Strategic, Planned Work

Pre-work from home there was a lot of variability in working towards unplanned work versus planned work (this is also known as epic-focus or roadmap focus). Unplanned work can come in the form of one-off features or an escaped defect, for instance. Since the transition (March onward), the data shows more consistent focus on planned work, as 40-50% work is planned.

Hypothesis: Progress towards longer-arched tasks can be explained by fewer interruptions throughout the day, where managers are no longer able to “steal time” from developers in between meetings and tasks.

Insight: WFH Had No Impact on Developer Throughput

The data indicates there’s been steady throughput for developers merging code into the codebase. The WFH shift had no change on developers’ ability to push code. Based on the information gathered in part II of this study, organizations have seen success leaning into the tools and processes their organizations already had in place.

Insight: After Initial Slip in Delivery Dates, Normal Delivery Timelines Returned

At the beginning of the COVID-19 transition, there were spikes in project delivery slippage, but delivery returned to normal around April 27; the typical range for slippage is 5-10 days. This implies that the study participants were able to deliver on their respective “normal” outcomes.

It’s great to see that organizations have been able to continue to track towards their company initiatives without significant disruption. Here are a few more suggestions on how to maximize your team outcomes.