3 Reasons IT Ops Uses Lean Flow (Kanban for Devops Part 1 of 3) February 12, 2015 by Dominica Degrandis

3 Reasons IT Ops Uses Lean Flow (Kanban for DevOps part 1 of 3) February 12, 2015 by Dominica DeGrandis

In this three-part series on Kanban for DevOps, Dominica DeGrandis, Director of Training and Coaching at LeanKit, explains three key reasons why IT Ops teams and those implementing a DevOps value chain use a lean flow approach to product development. Here’s part one.

Reason #1: The work isn’t done until it’s working right in Production. There’s a joke in software development about being “done” and being “done-done.” Single-done is the platitude; double-done is the retort. Corks pop, banners fly, the new feature is “done” — but real work still remains. Work requiring someone from operations to stay behind to get it “done-done.” Double-done signifies the real finish line, when everyone can relax and make merry — often quite a while after the code is shippable or even delivered to production. By the time all the operations tasks are truly done, the party is usually over and there’s nowhere to go but home, to catch a few hours’ sleep before getting up to go to work again.

Lean flow — also known as Kanban for knowledge work — lifts operations out of the double-done muddle. As a systems thinking approach, it encourages development and operations to map the entire work stream and consider all relevant tasks, including those routinely relegated to post-production. This unified work stream increases the odds that everyone can celebrate crossing the finish line together.

Why “Double-Done” Completion Practices Exist Before we explore reasons why Operations teams use a lean flow approach, let’s take a closer look at the business reasons for commonplace “double-done” completion practices: marketing, maintenance, and risk.

Marketing Generally speaking, customers disregard the fanfare of the single done. They don’t care if the code is in a shippable state. For customers, the new feature is not truly done until after the code is delivered to production, stabilized, and performing correctly. In the context of feature marketing, however, the single-done gives a fair signal to obtain feedback, to amp up the market buzz, and to anticipate the double-done.

Maintenance To remain competitive, businesses need a certain level of confidence to ensure that the new feature is resilient— that it is being monitored and maintained. The double-done model gives businesses a window (usually too short a window) to acquire the necessary elements for resilience, such as sufficient hardware capacity, storage, and security, as user volume increases.

Risk To avoid the risk of bungled updates, multiple team members must learn to troubleshoot and fix problems related to the new feature. To facilitate future support capability, some form of documentation is needed. Double-done practices provide a place to verify that support systems are adequate to anticipate and reduce risks. While double-done completion practices do serve some business needs, a lean flow approach addresses all of the above and more, within a fully transparent workflow.

Kanban for DevOps: Crossing the Finish Line Together Kanban provides end-to-end visibility of all the states the work must go through in order for the feature to be reliable for customers and safe for businesses.

A Kanban Board for DevOps Example Lean flow methods use Kanban boards to provide a visible connection between all the work states in the system. Blocked work that is preventing other work from being completed becomes self-evident. Dependencies between teams or with third-party vendors emerge as indisputable. And furthermore, because all the skill sets required for a feature’s completion have a voice in the process, it becomes a delightful experience for employees.

The Bottom Line With Kanban’s end-to-end, lean flow approach, “done-done” can become truly done. When we are able to visualize and consider the entire work stream, bottlenecks previously invisible can be addressed, tasks required after code is deployed to production can become mainstream, and everyone can join the party at the finish line.

Find out the second reason why IT Ops uses lean flow — read part two in the Kanban for DevOps series.

3 Reasons IT Ops Uses Lean Flow (Kanban for DevOps part 2 of 3) March 11, 2015 by Dominica DeGrandis

Reason #2: Staying focused when flooded with conflicting priorities. Imagine a five-gallon bucket poured through a narrow funnel. The funnel can hold only so much before it overflows into a messy spill. For operations teams, the funnel is often overflowing, because they support a flood of multiple and competing requests, servicing a wide variety of customers.

From configuration management to capacity expansion, and from security compliance to automation, operations teams work on so much more than new product development. Such broad usefulness can result in a barrage of interruptions that mess up a team’s flow. In every workshop I facilitate, operations teams cite the constant “context switching” from interruptions as one of their top frustrations. Context switching occurs when priorities conflict. When today’s number one priority routinely usurps yesterday’s number one priority, the purpose becomes murky and frustration, endemic. To get back on track, it’s necessary to clarify our prioritization processes (or lack thereof) and illuminate conflicting priorities.

But before we do that, let’s look at why the problem of conflicting priorities exists in the first place, why it’s such a tough problem to solve, and what happens when it doesn’t get solved — or at least improved.

Why conflicting priorities exist and why they’re a problem Many priority conflicts arise from three common causes.

1. Too much demand. Conflicting priorities can occur when there is more demand than we can handle. Of course, we desire demand, because if no one demands our product or services, we may be out of work soon. Demand is life, but when left unchecked, it overloads our capacities. Wanting to do the right thing, we attempt to do too much work too fast. Heroic efforts appear praiseworthy in the short term, but such efforts inevitably lead to expensive mistakes. Recall the adage: When we attempt to get everything done, nothing gets done.

2. Shifts in business direction. Conflicting priorities often seem to go hand in hand with uncertain market conditions. To stay competitive, business owners must hustle to change direction quickly. As a result, teams working on Project A may be asked to begin Project B before discovering that Project A may soon be abandoned. 3. Unrealistic project timelines. Conflicting priorities can also come from inappropriate deadlines and targets. For example, when a sales manager promises a new feature to a customer – without asking impacted teams – the new work can knock other committed projects off their timelines.

Likewise, if a hopeful vendor naively says, “Sure, we can do that – no problem!” — an overly optimistic target may land the work in the “abandoned” bin.

Why the problem of conflicting priorities is tough to solve People in other parts of the company who submit requests to operations teams are often unaware of all the other requests. Everyone just wants to get his or her job done. Urgently.

When the roiling sea of requests hits the funnel, everything clearly cannot all be done at the same time. We need some form of prioritization to monitor the flow. But how does prioritization happen?

It’s not unusual to see the following methods:  Prioritization by he or she who yells the loudest  Prioritization by who has the most political authority or influence  Prioritization by who is best friends with the person whose skill set is in need

While these methods can work, they exemplify the kind of command and control behavior that has people updating their resumes. On a more fundamental level, these methods mask problems — like not understanding which work request, when well executed, represents the best choice for the company.

I recommend the following method instead: Prioritization by examining the over-arching business value to the entire company, rather than one particular department.

Business value (and/or cost of delay) is often hard to decipher, and it’s too big a topic to address in this post. Suffice it to say that when all concerned parties consider the whole picture, it’s easier to be on the same side of the fence.

What clear priorities can do for teams and how to get them When workers request project priority, a “do all the things” answer can do more harm than good. Attempting to increase the workload through a funnel that can only handle so much is a futile exercise — it only increases interruptions that cause further delays.

Clear priorities are vital. They help reduce questionable tasks and relieve workers of expensive context switching, which in turn improves the flow of work. But it’s important that these priorities come from the top.

Business owners (or the leadership team) with collective responsibility for attaining common goals for the organization should identify which few items to place in the funnel. Operations leadership needs a seat at this leadership table.

Removing the burden of guessing what is the highest business priority frees teams to focus on the execution of the requests. Escalated requests bounced back and forth between departments are reduced because of the priority consensus at the leadership level. High-level prioritization is an action for leadership to determine — not downstream teams.

Why we need to shine a light on conflicting priorities When a work item takes longer than expected, or it sits idle, it consumes space in the funnel. This prevents progress on other work and blocks the funnel. A blocked funnel is impossible to ignore. Like a clogged kitchen drain, the plumbing gets the attention it deserves. To clear the funnel, a discussion is needed to decide if the work should be escalated or pulled out.

Here’s an exercise to try: Query your ticketing system to reveal all work items that haven’t been touched for a period of time (e.g., 30 days). Compare that outcome with the work that is getting done. Is the result surprising? If yes, it might be time to revisit your organization’s prioritization method and have a good old-fashioned conversation about conflicting priorities.

How to shine a light on conflicting priorities “When one party begins to feel pain in synchrony with the other, the problem will eventually find its resolution.” — Gerald Weinberg

Talking about conflicting priorities may be uncomfortable, but shared pain is an effective way to focus attention on problems and begin to resolve them.

Making pain visible is the most comprehensive way to share pain. Shining a light on conflicting priorities requires a visual representation of all the work that is currently in the funnel. When everyone is looking at the same picture, visibility makes priorities obvious: Everyone can see what’s being worked on and what’s not.

A Kanban board showing team priorities

In the Kanban board example above, three lanes represent work currently in the funnel: “In Work,” “On Hold,” and “Abandoned.”

The “On Hold” lane exists below the “In Work” lane to explicitly disclose when work cannot make any progress. Additionally, we can discern that the work in the “On Hold” lane is sitting idle because people are busy working on higher priority items above. The “On Hold” lane can also be used to show items that are delayed for reasons outside of the team’s control, such as purchase order approvals or third-party services.

The “Abandoned” lane reveals work that was started but is now de-prioritized to the point of no return. Showing the state (i.e., the column or lane) where the work stopped allows us to better understand the cost of beginning but not finishing work. Adding a policy to visualize displaced work causes necessary prioritization discussions and highlights the potential morale cost.

The bottom line Using Kanban boards to reveal all the work in progress helps operations teams monitor priorities more effectively. But first, business owners and leadership teams (which include operations) need to determine which few priorities make it into the funnel. Clarity is critical. When priorities become clear and work-in-progress limits are adjusted appropriately, we can finally focus. Uninterrupted focus is priceless when it comes to getting things done.

Learn the third reason why IT Ops uses lean flow — read part three in the Kanban for DevOps series.

3 Reasons IT Ops Uses Lean Flow (Kanban for DevOps part 3 of 3) by Dominica DeGrandis

Reason #3: People responsible for product support have a voice during product development. Accountability with no authority sucks. To be responsible for keeping production running smoothly — but have no voice in the changes rolling into production — sucks energy, sucks morale, sucks cash. And that sucking noise means organizational health is heading south.

The problem is twofold: 1) The economic cost of owning something is not necessarily reflected in the building of it, but in the support and future maintenance of it. We often don’t understand the effort required to support products.

2) The prosperity of organizational health (i.e., a company’s economic performance, job satisfaction, levels of trust and cooperation, and tolerance for experimentation) is dependent on the alignment of connected teams. When teams ignore or compete with each other — instead of working to improve overall success and growth — organizational health deteriorates. Let’s not forget that the intent of the DevOps movement is to improve the system as a whole.

Product support impacts economic cost How much budget is needed for maintenance? Who knows?

Let’s define maintenance. Maintenance includes work to provide enhancements, fix defects, run tests, support customers, expand capacity, migrate platforms, optimize performance, reengineer features, update security/compliance, localize global elements, and decommission servers. Maintenance almost always includes some form of operational support. And often enough, some form of design, development and testing is necessary.

Research¹ on budgeting for software maintenance shows that product support needs anywhere from 20-75% of the initial cost of the product development. The variance of the range is due to many factors, including size, skill level, complexity, duration and scope.

Even though it’s difficult to clearly define the maintenance impact of a new product, it is worthwhile to get thoughts from people responsible for its support. They are the ones who get paged at 2:00 a.m. to stabilize the product in production, to handle performance issues and to react to customer complaints. Asking them what they think early on allows for differences of opinions to be aired and debated — a fundamental feature of organizational health.

Product support impacts organizational health “If people don’t weigh in, they can’t buy in,” asserts Patrick Lencioni in his book, The Advantage

Lencioni correctly observes that people do not actively commit to a decision made on their behalf unless they have had the opportunity to provide input, ask questions, and understand the rationale behind it.

When people are aware of new product development and have a voice during the development of it, they can avoid being blindsided with, “Oh, by the way, you are inheriting a new product tomorrow,” Giving people a chance to anticipate support needs leads to all sorts of goodness. They can allocate capacity for support and have time to consider a proper solution. They can identify load balancing requirements and security implications. Authority enhances their accountability when they are empowered to suggest alternative improvements that are likely to reduce overall costs and risks.

People want to know what’s headed their way and have an opportunity to weigh in on decisions. Even if their suggestion isn’t implemented, they know that their contribution was considered. This alone can improve trust and cooperation — essential ingredients for alignment.

How to create alignment across the organization Creating alignment begins by creating clarity so everyone can understand why the organization is moving in a particular direction. Clarity on the company’s intent reduces confusion and ambiguity. With a straightforward and consistent message from leadership, employees can freely execute their responsibilities. There are few more destructive obstacles for employees than having to continually maneuver through shifting or contradicting communications from misaligned leaders.

Communication gives people the clarity they need to move forward as an aligned organization. Organizations can boost communication by creating visual representations of their priorities, risks and workflows. We’ll explore workflow visuals here.

Solving misalignment problems with lean workflow visuals One way to solve misalignment problems is to look at the way work moves through an organization. Analyzing how the iterative bits of work flow across the whole value stream and into the hands of the consumer brings clarity to and supports consequent alignment across teams. From design, to build, to release, to product support — people use lean workflow visuals (such as Kanban boards) to help them see problems related to handoffs, waste, rework and blockages.

A Kanban board showing development, operational impacts, and support work

Mapping a cross-functional view of iterative product bits moving through the organization on their way to production reveals how traditionally unrelated processes function and how decisions made upstream impact the flow of work downstream.

In the Kanban board above, the support lane at the far right calls attention to the support responsibilities the team will need to prepare for. In true DevOps fashion, we are reminded that product development is not done until it’s working right in production and fully supportable.

Additionally, the data flow model and data storage tasks call attention to themselves in this view so that they can be reviewed by those impacted. By inviting broader participation in the building of the product, we can avoid troublesome retrospective questions, such as “Why did they design the data flow model like this? It will take at least six weeks to procure the storage needed to handle the data integration — and we want to go live when?!?!”

An anticipatory understanding about what is involved to release and support a product requires early and regular engagement from those impacted.

Ways to build team engagement To engage impacted teams, organizations are embracing one of two effective strategies: 1) Embedding operations and infrastructure talent into product development teams 2) Insisting that product teams take responsibility for the operation and support of the product.

In the first case, the level of trust and cooperation obtained from having a voice in the matter increases quality and job satisfaction. In the second case, the accountability obtained from servicing the product increases quality and autonomy. Either way, these changes to traditional operational support methods put companies at an advantage where they can react and adapt faster.

The bottom line In order to prosper, grow, and increase cooperation and job satisfaction, we need alignment across the organization. We need everyone rowing the boat in the same direction. Lean flow helps us anticipate what’s headed our way and gives people a voice in the process. Empowered, engaged accountability means organizational health is headed north.

Source: Hayes, Jim. (2014, December 3) How Much Budget Do I Need for Software Maintenance?