All the way back in 1997, when I worked in the land of consulting, I established a practice that I called ‘Software Engineering Process Improvement’ (SEPI) to address what I saw was a growing trend in the industry: the need to help coach organisations in ways to make their software development processes and practices more efficient and to deliver more value.
I’ve had the privilege of watching the trend of software development practices change over time addressing ever-evolving technologies and business realities. Here are my predictions for agile:
1. Agile must evolve
Has agile lived up to its hype? Well, from my experience, backed by Gartner’s Hype Cycle results on agile, it doesn’t seem that agile is ‘complete’ yet. There is no doubt that agile (in all of its forms—from engineering practices to team communication practices—has drastically influenced how almost all software is being created today.
For example, agile has worked its way in to the Project Management Institute (PMI) and is by far the fastest growing form of software development methodology used today. However, agile still has its problems. First, you can’t simply flick a switch and be agile. Organisations have begun to understand that adoption of agile is very much about organisational change: it needs to be well positioned, teams need to be appropriate trained, and it should be rolled out incrementally.
In addition, enterprise agile and agile at scale has yet to become mainstream, as we see the majority of agile adoption happening at the team level versus the organisational level. Agile will require some additional tweaks regarding end to end maturity, adoption and scale to bring it out of the Trough of Disillusion.
2. DevOps influence on agile
The latest buzzword in the software development community is DevOps. DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals. DevOps is a response to the interdependence of software development and IT operations. What is key to understand is that the concept of development working closely with QA and IT to produce solutions is not a new concept, existing for decades stemming from a more holistic mindset of process and flow (more on that later when I talk about Lean).
DevOps involves more than just software deployments: it’s a set of processes and methods for thinking about communication and collaboration between departments. This is really one of the key missing ingredients in the traditional (simple) view of agile today. Most organisations don’t want to deal with DevOps for a few reasons: the nature of organisational silos as well as the added complexity to agile adoption.
Most agile methodologies promote tight integration between the business and developers, but it doesn’t stress how QA and IT should also be working as part of the entire software lifecycle. By all means, the traditional teachings of agile really do focus on the development teams. In today’s world, we’re seeing a collapse of departments; increasingly the line between development, QA, and operations is becoming very blurry and these silos will further collapse with the DevOps mindset.
This isn’t a surprising trend, and in some ways its saddens me that it’s only becoming a trend now. So, why DevOps now? There are a few reasons for this. First, it’s the recognition that there is waste and risk between the boundaries of traditional development and operations. The handover to operations is full of waste in both directions. Has the system been developed for the correct operational environment? Is the development team taking into consideration operations and maintenance cycles? Does the operational team understand the requirements of the system? What is required for operational and maintenance teams to properly deploy, maintain and troubleshoot the system? This recognition has been forced by an even greater need of efficiency and value production in software development, driven either by shrinking budgets or a ‘startup’ mentality.
Second, DevOps can help us really understand how our system is being used. Many teams have begun using deep functional instrumentation (some call it application telemetry) to help understand usage patterns of the application. This telemetry, implemented in a production and operational setting, feed back into the product teams giving them insight into how users ACTUALLY use the software. This can not be done without an operational mindset.
3. Agile for the CEO—the agile enterprise
For many years, agile has been the appeal of the development team. Agile was considered by senior management as “something the techies do”. This year, agile will come to senior management, as long as it continues to mature and encompass a greater representation of DevOps while being finely tuned to the needs of the business. In fact, taking this even further, businesses will start to embrace the fact that they must work hand in hand with IT in order to truly push the boundaries of business and innovation—and ultimately survival—a trend some are already calling business-agile enterprise.
Ultimately, agile is about producing the highest amount of business value with the smallest investment, to maximise return of investment by maximising throughput, minimising operation and maintenance costs, with the smallest investment possible. IT projects have become the lifeblood of new business operations and for these reasons why wouldn’t the CEO want to have a deeper understanding of agility?
With that said, agile needs to be repositioned. Many years ago I was escorted out of a building after I suggested to the CEO of the company that I will be introducing agile techniques to his IT teams and the CEO believed that agile was just an excuse for his developers not to work hard and produce results when he needed them. Since then, I have never used the term agile with a CEO, instead I would borrow from the land of lean and talk about removing waste, focusing on the production of business value, reducing risk by having constant review and accurate progress metrics, ultimately getting more value out of the IT investment.
As we see agile move into the mainstream, coupled with influences such as lean thinking, this is a perfect opportunity to ensure that we are thinking holistically about the business from the top down as well as the bottom up.