Maark Blog Agile: A Square Peg in Circular World

"[You] insist upon forcing a square peg into a round hole, because in a round hole you, being a round peg, feel tight and comfortable. Now I call that irrational."Edward Bulwer-Lytton, Kenelm Chillingly, His Adventures and OpinionsAgile is the new black. Or at least that's what its proponents from the software development community proclaim. Since its introduction back in 2001, the Gospel of Agile has gradually moved with evangelistic fervor from engineering teams to other parts of organizations, all the way up to the C-suites of F ortune 500 companies. Beyond its merits, Agile is undoubtedly aided by its name - after all, who doesn't want their organization to be associated with a word like "agile"? In fact, Agile has become so dominant in the industry that its critics can be looked down upon on social media as being the engineering equivalent of a "flat earther".On the surface, Agile would seem to be the ideal, forward-thinking set of principles and methodologies for digital transformation and web and mobile development engagements that we work on every day at Maark. However, based on our own experience and witnessing the experiences of others, I have grown to question that perspective. While I believe that certain aspects of Agile principles and methodologies can be highly beneficial in development projects that we undertake in the services business, a religious adherence to Agile is much like trying to fit the proverbial square peg into a round hole.A Quest for the Silver BulletAgile was first introduced in the early 2000s as a modern response to the old school waterfall methodology. The waterfall approach, which has its origins in manufacturing and industrial engineering, was brought over to the world of software development during the early years of computer programming. A waterfall project has a series of sequential phases (requirements, design, development, testing, and delivery). Since a phase is dependent on what came before it, each phase must be completed before the next starts.The primary benefits of waterfall are that the scope and timeframe are agreed to and known beforehand; it's predictable, driven by a plan, and has built-in methods of accountability. However, the waterfall approach to software development has long been derided because of its inflexibility in dealing with changes, inevitable unknowns until the full product is built, and a tendency to sacrifice testing when time runs short.Enter Agile. While the waterfall approach focuses on upfront planning and a strict adherence to that plan, Agile is all about flexibility and iteration. With Agile, a product evolves over time, and plans are constantly reevaluated to adapt to changing conditions. Priorities are set together on a collaborative cross-discipline team, and then implemented through small phases called sprints. A product using the waterfall approach will be releasable only at the very end of the project, but an Agile-driven product will be releasable at the end of each sprint.The promises of Agile are alluring and seductive: adaptability, responsiveness, a launchable product at end of each sprint, and a transparent, collaborative environment in which business stakeholders and the development team work closely together to plan and prioritize. Fueled by such potential, Agile has ruled the roost for over 15 years. In fact, a whole cottage industry has developed around it: over 40 variants of Agile; consultants, books, and certification programs; and even new job titles. Terms like sprint, scrum, and user story have become a natural part of the vernacular for most any professional in the tech space.However, in spite of the accolades, be careful not to think of Agile as a silver bullet and appropriate for all contexts. It's not.Something is Rotten in DenmarkAs Agile spread in popularity in the 2000s, we at Maark began to incorporate Agile principles and practices into our teams and projects. We assigned scrum masters, created user stories in Jira, restructured projects into sprints, and even experimented with planning poker for time estimates. But, in project after project, its benefits proved elusive.One might offer a critique and say that Agile didn't work for us because we weren't committed enough to its principles. Or perhaps we didn't follow the methodology rigorously enough or implement the right variant of Agile. Sure, there were many ways in which we could have done Agile better. But, at the same time, I don't believe any tweaks or changes to our approach would have had a significant impact on our results. Instead, I would argue that we uncovered a deeper, more fundamental problem: that there are almost intractable tensions between the principles and practices of Agile and the real world in which we operate.First, Agile principles almost inevitably run counter to stakeholder expectations. When we engage a client for a particular software deliverable - whether it is an app or website - the client is chiefly concerned with two things: our ability to deliver the product vision, and the timeline and budget required to do so. The vision often evolves as we work with the stakeholders during the planning stages, but, at the end of the day, the stakeholders want to know where we are going to finish before the first line of actual code is written.The reality is that much of the larger world outside of software development runs upon predefined plans and predictive results. And most stakeholders who inhabit that world naturally expect to structure engagements in a similar fashion. Therefore, trying to force Agile into this environment naively assumes others will (or even can) adjust to these values and methodologies.Second, Agile priorities often clash with the priorities of the stakeholder. One of the primary tenants of Agile is to empower teams and create a learning organization. As such, misdirections or deadends are considered not so much failures as much as they are educational opportunities. This perspective contrasts starkly with stakeholders that I spend time with. Misdirections made during development will never be thought of by clients as "valuable learning experiences". Instead, they are major blunders that could potentially cost them a strategic opportunity, sales, or, in extreme cases, even their job.Third, Agile tries to combine two worlds that don't easily mesh. Agile requires everyone involved with the product to be part of a collaborative decision-making process. The theory is great, but it places a huge demand on all of the parties. It requires the stakeholders not only to be intimately involved in developing the product, but also to be savvy in both Agile as well as the principles of software development. In the end, clients engage us because they don't have bandwidth or expertise for this level of involvement.Others are also reaching similar conclusions on the shortcomings of Agile. Over the past few years, I have seen a steady increase in developers reacting against it. Blog posts with scathing titles such as "Run Away from Agile" or "Agile Is The New Waterfall" illustrate the frustration that some have with the methodology. Even Andy Hunt, one of the original authors of the Agile Manifesto back in 2001, believes it has lost its way in his "The Failure of Agile" blog post.The S.W.A.T ApproachAt Maark, we've taken a pragmatic approach to the shortcomings of waterfall and Agile and adopted principles and techniques that work best for the realities of the services business. We don't do Agile; instead, we are focused on being agile, moving quickly across an enterprise as a "S.W.A.T. team" to deliver pragmatic value to our clients. In my follow-on post, I will dive into how we moved from Agile to develop app and website solutions using a S.W.A.T. approach.Rich Wagner is the Managing Director of Engineering of Maark where he leads the development team and spearheads the technology strategy of the agency. He has led the development of multiple enterprise apps, mobile apps, and websites for Fortune 500 companies. Rich has also authored over 20 books on software development and other subjects.Photo credit: LiveTrucking