The Cranky Admin
Agile Development and the Admin
The good and bad of the shortening software cycle, from an administrator's perspective.
Software used to be reasonably straightforward for customers. The advent of the agile programming methodology and the rise of the cloud-driven subscription model have changed all of this. Thanks to agile, change management, risk, trust and expertise have become wrapped up in a tangled debate over the future of the datacenter and our approach to IT in the business.
A simplified view of traditional development methods had them creating a fixed plan for what would be in an application. Developers then coded until the goals of the plan were achieved; the software was then released. After the initial software version was released, developers would ideally split into two groups: those fixing bugs in the existing version and those working on the new version.
The next version of the software would proceed much as the first. A plan would be created that detailed the new features and enhancements. Developers would code until the goals of the plan were achieved, and the new version would be released.
This approach remains an excellent way to develop Commercial, Off The Shelf (COTS) software for mass market consumption. Unfortunately, it had posed major problems for in-house developers as well as those making software for a limited audience.
Why Agile?
Modern agile and agile-descendant approaches to development take a different approach to traditional development methods. Seminal documents about the agile programming methodology, such as the
Agile Manifesto, focus on the great bugbear of developers: managing change requests.
Having been a developer who has worked on a number of multi-year projects, I can certainly empathize. There is little in life more frustrating than being 90 percent of the way toward coding an application to specification, then getting handed a stack of change requests. In many environments, the needs of the clients change faster than new versions can be released, so it can often feel like reaching a stable, polished, "finished" version of the software you're building is impossible.
In-house developers working on middleware or any developers building industry-specific software will know what this is like. It's just the nature of the beast.
Agile's solution to this is to shorten release cycles. Instead of years between versions, software has just months or weeks between releases. Developers meet regularly with other stakeholders to discuss requirements and actively solicit change requirements on an ongoing basis.
The goal is to regularly release software with features that matter to customers in as close to real time as possible, and prevent wasted effort on the part of developers. This solves real-world problems, and this makes agile a Good Thing.
Agile is particularly beloved of cloud-native developers. They like to break applications up into microservices and charge subscriptions. The theory behind microservices is that they can be updated independently of one another. This is great for the agile programmer, because new features can be rolled out quickly without having to make changes to a much larger application. More changes with fewer disruptions makes them happy.
The subscription model also makes agile developers happy. Not only is recurring revenue something that every business likes, but it also frees developers from having to demonstrate value with each release. Instead of a series of fixed releases each with their own associated price tag, customers just pay in perpetuity and receive new features as they're developed.
An Agile Fit
Agile isn't a good fit for every use case. Additionally, numerous development houses apply agile in a manner that is decidedly non-optimal.
Perhaps most importantly, agile has become something of a religion in IT, one closely interlinked with the parallel faiths of "newer is automatically better" and the cloud as the solution to all ills. Agile adherents and evangelists don't tend to publicly discuss the limitations and misapplications of agile; it's an oversight that only perpetuates the cycle.
One place agile is great is young projects. New applications that not only need to get funding fast to speed development, but feature requests come at the developers hard and fast. The flip side of this is that mature applications with few real change requests are a terrible place to inject agile.
Software which sits at the center of a number of integrations must either demonstrate stability or be under active development by programmers that are responsive to customer requirements. Office packages, operating systems, financial systems, middleware and software for embedded devices all fit the bill.
Middleware developed in house could meet the needs of serving as a hub of integration while adopting an agile approach. If an application that integrates with the in-house middleware changes in some meaningful fashion -- one requiring code changes to the middleware suite to compensate -- then someone can go bang on the developers' door until these critical changes are handled.
Trust
Organizations don't have this luxury with COTS software. When you can't simply walk down the hall and bang on the door until a necessary change in an application is made, then adoption of an agile by a developer means that customers are forced to trust that developer.
This is problematic. If I need a change made in Windows, for example, I will be ignored. I could gather together hundreds, even thousands, of other systems administrators and small business owners and we would still be ignored.
Microsoft does exactly as Microsoft wants, and only the largest corporations in the world have enough pull as customers to even potentially affect their judgement. Even governments can't get Microsoft to change their mind on issues important to millions of Microsoft's customers.
Does anyone remember the last Patch Tuesday where some Windows patch didn't break something? We've even had bad patches that try to fix bad patches and end up creating boot loops. Microsoft doesn't seem to care. They still force cumulative updates on us, and we only get control of our update process if we buy Windows Enterprise Edition, which only comes in a minimum of five licenses.
Microsoft could easily give its customers enough control to compensate for Microsoft's own crashing ineptitude, but they won't. Microsoft's desktop operating system and office package monopolies mean they can do as they please and demand that other developers in their ecosystem adapt to them.
If Microsoft changes an API and thousands of applications break, well that's up to the developers of those thousands of applications to solve, not Microsoft. Microsoft can misapply agile to the detriment of everyone, and we collectively have to accept it. Microsoft doesn't have customers, they have hostages.
Shifting Risk
Sadly, Microsoft's arrogant misuse of agile is increasingly being used as an example by others. Every developer wants to be able to force their customers to keep up to date. The argument that security patches are important for everyone is easily twisted to claim that functional patches are good for everyone.
If a developer could ensure that all their customers were running the same version, with the same patch level, then the support burden for that developer decreases greatly. Similarly -- and more importantly -- the Microsoft approach to agile allows developers to divest themselves of risk.
With traditional application development models, developers bore all the risks. When they set out to build an application in the first place, they risked creating software that nobody would want to use. Every time they invested in a new version of their software, they took the risk that customers would prefer to stick with the old version.
What's more, customers expected developers to thoroughly test their software. Customers didn't accept excuses for new versions that broke interoperability with third-party applications, nor do they appreciate radical changes in user interfaces.
This was especially true in the days before subscriptions. Back then, customers viewed software as something they owned. They treated software like they did hardware. It was a fixed asset, something they built a business around and they would run it until it stopped delivering value.
Change equals risk. If the developer is offloading that risk to their customers, then the customer is forced into a position of trusting the developer.
And trust, once lost, is nearly impossible to regain.
Cloudy With Additional Questions
Customers subscribing to modern applications should ask themselves many questions before proceeding. How much risk are you, as a customer, willing to bear? Do you feel that the value you're receiving from your software vendors is equal to the both the subscription fee you're asked to pay and the risk you're asked to accept?
Have you ever qualified the risks of the software you subscribe to? What is the impact to your organization if the developer makes changes that breaks an integration you rely on? Do you trust the developer to revert the change, or the developers of the applications that integrate with the changed application to themselves change their software according to your requirements?
What contractual guarantees do you have if the developer makes a change detrimental to you? Can you stick with an older version of the software? For how long? Are you compensated if the developer breaks something and refuses to fix it?
Many of these questions should be asked of any software a customer licenses, regardless of the application development method. With traditional software, however, the customer typically had the opportunity to refuse to update. They could sit on the old version until either they or the developer found a solution to the problem.
Agile can be a powerful way to approach application development, but the balance between risk and rewards for the customer is delicate. The close relationship between agile, cloud-delivered Software-as-a-Service (SaaS) and subscriptions further complicates matters. Customers consuming SaaS applications generally have no control whatsoever, and they bear all the risk.
Agile isn't the be-all and end-all of application development, the public cloud won't solve every problem and newer is emphatically not always better. Yet each in turn -- agile, the public cloud and novelty -- are valuable.
The experienced systems administrator remains relevant to today's successful organizations because one aspect of their job remains unchanged: the need to be educated about the details -- and especially the failure modes -- of the technological solutions we bet our businesses on. This remains true whether the solutions we review are virtualization, cloud or something else entirely.