The Cranky Admin

There Is No 'Silo' In Team

Holistic IT seems like a laudable goal, but how do you get there?

• By Trevor Pott
• 05/22/2017

A management approach that treats all IT infrastructure, software and services as a single entity -- holistic IT -- is the goal of many organizations. The diametric opposite of traditional IT, holistic IT is often associated with modern buzzwords such as DevOps. Buried underneath layers of marketing and hype are some logical and rational reasons to embrace holistic IT that are grounded in solving real-world problems faced by everyday practitioners.

Traditional IT is siloed. Each group of warm bodies is its own world. Developers, designers, project managers, QA techs and systems administrators all have roles to play in IT. In a traditional IT shop, however, each group lives on the other side of the wall from one another.

Even within these departments there are silos. Consider how systems administrators have become specialized. In a single organization you may find an administrator or multiple administrators focusing on individual applications, operating systems, Web sites, or specific aspects of physical infrastructure.

The communications barriers, rivalries and mutual distrust fostered by this approach are so universal they have become a permanent part of our popular culture. There are Internet memes and entire vocabularies of thinly-coded speech dedicated to pointing the finger of blame at the next room over. That's where the black magic happens; and when problems arise, throwing a rival group under the bus is an easy out.

It's not efficient, but it is very human. Unfortunately, our competition these days is from shell scripts and robots, so us puny fleshbags need to up our game.

Thinking Holistically
The first steps along the path towards holistic IT don't involve drum circles, talking sticks or even magic crystals. There's no sharing of feelings, or even a need to play nice. Starting down the long road to joined-up thinking requires little more than buying the right software and ensuring everyone uses it.

Of course, finding the right software for your particular organization's circumstances has a difficulty level of "reality bending," but let's put a pin in that for the moment. The software is important because it not only forms the skeleton upon which to build cross-team interactions, but it gives everyone something to hate that isn't one another.

Shifting the focus of emotion is important. Any attempt at holistic IT will repeatedly touch the raw nerve of job security for all involved. When you are a specialist working in a silo, it's easy to make yourself indispensable. When you work in a cross-discipline fashion with a large number of other intelligent people, it's easy to feel like you're under constant scrutiny and/or being turned into a disposable cog in a vast machine.

It's also important to recognize that the push towards holistic IT is happening alongside some other important transitions within our industry. Hybrid IT is the new normal and it makes troubleshooting all the more difficult.

Cloud-in-a-can is a thing that we can now buy, which will ultimately end up transforming every aspect of how organizations deliver IT. All of this cloud talk is changing how we think about everything in IT, including economics and supply chains, and it has brought both systems integrators and trust back into every serious discussion about IT's future.

All of this is happening against a backdrop that includes a history of organizations treating IT staff terribly, and generations of conflict between end users/customers and IT practitioners. Holistic IT is about people, and that is always a delicate balancing act.

Software Feeds, Speeds and Needs
Helping different silos work together requires -- at a minimum -- infrastructure discovery, e-discovery, secure archiving, reporting, auditing, monitoring, ticketing, testing, orchestration, backup and recovery software. All of this has to cross disciplines. At the core of each rest concepts like profiles, recipes, role-based access control (RBAC) and a huge dollop of “covering your butt.”

Holistic IT is about people, and nobody wants to be target practice for the blame cannon. Taken together, infrastructure discovery, e-discovery, secure archiving, reporting and auditing are tools focused on figuring out what state everything is in, and then archiving it in a secure fashion so that one can prove what happened in the past.

These are tools that management doesn't often see the value in, or may even consider to have negative value, as their primary purpose in the real world is for IT practitioners to say "Yes, I did in fact warn you about this horrible thing, you did nothing, and guess what happened?" Bad for penny pinchers. Great for morale.

Monitoring and ticketing software are likely to be the most difficult software purchases any IT organization makes. Both need to work together: to draw all stakeholders in a problem together without assigning blame or giving anyone any easy out to ignore the problem, but still requiring someone to take charge and shepherd the ticket to resolution.

Problems in a modern datacenter can be complex. Issues experienced by users can involve dozens or even hundreds of different technologies, crossing potentially dozens of individual silos within an organization and possibly involving external suppliers or vendors as well.

Keep the Problem Inside the Walls
Ticketing and monitoring software needs to be able to help practitioners narrow the scope of the problem to a few likely technologies and make it easy to engage the individuals responsible, all while providing a solid trail of indisputable evidence that doesn't allow anyone to "throw the problem over the wall" in order to get rid of it.

Each organization's internal business processes and IT mix will determine which solutions are required to pull this off. Lashing those solutions together into a workable whole is the hard part, and in many situations will require custom code.

Testing and orchestration software comes in a few different flavors. It can be as simple as copy data management or private cloud software. Both should allow infrastructure administrators to define templates or recipes for workloads, often including backup regimens, storage profiles and more. Developers and application administrators can use a self-service portal to generate new workloads, copy existing ones, move workloads and so forth.

Ideally all of this is backed by robust RBAC which controls who has access to what, and what they can do with the data. This in turn feeds into the butt-covering software stack, making audits -- and ultimately, compliance -- easier on everyone.

Testing and orchestration software can also be much more robust. It can go down the route of desired state configs, with a much more programmatic approach. Automated regression testing, chaos monkeys, versioned configurations and more are part of bleeding edge, holistic IT.

Backup and disaster recovery software give everyone the ability to sleep at night. Far from being simple holes into which all of the day's changes are poured, backups and disaster recovery have almost no value if nobody has ever bothered testing them to see if they can restore an organization to a working state.

From recovering a single file to an entire datacenter, there is often more red tape around backups and disaster recovery software than any other part of the organization. In part, this is simply because it isn't regularly used. Some aspects, such as SMB file recovery, have evolved end user self-service portals as a basic expectation. Others -- such as the restoration of active workloads -- are still very much guarded by a priestly case of paranoid and testy administrators wielding forms to be signed in triplicate, which exist primarily because the restore process lacks any semblance of formal testing.

Putting It Together
The end goal of holistic IT is to quantify not only what each workload consists of, but all of its interactions. What happens when a customer pushes "submit" on a form on the Web site? How many systems are involved in making that service available to the customer? Is the entire stack instrumented? Is each part of the form tested as part of ensuring service availability? Are those tests built into the monitoring and ticketing system, and will they survive a disaster recovery event?

If IT is a single entity -- one giant, interwoven machine -- then for it to function, all parts must act in harmony. Harmony is enabled by ease of use, and ease of use is very, very difficult. Documentation, adherence to standards and procedures, and nearly obsessive attention to automation is all required.

Holistic IT is not about job elimination. It isn't about turning us into faceless drones. It is instead about the simple admission that today's IT -- even in the smallest shops -- is composed of so many moving parts that it's simply too much for any one human mind to hold. We are, with very rare exceptions, simply not capable of knowing everything that relies on the piece of IT for which we are responsible, nor everything upon which our chunk of the datacenter relies.

We need software to help bridge the gap. We need to augment our very human memory with documentation and discovery, history and testing. We need to be able to prove to ourselves where the problem lies so that we can have rational, professional conversations with our peers and solve problems for our customers.

Above all, those of us championing this future of IT need to be aware that these changes are big, and big change is scary. Holistic IT makes life easier, but it is ultimately all about people. Keep the feelings of your human compatriots in mind and make the transition slowly. One piece of software at a time.