Tracking enterprise infrastructure's evolution in the AI era

In-Depth

An Interview with Chris Wolf -- Beyond the Hype: Three Years of Enterprise Private AI Realities

• By Tom Fenton
• 09/14/2026

VMware Explore was extra special this year because I got a chance to sit down and interview former Virtualization & Cloud Review columnist and longtime industry analyst Chris Wolf, Global Head of AI and Advanced Services for Broadcom's VMware Cloud Foundation division.

After a quick chat about exploring the beauty of Utah's outdoors (he uses four wheels to do so, while I use two), we sat down to talk tech in general and AI specifically.

Longtime readers will remember Chris as one of Virtualization & Cloud Review 's original columnists. Since then, his career has taken him from Burton Group and Gartner to VMware and now Broadcom, where he leads AI and advanced services for the VMware Cloud Foundation division.

Wolf told me he was the VMware leader who first pitched Private AI. At the time, enterprise generative AI was still dominated by cloud API experiments and plenty of inflated promises. Three years later, enterprise IT is dealing with the less glamorous realities of data sovereignty, GPU costs, power density, and operational risk.

The Foundation: Open-Source Runtimes and Image Registries
Wolf said that, looking back three years later, some of VMware Private AI's technical foundations came down to a couple of pragmatic software decisions. While many vendors were developing proprietary AI management tools, VMware's engineering team chose the open-source vLLM project as a model-serving runtime and used Harbor as the model store.

Treating models as standard Open Container Initiative (OCI) artifacts inside Harbor might not have been the obvious choice then. For virtualization and platform administrators, though, the approach feels familiar: standardized tools, push-and-pull workflows, role-based access controls, and existing CI/CD processes.

Chris said the decision to adopt vLLM, which VMware had already used internally for benchmarking before rolling it out more broadly, has also held up well. Its memory-efficient PagedAttention mechanism and strong community adoption have helped make vLLM one of the leading open model-serving runtimes.

The early strategy may have moved faster than the market, however. In 2023, the performance gap between the best cloud models and many open-weight models was substantial, and enterprises often had good reasons to use public cloud APIs. Over the past couple of years, open-weight models from Meta, Mistral, and others have narrowed that gap considerably for many enterprise and domain-specific workloads, making private and hybrid deployments increasingly practical.

Virtualization Overhead and the Consolidation Dividend
A persistent question among infrastructure architects is whether enterprise AI workloads should run on bare metal rather than in virtual machines. Much of that thinking goes back to high-performance computing, where the assumption has long been that adding a virtualization layer introduces an unacceptable latency or compute penalty.

Wolf told me that internal and customer testing comparing modern VMware virtualization with bare-metal deployments for AI workloads showed a performance delta of around 1.5 percent. That difference is small enough that the operational benefits of virtualization can become more compelling.

Wolf argued that the small performance penalty can be more than offset by consolidation. For example, workloads that previously required two physical servers could be consolidated onto a single, higher-density virtualized server. Depending on the workload and configuration, the potential savings in rack space, power, cooling, and host management can easily outweigh a small amount of compute overhead.

That balance is one reason I think the discussion about running AI on bare metal versus virtualized infrastructure is becoming more nuanced. If virtualization can deliver near-bare-metal performance while letting infrastructure teams consolidate and manage resources with the tools they already know, the choice isn't as obvious as it once seemed.

Pivoting from High-End vGPU to Practical PCIe Pass-Through
Not every initial architectural assumption held up the same way in customer data centers. Early iterations of VMware Private AI leaned heavily on NVIDIA vGPU technology, which lets physical accelerators be shared among multiple virtual machines but also adds software and licensing complexity.

As companies moved from AI pilots to deployments across data centers and regional branch offices, the requirements changed. Many distributed sites did not need complex multi-tenant GPU sharing. They needed to run a particular model on a dedicated accelerator.

VMware has increasingly emphasized direct PCIe GPU passthrough for use cases where customers do not need to share a GPU among multiple VMs. The interesting part is not simply assigning a PCI device to a VM; the challenge is making sure normal infrastructure operations, including maintenance mode and patching, continue to work with dedicated accelerator hardware.

Passthrough configurations can also eliminate the need for some GPU virtualization licensing costs on specialized inference nodes. Wolf pointed to configurations using GPUs such as dual NVIDIA RTX 6000 Ada cards as an example of how enterprises can build inference systems without automatically moving to much more expensive data-center accelerator hardware.

Tokenomics: The Real Math Behind Private Hosting
Over the past three years, the financial discussion around enterprise AI has matured. During the early generative AI rush, organizations could experiment with consumption-based API tokens and charge them to R&D budgets. Once those applications moved into production, however, the recurring cost of inference became much harder to ignore.

Wolf said some enterprise customers running high-volume inference are seeing private infrastructure deliver an eight- to ten-times cost advantage over public-cloud consumption. The actual economics, of course, depend heavily on workload volume, GPU utilization, model choice, power costs, how the infrastructure is amortized, and many other variables.

Chris also pushed back on the idea that private AI automatically means multimillion-dollar, liquid-cooled hardware clusters. Training the largest foundation models can require rack-scale systems, high-density accelerators, liquid cooling, and enormous power budgets. But fine-tuning and domain-specific inference can often run on standard air-cooled PCIe hardware.

A dual-socket server with modern enterprise PCIe GPUs can host quantized open models and mixture-of-experts architectures in a standard data-center rack, depending on the model and workload. That makes the entry point for enterprise inference considerably more approachable than the infrastructure required to train a frontier foundation model.

The Agentic Shift: Taming Tenacious Loops and Guarding Production
Conversational chatbots have become table stakes for many enterprises. The next step is agentic AI, where models can call tools, inspect logs, query endpoints, and potentially trigger remediation actions.

Wolf made an interesting distinction between traditional automation and agents. A script generally follows the path you gave it. An agent can decide that the path isn't working, try something else, and keep going. That helps when the agent is doing the right thing, but it can be expensive or dangerous when it isn't.

An unconstrained agent could repeatedly query a model, consume an API quota, or keep using infrastructure resources long after an administrator expected it to stop. That is why runtime boundaries matter. Wolf pointed to controls such as token quotas, execution budgets, circuit breakers, and comprehensive compliance logging.

The risk becomes greater with shadow AI inside organizations, where developers may give experimental agents access to service accounts. An agent with too much authority could make a serious configuration change or damage data while attempting what it believes is an appropriate automated fix.

Wolf told me VMware support operations already use internal AI assistants and diagnostic agents to analyze telemetry, parse system logs, and cross-reference support information. According to Wolf, these systems can surface likely root causes in minutes for problems that previously required engineers to spend considerably longer working through telemetry and logs.

The important part is what happens next. The agent may diagnose the problem and prepare remediation commands, but a human administrator must validate and authorize the change before applying it to production infrastructure.

The Standardized Stack Versus Snowflake Architectures
To improve interoperability, VMware has aligned its private AI stack with open interfaces, including Kubernetes-conformant orchestrators, OpenAI-compatible APIs, and the open Model Context Protocol (MCP). MCP provides standardized client-server interfaces through which AI applications can interact with data and tools, subject to the organization's security and authorization controls.

Wolf also warned me about what I call the snowflake problem: building an AI platform from dozens of independently sourced components that nobody wants to own six months later. A collection of open-source projects can make for an impressive demonstration, but maintaining, securing, patching, and supporting a customized production architecture is a different matter.

As enterprises move from proof-of-concept experiments to production, the fundamentals of enterprise infrastructure still matter. High availability, rolling patch management, predictable security boundaries, and disciplined cost control remain baseline requirements, whether the workload is a traditional relational database or a distributed neural network.

What I Took Away
After about half an hour, our time was up. What stuck with me was how much the conversation had moved away from the early AI hype and toward the same things we have always worried about in enterprise infrastructure: cost, utilization, lifecycle management, and security.

Wolf’s experience suggests that some of VMware’s early architectural decisions, particularly the use of vLLM and Harbor, have held up well. In contrast, others have evolved as customers moved from experiments to production. The virtualization discussion was equally interesting. If enterprises can get near-bare-metal AI performance while gaining the consolidation and management benefits of virtualization, running every AI workload directly on a physical server becomes a much harder argument.

The biggest change may be what comes next. Autonomous agents can do considerably more than a chatbot, but that additional capability comes with additional risk. Runtime limits, circuit breakers, logging, and human approval aren't optional details when an AI system can make changes to production infrastructure. For me, that was the most useful takeaway from the conversation: enterprise AI is becoming less about the hype around what models can do and more about the infrastructure discipline required to let them do it safely.