In-Depth
How to Scale Backend Infrastructure for the Age of Agentic AI
As autonomous agents and AI-driven systems take center stage, backend infrastructures are feeling the heat. Traditional architectures are being pushed to their limits by API traffic that is exploding not just in volume, but also in complexity and unpredictability.
In a virtual summit presented today from Virtualization & Cloud Review titled Managing API Traffic in the Age of Agentic Systems, we explored the challenges of scaling for machine-driven workloads. Presenter Greg Schulz, founder of Server StorageIO, detailed how the industry is moving from software-defined to API-defined infrastructure to meet these demands. It's being made available for replay thanks to the sponsor A10 Networks.
We're going from not just having software-defined data infrastructures to API-defined infrastructures as well as just regular workloads."
Greg Schulz, founder of independent IT analyst firm Server StorageIO
The Rise of API-Defined Infrastructure
The transition to agentic systems is fundamentally altering the demand drivers for modern workloads. Schulz noted that the industry is evolving "from Software Defined to API Defined Infrastructure." This shift is characterized by three primary metrics: volume, velocity, and variance. Volume refers to the absolute number of calls being made. Velocity describes the "speed and rate of calls being made." Variance encompasses the diverse types of requests, sizes, and specific service needs required by AI applications.
[Click on image for larger view.] The Big Picture - Problem, Issues and Challenges - More Demand and Workloads (source: Greg Schulz).
Autonomous Reasoning and Agentic Workloads
Modern workloads are "evolving from automated to autonomous reasoning." Unlike traditional bots, AI agents utilize Large Language Models (LLMs) as a "brain" and Retrieval Augmented Generation (RAG) as a "memory" to perform complex tasks. Schulz described an agent as a "module, app, plugin, piece of software that can do some functions" on behalf of a user. This results in a significant increase in Machine to Machine (M2M) and Agent to Agent (A2A) activity. Schulz defined A2A as "One agent making a call to another agent, or to another service or another task via an API."
[Click on image for larger view.] Applications and APIs - Big Pictures Fundamentals and Refresh (source: Greg Schulz).
Scaling via Federated API Management
To address these scaling challenges, Schulz recommended the use of API gateways to "provide abstraction & transparency to remove complexity." These gateways facilitate rate limiting, caching, and load balancing for high-velocity traffic . In distributed environments spanning edge, on-premises, and multi-cloud locations, Schulz suggested a "federated API management approach." This model allows organizations to orchestrate across different management frameworks and platforms . "Gain situational awareness, timely insight and observability into your environment" to avoid "flying blind," Schulz told the audience .
[Click on image for larger view.] API Management - Gateways, Management and Federation (source: Greg Schulz).
Economic and Operational Considerations
Managing agentic traffic also requires an understanding of "API economics." This involves tracking the costs associated with third-party API calls and potentially monetizing internal services. Schulz emphasized that as usage increases, organizations must "understand threat risks, and vulnerabilities" associated with new endpoints. Effective management should provide insight into resource usage across network, compute, memory, and storage layers.
And More
Beyond the technical shift toward agentic workloads, Schulz emphasized the critical role of "situational awareness" and observability in modern infrastructure. He warned that many organizations are currently "flying blind" and must prioritize high-quality telemetry to feed into their management tools. This awareness is essential for identifying "degraded items or layers" and managing "planned as well as un-planned maintenance" for APIs. According to Schulz, a robust management strategy must also encompass the "API lifecycle," providing the orchestration needed to handle troubleshooting, diagnostics, and remediation across distributed environments.
Schulz also delved into the evolving concept of "API economics," noting that every call made to a third-party service carries either a direct or hidden cost . He urged organizations to track these metrics closely to understand the cost of usage versus the potential for monetization . Security remains a paramount concern, as the proliferation of APIs creates more "points of vulnerability or exposure." To mitigate these risks, Schulz recommended using API gateways to enforce "phishing resistant" authentication and establish "guard rails" that reduce the chances of AI hallucinations or malicious back-end exposure .
All of the above and much more can be found in the replay. And while replays are fine, especially if timely (this was just today, after all), one of the best things about attending such online education summits and events live is the ability to ask questions from the presenters, a rare opportunity for expert, real-world, one-on-one advice (not to mention the chance to win a great prize, in this case a $300 AMEX gift card, provided by sponsor A10 Networks, a leader in security and infrastructure solutions, which also presented a session).
With that in mind, a list of upcoming events from our parent company can be found at the Redmond summit list.
Upcoming 2026 Virtual Summits
About the Author
David Ramel is an editor and writer at Converge 360.