In-Depth

Copilot in Azure

The always-skeptical Paul worries about this amazing AI tool dumbing down junior admins too much, but he's also impressed with the preview and excited about the possibilities, giving it the ultimate compliment: "It's really cool."

• By Paul Schnackenburg
• 07/03/2024

There are several scenes in "Star Trek TNG" where Geordi or Data talks to the computer and says "run a level 5 diagnostics on this system" or words to that effect. As an IT teacher, for many years I've told my students how nice it would be to be able to do exactly that with our IT infrastructure, never expecting this to become reality.

At Microsoft's Build 2024 they made the public preview of Copilot in Azure open to all users, no sign-up required (I've been in it for about six months). Let's see if we're living in the future, and I can indeed use Copilot in Azure to automate diagnostics and other infrastructure tasks successfully.

A Free Copilot?
Microsoft is surfing the AI hype hard and putting Copilots into all of their applications / services / platforms that shows signs of life -- it'll be interesting to see when the dust settles, and we head into the "trough of disillusionment" which ones of these quietly die off, and which ones thrive. But Copilot isn't one product; even within one suite of products they behave slightly differently. For example, Copilot in Microsoft 365 shows up in Word, Excel, PowerPoint and Outlook (both new and old), but it's the team that maintains Word that customizes that Copilot and that's a different group from the Outlook team and so on. And Microsoft 365 Copilot is a paid subscription, $30 per person, paid in advance for one year.

The most expensive Copilot is definitely the one for security, weighing in at $4 per Secure Compute Unit (SCU), a very nebulous concept, per hour. Microsoft recommends three SCUs for a normal deployment, and also recommends that you keep it running 24x7x365 for a neat bill of a bit over $100,000 per year.

There's been no word on cost for Copilot in Azure yet, and normally pricing is announced when a product / service becomes Generally Available (GA). If I was a betting man, I would say it'll remain free, but then again it takes compute resources to process every prompt, so who knows? The "here's a free public preview to get you hooked, followed by the cost shock at GA" is by now a well-known drug dealer, giant tech company playbook.

Copilot Everywhere
Since the announcement of Copilot for Azure back at Ignite 2023, it's proliferated to more parts of Azure and can now help you deploy VMs, web apps and AKS (Azure Kubernetes Service) clusters, and it understands and can generate CLI scripts, PowerShell scripts and Kubernetes YAML files, Kusto Query Language (KQL) queries, with Bicep and Terraform support coming soon.

Copilot works across the Design, Operate, Optimize and Troubleshoot phases of public cloud usage. During the preview there are some limitations to be aware of. You can only make 15 requests during a chat, and only do 10 chats in a 24-hour period, and actions you take on more than 10 resources have to be performed outside of Copilot (perhaps using a script generated by Copilot).

Just like any other Copilot, this one only has access to resources and actions on those resources that the individual user account has. By default all users in a tenant (that have Azure portal access) have access to Copilot, but an administrator can limit this to specific users and/or groups.

All activity taken by a user is recorded in the Activity log for the resource, and the audit log. Copilot is also trained to work with forms in the Azure portal, and as you know, there is always a form to fill in when you're creating a new resource.

Getting the Lay of the Land
Understanding what Resource Groups (RGs) and resources are present, and what their state is, can be important, especially if the tenant / subscription was created by someone else. Here I'm figuring out what RGs I have in this tenant, and in what region they're located.

Note that Copilot generated a KQL query for the Azure Resource Graph and ran it automatically to gather the information, as well as showed me the query so that I can copy it out and customize it if desired. The results are limited to the top five RGs (a public preview limitation). Copilot also gave me some suggested follow-up prompts, including finding out what resources were inside the web app RG. Following that output I clicked on the link for the web app, which took me to that page in the main window, whilst leaving Copilot active on the right.

The follow-on question is how many daily visits this web app has (it's my business web site, the very definition of a low-traffic site), showed me zero visits in the last 24 hours. I then asked for the same data for a month, which didn't work and again only showed me one day in the Copilot window but clicking the button "Open in Metrics Explorer" gave me the full monthly graph.

The point of showing you the steps I took aren't the specific prompts and answers, but the process that a person would go through to learn about an Azure infrastructure. Obviously, I could have browsed to these RGs, found the resources and looked up in the documentation how to graph the daily visits to the site, but this would have taken a lot longer.

Creating New Resources
Finding existing stuff is good, but how about creating new infrastructure? Here I'm asking Copilot to create a PowerShell script to deploy an Azure SQL Database for me.

Clicking the Run button opens the Cloud Shell and, in this case, gave me errors. Note also that it is creating a managed SQL DB instance, rather than a PaaS Azure SQL DB (reading the comments underneath the output from Copilot is important). I also tried asking Copilot to generate a script to create a VM, with similar errors in the output.

Switching to GUI, I went to the Azure Cosmos DB page to create a new database and asked it to help me create a free tier database, and it gave me the step-by-step instructions but no actual help in the UI.

So, Cosmos DB might not have a specific plug-in for Copilot yet, but I know that VM creation does. Here I asked Copilot to help me create a cost-effective VM, and not only did I get instructions, but also pointers in the UI. Here it's suggesting I enable Azure Spot discount, with a button to enable that option, then followed by suggesting that I use a Standard_B2s-v2 (a very small and cost effective VM), and finally enable auto-shutdown. Note also that it was automatically given the VM the name "MyLowCostVM," but it didn't suggest I enable Azure Hybrid Benefit (presumably because you need to have Software Assurance licensing to use that). Once I enabled auto-shutdown using the Copilot provided button it suggested I continue directly to Review and Create.

So far, so good, these are all indeed sensible suggestions if I need a cost-effective VM.

Copilot for Databases
Copilot is also available for databases (private preview at the time of writing), helping you deploy databases, check them for performance issues and optimize queries (yes, Copilot speaks T-SQL), and this is available for Azure SQL DB as well as Cosmos DB (with others coming as well).

Here's Copilot helping to optimize a T-SQL query that was using a lot of CPU time.

The coolest demo I've seen, however, was Copilot assisting with asking if an on-premises, Arc-connected database had been backed up lately. And here's the prompt where it's checking if an Availability Group in SQL Server on-premises is working as expected.

Kubernetes + Copilot
Azure Kubernetes Service (AKS) is another one of the early adopters. Copilot knows how to read and generate Yaml for Helm charts, as well as how to use Kubectl, the Kubernetes CLI.

Here's an example of Copilot using Kubectl to change the configuration of an AKS cluster.

Managing Costs
An obvious use case is understanding your usage of various Azure services, and what you can do to manage that cost. Here I start by asking how much I've spent in this subscription this month, followed by asking it to forecast what my spending will be in the coming month.

When I then asked a follow-up question to optimize costs, it asked me to set my Cost Management scope, and also explained what that is. This also highlights another great feature of Copilot: It's aware of the context that you're working with in the main area of the portal, so if you're working with Costs, it'll take you there. If you're working with a VM, and you're asking questions about "a VM" it knows that you mean the one (or ones) that are shown in the left area of the portal.

And when you're asking it to act, restart a cluster, create a database or change a configuration, it's always double-checking with you in the Copilot pane to show you what's about to happen and verifying that you're OK with it.

Copilot for Azure is also available in the Azure mobile app -- here you can see it on my Samsung phone.

Conclusion
Whilst not available yet, Copilot will also show up in the Azure CLI / PowerShell interface. At the moment Copilot in Azure (like many other Copilots, frankly) is a bit hit and miss. Sometimes it's amazing, almost like its reading my mind and figures out exactly what I want. Other times it's frustratingly "stupid" and unable to do the most basic of tasks. It is in preview and its capabilities have improved considerably between when I first tried it some months ago and now.

If it does fulfill its potential at some point in the future, I worry about this becoming another layer between "how it actually works" and the appearance for the user / admin. We see this with programming languages, with further layers of abstraction added, combined with ready-made (often open-source) libraries providing easy to use building blocks -- but no one deeply understands how the whole thing works. Until a hacker sits down, figures out how it actually works under the hood and exploits that to their advantage.

I fear we will have junior admins in a few years, guided by well-meaning Copilot interactions, creating infrastructure and resources without the ability to really judge whether the output is actually good or not.

And today, whilst it demos well to create a VM with the assistance of Copilot, in an Azure production environment you don't click next, next, next in a wizard. There are templates, and guardrails and policies, so it'll be interesting to see where Copilot finds its usefulness there.

Overall though, it's really cool, and I can definitely see Copilot bringing added efficiency to designing, operating, optimizing and troubleshooting Azure deployments.