In-Depth
Why Context Is More Important Than Prompt Engineering
Recently, I have read quite a few articles that describe prompt engineering as being the number one skill that users will need in the AI-driven workplace. However, while I cannot dismiss the importance of good prompt engineering skills, my belief is that prompt engineering will soon give way to context engineering.
The idea behind this philosophy is simple. At first glance, LLM-based chatbots would seem to allow us to converse with a machine in the same way that we might talk to another human, and indeed there are similarities. However, there are also some major differences between the way that we interact with humans and chatbots, and prompt engineering might be described as a tool for overcoming these differences.
In order to better illustrate what I mean by this, let me share with you a story from a few years ago. One night, I was having dinner with several friends and someone else that I knew who was visiting from Australia. At some point during the meal, someone made a joke about "Monday morning quarterbacks." Everyone immediately got the reference, except for my Australian friend who had never heard the expression before. To put it another way, she lacked the cultural context needed in order to understand the joke.
That's a lot like what happens with prompt engineering. Prompt engineering usually works fine if you are doing something simple, but if you are asking for something that is difficult or specific, you may find yourself having to over-explain to the chatbot what it is that you are trying to achieve. Human conversation is often based around things like cultural knowledge, shared experiences, and previous interactions. A chatbot may lack these types of shared knowledge.
The way that we engage with a stranger is different from the way that we engage with a close friend simply because the stranger lacks the type of connection that can only come through shared experiences. Such a conversation can feel awkward and unnatural until we establish a connection with the person. The same thing happens with AI. Prompt engineering may feel similarly unnatural and cumbersome because we are trying to compensate for missing shared context by over-explaining, providing excessive detail, and using repetitive clarification.
But what if we truly could converse with a chatbot in the same way that we engage with other humans? What if instead of writing verbose prompts, we could just say whatever is on our minds and know that the chatbot will understand exactly what we mean?
The key to accomplishing this would be to provide relevant context, but without going overboard. Let me give you another example from real life. Quite a few years ago, I took a job working for the military. For the first few weeks, I was constantly getting myself into trouble. I didn't understand what the various ranks meant, I wasn't familiar with the countless acronyms, and I had never heard most of the military slang terms that seemed to suddenly dominate every conversation. I was a total misfit, not because I was stupid, but rather because I had never been exposed to the military in any meaningful way.
Now imagine how the situation would have been different if, before starting the job, I had taken the time to learn about ranks, military jargon, commonly used acronyms, and military culture. I'm sure that I would still have had a bit of a learning curve, but there is little question that my transition into my new role would have been much smoother. This is what context is all about.
So what exactly is context in the world of AI? I have seen the term used in a few different ways, but at its simplest, context refers to the background information that helps the AI to understand what you are asking and why. The actual request is stored in the prompt, but it is the context that helps the AI to know how best to answer the prompt. The context can vary widely depending on what the AI is designed to do. The context might, for example, include the conversation history, a collection of documents, project details, a dictionary of organizational terms, or just about anything else that might be helpful in determining a prompt's meaning and intent.
Let me give you a really simple example. Suppose for a moment that I asked an AI chatbot to give me some restaurant recommendations. The chatbot would do its best to come up with a list of restaurants, but the list would be far from perfect because the chatbot is missing some key pieces of information (context). For example, the chatbot may not know where I am in the world, so that it can recommend restaurants close by. The chatbot probably also does not know that I have Celiac, meaning that I have to avoid gluten.
So why not just include all of this within the prompt? To be fair, building prompts around these types of details is commonplace. But remember, the goal is to make AI interactions more natural and more like conversing with a human. Think about it this way: when I am out with friends and we are trying to figure out where to go for dinner, I don't have to remind them that I can't have gluten. They already know that.
This is the key difference between prompt engineering and context engineering -- prompts are fleeting, while context is pervasive. Once you include certain details in the context, the AI "knows" those details and can apply that information when responding to prompts. Hence, in the restaurant example, the prompt goes from "Where should I go for dinner? I am in Rock Hill, South Carolina, and I cannot have gluten." to simply, "Where should I go for dinner?"
By using context, you can help to avoid making prompts bloated, repetitive, and unnatural. Establishing good context also helps to avoid the problems that can come from accidentally omitting an important detail within a prompt.
Just as prompt engineering can be an art form, so too can context engineering. When constructing context, it is important to give the AI the information that it needs, but without just "throwing the kitchen sink at it."
When you give an AI too much context, the sheer volume of data dilutes the information that actually matters. The AI might struggle to figure out what information to use in answering the prompt, and having excessive irrelevant context can also increase the likelihood of hallucinations.
The key to good context engineering is making sure that the information within the context is relevant to what you are trying to achieve. It's also important to keep the context current. Remember, providing outdated information can lead to the AI drawing incorrect conclusions. Finally, you should try to make the context data structured, well organized, and concise. The better job you do of this, the easier it will be for the AI to make sense of the context.
Ultimately, context engineering is about giving AI the background information that it needs in order to respond to prompts in a more useful way. In the future, getting good results from AI is going to depend far less on prompt construction and more on outfitting the AI with relevant context. It will no longer be about teaching humans how to talk to machines, but rather equipping machines with shared understanding that will enable them to behave in a more human-like way.
About the Author
Brien Posey is a 22-time Microsoft MVP with decades of IT experience. As a freelance writer, Posey has written thousands of articles and contributed to several dozen books on a wide variety of IT topics. Prior to going freelance, Posey was a CIO for a national chain of hospitals and health care facilities. He has also served as a network administrator for some of the country's largest insurance companies and for the Department of Defense at Fort Knox. In addition to his continued work in IT, Posey has spent the last several years actively training as a commercial scientist-astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space. You can follow his spaceflight training on his Web site.