The Best Prompt Engineer Is a Domain Expert

In 2023 and most of 2024, “prompt engineer” was a real job title. I knew people who held it. The recruiter pitch was that the role required a rare combination of technical fluency and language craft: somebody who could coax an LLM into producing what an organization actually wanted, where most people couldn’t. Companies hired them. Some of them got paid extremely well. There was a moment in early 2024 when the median salary for the title in San Francisco crossed three hundred thousand dollars, and venture capitalists were writing memos about the new white-collar profession.

The job title is fading now. Not because companies stopped wanting good prompts (they didn’t) but because the people who were actually producing useful prompt-driven systems turned out, on inspection, to mostly not be the prompt engineers. They were domain experts who had figured out how to articulate, in writing, what they did all day. The “prompt engineer” was usually a generalist sitting next to them, transcribing their judgment into something the model could read. When the domain expert left the project, the prompts decayed. When the prompt engineer left, the prompts kept working. The relationship was clear; the title was a misnomer.

This essay is about the staffing pattern that follows from internalizing that observation. The argument: the leverage on AI product quality is not in hiring people unusually good at LLMs. The leverage is in identifying who in your organization actually knows what good output looks like, and giving them the tools and authority to encode that judgment into the system. Everything else (the prompt phrasing, the evaluation infrastructure, the retrieval tuning) is downstream of that one decision.

The strongest version of “prompt engineering is a specialty”

The case for prompt engineering as a specialized discipline is real and worth engaging with.

The argument goes like this. Prompts are a craft. There are non-obvious techniques (few-shot examples, structured output cues, chain-of-thought scaffolding, role priming) that affect output quality in ways that are not intuitive to domain experts. A senior radiologist may know exactly what makes a good radiology report, but she doesn’t necessarily know how to write a prompt that gets a frontier model to produce one. The prompt engineer is the translator between expert judgment and model behavior.

There is also a version of this argument that says that prompt engineering is a transferable skill. Someone who is good at it gets good across domains, because the underlying tricks are domain-independent. You hire one prompt engineer, you put them on whatever the company is currently building, and the same craft transfers.

Both points have partial truth. There are non-obvious techniques. There is a craft. A 2023 prompt written by someone who knew what they were doing did outperform a 2023 prompt written by a domain expert with no LLM experience, holding the underlying judgment constant.

The part that doesn’t survive contact with how good 2026 models are is the assumption that those non-obvious techniques are still where most of the value lives. They aren’t. Frontier models in 2026 are dramatically better at understanding intent than 2023 models were. The “tricks” that were necessary to elicit good behavior (the elaborate role primings, the forced chain-of-thought, the carefully escaped JSON formatting) are largely subsumed by the model’s native capability. A 2026 frontier model given a clear, plainly-written specification of what the domain expert wants will produce output of comparable quality to the same specification dressed up in three pages of prompt engineering. Stanford’s DSPy team and others have published replicable results showing that systematic prompt optimization past a reasonable baseline tends to produce single-digit-percentage-point gains, with the bigger improvements coming from optimizing the surrounding harness rather than the prompt text itself. That is the size of the prompt-craft contribution in 2026. Real, measurable, and small.

What grew, in inverse proportion, is the value of clearly articulating what you want. The model doesn’t need clever tricks anymore. It needs an unambiguous statement of the goal, the criteria for success, the edge cases that matter, and the failure modes that are unacceptable. Producing that statement is not a craft that transfers across domains. It is the work of a person who deeply understands the domain. The radiologist can do it. The senior claims adjuster can do it. The tax accountant can do it. The “prompt engineer” who has never read a radiology report cannot, no matter how clever they are with the formatting.

Cut to it: the argument is right for 2023 and wrong by 2026. The craft of prompt engineering has been absorbed into the model’s improving capability. The work that remains (and is now where the leverage actually lives) is the work of domain experts articulating judgment in writing. That work is the bottleneck.

What domain experts actually contribute

What a domain expert provides in an AI product is more concrete than “knowledge.” Four specific contributions, none of them substitutable by anyone else on the team:

Domain experts provide the gold standard. They are the people who can look at an output and say, with authority, “this is correct” or “this is wrong”, and write down the criteria they used. The gold standard is what the eval suite is built against. Without it, the eval suite is comparing outputs against the wrong target, and you end up optimizing for something that looks good to a generalist but fails the actual users.

Domain experts provide the failure mode taxonomy. They know what kinds of wrong answers are catastrophic versus merely annoying, what edge cases matter, what categories of error a real user will notice. The ML engineers cannot know this, because they don’t see the work the way a practitioner sees it. A claim that looks similar to a layperson (“the system mis-classified this loan application”) is, to a senior underwriter, either “a routine model mistake we’d catch in QA” or “a regulatory exposure that could cost us a fine.” The difference is not technical. It is domain-specific.

Domain experts provide the criteria for abstention. Where should the system say “I don’t know”? What categories of question are out of scope? What kinds of inputs should trigger an escalation to a human? The model cannot decide this on its own, and the product manager cannot decide it without consulting an expert. The threshold between “the AI handles it” and “the AI escalates” is set by someone who understands what a human in that role would do.

Domain experts provide the rubric maintenance over time. The world changes. Regulations change. Best practices change. Internal policies change. The eval rubric that was correct last year is partially wrong this year. Someone has to keep it current, and that someone is not the ML team, because the ML team doesn’t read the trade journals or sit in the practitioner meetings where the changes get communicated. The domain expert is the only person who can keep the gold standard alive.

Notice what is absent from that list. I have not mentioned writing prompts. I have not mentioned formatting JSON. I have not mentioned chain-of-thought tricks or few-shot example design. The domain expert’s contribution is upstream of all of that. They produce the criteria; the engineering team encodes the criteria; the prompt becomes whatever the encoding requires. The prompt is downstream of judgment, and judgment is what the domain expert owns.

A staffing rubric for AI projects

The staffing pattern below is what I now recommend, with what each role actually owns. It is different from how most enterprises currently approach this. The difference is one of the strongest predictors of project success I have encountered.

The domain SME owns: the gold standard, the eval criteria, the success rubric, the abstention thresholds, the categorization of failure modes, and the periodic re-validation of the criteria over time. This person is the benevolent dictator I mentioned in the evals essay. Their judgment defines what the product is for. They do not need to be technical. They need to be willing to write down what they think, in concrete enough terms that an engineer can translate the writing into evaluations.

The AI/ML engineer owns: the evaluation infrastructure, the retrieval pipeline, the model integration, the gateway-level observability, the prompt encoding of the SME’s criteria. This person turns the SME’s gold standard into a running system. They do not need to know the domain at expert depth, but they need to know enough of it to ask the SME good questions when ambiguity arises.

The product manager owns: the workflow design, the trust scaffolding from the user-process essay, the escape hatches, the confirmation steps, the user-visible success metrics. This person turns the working system into something the actual users will adopt. They negotiate between the SME’s accuracy demands and the user’s tolerance for friction.

The operations / SRE owns: cost, latency, rate limits, on-call rotation, incident response. This person keeps the system running and the budget in line.

Four roles. The most common anti-pattern in enterprise AI is that the AI/ML engineer is hired to own all four: the SME is “consulted” rather than authorized, the product manager is overruled when their UX concerns slow down the engineering team, the operations role is filled by the same engineer at night. The result is a technically sophisticated system, optimized for the wrong target, deployed without trust UX, that goes over budget six weeks after launch. The fix is to staff the four roles distinctly and to give the SME, in particular, the authority their role actually requires. Most enterprises don’t, because the SME is usually a busy senior practitioner who is doing their day job and “consulting on the AI project” is the third item on a list of priorities. The teams that win clear time on the SME’s calendar and treat them as the product owner. The teams that lose treat the SME as a tester.

Why the SME’s calendar is the bottleneck

If the SME owns the gold standard and the SME cannot give the project meaningful time, the entire pipeline downstream of them is operating against a missing input. This is, in my experience, the single most common reason ambitious enterprise AI projects fail. The technology works. The retrieval is tuned. The evaluation infrastructure exists. The judge prompts are written. The thing that is missing is somebody whose taste is encoded into the judge prompts, and that absence is invisible until you look at the outputs and ask “who decided this was correct?”

The pattern I now push for is concentrated SME time at the start of the project, distributed SME audit time over the long run. The SME spends a week or two upfront generating gold-standard examples, writing the rubric, walking the engineering team through failure modes. After that, they audit a sample of outputs once a month, mark drift, and update the rubric. The total ongoing SME time is small, and most senior practitioners are willing to commit to a few hours a month for a product they had a hand in shaping. What they will not commit to is being the constant on-call quality reviewer, which is what they get asked to be when their initial role wasn’t structured correctly.

The hard part is the upfront commitment. A senior radiologist, a senior tax partner, a senior underwriter: these people are expensive and busy. Getting two weeks of their concentrated attention is a real ask, and a lot of organizations balk at it. The organizations that don’t balk produce AI products that work. The organizations that try to substitute the SME’s time with “research” or “user interviews” or “the engineering team’s intuition” produce AI products that look like they should work and don’t, and the post-mortem never quite identifies the missing ingredient because the ingredient is not technical.

What this requires architecturally

The architectural piece that makes the SME staffing pattern scalable is the ability for the SME to work asynchronously rather than in real-time pair-programming mode. The SME should be able to log into a system, see a sample of production traces, mark them correct or incorrect, optionally add a note explaining the decision, and have that signal flow into the eval pipeline without sitting next to an engineer for sixty hours. The work the SME does is judgment; the work of plumbing the judgment into the system is engineering, and the two should not be locked together.

This is what the gateway-as-instrumentation argument enables on the build path. AOCore captures every call (the prompt, the retrieved context, the model output, the user reaction) and exposes it in a form the SME can review. The SME does not need to understand the code. They need to be able to look at the trace, decide whether the output was right, and click. The engineering team takes care of the rest. Most teams trying to involve SMEs in AI quality fail at this step because the SME has no way to see the data; the gateway is what makes the data inspectable to non-engineers.

For organizations that aren’t going to build this infrastructure from scratch, AODex’s persona library is the SME-pattern-at-product-scale. A senior practitioner inside the buying organization can configure a persona (voice, system prompt, attached knowledge base, behavioral guardrails) once, share it across the team, and have that persona deployed against their domain. The pattern lets domain expertise be codified once and consumed by everyone who needs it, without an engineering team sitting between the SME and the deployed product. Combined with persistent memory and team workspaces, the SME pattern becomes a feature of the platform rather than a project organizations have to staff.

The headline of this essay is one I believe more strongly the longer I work in this space. The best prompt engineer is a domain expert with a quiet hour, a clear specification of the problem, and the authority to decide what good looks like. The cleverness was always in the judgment. The model has just made the judgment finally, durably, encodable.

Find your domain experts. Clear their calendars. Build the system around their taste. The rest is engineering.