Why the quality of AI depends on the quality of human expertise behind it
Lately, I’ve been spending quite a bit of time helping several of my former client organizations improve their internal AI implementations. Concurrently, I’ve been doing part-time work with several major AI startups, at my own pace (mostly nights and weekends), evaluating and helping improve their large-scale AI models. Along the way, I’ve also taken advantage of some free AI-related training and certification opportunities. Most recently, I completed a course on mastering Prompt and Context Engineering, including a practical project focused on improving AI-generated Scrum Sprint assessments.
These experiences have been interesting because they allow me to look at AI from two different perspectives: how AI models are developed and improved, and how companies attempt to use these models to solve real business problems.
One thing keeps coming up, and frankly, I find it concerning.
I’m seeing more and more organizations introducing AI capabilities into tools like Jira, Azure DevOps, Asana, and other platforms to support what people commonly call “Agile stuff”: backlog management, user story creation, sprint assessments, reporting, forecasting, dependency management, and even agile coaching.
The opportunities are significant. However, I’m not particularly happy with how many organizations are approaching this and how much money the spend doing things incorrectly.
Too often, companies simply activate an AI feature, connect it to their existing tooling, provide some generic instructions, and expect meaningful improvements. What they frequently overlook is the quality of the professional knowledge, assumptions, and mental models that guide the AI in the first place. And this is where things can go wrong.
In my recent work, I’ve been experimenting with several closely related disciplines that can make a significant difference:
Prompt engineering is about designing effective instructions for AI. It means clearly explaining what we want the AI to accomplish, what constraints it must respect, what it should avoid, and what a successful result should look like. A prompt such as “Analyze our Sprint performance” leaves too much room for interpretation. A better prompt explains which evidence to examine, how to interpret it, which conclusions would be inappropriate, and what questions to raise when information is missing.
Context engineering goes further. It involves supplying AI with the right background information, professional knowledge, business context, and supporting evidence. For example, an AI system assessing Scrum Sprint results needs more than Jira statistics. It needs to understand the sprint Goal, the team’s Definition of Done, organizational dependencies, historical delivery patterns, the difference between team performance and individual performance, as well as other systemic impediments. Without this context, even a sophisticated model can produce misleading conclusions. AI content engineering focuses on creating and structuring the knowledge and examples that guide AI behavior. This includes authoritative reference materials, practical examples of good and bad outputs, decision rules, exceptions, and evaluation criteria. For example, if we expect AI to generate meaningful user stories, we should provide examples of properly sliced, customer-valued stories, explain why they are good, and contrast them with poorly written stories that merely follow a template.
AI content and knowledge management is about keeping this information accurate, relevant, consistent, and current. It involves maintaining knowledge repositories, managing versions, resolving conflicting instructions, updating business rules, and removing outdated or misleading guidance. It also means establishing which sources are authoritative and ensuring that new organizational decisions do not get confused with universal agile principles.
An especially important element is defining examples of goodness and exception boundaries. AI needs to understand not only what good looks like, but also when a commonly recommended practice should not be applied. For example, INVEST is useful for evaluating user stories, but not every product backlog item (PBI) must be written as a user story. A Definition of Ready (DoR) may help some teams, but it is not mandatory in Scrum. Velocity can support a team’s planning, but it should not become a performance comparison between teams.
These distinctions may seem obvious to experienced practitioners. Unfortunately, they are not always obvious to the people configuring enterprise AI solutions.
One company I recently helped went even further. It had developed an AI “super-agent” responsible for coordinating and instructing other specialized AI agents. The problem was that the supervising agent itself had been supplied with questionable agile knowledge. The human quality assurance process was limited, and some of the people evaluating the results lacked sufficient subject-matter expertise.
It was a classic example of the blind leading the blind, except now the blind were operating at machine speed.
My concern is simple: when organizations automate misunderstood or diluted agile practices, AI does not necessarily correct those misconceptions. It can amplify and institutionalize them.
To illustrate what I mean, here are ten classic examples from agile product management and tooling where better prompt engineering, professional knowledge, and effective context management can make a substantial difference.
1. User Story Writing: Good Grammar Doesn’t Mean Good Requirements
Many AI tools can generate user stories in seconds. Unfortunately, they often produce nicely formatted statements that are vague, oversized, or disconnected from meaningful customer needs.
As a practical example, from my personal observations, imagine that an AI assistant generates the following Jira story: “As a user, I want a comprehensive dashboard so that I can efficiently manage my activities.”
It follows the familiar user story template, but what does it actually mean? Who is the user? Which activities? What problem are we solving? What value will the dashboard provide?
A stronger story might be: “As a customer service representative, I want to see my unresolved customer requests so that I can identify which customers need immediate attention.”
This is more specific, easier to discuss, and potentially small enough to deliver incrementally.
What’s missing from the AI configuration?
The AI needs authentic knowledge of user stories, including the Three Cs (Card, Conversation, Confirmation), INVEST principles, customer value, and the importance of shared understanding. A useful prompt instruction would be: “Do not evaluate a user story solely by its grammatical structure. Assess whether it identifies a meaningful user need, provides clear business value, and is suitable for incremental delivery. Flag vague language and ask clarifying questions instead of inventing missing requirements.“
The supporting knowledge should contain examples of strong and weak user stories, explanations of INVEST characteristics, and exceptions where a traditional user story format is unnecessary. The stronger AI capability would act as a knowledgeable refinement assistant, not merely a story-writing machine.
2. Story Decomposition: Breaking Work Into Technical Tasks Instead of Customer Value
AI-generated decomposition frequently reflects traditional project management thinking rather than Agile product development.
As a practical example, from my personal observations, imagine that we have an epic in Azure DevOps: “Enable customers to manage their online banking accounts.”
A typical AI decomposition might produce:
- Design database tables
- Develop backend services
- Create user interface
- Implement business logic
- Perform integration testing
These are technical activities, not independently valuable customer capabilities. A better decomposition would identify functional slices such as viewing a checking account balance, reviewing recent transactions, or viewing a mortgage payment schedule. Each slice should include whatever technical work is necessary to deliver a small, testable, customer-facing capability.
What’s missing from the AI configuration?
The AI needs to understand vertical slicing, incremental delivery, functional decomposition, and the difference between user stories and technical tasks. A useful prompt instruction would be: “When decomposing an epic, prioritize small, customer-valued, end-to-end capabilities. Do not automatically split work by technical layers, components, or development disciplines. Identify dependencies and explain whether each proposed slice can deliver meaningful value independently.”
The supporting knowledge should include examples of vertical versus horizontal slicing, functional story maps, CRUD-based decomposition, and appropriate exceptions. A stronger AI assistant would help reduce dependencies and accelerate value delivery instead of reproducing traditional functional silos inside Jira.
3. Acceptance Criteria: More Text Doesn’t Mean Better Quality
AI is exceptionally good at generating acceptance criteria. Unfortunately, it is equally capable of generating vague, redundant, or unnecessarily prescriptive acceptance criteria.
As a practical example, from my personal observations, consider this criterion: “The application should provide a fast, intuitive, and secure checkout experience.”
It sounds professional but is not sufficiently testable. For a shopping-cart story, a better criterion could be: “Given a customer has three items in the shopping cart, when one item is removed, then the cart displays the remaining two items and recalculates the total correctly.” This describes observable behavior. However, AI should not invent additional business rules, such as refund policies, discount calculations, or payment restrictions, without supporting information.
What’s missing from the AI configuration?
The AI needs to understand acceptance criteria as verifiable conditions of satisfaction, the difference between requirements and implementation details, and the distinction between acceptance criteria and Definition of Done. A useful prompt instruction would be: “Generate clear, testable acceptance criteria based only on confirmed business requirements. Include relevant positive, negative, and boundary scenarios. Do not invent business rules or prescribe technical implementation unless explicitly required. Identify missing information as questions for clarification.“
The supporting knowledge should contain good and bad acceptance criteria, Given/When/Then examples, and guidance on distinguishing functional requirements from technical implementation decisions. A stronger AI assistant would improve requirements quality and testability without silently making product decisions on behalf of the Product Owner.
4. Backlog Prioritization: Automating Politics Instead of Business Value
Many organizations want AI to automatically prioritize their Product Backlogs. The problem is that AI may simply reproduce the organization’s existing prioritization habits, including those driven by hierarchy, urgency, or political influence.
As a practical example, from my personal observations, imagine a Jira backlog that contains three items:
- A senior executive requests a new dashboard.
- Customer research identifies a recurring checkout problem causing abandoned purchases.
- A technical improvement could reduce recurring production incidents.
A poorly configured AI may rank the executive request first because it has a high-priority label and an aggressive deadline. But does that make it the most valuable item? Not necessarily.
What’s missing from the AI configuration?
The AI needs knowledge of product strategy, customer value, cost of delay, risk reduction, learning opportunities, and prioritization techniques such as MoSCoW. A useful prompt instruction would be: “Evaluate backlog priorities using customer value, product goals, available evidence, risk, and dependencies. Do not assume that executive requests, deadlines, or existing priority labels automatically indicate greater business value. Explain the trade-offs and identify missing evidence.” The supporting content should include the current product strategy, Product Goal, customer research, business objectives, and agreed prioritization criteria. A stronger AI assistant would support Product Owner decision-making rather than automatically converting organizational politics into backlog priorities.
5. Sprint Metrics: Turning Empirical Evidence Into Performance Ratings
This is one of the most troubling examples I’ve encountered, and more than once. Organizations increasingly use AI to assess sprint performance based on Jira or Azure DevOps data. Unfortunately, many AI-generated assessments reflect a poor understanding of Scrum and empirical process control.
As a practical example, from my personal observations, imagine a team plans 40 story points and completes 30. The AI reports: “The team achieved 75% of its Sprint commitment and underperformed by 25%. Better estimation and stronger accountability are recommended.” This is a questionable assessment. What if the Sprint Goal was achieved? What if the remaining work was blocked by an external dependency? What if an unexpected production incident required the team to redirect its efforts? And since when did story points become a standardized measure of team performance?
A more useful assessment might state: “The Sprint Goal was achieved. Two selected backlog items remained unfinished because an external integration dependency was not resolved. Similar dependencies affected the previous two Sprints, suggesting a recurring organizational impediment.”
What’s missing from the AI configuration?
The AI needs a sound understanding of Scrum empiricism, Sprint Goals, Definition of Done, velocity limitations, and the difference between team-level delivery evidence and individual performance assessment. A useful prompt instruction would be: “Do not use velocity, story-point completion, or Sprint Backlog completion percentages as direct measures of team productivity. Evaluate Sprint Goal achievement, completed Increments, impediments, and relevant historical trends. Separate observations from interpretations and avoid unsupported performance judgments.”
The supporting content should include authentic Scrum knowledge, team-specific metric definitions, Sprint Goals, historical impediments, and examples of good and misleading Sprint assessments. A stronger AI assessment would help teams inspect and adapt rather than turn Agile metrics into another management stick.
6. Definition of Done: Confusing Completed Tickets With a Done Product
Many AI tools assume that when a Jira issue reaches Done, the corresponding work is genuinely complete. That assumption can be dangerously misleading.
As is a practical example, from my personal observations consider a team that completes all its planned development stories. Jira reports 100% completion. The AI congratulates the team on excellent delivery performance. However, integration testing, security validation, and deployment remain the responsibility of other departments. The product cannot yet be released. The team may have completed its local activities, but the organization still has substantial Undone work.
What’s missing from the AI configuration?
The AI needs to understand Definition of Done, potentially releasable Increments, cross-functional teams, organizational dependencies, and the difference between local task completion and integrated product delivery. A useful prompt instruction would be: “Do not equate workflow status with product completion. Evaluate whether completed work satisfies the applicable Definition of Done and whether an integrated, usable Increment exists. Identify remaining Undone work and distinguish team-level limitations from organizational constraints.“
The supporting knowledge should include the actual Definition of Done, release workflow, quality standards, integration responsibilities, and known organizational impediments. A stronger AI assistant would expose hidden delivery problems instead of celebrating green dashboards that conceal unfinished work.
7. Definition of Ready: Turning an Optional Practice Into a Mandatory Gate
Some AI-enabled Agile tools automatically evaluate whether backlog items are ready for development. This can be useful, but it can also recreate traditional phase-gate management.
As a practical example, from my personal observations, imagine an AI assistant that refuses to mark a story as sprint-ready because it lacks detailed technical design, complete documentation, and several stakeholder approvals. The team understands the customer problem and is capable of delivering a small increment, but the tooling blocks progress. The organization has effectively automated bureaucracy.
What’s missing from the AI configuration?
The AI needs to understand that Definition of Ready is optional in Scrum and should not automatically become a rigid approval gate. It should distinguish between information essential to begin useful work and details that can emerge through collaboration and development. A useful prompt instruction would be: “Assess whether a backlog item is sufficiently understood to begin meaningful work. Identify genuine blockers and important uncertainties, but do not require exhaustive documentation or impose a universal Definition of Ready. Treat readiness as a discussion aid unless the organization has explicitly established additional requirements.” The supporting knowledge should include the differences between Definition of Ready and Definition of Done, examples of appropriate readiness considerations, and warnings about excessive upfront analysis. A stronger AI assistant would support just-in-time refinement rather than enforce unnecessary administrative controls.
8. Forecasting: Producing Precise Dates From Unreliable Assumptions
AI-powered forecasting is becoming increasingly popular. Unfortunately, sophisticated-looking forecasts can create an illusion of certainty.
As a practical example, from my personal observations, consider an AI assistant that analyzes Jira data and announces: “The release will be completed on March 17.” But the backlog contains several large, poorly understood items, historical throughput varies significantly, and two critical dependencies remain unresolved. The date may look authoritative, but the underlying forecast is weak.
What’s missing from the AI configuration?
The AI needs knowledge of empirical forecasting, throughput, cycle time, variability, uncertainty, and the limitations of story-point-based predictions. A useful prompt instruction would be: “Use relevant historical delivery evidence to generate forecasts. Explicitly identify assumptions, uncertainty, and data limitations. Prefer probability-based ranges when sufficient data is available. Do not present a single completion date as a guaranteed commitment.” The supporting content should include historical delivery data, current backlog information, known dependencies, and examples of responsible probabilistic forecasting. A stronger AI forecasting capability would improve business decisions by communicating uncertainty rather than hiding it behind artificial precision.
9. AI Scrum Masters and Agile Coaches: Giving Advice Without Understanding the System
This is another area where I see considerable risk. Some organizations are experimenting with AI Scrum Masters and automated Agile coaching assistants. The problem is that these systems often generate generic advice without understanding the organizational environment.
As a practical example, from my personal observations, a team repeatedly carries unfinished work between Sprints. The AI Scrum Master recommends: “The team should improve estimation accuracy, increase accountability, and commit to fewer story points.” But suppose the real problem is that the team depends on three external departments for testing, integration, and deployment. Better estimation will not eliminate those dependencies. The recommendation addresses symptoms rather than causes.
What’s missing from the AI configuration?
The AI needs genuine Agile coaching knowledge, systems thinking, organizational design principles, cross-functionality, self-management, and the ability to distinguish team impediments from management-created constraints. A useful prompt instruction would be: “Before recommending corrective actions, examine potential systemic and organizational causes. Do not automatically attribute delivery problems to team behavior, estimation, or individual accountability. Ask diagnostic questions and identify impediments that require management or organizational intervention.” The supporting content should include examples of organizational impediments, feature-team versus component-team structures, coaching questions, and cases where the appropriate response is to escalate rather than prescribe. A stronger AI coach would help people understand and improve their system of work instead of repeating generic Agile advice.
10. Product Definition and Organizational Design: Making the Wrong Operating Model More Efficient
Perhaps the most serious mistake occurs when AI accepts an organization’s existing product and team structures without questioning them. Many companies have historically organized technology work around components, departments, and specialized functions. During Agile transformations, these structures are sometimes simply renamed as products. AI then begins optimizing the resulting backlogs, roadmaps, dependencies, and metrics.
As a practical example, from my personal observations, consider an organization defines separate products for: Database Services, Middleware, Front-End Development, and Integration Services. Each has its own backlog, management structure, and AI-generated reporting. Meanwhile, a single customer-facing feature requires coordinated work across all four groups. The AI may produce excellent reports for each individual group while completely missing the underlying problem: the organization is structured around technical components rather than end-to-end customer value.
What’s missing from the AI configuration?
The AI needs knowledge of customer-centric product definition, value streams, cross-functional feature teams, organizational design, and the relationship between product boundaries and delivery dependencies. A useful prompt instruction would be: “Do not automatically assume that every organizational unit, technical component, or platform is a customer-facing product. Evaluate product boundaries based on identifiable customers, user needs, value propositions, and end-to-end delivery capabilities. Identify organizational structures that create unnecessary dependencies.” The supporting content should include product vision and strategy, customer journeys, product definition criteria, value-stream maps, and examples of component-oriented versus customer-oriented organizational structures. A stronger AI assistant would help organizations challenge structural complexity instead of merely automating its administration.
The Bigger Concern: AI Training Other AI
The examples above become even more important when organizations introduce multi-agent AI systems. Imagine a supervising AI agent responsible for coordinating several specialized agents: one for backlog management, another for sprint assessments, another for forecasting, and another for agile coaching. If the supervising agent has been given incorrect Agile knowledge, it may distribute those misconceptions to every subordinate agent.
For example, if it believes velocity is a valid measure of team productivity, the Sprint Assessment Agent may use it to judge performance, the Forecasting Agent may use it to compare team capacity, and the Agile Coaching Agent may recommend increasing velocity. Now the same misconception is being reinforced across multiple AI capabilities. This is why I believe organizations need much stronger human quality control over the professional knowledge used to configure and evaluate AI. The fact that several AI agents agree with one another does not mean they are correct. They may simply be reproducing the same flawed assumptions. And if the humans responsible for quality assurance lack sufficient subject-matter expertise, the organization may have no reliable mechanism for detecting the problem. In my view, this is one of the most significant risks of scaling enterprise AI too quickly.
Conclusion: The Quality of the AI Brain Depends on the Quality of the Human Brain Behind It
My recent work with AI startups and enterprise AI implementations has reinforced something I’ve believed for a long time:
AI can be a powerful amplifier of human expertise. But it can also be an equally powerful amplifier of human misconceptions.
Organizations are understandably excited about introducing AI into product management, Agile tooling, and software delivery. The potential benefits are substantial: less administrative overhead, better requirements, more meaningful analysis, improved forecasting, and faster access to professional knowledge. However, simply unpacking an AI capability, connecting it to Jira or Azure DevOps, and providing a few generic prompts is not enough. Effective AI enablement requires deliberate investment in the quality of the AI’s knowledge and instructions. It requires well-engineered prompts, carefully selected context, authoritative professional content, examples of goodness, clearly defined exceptions, and meaningful evaluation criteria. It also requires ongoing content management, knowledge maintenance, and qualified human oversight. Most importantly, it requires genuine subject-matter expertise.A person who misunderstands Scrum is unlikely to create a reliable AI Scrum Master. Someone who treats velocity as a productivity metric is unlikely to configure an AI system that interprets Sprint metrics correctly. Someone who cannot distinguish vertical story slicing from technical task decomposition is unlikely to build an effective AI backlog refinement assistant. And an organization that does not understand customer-centric product management is unlikely to create an AI solution that meaningfully improves its product operating model. The same principle applies to AI agents that supervise or instruct other AI agents. The sophistication of the technology does not compensate for weak professional knowledge at the source. I sometimes think of this as the relationship between the human brain and the AI brain. An AI model certainly has capabilities and learned knowledge of its own. But when we configure it for specialized professional work, the quality of the human expertise we provide—and the quality of the human judgment used to evaluate its results—become critical.
If the human brain supplying the knowledge is confused, the AI brain may become a highly efficient mechanism for reproducing that confusion.
On the other hand, when AI is guided by authentic expertise, meaningful context, strong examples, and sound evaluation criteria, it can become an extraordinary professional enabler. This is why I believe organizations, especially PMOs, Product Organizations, and Agile Transformation Offices, should be cautious about scaling AI before establishing a reliable foundation of professional knowledge and quality assurance. Before investing heavily in AI automation, invest in the quality of the knowledge being automated. Before scaling AI agents, make sure the people guiding and evaluating them genuinely understand the domain. And before celebrating how much work AI can perform, ask whether it is helping the organization do the right work, in the right way, for the right reasons.
The real competitive advantage will not come from having more AI agents. It will come from having better-informed AI agents, guided by better-informed humans.