When Artificial Intelligence Gets It Wrong

By John R. Kennedy · October 28, 2026

Why the quality of our decisions matters more than the intelligence of our technology

Have you ever asked artificial intelligence a question, received a convincing answer, and later discovered it was wrong?

I have. And what's particularly interesting is how confidently AI can present information that turns out to be inaccurate. The answer might be well-written, logically organized, and supported by what appear to be credible explanations. There's often nothing about the presentation that suggests we should question it.

We've become accustomed to associating confidence with competence. When someone speaks authoritatively about a subject, we're naturally inclined to believe they know what they're talking about. Artificial intelligence complicates that relationship because a system can communicate with extraordinary confidence without possessing the information necessary to justify its conclusions.

But I think there's a larger issue worth considering. What happens when we become so comfortable with these systems that we stop questioning their answers? And what happens when we begin allowing them to act on those answers?

Intelligence and Accuracy Are Not the Same Thing

Imagine asking an experienced financial advisor to evaluate a company's performance. You provide financial statements, customer information, and historical sales data. The advisor analyzes everything and produces a detailed assessment. The calculations are correct, the analysis is thorough, and the recommendations seem perfectly reasonable.

There's just one problem. Some of the information you provided was wrong.

Perhaps expenses were incorrectly categorized, an important liability was missing, or the customer records included accounts that hadn't been active for years. The advisor may have performed an excellent analysis of fundamentally unreliable information.

Artificial intelligence faces a similar challenge, although its ability to process enormous amounts of information makes the problem potentially much larger.

We tend to focus on the intelligence of AI systems because that's what makes them impressive. We marvel at their ability to recognize patterns, solve problems, and produce sophisticated responses. But intelligence and accuracy aren't interchangeable. An AI system can reach a logically consistent conclusion based on information that is incomplete, outdated, or simply incorrect.

Consider a manufacturing company that has been operating for several decades. Over the years, it has accumulated enormous amounts of information about customers, suppliers, inventory, production schedules, and equipment maintenance. Unfortunately, much of that information is scattered across different systems.

The accounting department maintains one collection of records. Production uses another. Purchasing has its own database, and the sales department tracks customer activity in a separate application. These systems were implemented at different times, by different people, for different purposes.

Like many established businesses, the company has learned to operate around their limitations. Employees know which reports are reliable, which figures require adjustment, and which information needs to be confirmed before making an important decision. Some of that knowledge exists only in the minds of experienced employees who have spent years learning how the business actually operates.

Now imagine the company introduces an AI system designed to improve purchasing and production.

The technology is impressive. It analyzes historical orders, identifies patterns in customer demand, evaluates supplier performance, and recommends changes to inventory levels. Management is excited about the potential savings and the opportunity to eliminate repetitive work.

But the AI system doesn't necessarily understand all the informal knowledge employees have developed over the years. It may not know that a particular supplier's delivery dates are frequently inaccurate, that certain inventory records haven't been reconciled, or that an unusual sales pattern represents a temporary event rather than a permanent change in demand.

The system produces recommendations that appear perfectly reasonable based on the information available to it. Management sees the sophistication of the analysis and begins to trust the results.

And that's where things can become interesting.

For years, experienced employees compensated for weaknesses in the company's information systems. They understood the exceptions and knew when something didn't look right. Their knowledge was an invisible part of the company's decision-making process.

Now the company risks automating those weaknesses.

The problem isn't necessarily the intelligence of the AI system. It's the quality and context of the information supporting its decisions. The technology may be performing exactly as designed while producing recommendations that don't reflect the reality of the business.

When Answers Become Actions

There's another dimension to this story that I find particularly important.

Suppose the AI system recommends reducing purchases from a particular supplier. A purchasing manager reviews the recommendation, recognizes that the supplier's delivery records are inaccurate, and decides not to follow it.

The AI made a poor recommendation, but the company's decision-making process worked. The employee contributed knowledge that wasn't available to the system, and the business avoided a potentially costly mistake.

Now imagine the company decides to take automation a step further. Instead of requiring employees to review purchasing recommendations, management authorizes the AI system to place orders automatically.

The same incorrect information could now cause the system to reduce purchases, delay production, or create shortages. By the time someone recognizes the problem, the consequences may already be affecting customers and suppliers.

Nothing about the underlying information changed. The AI didn't necessarily become less accurate or less capable. What changed was the authority the company gave it.

This distinction matters because we're beginning to move from AI systems that primarily provide information to systems that can increasingly act on that information. An AI assistant might coordinate with suppliers, negotiate purchases, schedule employees, or authorize transactions. Each additional responsibility creates opportunities for efficiency, but it also increases the consequences of mistakes.

An incorrect answer can be inconvenient. An incorrect action can be expensive, difficult to reverse, or harmful to people who never interacted with the AI system in the first place.

And the problem isn't limited to artificial intelligence producing information that isn't true. Sometimes the information itself is wrong, even when the AI interprets it correctly. Other times, the information is accurate but incomplete, or the system lacks the context necessary to understand what it means.

These distinctions are important because improving the intelligence of AI doesn't automatically resolve weaknesses in the information it receives.

The Value of What We Know

I keep coming back to the experienced employees in our manufacturing example.

They understood things that weren't documented anywhere. They knew which suppliers could be trusted, which customers required special handling, and which reports didn't tell the complete story. Over time, they had developed an understanding of the business that couldn't be captured simply by connecting an AI system to its databases.

That knowledge had value. In fact, it may have been among the company's most important assets.

Yet because much of it existed only in people's minds, it was largely invisible to the technology the company was introducing.

Imagine if management had approached the project differently. Instead of beginning with the question of what AI could automate, they might have started by asking employees how the business actually worked.

They could have identified the information employees routinely corrected, the exceptions they recognized, and the judgments they made that weren't reflected in the company's systems. They might have discovered longstanding problems with inventory records, inconsistent supplier information, or business processes that had never been properly documented.

Correcting those problems would have improved the company's operations even without artificial intelligence.

And once that work was completed, the company would have been in a much stronger position to introduce AI.

I think there's an important lesson here. We often discuss artificial intelligence as though the technology itself is the source of all the value. But AI depends on things that already have value, including the knowledge, experience, judgment, and information accumulated by people.

The better we understand and preserve those assets, the more effectively we can use artificial intelligence.

Good information is an asset before AI ever touches it.

This also raises a question about what it means to trust artificial intelligence. We tend to ask whether the AI system itself can be trusted, but perhaps that's only part of the question. We should also consider whether we can trust the information it receives, the authority it has been granted, and the processes governing its decisions.

An AI system might accurately interpret an outdated document, correctly calculate a projection using incorrect assumptions, or faithfully follow instructions that unintentionally authorize something the user never intended.

In each case, the technology may be functioning as designed. The failure exists somewhere else.

As these systems become more capable, I suspect we'll discover that many of the challenges we attribute to artificial intelligence are actually problems we've carried with us for years. Inaccurate information, poorly documented processes, unclear responsibilities, and assumptions that were never questioned didn't originate with AI.

But AI can expose those weaknesses, amplify their consequences, and act on them at a speed and scale we haven't experienced before.

Getting the Order Right

All of this has led me to a principle that I believe should guide how we approach artificial intelligence.

People first. Data quality second. AI third.

I don't mean that as a ranking of technological importance. I mean it as an order of responsibility.

We should begin with people because they're the reason we're developing these technologies in the first place. Our objective should be to improve their lives, expand their capabilities, and help them make better decisions. Technology should support human judgment and accountability rather than diminish them.

We should then consider the information upon which those decisions depend. Where did it come from? Is it accurate? Is it current? Does it reflect the knowledge and experience of the people who understand it best? And are we using it in ways that respect the interests of the people it represents?

Only then should we determine how artificial intelligence can help us accomplish our objectives.

This doesn't mean we need perfect information before using AI, or that people must personally approve every action a system takes. Neither would be practical. It means the authority we give these systems should reflect the reliability of the information they're using and the consequences of the decisions they're making.

I'm optimistic about what artificial intelligence will make possible. I believe it can help us solve problems, expand access to knowledge, and accomplish things that would have seemed extraordinary only a few years ago.

But I'm also convinced that intelligence alone isn't enough.

We wouldn't knowingly build a skyscraper on an unstable foundation simply because we'd developed an extraordinary new construction technique. We shouldn't approach artificial intelligence any differently.

The technology will continue to improve. Its capabilities will expand, and the opportunities it creates may be enormous. But the foundation deserves at least as much attention as the technology we're building upon it.

The goal isn't to build a world where people serve increasingly intelligent machines. It's to build a world where increasingly intelligent machines serve people.

And the quality of the information connecting the two will help determine whether we succeed.

Continue reading: The AI Revolution Is Bigger Than You Think