Chapter 12

Independent Verification

The problem began with a number.

Daniel was reviewing a proposal for a small distribution project when he noticed that the projected monthly revenue had been revised upward. The new figure looked reasonable. It was supported by a table, explained in a paragraph, and presented alongside a forecast showing how the business could grow over the following six months.

He copied the figure into his AI council and asked the Investigator to examine the assumptions behind it.

The response was encouraging. The projection appeared consistent with the stated sales targets, provided the business could reach the expected number of customers.

Daniel passed the findings to the Analyst.

The Analyst examined the pricing, the projected number of transactions, and the relationship between sales volume and revenue. It found no obvious arithmetic error in the model as presented.

The Challenger identified several risks: slower customer acquisition, lower repeat usage, unexpected operating costs, and the possibility that the sales target would not be reached.

The Verifier was asked to examine the key assumptions.

Its response noted that the forecast depended on customer demand that had not yet been demonstrated at the proposed scale.

Finally, the Synthesiser assembled the results.

The report was clear. The calculation was internally consistent. The major risks were identified. The forecast remained conditional on demand.

Daniel was satisfied with the process.

Then he noticed something.

Every role had worked from the same original proposal.

The council had examined the assumptions, challenged them, and identified a weakness. But it had not independently established whether the projected demand was realistic.

The number had travelled through five stages of analysis without acquiring new evidence about the world.

Daniel stared at the report.

He had designed a council that could think about a claim from several directions. He had not yet ensured that the council could test the claim against something outside the claim itself.

He opened a new conversation and asked an AI assistant:

What independent evidence would help establish whether this projected demand is realistic?

The answer suggested examining comparable businesses, customer purchasing behaviour, existing sales records, and evidence of actual demand at the intended price.

Daniel began looking.

He found information about the broader market, but much of it described the size of the industry rather than the number of customers likely to buy this particular service. He found businesses offering similar products, but their prices and operating conditions were not identical. He found statements about customer interest, but few records of completed purchases.

The search did not prove the forecast was wrong.

It revealed that the evidence required to support the forecast was missing.

Daniel returned to the report and changed its status.

The projection is mathematically consistent with its assumptions. The assumptions about demand remain unverified.

That distinction was more valuable than another polished explanation.

The number had not become false. It had simply stopped pretending to be established.

Agreement Is Not Independence

Later that week, Daniel met Samuel at a quiet café.

Samuel had been examining a financial opportunity and wanted Daniel to look at a report produced by three AI assistants.

“I asked three systems to assess the opportunity,” Samuel explained. “All three identified similar conditions that could support growth. They disagreed on timing, but they broadly agreed on the direction.”

Daniel read the report.

The analysis was well organised. It discussed recent price movements, demand, market sentiment, and several economic factors. Each assistant had expressed its conclusion differently, but the main argument was similar.

“How did you select the systems?” Daniel asked.

“I used three different assistants.”

“Did they use different evidence?”

Samuel frowned.

“I didn't check.”

“Did they independently retrieve the information, or were they all responding to the same summary?”

“I gave them the same summary.”

Daniel put the report on the table.

“Then you've tested whether three systems can interpret the same information in a similar way. You haven't established that the information itself is correct.”

Samuel leaned back.

“But if they reach the same conclusion independently, doesn't that count for something?”

“It can count as a useful comparison of reasoning. But we need to be precise about what was independent.”

Daniel explained that separate systems could share training data, public sources, common assumptions, or similar patterns of reasoning. Even when their outputs were generated separately, their conclusions could depend on the same underlying evidence.

If one inaccurate report had influenced the original summary, three assistants could reproduce its central error without ever communicating with one another.

They might disagree about the details and still inherit the same mistake.

Samuel looked at the report again.

“So I shouldn't count the number of assistants that agree?”

“Not without examining what their agreement represents.”

Daniel drew a line down the middle of a sheet of paper.

On one side, he wrote Different Interpretations.

On the other, he wrote Independent Evidence.

The distinction was simple but important.

Different interpretations could reveal that a question had several possible answers. One assistant might identify a risk that another overlooked. A second might expose a hidden assumption. A third might suggest a more useful way to frame the decision.

Independent evidence did different work. It could establish whether a factual claim was supported by a source, a record, a direct observation, a measurement, or another appropriate form of evidence that did not merely repeat the original assertion.

Both were valuable.

Neither should be mistaken for the other.

Samuel decided to revisit the opportunity. He would identify the claims most important to his decision, locate the underlying sources, and determine whether those sources actually supported the conclusions being drawn from them.

He would also examine what could not be known in advance.

The exercise would not guarantee that his eventual decision would succeed. It would give him a better understanding of the evidence on which he was relying.

For Samuel, that was a different kind of confidence: not confidence that the market would obey a prediction, but confidence that he knew why he was making a decision and where his knowledge ended.

The Source Behind the Source

Daniel's next lesson came from a claim that appeared in several places.

A report stated that a particular market segment had grown rapidly over the previous year. Two industry articles repeated the figure. A presentation used it to support a forecast. An AI assistant cited the articles as evidence that demand was increasing.

At first, the claim appeared well supported.

Daniel decided to trace it.

He opened the articles and followed their references. Both pointed to the same industry report. The presentation cited one of the articles, while the AI assistant had used the articles and presentation as supporting material.

Four documents appeared to confirm the claim.

But the apparent agreement rested largely on one underlying source.

Daniel located the original report.

Its figure was genuine, but its scope was narrower than the later documents suggested. It described a particular segment in a particular region. The presentation had applied the figure to a broader market without clearly explaining the change.

The claim had not necessarily been invented. Its meaning had expanded as it travelled.

Daniel wrote the sequence in his notebook:

Original source. Secondary report. Industry article. Presentation. AI summary.

Each stage could preserve the original meaning, narrow it, exaggerate it, or remove a qualification. By the time the claim reached the final answer, the source might still be named while the relationship between the source and the conclusion had weakened.

Daniel returned to the AI assistant.

He asked it to compare the exact wording of the original report with the claim made in the presentation.

The assistant identified the difference once the relevant passages were supplied.

Daniel noticed the significance of that detail. The system had not independently discovered the problem during its earlier analysis. It had been able to examine the mismatch when the task directed attention to it and the necessary evidence was available.

That was useful, but it was not the same as having verified the claim from the beginning.

Daniel added a new requirement to his council.

For every important factual claim, the Verifier should ask:

· What exactly is being claimed?

· What is the original or most authoritative available source?

· Does the source support the precise wording of the claim?

· Is the source relevant to the location, period, population, or situation being discussed?

· Are other sources genuinely independent, or do they repeat the same underlying information?

· What limitations or qualifications have been lost along the way?

The list was not a substitute for judgment. Different claims required different evidence.

A mathematical calculation might be checked by reproducing the calculation from the original inputs. A current regulation might require an authoritative government or legal source. A claim about whether customers will pay might require actual purchasing behaviour rather than a collection of articles about market trends.

The method had to fit the claim.

Daniel was beginning to understand that verification was not a search for a source that agreed with the answer.

It was an examination of the relationship between the claim, the evidence, and the conclusion.

The Evidence Marcus Didn't Have

Marcus called Daniel one afternoon with an update.

“I've found a supplier who says they can reduce my delivery costs,” he said. “They've sent a price list and examples of work they've done for other businesses. If the arrangement works, I can improve the margins on the service.”

“That sounds worth examining,” Daniel replied.

“I'm thinking of moving most of the work to them.”

“Have you tested their service?”

“They have references.”

“Have you spoken to the references independently?”

“Not yet.”

“Have you compared the full cost of their service with what you're paying now?”

“I've compared the prices.”

Daniel waited.

Marcus understood.

“Not the full cost.”

They arranged to review the proposal together.

The supplier's price list looked attractive. Its examples were professionally presented, and the references appeared credible. The supplier claimed that its process was faster and more reliable than the alternatives.

Daniel did not assume the claims were false.

Instead, he helped Marcus separate them.

The price list established the quoted prices, subject to the supplier's terms. It did not establish the total cost of using the service.

The examples showed work the supplier presented as completed. They did not, by themselves, establish the supplier's consistency or performance under Marcus's operating conditions.

The references were potential sources of useful information. Their value depended partly on whether they were genuine, whether the people had direct experience, and whether their situations were comparable to Marcus's.

The claim about reliability required evidence of reliability.

The claim about speed required a meaningful comparison of delivery times.

The claim about lower cost required the full set of costs that Marcus would actually incur.

They were no longer asking whether the supplier looked professional. They were asking which claims mattered to the decision and what evidence could support each one.

Marcus contacted two references using independently obtained contact details. Both confirmed that they had worked with the supplier. One described a positive experience. The other said the work had been satisfactory overall but that delivery times sometimes varied.

That information was more useful than treating the references as a simple vote for or against the supplier.

Marcus then requested a small trial under agreed conditions. He would compare the result with his existing process, recording total cost, delivery time, quality, and the amount of additional supervision required.

The trial did not require him to transfer the entire operation before he had evidence of performance.

When the first results arrived, the supplier did reduce one part of the cost. But the additional coordination required from Marcus consumed some of the expected savings.

The arrangement was not a failure. Nor had the original price list established that it would be profitable.

Marcus decided to continue with a narrower set of tasks while investigating whether the coordination burden could be reduced.

“Before, I would have asked the AI whether this supplier was a good choice,” he told Daniel. “Now I know that isn't one question. It's several claims that need different kinds of evidence.”

Daniel nodded.

“And some of the evidence only appears when you test the arrangement.”

Marcus smiled.

“Which means I don't have to guess everything before I start. I just need to avoid committing more than the evidence justifies.”

That was the point.

Independent verification did not require every uncertainty to be resolved before action. Sometimes the most useful evidence came from a limited, reversible test.

The challenge was to choose a test that could reveal something meaningful without exposing the business to unnecessary harm.

When the Evidence Cannot Settle the Question

Daniel eventually encountered a different kind of problem.

He was helping someone compare two ways of organising a project. Both approaches had credible advantages. One offered greater flexibility but required more coordination. The other created clearer procedures but made changes slower.

The AI council examined the alternatives and identified the relevant trade-offs.

Daniel checked the factual claims. The time estimates were based on past work. The costs were calculated from available figures. The main dependencies had been identified.

Yet the council could not establish which approach was best.

The remaining disagreement concerned priorities.

One approach suited a person who valued flexibility and could tolerate changing responsibilities. The other suited a person who preferred predictable procedures and wanted clearer accountability.

No additional source could decide which preference mattered more.

Daniel had to recognise the boundary between factual verification and human judgment.

Some questions were answerable through evidence. Others involved values, acceptable risk, competing objectives, or preferences that could not be reduced to a factual verdict.

An AI assistant could help reveal the consequences of different priorities. It could ask whether the chosen approach matched the person's stated goals. It could identify contradictions between what someone said they wanted and what their preferred plan would require.

But it could not establish a universal ranking of human values and apply that ranking as though it were a verified fact.

Daniel returned to his council design and added a distinction:

Verify factual claims. Examine interpretations. Make values explicit. Leave the decision with the human.

This prevented a common mistake.

A recommendation could contain accurate facts and still depend on a value judgment. If that value judgment remained hidden, the recommendation might appear more objective than it really was.

For example, a plan could maximise expected income while increasing stress and reducing available time. The calculation might be correct. Whether the trade-off was acceptable would remain a human decision.

The council's task was to make that trade-off visible.

It was not to disguise a preference as a mathematical conclusion.

How Much Verification Is Enough?

The question troubled Daniel more than he expected.

If every source could contain an error, every measurement could have limitations, and every conclusion could be challenged, where should verification stop?

He had seen what happened when he searched for certainty without defining its purpose. The investigation expanded. New questions emerged. The decision remained suspended while he tried to eliminate every possible doubt.

That was not a sustainable way to operate.

Daniel returned to the principle he had developed when designing the council: the consequences of error should determine the depth of review.

A low-stakes, reversible decision might require only a quick check. A significant business commitment could require independent sources, careful calculations, and a limited trial. A decision involving serious harm, legal obligations, or substantial financial exposure might require authoritative evidence and qualified professional review.

The standard was not the same for every situation.

Nor did the number of sources alone determine whether the evidence was sufficient.

Ten weak sources could be less useful than one authoritative record. Several independent observations could be more informative than a large collection of articles repeating the same claim. A direct experiment could answer a narrow operational question more effectively than a broad industry report.

Daniel began using a simple test before ending a verification exercise:

What remaining uncertainty could materially change this decision?

If an important uncertainty remained, he considered whether it could be reduced through further research, expert advice, or a controlled experiment.

If the uncertainty could not be eliminated, he considered whether the decision could be made smaller, delayed until a critical condition was met, or abandoned.

If the remaining uncertainty was minor relative to the stakes, continuing to investigate might add little value.

This did not turn verification into a mechanical calculation. It gave him a reason to stop that was connected to the actual decision.

He also learned to distinguish two different statements.

I have not found evidence that this claim is false.

And:

I have found sufficient evidence to support this claim for the decision I am making.

The first did not automatically establish the second.

A missing contradiction was not proof. An unsuccessful search was not confirmation. A plausible explanation was not necessarily an established fact.

The standard was not perfect certainty.

It was sufficient, appropriate evidence for a clearly defined purpose, with important limitations still visible.

The Council's Verification Record

Daniel eventually created a short record for consequential claims.

He called it the Verification Record.

It contained the claim itself, the source or sources examined, what the evidence actually established, whether the sources were independent, what remained uncertain, and how the uncertainty affected the decision.

He tested the record on the distribution proposal he had examined earlier.

The original claim had described an established network of retail relationships. The available documents showed a list of potential retailers and statements made by the partner. They did not establish confirmed orders or active distribution agreements.

Daniel did not mark the claim as false.

He marked it as unverified and recorded the evidence needed to strengthen it: confirmation from the relevant retailers, documented agreements where applicable, and evidence of actual distribution activity.

Later, the partner clarified that the retail relationships were prospective rather than confirmed.

The distinction mattered.

The original documents had not been enough to establish the claim, but the missing evidence did not prove that the partner was dishonest. The situation had become clearer because Daniel had preserved the difference between an unsupported claim and a disproven one.

He had also preserved room for new evidence to change the assessment.

A verification record should not become a permanent accusation. It should reflect what is currently supported, what is unresolved, and what would justify changing the conclusion.

That principle applied to AI outputs as well.

If an assistant made a claim and later produced evidence that supported it, the record could be updated. If a source proved unreliable, the claim could be downgraded. If a calculation was corrected, the conclusion could be reassessed.

The goal was not to defend an earlier answer.

It was to remain accountable to the evidence.

The Architect Checks the Foundation

Several weeks later, Daniel reviewed a decision he had made using the council.

The outcome had been acceptable, but he noticed that one assumption had gone unexamined. It had not caused a problem in this instance. Still, the assumption had influenced the decision, and he could not explain why he had accepted it.

He felt the familiar temptation to dismiss the issue because nothing had gone wrong.

Instead, he reopened the record.

He identified the assumption, examined the evidence that had been available at the time, and asked whether the assumption had been reasonable given the consequences. He then changed the council's instructions so that similar claims would be identified in future reviews.

He was not trying to create a system in which mistakes became impossible.

He was trying to create a system in which mistakes could be found, understood, and used to improve the next decision.

The difference mattered because an AI council could produce an impressive report without being correct. It could cite a genuine source that did not support the precise claim. It could perform a calculation accurately using poor inputs. It could identify several risks while overlooking the one that mattered most.

Independent verification reduced these dangers, but it did not remove them.

The human still had to examine the quality of the evidence and decide whether it was appropriate for the situation.

Daniel returned to the first principle of the council.

The machine could investigate.

It could analyse.

It could challenge.

It could help verify.

It could organise the findings.

But the labels did not guarantee that any of those tasks had been performed well. A council became trustworthy through the quality of its process, the suitability of its evidence, its willingness to expose uncertainty, and the human discipline required to act on what the process revealed.

The council was not an authority that Daniel could obey without thinking.

It was an arrangement that helped him think more carefully.

He closed the Verification Record and looked at the next section of his manuscript.

There was still one danger he had not fully addressed.

Even if the evidence was carefully examined, the final decision could be handed over to the system through a different route. Daniel could let an AI choose the objective, rank the values, select the acceptable risks, and recommend the action. He could then tell himself that he remained in control because he was the person who clicked the final button.

That would preserve the appearance of responsibility while surrendering much of the judgment that responsibility required.

Daniel opened the next chapter and typed its heading:

Keep the Human Upstream.

He now understood that independent verification helped establish what the evidence could support.

The next challenge was deciding who had the authority to determine what the evidence should be used for.