Choosing the Route

At 8.40 on Tuesday 1 September, Daniel had three windows open and no recommendation written. One showed the retrieval feature in the company’s existing knowledge-base software. Another held the proposal from the supplier who had demonstrated its product. The third was an internal estimate for a small build using a model service and the company’s own interface.

The sales director was waiting for a separate answer by Friday. His team had a tender to prepare. Leila had offered staff time for the retrieval comparison only if it stayed within the board’s four-day effort allowance. Daniel could see how easily a limited decision might become a procurement exercise larger than the problem.

Maya asked which route was best.

“For what we know today, or for everything we might want later?” he asked.

“For the next decision.”

The question was no longer whether the company should buy an AI assistant. It was whether advisers could locate and inspect the applicable clause more readily than with the repaired library’s ordinary search. The comparison could not contact customers, change case records or make eligibility decisions. It had to end on 11 September.

Daniel closed the supplier’s future-roadmap tab. It was interesting. It did not answer the question in front of them.

Choose the smallest route that answers the question

The existing feature already operated inside the knowledge-base environment. Its configuration could return source passages with document identity and effective-date fields. That did not make it approved for the proposed work. It did mean Daniel could examine the feature without first creating a new document store and a second administration process.

The demonstrated product had a clearer conversational interface and more options for future workflows. It would also introduce a new supplier relationship, a separate source collection and another place to maintain permissions. The small internal build offered control over the interface, but the team would own the integration, testing and support work that the estimate had only begun to describe.

Daniel’s first comparison sheet favoured the internal build because its initial licence estimate was low. Leila asked who would fix it during an incident. The proposed developer was already committed to a finance-system change. The estimate did not contain cover for leave or a maintained fallback. Daniel changed the entry from “low cost” to “initial cost estimated; support commitment unresolved”.

The discovery did not make internal development a bad route in general. It made the route incomplete for this decision. A prototype that one person could produce quickly was not yet a service the company had agreed to support. The missing work would have to be funded or the scope reduced before the route could fairly be compared.

They chose the existing feature as the first candidate for the bounded comparison. It added the fewest new arrangements while exposing the clause and its date. The demonstrated product remained an alternative if the existing feature failed the material requirements. The internal build was deferred, with its missing support commitment recorded rather than dismissed as technically inferior.

Choose the least new thing that can answer the actual question. “Least new” includes sources, permissions, skills, suppliers and obligations to maintain the service. It does not simply mean the lowest licence price or whatever the organisation already owns. An existing product with unsuitable data handling can be the wrong choice; a new product with a justified advantage can be the right one.

The current decision also had a deliberately short horizon. Selecting a route for comparison did not commit the company to a future agent platform. Daniel left the later possibilities in a separate note. They could return when a diagnosed problem required them.

Compare routes against the same requirements

Leila wrote the requirements in the language an adviser would use. Show the passage. Show which document it came from. Show when it applies. Let me open the full source. Tell me when the source set does not settle the question. Do not change a customer’s record while I am looking.

The team then added the operating requirements: approved access, a source owner, evidence of the configured data handling, a way to disable the feature and a manual search route that remained available. Each route was assessed against that same set. A long feature list could not compensate for a missing material requirement.

Their comparison did not use a weighted score to conceal a failed condition. Customer confidentiality and the absence of unauthorised actions were conditions of entry. Convenience and configuration effort mattered after those conditions could be met. If a route could not meet the intended boundary, it would be narrowed or rejected rather than awarded enough points elsewhere to average the defect away.

The existing feature was promising because it could expose the required fields and use the repaired library. The supplier’s product might be more capable at composing replies, but reply composition was outside the immediate comparison. The internal build might offer a better interface later, but nobody had established that the standard interface prevented an adviser from doing the present task.

Daniel kept one route table in his working paper. It recorded requirement, evidence, unresolved question and consequence for the choice. He did not colour every untested claim green because a provider had said the feature was possible. Configuration evidence and a feature description received different statuses.

A fair comparison also gives the ordinary process a chance to improve. Priya showed the team a saved search in the existing library that located a common clause quickly. That case stayed in the comparison. It was not removed because it reduced the apparent benefit of AI. If ordinary search already did a task well, that was information the company could use immediately.

For less familiar product terms, the new retrieval feature might have an advantage. That was the claim to test. The comparison would examine whether the adviser could reach an applicable source, inspect it and know when to stop. It would not reward a polished paragraph for work the adviser still had to undo.

Ask the provider about the actual configuration

At 10.30 on Wednesday 2 September, Daniel met commercial counsel with the existing provider’s order documents, configuration guide and written answers. He had highlighted the relevant feature, not the whole product family. Counsel asked which information the comparison would contain and whether the proposed settings were available under the company’s account.

The comparison would use the repaired procedure extracts and fictional questions. Customer records were excluded. That narrowed the immediate data question, but the procedure material still belonged to the company and needed appropriate handling. A later experiment with other information would require its own scope to be checked.

They traced where the feature processed a question, where the source material was indexed and what was retained in the configured service. They asked who could access it for support, which other organisations were involved in processing, what changes could occur without notice, and what records would be available if something went wrong. Each answer was tied to a document or a configuration check.

These were ordinary provider questions made specific to the use. There was no need to turn them into an encyclopaedic questionnaire about every possible AI risk. Nor was there a reason to accept “enterprise grade” as an answer to a question about retained content.

Daniel found an optional history setting enabled in the assessment account. The feature could keep user questions for convenience. The comparison did not need that history. He disabled it and recorded the setting, while counsel kept the provider’s separate operational-log terms in view. Switching off visible history did not, by itself, prove that no information existed anywhere else.

The team’s record distinguished content used for the feature, permitted operational metadata and information retained by the company for its own evaluation. Where a term remained unclear, they asked a narrower question. They did not silently convert “not used for training” into “not retained”, or “deleted from the screen” into “removed from every copy”.

For this fictional comparison, counsel accepted the documented handling of the restricted source set and Daniel verified the selected settings. That conclusion applied to this configuration and purpose. It was not an assurance about all products from the provider or a general rule that an existing contract covers every newly enabled feature.

They retained the evidence in the company’s controlled procurement record. The board paper contained the conclusion, the scope and the unresolved issues, with links to the records. It did not reproduce confidential contract terms in a widely circulated appendix.

Check the promise against the screen

At 1.15 that afternoon, Priya tried the configured feature with Daniel. It returned the correct current clause. Beside it was a date. At first glance the requirement appeared satisfied.

Priya opened the source. The date in the result was the file’s last modified date, not the procedure’s effective date. The underlying field was available, but the standard result layout had selected the wrong one. A pass against “shows a date” would have accepted a misleading display.

Daniel changed the mapping to the explicit effective-date field. He also labelled it in words. They reran the case and retained the before-and-after result. The new configuration showed what the team had actually asked to know. It still did not decide which period applied to every possible customer event.

A second test used an extract without an effective-date value. The feature initially presented it alongside complete records without making the omission prominent. Daniel changed the result layout to display “effective date not supplied” and prevented that source from appearing as an unqualified current instruction in the comparison collection.

Leila asked whether they could simply discard the incomplete file. For the comparison they could exclude it from the active source set, with the exclusion recorded. For the library itself, the owner still had to resolve the missing information. Hiding the defect from a search result was not the same as repairing the source.

The team kept the route but changed the configuration and its entry conditions. This was the discovery that made the comparison useful before any performance number was available. The existing feature could meet the narrow requirement, but the default display had not done so. Buying a new product would not have relieved the company of deciding what the date meant.

Ask someone who does the work to test the promise as it appears on the screen. “Source shown” can mean a document title without the relevant clause. “Date shown” can mean the upload date. “Human approval” can mean a person who never sees the proposed action. Replace these labels with an observable behaviour and a record of whether it occurred.

Do not make a configuration screenshot carry more than it can prove. It can show a setting or a result at a time. It does not prove that the setting cannot change, that every source has complete metadata, or that a provider will preserve the display after an update. Those become operating questions for the next stage.

Give the tender thread a bounded yes

At 9.00 on Friday 4 September, the sales director came to Maya with Daniel and commercial counsel. He had brought the first tender proposed for the new route and the requirement-mapping sheet his team already used. The original trial account remained disabled.

Counsel had reviewed this customer’s terms and the selected enterprise tool’s agreement. In this fictional case, the customer terms permitted the proposed processing in that approved environment. The provider agreement and the activated configuration allowed processing without retaining the submitted tender content or generated output after the request, and without using either for model training.

Daniel showed the account settings and the written confirmation for the specific service path. Counsel had checked the treatment of content across processing and support, rather than relying only on a “no retention” label. Limited non-content operational metadata was described separately. The company would retain its own tender and review records under its existing arrangements.

This was not a universal conclusion about tender documents or enterprise tools. It applied to the reviewed customer, document set, purpose and configuration. A different customer’s restrictions could lead to a different answer. Documents with unresolved permissions would stay outside the approved route.

Maya asked who would approve the answer sent to the customer. The sales director would remain accountable. The tool could assist with identifying requirements and drafting a mapping. A named bid reviewer would check each requirement against the exact source clause before the work entered the proposal. The tool could not send the tender, commit a price or accept an obligation.

The sales director pointed to the checking time already included in his baseline. “We’re not inventing review because there’s a model,” he said. “We’re making sure this review catches the mistake it can introduce.”

He was right. The new check had to fit the actual workflow rather than appear as an additional signature at the end. A reviewer would mark each mapped requirement as supported, disputed or missing. A link to a nearby clause would not count as support merely because it looked plausible.

Maya authorised the first reviewed tender and a limited continuation for documents meeting the same recorded conditions, under the board’s delegation of 28 August. The review date was Friday 18 September. Any new customer terms, changed processing path or failure of the citation check would require clarification or a pause before further use.

The sales director left with a yes he could explain to his team. He also left with a boundary more precise than “use AI carefully”. The original incident had not proved that tender assistance should never be used. It had shown that permission, source checking and a record of the work could not be assumed.

Put the check where the error becomes visible

At 2.10 that Friday, the bid reviewer found a requirement that the approved tool’s first draft had linked to a general service clause rather than the customer’s more specific exception. The linked text existed. It did not establish the proposed response.

The reviewer marked the mapping disputed and returned it to the sales director. They corrected the reference before the proposal was used. The discrepancy and its correction stayed in the tender review record. They did not use this caught defect to claim a reliable error rate or to conclude that every future error would be equally visible.

Maya asked whether this meant the authorisation had failed. The sales director said the bounded route had produced a defect that its intended review caught. The result supported retaining the source check. It did not establish a net time benefit, because the team had not yet completed a comparable timed tender with the new workflow.

That answer was more useful than either celebration or an automatic shutdown. The team examined the defect’s consequence, whether the required check had worked, and whether the boundary still held. No unsupported requirement had reached the customer, and no unauthorised processing had been found in this instance. A missing review, an external release or a changed data-handling condition would have called for a different response.

The reviewer also compared the original requirements list with the mapping, so the check could find an omission as well as a wrong citation. Looking only at the tool’s selected rows would not reveal a requirement it had left out. This completeness check came from the two omissions found in the pre-trial baseline.

The authority mode was Assist. People prepared, checked and approved the tender. The model output was material they worked with, not an accepted statement of what the company owed or could promise. The practical boundary was expressed through access, review and release permissions rather than through the mode’s name alone.

A checked first tender would be evidence about that tender. It would not justify changing the register entry to “safe” without qualification. Daniel recorded the approved purpose, configuration, owner and review date. Future observations would accumulate beside that decision.

Daniel did not describe the whole provisional register as verified. The wider discovery record still contained overdue checks from 21 August. He named them as overdue in the status note and gave Maya the remaining owners and follow-up actions. Approval of the reviewed tender route did not settle those unrelated reports.

He also kept the disabled trial account separate from the newly approved enterprise route. Its available history remained preserved, and complete disposal had not been established. Formal retirement would still require checking retained records, provider obligations and access closure. The new yes did not erase that unfinished work or make the earlier unrecorded incident disappear.

Write down why the remaining uncertainty is acceptable

For the retrieval route, Maya drafted a decision record before the comparison was complete. She wanted to distinguish choosing what to examine from deciding that it should be used with customers. The first decision could be made with less evidence because the scope was narrower and the consequences were contained.

She wrote the decision in one sentence: configure the existing knowledge-base retrieval feature for a bounded comparison with ordinary search, using the repaired source set and fictional questions, within the board’s four staff-days and AUD 1,500 ceiling, ending 11 September.

Then she added the three fields Nadia had asked her to make explicit. What was still uncertain? Why was that acceptable for this decision? What would make the company reconsider? The questions forced her to explain permission in relation to evidence rather than write “risk accepted” and leave the reasoning invisible.

FIG-4.1

Explain why this decision can proceed

1. Still uncertain
   Benefit over repaired search; unusual wording; adviser checking effort.
2. Why acceptable now
   Bounded comparison; fictional questions; approved sources; no live case changes or customer messages; effort ceiling and end date.
3. Reconsider if
   Processing departs from permission; source or effective-date information disappears; scope or effort expands; no useful result by 11 September.

Still uncertain: whether retrieval improves an adviser’s source-finding work over the repaired library’s search; whether unusual product language produces misleading matches; and how much checking the displayed sources require. The available tests did not establish broad performance, live customer benefit or sustained operating effort.

Why acceptable: the current activity was a time-limited comparison with fictional questions and an approved procedure subset. It had no customer communication, financial authority or connection that could change a live case. The existing search remained available. The comparison could answer the uncertainty without first exposing customers to the proposed use.

Reconsider if: the selected configuration processed information outside the approved route; source identity or effective-date information disappeared; the comparison required live customer data or record access; effort exceeded the ceiling; or no useful distinction from ordinary search emerged by the end date. Daniel could disable the feature and inform Maya. A broader proposal would return for a new decision.

The three fields did not replace the evidence. They explained why the evidence was enough for this action and not for a larger one. A later reader could see what the decision depended on, instead of guessing from an approval signature.

The same fields appeared in the tender record with different answers. Remaining uncertainty concerned benefit and error frequency across eligible tenders. Limited use was acceptable because the specific processing permission and configuration were evidenced, source checking covered every requirement, and people retained release authority. A permission change, failed check or unauthorised disclosure would interrupt that route. The retrieval decision could not be copied across unchanged.

Stop asking questions that cannot change this decision

By Monday 7 September at 11.20, the team had collected another page of possible provider questions. Some concerned functions they had disabled. Others concerned a future deployment across the company. Daniel worried that he would spend the remaining comparison time describing possibilities rather than testing the task.

Nadia asked him to put each question beside a consequence. Would the answer change permission for the current comparison, change how it was configured, change what it measured, or change the response to a failure? If so, it belonged now. If it concerned a later expansion, it belonged with that later decision.

They retained the unresolved question about source refresh because it affected whether the comparison used the intended procedure version. They deferred a question about bulk customer-message dispatch because there was no dispatch connection in the proposed route. If a later proposal requested one, the question would return with the new authority.

They also dropped a request for a broad certification claim that would not answer their particular configuration question. Counsel already had the applicable handling terms. Daniel still needed to show that those terms and settings covered the actual feature path. More general assurance material would not substitute for that missing link.

Knowing when to stop gathering evidence requires a stated decision boundary. Without it, every possible future use can become a reason to delay the present one. With it, the team can say which uncertainty it will learn through a contained activity and which uncertainty prevents that activity from beginning.

This is not permission to stop because the paper is long or the deadline is near. If a material question remains unanswered, narrow the action, change the route or defer it. The reason to proceed is that the remaining uncertainty fits within the permitted scope and its protections, not that the organisation has become tired of asking.

Maya kept the outstanding questions with their future triggers. They were neither forgotten nor treated as prerequisites for work they could not affect. Daniel could return to the comparison.

Compare the whole act of finding a source

On Thursday 10 September at 2.30, Priya and another adviser completed the final comparison session. Daniel had prepared twelve fictional questions against the repaired source set. Eight represented ordinary cases, two involved exceptions and two could not be settled by the supplied material. The categories were chosen to expose specific questions, not to reproduce the company’s workload proportions.

Each adviser used both ordinary search and the configured retrieval feature on matched variants, with the order alternated. The team still expected learning effects and did not claim a controlled trial. They recorded time until the adviser had inspected an applicable source, not time until the first text appeared on the screen.

The feature helped on some less familiar wording. It offered little advantage where a saved search already led directly to the clause. In an unsupported case it returned related material, which the adviser had to recognise as insufficient. Priya refused to call that a resolved question merely because the display was populated.

Daniel retained the timings, outputs and source checks as an exploratory comparison. The set was too small and deliberately constructed to support a company-wide productivity claim. It was enough to show that the configured feature could expose the required source information and that some of the residual search difficulty remained worth examining with more realistic work.

The result changed the next proposal. The team would not ask for a general customer-answering assistant. It would ask for a bounded experiment on source retrieval, with reply drafting considered separately. The common saved searches would remain available, and an unsupported result would require an adviser to use the existing escalation route.

The provider’s original 96 per cent figure did not appear in the recommendation. The team now had evidence closer to its actual task, albeit limited. It also had a clearer question for the experiment: under ordinary service conditions, does this feature help advisers reach and check the applicable source without creating disproportionate review work or misleading certainty?

Keep the cost of waiting visible

The sales director’s Friday authorisation had removed one avoidable delay. The service comparison had a different end point. Leila wanted advisers to benefit from the source display soon, but she also needed the team to define the experiment’s participants, information and stopping conditions before introducing it into customer work.

Maya recorded the cost of that remaining delay as a question for the next decision. The process repair was already helping customers; the residual source-search work continued manually. They would measure the additional review work required by the proposed experiment rather than assume that more checking could only help.

This did not settle the later choice about blanket second checks. That judgement still lay ahead. It did establish that waiting and review consumed something real: staff attention, service capacity and customer time. Those costs belonged beside the harms the team hoped to prevent.

A fair decision record should let someone disagree intelligently. The dissenting director could point to the reversible nature of internal retrieval. Nadia could point to unanswered questions about use under pressure. Neither argument had to be caricatured for the organisation to choose a limited next step.

Close the comparison and preserve the authority boundary

At 9.15 on Friday 11 September, Maya reviewed the comparison record with Leila, Daniel and Nadia. The work had used three and a half staff-days and AUD 900 of the authorised allowance. The saved work log and external-support invoice were attached. The unused allowance did not become permission to extend the trial indefinitely.

They selected the existing feature as the route to take into the experiment proposal. They did not authorise customer-facing use in that meeting. The board had authorised a comparison; the next proposal would specify the experiment budget, scope and participants for the board’s decision. Maya could implement an approved experiment within delegation, but could not create that authority herself.

Daniel disabled the comparison access pending the next approved stage, preserved the evaluation records and confirmed that ordinary search remained available. The repaired library continued operating. The tender route also continued within its separate authorisation and review date. Stopping one temporary activity did not mean stopping all useful work.

Leila asked for the source-maintenance time to appear in the experiment proposal. The feature had not removed the need to maintain effective dates or resolve contradictory material. She would not accept an experiment budget that treated her team’s source work as free simply because it did not arrive on a supplier invoice.

Maya agreed. The next paper would carry the residual question, the selected route and the proposed controls. It would also carry the cost of those controls. Choosing the technology had made the decision more concrete, not completed it.

Before the meeting ended, Nadia asked who would receive the decision. Daniel needed the configuration boundary, advisers needed to know that the comparison had ended, and the sales team needed confirmation that its separate permission still stood. Maya sent a short operational message to each group and retained the fuller record for review.

An approval held only in a meeting note would have left people guessing. The message named the permitted activity, its owner and the next review, using the same words as the decision record. It also told users where to raise an unexpected result. Communication was part of making the decision effective, rather than an announcement saved for a later launch.

Complete the Decision Record

The worked retrieval record stated the decision and its limits first. It named the existing feature, the comparison purpose, the approved source set and the absence of live customer actions. It linked to the board’s 28 August authority, the configuration checks, the provider review and the saved comparison results.

It then recorded the alternatives. Ordinary search remained the fallback and comparator. The demonstrated product was retained as an alternative if a material requirement could not be met. The internal build was deferred because its support commitment was unresolved. Those conclusions concerned the present problem; none was a universal ranking of the three routes.

The record’s three evidence fields explained remaining uncertainty, why it was acceptable for the limited comparison, and what would trigger reconsideration. Daniel owned configuration and disabling the feature; Leila owned the service outcome and procedure sources; Nadia challenged the interpretation; Maya owned the decision within the authority granted. The next decision was whether to approve the bounded experiment, not whether the company had “adopted AI”.

Appendix D provides the blank form and a compact worked version. Use it when the evidence has become sufficient for a specific action, even if it remains insufficient for a larger ambition. Write the boundary so the next person can distinguish the two. Then allow the authorised work to proceed, and stop gathering evidence that belongs to a different decision.