Start with the evidence
A plausible answer cannot establish the current state of one work order.
Suppose a planner at Elm River Precision asks, “Can WO-1842 ship Friday?” A conversational model can explain what a planned ship date means. It can describe common causes of delay and produce a professional status note. None of that establishes whether final machining is complete, whether an inspection hold is active, whether the date is still current, or whether anyone has made a customer commitment.
Those facts belong to current operating records and accountable people, not to the model’s general language capability. The fictional Academy case says that final machining is complete, hold H-291 remains active, October 16 is a planning value rather than a confirmed promise, and Quality owns the hold decision. The example is a teaching record—not a deployed Elm River system, a customer result, or evidence that any commercial product can reach a manufacturer’s ERP or QMS automatically.
A practical mental model
The phrase “AI knows our business” hides six different jobs.
| Layer | What it can contribute | What it cannot establish by itself |
|---|---|---|
| 1. Model | Learned capability to interpret input and generate a prediction, classification, recommendation, or draft. | The current state, revision, approval, or authority inside your factory. |
| 2. Working context | The request, instructions, supplied records, selected history, and returned tool data available for one response. | Information that was omitted, inaccessible, stale, or displaced from the finite context. |
| 3. Retrieval or connection | A controlled path to current permitted evidence from a named source. | Truth, completeness, permission beyond the caller, or authority to change another system. |
| 4. Configured tool | A bounded read or action capability with a defined input, identity, permission, prohibited action, and failure behavior. | What any particular call returned—or whether that result is current, complete, or sufficient. |
| 5. Returned evidence | The actual record, value, source identity, effective time, status, receipt, or explicit error produced by one permitted operation. | Whether the evidence supports the consequential decision or resolves a conflict with another source. |
| 6. Accountable person | Contextual review, exception handling, correction, escalation, and the reserved decision. | A guarantee that the preceding system design, evidence, or review process is adequate. |
This separation is useful because each layer can succeed while another fails. A strong model can receive an obsolete procedure. Correct retrieval can return a record the user is not authorized to act on. A technically successful action can still exceed business authority. A named reviewer can become ceremonial if that person cannot see the source, recognize the error, or stop the outcome.
Google’s production machine-learning guidance treats pipelines for serving predictions, preparing data, training, and validation as distinct operating work around models. Microsoft’s current Copilot documentation provides one product-specific example in which the application grounds a prompt with information available through Microsoft Graph under the signed-in user’s permissions before a model generates a response. Neither source says that every AI product or every factory follows the same architecture.
Fictional manufacturing example
Follow the WO-1842 evidence path, not the fluency.
The useful design question is not whether the assistant sounds informed. It is what each additional source or capability allows the internal brief to say honestly. The progression below uses the entirely fictional Elm River Precision records from the Academy.
| Available layer | What the system can support | What remains unresolved |
|---|---|---|
| Model only | General explanation of work orders, inspection holds, planning dates, and status-review structure. | Every current claim about WO-1842. |
| One schedule export | What that supplied export reports, with its file identity and effective time. | Current inspection state, missing records, later changes, and customer commitment. |
| Read-only ERP and QMS retrieval | The current planned date, operation state, active hold, responsible owner, source identity, and observation time returned by the configured reads. | Hold disposition, schedule feasibility after disposition, and any external commitment. |
| Draft-request tool | That one internal review request was created—or failed—with a receipt or explicit error. | Approval, hold release, schedule change, or customer communication. |
| Authorized review | Quality can resolve the hold; Production Control can evaluate schedule impact; the authorized commercial owner can approve a customer message. | Anything outside each role’s evidence, competence, and delegated authority. |
The resulting internal brief can be modest and still useful: final machining is complete; hold H-291 remains active; Friday is the current planned date rather than a confirmed promise; Quality review comes next; shipment status remains unresolved. That answer is stronger because it exposes the evidence and the limit, not because it adds certainty.
Use it in the next meeting
Ask five questions before believing a factory AI demonstration.
- What exact manufacturing question is the system answering, and which consequential decisions are excluded?
- Which authoritative source can establish each current claim, including its identity, revision, status, and effective time?
- How does permitted evidence cross into the working context, and what happens when retrieval is denied, stale, unavailable, or conflicting?
- Whose identity and technical permissions govern every read or action, and which capabilities are deliberately absent?
- Who can inspect the returned evidence, correct or reject the proposal, handle the exception, and make the final decision?
| Role perspective | Decision the questions improve |
|---|---|
| Daily practitioner or Quality reviewer | Whether the answer points to a current governing source and preserves a gap that needs review. |
| Owner or operating leader | Whether a demonstration solves one defined problem with a measurable result instead of merely producing impressive language. |
| IT, security, or MSP | Whether identity, source access, tool scope, logging, failure behavior, support, and recovery are explicit. |
| Workforce or learning sponsor | Whether employees can explain the boundary, practice safely, and return decisions to the responsible role. |
Move forward deliberately
Build the evidence path before adding more autonomy.
The constructive response is not to keep AI away from factory information. It is to begin with one question whose source, reviewer, failure, and value can be named. A useful first version may be a person supplying one approved record to produce a draft. A later version may justify read-only retrieval. Action tools and branching automation should arrive only when the requirement and controls earn them.
- Name one recurring question and the person who currently answers it.
- Identify the current governing sources and the conditions they cannot establish.
- Define a reviewable draft that preserves source identity, time, status, and unknowns.
- Start with the smallest permitted information path and explicit failure behavior.
- Keep the manual process and consequential authority with the responsible roles.
- Compare evidence-finding time, missing facts, corrections, exceptions, review time, and downstream rework with the current baseline.
That sequence gives leadership a buying test, operations a bounded workflow, IT and an MSP a connection boundary, Quality a reviewable evidence path, and employees a practical rule: do not ask whether the AI sounds as if it knows the factory; ask what current evidence the system can show and what still belongs with you.
Evaluate the operation
Measure whether the evidence path improves the work.
A technically connected answer is not automatically a better workflow. Compare the assisted path with the current one using a small set of operating measures: time to locate governing evidence, material fields supported, missing or conflicting facts preserved, corrections introduced, exceptions routed correctly, reviewer effort, and downstream rework. Record denied, stale, unavailable, and uncertain results instead of excluding them from the pilot story.
The decision after a pilot may be to continue, narrow, repair, use conventional automation, or stop. That is useful evidence. The purpose of the evidence path is not to prove that AI belongs everywhere; it is to make one information task more reviewable without confusing computational assistance with knowledge or authority the system does not possess.
References
Sources and further reading
- What is AI? Can you make a clear distinction between AI and non-AI systems? Updated AI-system definition and explanation of input, model, inference, and output · Technology-neutral explanation of how an AI system infers outputs from inputs
- ML pipelines Serving, data, training, and validation pipeline explanation · Production-model guidance showing that useful operation depends on pipelines around a model
- Microsoft Copilot architecture and how it works Prompt, grounding, Microsoft Graph, signed-in-user permission, model, and response flow · A current product-specific example, not a universal architecture or configuration guide
- AI Risk Management Framework Core Govern, Map, Measure, and Manage outcomes for context, oversight, knowledge limits, evaluation, and responsibility · NIST states that AI RMF 1.0 is being revised; verify the current framework before operational use