A manufacturing AI project often has a clearly defined beginning.
A team identifies a use case. Data is collected. A model is trained. Engineers test it. A pilot demonstrates promising results. Management approves production deployment.
- Then the real operating problem begins.
- The machine changes.
- The process changes.
- Input data changes.
- A sensor fails.
- An operator learns to work around the system.
- A model is updated.
- A new product variant enters production.
Or the AI encounters a condition that was poorly represented in its development data.
None of these events necessarily means the original AI system was badly designed. They are consequences of putting an adaptive software system into a physical operating environment.
That changes the management question.
Instead of asking only:
“Does the AI work?”
Manufacturers need to ask:
“How do we know when the AI is working as intended, who is accountable when it is not, and what happens when conditions change?”
That is the operational side of industrial AI.
Production AI is an operating system, not a finished model
A model is only one component of an AI-enabled manufacturing system.
The production environment also contains:
- Sensors,
- Data pipelines,
- PLCs and control systems,
- Machines,
- Network infrastructure,
- Operators,
- Engineers,
- Maintenance teams,
- Quality processes,
- Business rules,
- Cybersecurity controls,
- Escalation procedures.
An AI model can perform exactly as tested while the larger system becomes unsuitable.
For example, a predictive-maintenance model might continue generating technically valid probabilities. But if a sensor has degraded, the maintenance team has changed its inspection process, or the machine is now operating under a different production regime, the operational meaning of those predictions may have changed.
This is why AI governance cannot be reduced to a model-development checklist.
NIST’s AI Risk Management Framework treats risk management as a lifecycle activity. Its four functions;
Govern, Map, Measure, and Manage, are intended to help organizations address AI risks across development, deployment, and use. NIST also recommends testing before deployment and continued measurement while AI systems are operating.
For manufacturing, that principle has a practical consequence:
Production AI needs continuous operational control.
Start by defining accountability
One of the easiest mistakes is assigning ownership to the wrong organizational layer.
An AI system might be developed by a data-science team, deployed by IT, connected to operational technology by engineering, and used by production personnel.
So who owns it?
“Everyone” is usually another way of saying nobody has clear accountability.
A production AI system needs named responsibilities for at least five areas:
Business ownership
Someone should own the outcome the system is supposed to improve.
- For a quality system, that might involve quality leadership.
- For maintenance prediction, it could sit with asset or maintenance operations.
- For production optimization, manufacturing operations may be responsible.
Technical ownership
Someone needs responsibility for the software, infrastructure, integrations, model version, and technical changes.
Data ownership
Someone must be accountable for whether the data feeding the system remains available, appropriate, and trustworthy.
Operational ownership
Operators and engineering teams need defined responsibilities for using, challenging, overriding, or escalating AI recommendations.
Risk ownership
Someone must coordinate decisions when AI behavior creates safety, quality, cybersecurity, compliance, or operational risk.
NIST’s AI RMF guidance explicitly emphasizes documented roles, responsibilities, and communication structures for AI risk management.
The important principle is not which department owns the system.
It is that ownership must be explicit.
Define what the AI is allowed to doDefine what the AI is allowed to do
An AI system should have an operating boundary.
That boundary should answer four questions:
- What can the system observe?
- What can it recommend?
- What can it change automatically?
- What requires human approval?
These questions become increasingly important as AI moves closer to physical control.
Consider three hypothetical systems.
System A: Maintenance recommendation
The AI identifies a machine that may require inspection.
A technician reviews the recommendation.
The risk of an incorrect recommendation may be manageable if the workflow includes appropriate checks.
System B: Quality disposition recommendation
The AI recommends whether a product should be held for inspection.
Because quality decisions can have downstream consequences, organizations may require defined human review for certain categories of output.
System C: Automatic process adjustment
The AI changes a production parameter without human approval.
The control boundary is much more significant because an incorrect action can affect equipment, quality, throughput, or safety.
The lesson is straightforward:
AI autonomy should be proportional to the consequence of error.
That does not mean humans should approve every AI output forever.
It means autonomy should be earned through evidence, validation, safeguards, and operational experience.
Human oversight must be designed, not assumed
Many AI deployments include a nominal human-in-the-loop arrangement.
The system produces a recommendation.
↓
A person clicks approve.
That is not necessarily meaningful oversight.
Suppose a technician receives hundreds of AI-generated alerts every day.
If most are low value, the technician may begin accepting them automatically.
The formal human control still exists.
The practical control may not.
Effective human-AI collaboration requires answers to questions such as:
- What information does the operator see?
- Why is the recommendation being made?
- What confidence or uncertainty is communicated?
- Which situations require escalation?
- How much time does the operator have to intervene?
- What happens when the operator disagrees?
- Is disagreement recorded?
- Is repeated disagreement investigated?
The World Economic Forum’s 2026 Human-Machine Collaboration Framework describes how industrial work is changing as intelligent systems take on more tasks, with human contributions increasingly involving judgment, accountability, exception handling, and governance.
This suggests a useful design principle:
- Don’t ask where to insert a human after designing the AI workflow.
- Design the human-machine workflow together.
Monitor the environment, not just the model
One of the most common operational mistakes is monitoring the model as if it were an isolated software component.
Traditional software monitoring might focus on uptime, latency, errors, and resource consumption.
AI systems require additional signals.
Consider a visual-inspection model.
Its performance could deteriorate because:
- Lighting changes,
- Camera position changes,
- Surface characteristics change,
- Product variants change,
- Image quality declines,
- New defect types appear.
The model itself has not necessarily changed.
Its operating environment has.
Therefore, an AI monitoring system should consider several layers.
Input Health
- Are required signals available?
- Have distributions changed?
- Are sensors operating within expected ranges?
Model Behavior
- Have prediction distributions changed?
- Are confidence levels shifting?
- Are error rates increasing where ground truth is available?
Workflow Behavior
- Are operators increasingly overriding the AI?
- Are recommendations being ignored?
- Are users developing workarounds?
Business Outcome
- Is the targeted production metric improving?
- Are false alarms increasing?
- Are quality escapes declining?
- Is downtime actually changing?
NIST’s AI RMF specifically describes measurement as an ongoing activity and recommends assessing AI risk and trustworthiness regularly rather than treating evaluation as a one-time exercise.
Build a change-management policy for AI
Manufacturing systems already have change-management practices.
AI needs to fit into them.
A model update should not be treated exactly like a routine software patch if the change can alter production decisions.
The organization should establish criteria for changes such as:
- Model retraining,
- New training data,
- New product variants,
- Sensor replacement,
- Process changes,
- Software dependencies,
- Changes in operating range,
- Changes in the ai’s autonomy.
Some changes may require simple validation.
Others may require a more substantial re-evaluation.
The important point is to
╳
Avoid an uncontrolled transition from
Validated system
To
Different system that happens to carry the same name.
Versioning, documentation, testing, approval, and rollback procedures therefore matter.
Define what happens when the AI fails
Every production AI system needs a failure strategy.
This does not mean predicting every possible failure.
It means defining sensible responses to foreseeable classes of failure.
For example:
Data failure
If required sensor data disappears, does the system:
- Stop making predictions?
- Issue a warning?
- Switch to a fallback rule?
- Require human review?
Model failure
If monitoring detects unusual behavior, does the system:
- Reduce autonomy?
- Revert to an earlier model?
- Switch to deterministic logic?
- Stop the automated action?
Connectivity failure
- If the AI service becomes unavailable, what continues operating?
- This question is especially important when cloud or network dependencies interact with industrial operations.
Human disagreement
- If experienced operators repeatedly reject recommendations, who investigates?
- The disagreement could indicate operator error.
- It could also indicate that the model’s assumptions no longer match production reality.
- A mature system needs a mechanism for learning from both possibilities.
NIST’s AI RMF recommends documented risk responses and prioritizing risks involving physical safety, legal obligations, regulatory compliance, and other high-impact consequences.
Cybersecurity is part of the operating model
Industrial AI creates another layer of connection between information technology and operational technology.
That connection can create new attack surfaces and dependencies.
ISA’s 2025 position paper on Industrial AI identifies human safety, system reliability, data quality, explainability, information protection, and AI-specific attack vectors among the important issues for industrial deployment. It also highlights ISA/IEC 62443 as a framework for addressing cybersecurity in industrial automation and control systems.
The practical implication is that an AI security review should consider more than access to the model.
It should examine:
- Training data,
- Inference data,
- Model interfaces,
- Apis,
- Network paths,
- Credentials,
- Connected industrial systems,
- Update mechanisms,
- Remote access,
- Logging,
- Third-party dependencies.
A model can be mathematically sound and still participate in an insecure system.
Do not confuse explainability with operational usefulness
Explainability is frequently discussed as though every AI system needs the same type of explanation.
That is not practical.
The explanation required by a maintenance technician may differ from the information required by a process engineer or risk manager.
For an operator, the useful information might be:
- Which signals changed,
- How unusual the condition is,
- What action is recommended,
- What would happen if no action is taken.
For an engineer, a deeper diagnostic view may be necessary.
For management, the important issue may be performance trends and exceptions.
Therefore, the goal should not be “maximum explanation.”
It should be appropriate information for the decision being made.
A practical production-control model
A useful operating model can be organized around seven controls.
1. Ownership
Who is accountable for the system and its outcome?
2. Scope
Where is the AI validated, and where is it not?
3. Authority
What can the AI recommend, trigger, or change?
4. Observation
What technical, operational, and human signals are monitored?
5. Intervention
When does a person review, override, or escalate?
6. Change
What changes require revalidation or approval?
7. Recovery
What happens when the AI, data, network, or surrounding workflow fails?
These seven controls provide a more useful production conversation than simply asking whether the model achieved an acceptable validation score.
A hypothetical example:
AI-based maintenance recommendations
Consider a plant using AI to identify machines at elevated risk of failure.
The model performs well during validation.
A weak deployment might connect it directly to maintenance scheduling.
A stronger operating design could establish:
- Scope: The model covers specified machine families and operating conditions.
- Data controls: Required sensor streams must meet defined quality thresholds.
- Recommendation: The AI generates a risk score and recommended inspection window.
- Human review: A technician evaluates the recommendation alongside machine history and current operating conditions.
- Override: Technicians can reject the recommendation and record a reason.
- Monitoring: The system tracks prediction performance, missing data, overrides, and eventual failure outcomes.
- Change control: Major equipment changes or new machine configurations trigger a review.
- Fallback: If the AI becomes unavailable, existing maintenance procedures continue.
The AI is still imperfect.
The difference is that its imperfection is bounded by the operating system around it.
Common mistakes in production AI governance
Treating deployment as the finish line
A successful pilot proves that something can work under defined conditions.
It does not prove that it will remain appropriate indefinitely.
Giving responsibility to the AI team alone
AI performance is connected to operational processes, data, users, and physical systems.
Ownership needs to cross those boundaries.
Measuring only model accuracy
A model can have strong statistical performance while producing little operational value—or while becoming unreliable under changed conditions.
Adding a nominal human approval step
If users cannot understand, challenge, or meaningfully override the recommendation, the human is functioning as a rubber stamp.
Ignoring overrides
Repeated overrides are valuable operational signals.
They may indicate model problems, workflow problems, training issues, or changing process conditions.
Treating model updates like ordinary software updates
A new model can change operational behavior.
The change therefore deserves an appropriate validation process.
Separating cybersecurity from AI deployment
Industrial AI interacts with OT environments.
Security needs to account for the entire system, not just the model endpoint.
When more autonomy is appropriate
Not every AI system needs maximum human oversight.
If an AI system has been extensively validated in a constrained environment, produces low-consequence recommendations, and operates with effective monitoring and fallback mechanisms, greater automation may be reasonable.
Conversely, high-consequence systems may require tighter boundaries.
The relevant question is not:
“Should humans always stay in the loop?”
It is:
“What level of human involvement is appropriate for this decision’s consequence, uncertainty, and operating environment?”
That produces a more useful design discussion.
Human oversight can range from:
- Passive monitoring,
- Review before action,
- Approval of exceptions,
- Intervention when confidence falls,
- Complete human control.
The appropriate point depends on the system.
AI governance should be built into normal plant operations
The best AI governance will not feel like a separate compliance exercise.
It should connect naturally with existing practices for:
- Quality management,
- Maintenance,
- Process safety,
- Cybersecurity,
- Engineering change control,
- Incident management,
- Operator training,
- Equipment validation.
This is particularly important because manufacturing already operates through controlled procedures.
AI should become part of that operating discipline rather than creating an isolated governance structure that plant personnel rarely use.
NIST explicitly describes its AI RMF Playbook as voluntary and adaptable rather than a one-size-fits-all checklist.
That is useful guidance for manufacturing organizations: adopt controls according to the actual system, its risks, and its operating context.
Questions to answer before putting AI into production
Before an AI system influences a production decision, a team should be able to answer:
- Who owns the system?
- What exact decision does it influence?
- Under which operating conditions has it been validated?
- What is outside its approved scope?
- What can it do automatically?
- What requires human approval?
- How is uncertainty communicated?
- What signals indicate degraded performance?
- How are operator overrides recorded?
- What changes require revalidation?
- What happens when data becomes unavailable?
- What happens if the AI service fails?
- How is the system secured?
- Who investigates incidents?
- When should the system be retrained, restricted, replaced, or retired?
If these questions cannot be answered, the remaining work is not merely technical.
It is operational design.
The real measure of trustworthy industrial AI
Trustworthy AI does not mean an AI system that never makes mistakes.
That standard would be unrealistic.
A better standard is whether the organization can understand the system’s intended role, recognize meaningful deviations, limit its authority, intervene when necessary, recover from failures, and learn from incidents.
NIST’s AI RMF describes trustworthy characteristics including validity and reliability, safety, security and resilience, accountability and transparency, and explainability and interpretability
Industrial AI adds another important dimension: physical consequence.
When AI participates in a manufacturing workflow, errors can affect equipment, products, workers, schedules, energy consumption, and downstream operations.
That makes operational boundaries particularly important.
From AI deployment to AI operations
The mature question for manufacturing organizations is no longer simply whether an AI model can perform a task.
It is whether the organization can operate the entire AI-enabled process responsibly over time.
That requires a shift from:
MODEL PERFORMANCE TO SYSTEM PERFORMANCE
and from:
DEPLOYMENT TO CONTINUOUS OPERATION.
The strongest production AI environments will therefore have clear ownership, explicit operating boundaries, meaningful human oversight, continuous monitoring, controlled change, cybersecurity, and documented recovery procedures.
The goal is not to eliminate every AI failure.
It is to make failures visible, bounded, recoverable, and useful for improving the system.
A manufacturing AI system becomes operationally mature when the organization can answer a simple question at any moment:
What happens when the AI is wrong?
If the answer is specific, tested, and owned, the organization is no longer treating AI as a model sitting inside a factory.
It is treating AI as part of the production system.
FAQ
Does every manufacturing AI system need a formal AI governance program?
Not necessarily. The controls should reflect the system’s risk, autonomy, physical consequences, and organizational context. A low-risk analytical assistant may need substantially different controls from an AI system that influences machine operation.
Who should own a manufacturing AI system?
Ownership should be explicit rather than automatically assigned to IT or data science. Business, technical, data, operational, and risk responsibilities may be distributed across teams, but accountability and communication paths should be documented.
How often should a manufacturing AI model be retrained?
There is no universal interval. Retraining should be driven by evidence such as data drift, changing operating conditions, declining performance, new product variants, or changes in the process. Retraining itself should be but AI can introduce additional concerns involving model interfaces, data pipelines, training data, AI-specific attacks, and changing model behavior. ISA specifically identifies information protection and AI-specific attack vectors controlled and validated.
What should happen when operators frequently override AI recommendations?
Treat repeated overrides as a diagnostic signal. Investigate whether the model is wrong, the operating conditions have changed, the recommendation is poorly presented, or the workflow is asking the AI to solve the wrong problem.
Is AI cybersecurity different from traditional OT cybersecurity?
There is overlap, but AI can introduce additional concerns involving model interfaces, data pipelines, training data, AI-specific attacks, and changing model behavior. ISA specifically identifies information protection and AI-specific attack vectors alongside traditional industrial safety and reliability concerns. isa.org.
Can AI operate autonomously in manufacturing?
In some constrained environments, autonomous operation is technically possible. The appropriate level of autonomy depends on validation evidence, consequence of error, system boundaries, monitoring, fallback mechanisms, and applicable safety and operational requirements. Greater autonomy should not be treated as automatically better.
