Deploying AI in business: a 10-point checklist from pilot to production

The short answer

An AI pilot that works is not the same as a system ready for production. To go live, a business needs answers to ten questions: who owns it, how success is measured, where the data comes from, which test cases it is evaluated on, where people approve, whether security has reviewed it, how it is monitored, what the cost budget is, how to roll back in an incident, and whether users are trained.

This guide explains why projects tend to stall after the pilot, gives a 10-point checklist table and shows how to use it as a gate before switching the system on for real users.

Why many AI projects stall at the pilot stage

In a press release dated 29 July 2024, Gartner predicted that at least 30% of generative AI (GenAI) projects would be abandoned after proof of concept by the end of 2025. Gartner named four causes: poor data quality, inadequate risk controls, escalating costs and unclear business value.

In the same release, Gartner said GenAI costs are less predictable than other technologies and depend on spending level, the use cases chosen and the deployment approach. In an article dated 26 January 2026, Gartner updated this: by the end of 2025 at least 50% of GenAI projects had been abandoned after proof of concept, for the same four groups of reasons, and it added another failure point, poor change management, which leaves even good tools with little adoption. These causes match what typically happens when a demo meets real data, real users and a real budget.

• The pilot ran on curated data; production data is far messier.
• There was no baseline, so value cannot be proven.
• Nobody owns the system once the pilot team disbands.
• Security and legal are consulted only at the last minute.
• Model costs grow with real traffic and nothing caps them.

The 10-point checklist before going live

Use the table below as a gate: each row needs a written answer and evidence. If a row is missing, record the accepted risk and who accepted it.

Points 1–2: ownership and metrics

An AI system without an owner degrades as soon as the data or process changes. NIST's AI Risk Management Framework (AI RMF 1.0), a voluntary framework, places the GOVERN function — accountability, policies and oversight — underneath all other AI risk management activities.

Metrics should tie to business outcomes (handling time, manual correction rate, cost per case), not just model accuracy. Above all, measure a baseline before switching AI on; without it, any claim of improvement cannot be verified. See our guide on measuring AI Agent ROI.

Points 3–4: data and the evaluation set

Data quality is the first cause on Gartner's list. Before going live, list every data source the system reads and writes, classify its sensitivity and confirm who has authority to grant access. See also data governance basics.

An evaluation set is a collection of examples with expected answers, drawn from real data and including hard and exceptional cases. Every time the prompt, model or tools change, re-run it before the change goes live. Anthropic recommends extensive testing in sandboxed environments with appropriate guardrails before giving agents autonomy.

Points 5–6: human approval and security review

Every hard-to-reverse action — moving money, deleting data, sending information outside the organisation, committing to a customer — should have a person approve it before execution. How to design approval points is covered in our human-in-the-loop guide.

For security, the OWASP Top 10 for LLM Applications (2025 edition) is a useful starting point. It ranks prompt injection first and includes Excessive Agency: when a system is granted too much functionality, permission or autonomy, a manipulated model can take harmful actions. It also lists Unbounded Consumption, which bears directly on cost.

• Least privilege: grant only the tools and data the task requires.
• Treat model output as untrusted data; validate it before passing it to other systems.
• Log every AI action for traceability — see access control and audit logs for AI Agents.
• Keep external content (email, documents, web pages) separate from system instructions.

Points 7–8: monitoring and cost budget

NIST notes that AI risks differ from traditional software partly because data can change over time and alter system behaviour. Post-deployment monitoring is therefore essential: track error rate, hand-off rate and latency, and sample quality regularly. Every alert needs an owner and a response.

Escalating cost is one of the causes Gartner names. Set budgets per case and per month, hard limits at the API layer, and alerts as you approach the threshold. Ways to optimise model spend are covered in our guide on LLM cost control.

Points 9–10: incident plan and change management

A rollback plan must be written and rehearsed before it is needed: who may switch the system off, how, how work moves back to the manual process and who gets notified. Keep the manual fallback available at least until the system has run stably for a while.

Finally, users need to know what the system can and cannot do, when to double-check, and where to report problems. A good tool that users either distrust or over-trust is still a failure.

A staged rollout path

Rather than switching on for all users at once, move through stages and advance only when the checklist passes.

• Shadow mode: AI proposes, people keep working as before; compare results.
• Approved: AI acts, a person approves each result before it applies.
• Threshold: AI handles low-risk cases alone; above-threshold cases go to a person; sample checks continue.
• Expand: add request types, user groups or departments, re-running the checklist each time.

How EYC helps

EYC advises on and builds AI and AI Agent automation, including reviewing a pilot against this checklist before it goes live. Infrastructure, monitoring and cost can be combined with our cloud and data infrastructure service. Service levels, data restore and independent audits are defined under contract.