AI Readiness Assessment: 8 Checks Before You Fund a Pilot

An AI readiness assessment checks whether a business can use AI for a clear task with a clear gain and manageable risk. It examines the process, data, people, technology, and running plan before the business commits to a build. The result should be a useful choice: proceed with a pilot, fix named gaps first, or use a simpler solution.

A company can have modern software and still lack the data or ownership that an AI project needs. Conversely, a small team can run a useful pilot when it chooses a narrow task and clear review rules. This guide explains eight proof-based checks and shows how to turn the findings into an actionable pilot brief.

Start with one use case, not a company-wide score

Readiness depends on the task. A tool that drafts internal meeting notes has different needs from a system that recommends credit choices. Therefore, begin with a named user, an existing process, and an outcome that the business can measure.

For example, an equipment supplier might want to draft answers to routine support questions. The assessment should ask whether approved manuals exist, whether staff can verify a draft, and whether the tool can show which source supports an answer.

That scope keeps the work grounded. It also avoids the trap of declaring an entire company ready for every type of AI. A useful AI readiness framework organizes proof, but it should not turn a complex choice into an unexplained total score.

1. Confirm that AI addresses the actual problem

Write the problem without mentioning AI. For example, support agents spend time finding the current product manual before they can answer a client. Next, find the part of that process that needs judgment, prediction, or generation.

A search improvement or a clear rules engine might solve the issue. If a missing client reference causes most delays, generating better text will not fix the bottleneck. Therefore, check the AI idea with a simpler process or software change.

NIST’s AI RMF guidance on managing risk includes considering whether an AI system is right for its intended purpose. Use that principle to make the first choice explicit rather than treating AI adoption as the goal.

Record the current baseline. Measure the useful task on a typical sample, using a clear definition. A pilot cannot show value if the team never agrees how the existing process performs.

2. Check data access, quality, and ownership

List the data the planned system needs and where it lives. For each source, name an owner, the update process, access restrictions, and known quality problems. A large folder of files does not on its own form a usable knowledge base.

Next, inspect a sample. Look for missing fields, duplicate records, conflicting versions, and outdated guidance. If two manuals disagree, decide who resolves the conflict before an AI tool uses both.

Access rights matter as much as completeness. A service agent may need public product guidance but not confidential contract terms. The app should keep those boundaries when it fetches data or stores a conversation.

Finally, assess ongoing upkeep. A source that looks clean today may become stale after the next product release. Readiness means the business can keep the data current, not simply assemble a one-time export.

3. Define acceptable output and human review

Describe what a useful result looks like. For a draft support response, the criteria might include the correct product, a supported answer, plain language, and a clear escalation when proof is missing.

Then define what the system must never do. For example, it should not invent a warranty promise or expose another client’s account details. These rules help the team design both the interface and the test cases.

Human review needs a real workflow. Who sees the draft? What proof can they inspect? Can they correct it quickly? If review needs more effort than writing the answer, the use case may need a narrower scope.

Our guide to LLM evaluation explains tests for deployed language apps. During readiness work, focus on whether you can define and collect the proof needed for those tests.

4. Assign choice rights and accountability

Name a business owner who can accept the pilot’s gains and limits. Also name the people responsible for data, technical delivery, user training, and incident handling. A general steering group does not replace those working roles.

The NIST AI RMF Playbook provides voluntary actions organized around Govern, Map, Measure, and Manage. It is not a mandatory readiness checklist. You can use useful actions to find missing ownership and review practices for your own project.

Keep governance proportional to the use case. An internal drafting assistant and a high-impact automatic choice do not need identical controls. However, each still needs someone who can pause the feature when the proof no longer supports its use.

For a broader operating model, see our AI governance framework guide. The readiness assessment should find the needed owners and gaps, rather than duplicate that full governance program.

5. Review integration and technical constraints

Map the journey from user input to final action. Find the systems the AI feature reads, the systems it writes to, and the points where a person checks the result. This map often reveals dependencies that a standalone demo hides.

For example, a tool may draft a service reply well but fail to fetch the correct account because the CRM has mixed IDs. That is an integration gap, not a prompt-writing problem.

Check response time, usage limits, and planned demand. Also decide what happens when a provider becomes not available. Staff need a fallback path that keeps their work and makes the failure clear.

Our article on AI infrastructure covers the delivery components in more detail. At this stage, confirm that your team can support the smallest safe pilot before planning a larger tool.

6. Assess staff capacity and change effort

Ask users how the planned feature changes their daily work. A helpful tool should fit a real task, not need staff to invent a new reason to use it. Interview the people who handle exceptions as well as those who perform the standard process.

Training should cover verification, escalation, and limits. For example, show staff how to find an unsupported answer and when to ignore the draft. A short demo of attractive outputs will not prepare them for ambiguous cases.

Next, check capacity. Who will label test examples, review pilot results, and answer user questions? Those tasks consume working time. Include them in the plan rather than assuming employees will absorb them between existing duties.

Finally, involve users in the choice to proceed. Their feedback may reveal that the planned feature saves typing but adds several sign-off steps. Readiness includes the ability to change the workflow, not just install a tool.

7. Estimate the full running cost

Count more than model usage. A pilot may need data preparation, integration work, review, logging, staff review, support, and ongoing content upkeep. Separate one-time setup from recurring effort.

Use a clear unit for cost and value. For a support assistant, that might be cost per reviewed answer and time per resolved case. However, do not assume every draft saves the same amount of time.

Test several demand levels. What happens during a product launch or seasonal peak? What happens if users submit longer documents than planned? A low-volume demo can hide cost patterns that matter in production.

Also include the cost of mistakes and rework where you can estimate them sensibly. If you cannot quantify a key impact, record the uncertainty and assign a review step. An honest range supports a better choice than an exact-looking number without proof.

8. Create a pilot with clear stop conditions

A readiness assessment should end with a bounded experiment. Define who will use the feature, which data it can access, what it may do, and when the team will review the result.

For example, a supplier could pilot draft-only support answers for one product family. Agents would approve every response, and the tool would escalate questions that lack an approved source. The pilot would not change orders or send messages on its own.

Set success criteria before launch. Also set stop conditions, such as repeated exposure of restricted content or a review burden that exceeds the baseline. These conditions help the team respond consistently instead of debating every event from scratch.

Keep the pilot reversible. Staff should keep a way to complete the task without the AI feature, and the team should know how to disable it. That fallback makes the experiment easier to manage and its outcomes easier to interpret.

A useful readiness evidence sheet

For each row, use ready, ready with conditions, or not ready. Add the proof and the gap owner beside that status. This simple approach makes uncertainty visible without pretending that all readiness areas carry the same weight.

For instance, good training cannot compensate for data you have no right to use. A single critical blocker may stop the pilot even if several other areas look strong. Therefore, review critical conditions separately from general improvement opportunities.

Turn findings into a delivery brief

Summarize the use case, users, proof, unresolved gaps, and recommended next step. Include a named owner and due date for each prerequisite. Keep the brief short enough that the sponsor and delivery team can use it in one working meeting.

Then decide whether to pilot, prepare, or stop. Preparing may mean cleaning source documents, adding access controls, or clarifying the process. Stopping may mean a conventional search tool or rules-based automation fits better.

Revisit the assessment when the scope changes. Adding automatic actions, new users, or more private data can change the risk and running needs. Readiness is a choice about a defined use case, not a permanent badge for the company.

Keep a record of assumptions

Alongside the pilot brief, list what the team has not yet proved. For example, staff may expect that source manuals cover most questions, but nobody has checked a sample. Mark that as an assumption and assign a test.

Next, separate an unknown from a known blocker. An unknown needs proof. A blocker needs a change before the pilot can proceed. This distinction helps sponsors fund the right next task instead of treating every gap as a reason to buy more technology.

Make the next choice clear

A useful AI readiness assessment replaces enthusiasm and uncertainty with proof. Choose one task, inspect the prerequisites, and define a reversible pilot. The best outcome is a clear next step, whether that means building, preparing, or choosing simpler software.