Describe the task before naming the technology
Write one sentence describing the current workflow: who does the work, what information they need, what they produce and who checks it. Add the time spent, the common exceptions and the cost of a mistake.
Compare three possibilities: improving the existing process, using a rules-based automation, or using AI. A fixed routing rule may be easier to maintain than a model. A messy knowledge task may need better source material before it needs a new interface.
Choose a boundary you can explain
A first pilot might help a staff member find an approved product answer or prepare an internal draft for review. Define what it is allowed to do, what it must decline and when a person takes over. Keep the initial task narrow enough to inspect its outputs.
Write down the permitted data, the people allowed to access it and where it will be processed. Confirm the relevant tool settings and agreements before entering private business or customer information. Publicly available information is not automatically suitable for every use.
Create a small test set before launch
Collect representative questions or examples with expected answers. Include incomplete information, conflicting sources and requests outside the tool’s scope. A useful system should make uncertainty and escalation visible.
Keep test material separate from the examples used to develop the workflow. Have a knowledgeable person review the outputs and record the kinds of errors. A single overall score can hide a failure in the task that matters most.
Measure the whole task, including corrections
Compare the existing process with the assisted process using a consistent task definition. Count review and correction time, ongoing software costs and work created by exceptions. Faster draft generation is only one part of the result.
For example, record time to an approved answer, the proportion requiring material correction and the number of cases escalated to a person. Define an acceptable result and a stopping condition before the pilot begins. Expand only when the evidence supports it.
Keep responsibility visible
Name the person responsible for the workflow, how staff report a problem and when it will be reviewed. Document source changes and decide what happens when the system is unavailable. Responsible implementation is an ongoing operating practice.
For further reading, the NIST AI Risk Management Framework provides a voluntary framework for considering trustworthiness throughout AI design, development, use and evaluation.
TAKE THE NEXT STEP
A little structure helps.
Use this plain-text worksheet with your team, or bring it to a project conversation.
Download the AI pilot planner ↓Free download · editable text file · no signup
Explore the related service
Custom AI agent design & development ↗