Daedalus Development Group · · 2 min read
Before an AI Pilot, Measure the Work You Already Do
Build a simple baseline for response time, review effort, and exceptions before automating a business workflow.
In this article
An automation pilot needs a comparison. If a team cannot describe its current process, a fast demonstration can look impressive without establishing that the work became easier. A modest baseline helps distinguish a useful improvement from effort that has simply moved into a new review queue.
The aim is not to instrument every action. Choose one recurring task and gather enough representative examples to understand the normal path, the difficult cases, and the decisions that should remain with a person.
Map the existing handoff
Follow a request from arrival to completion. For a service inquiry, that might mean reading the message, checking the service area, requesting missing details, preparing a response, and recording the outcome.
Name the owner at each step. Record where information comes from and which system is treated as authoritative. An assistant cannot reliably resolve conflicting records if the team has not decided which record wins. That is a workflow decision, not a prompt-writing exercise.
Measure waiting and working separately
A request may wait hours even when preparing its response takes minutes. Track both the time spent doing the task and the elapsed time before the customer receives a useful answer.
Use a small sample that includes quiet and busy periods. Keep the original inputs and the human result, removing unnecessary personal information from any shared evaluation set. Describe the sample honestly. A handful of selected examples supports a pilot discussion; it does not establish an organization-wide productivity claim.
Count the review work
For each proposed output, ask whether the reviewer can check it against the source. Record corrections, missing facts, and the time needed to decide whether the draft is usable.
An assistant that produces more drafts can create more work if each draft requires reconstruction. A useful pilot should make the review path shorter or clearer. Include rejection and escalation rates alongside speed, rather than reporting only how often the model generated an answer.
Make exceptions part of the pilot
Add examples with missing fields, contradictory instructions, unavailable source systems, and requests outside the agreed scope. Decide what the system should do: ask a question, hold the item, or send it to a named person.
For external messages, commitments, spending, or account changes, identify the approval boundary in advance. Begin with drafts or read-only summaries where that suits the task. More autonomy is a separate decision that should follow evidence from the actual workflow.
Define a stopping rule
Agree on what would justify continuing, changing the approach, or stopping. That might be fewer repeated lookups, shorter review time, or more complete intake. Avoid selecting a success measure only after seeing the results.
A small baseline makes the next conversation concrete: what improved, what still failed, and which cases need a person. Explore business automation or read One Workflow First for a practical way to narrow the initial scope.