An automated Excel workbook issuing invoices. A chatbot responding to customer emails. A fraud-detection algorithm scanning transactions. All three are lumped into the same bucket: “automation.” Yet their nature, costs, and risks differ wildly. How to choose?
The classic mistake: conflating end-to-end automation with task automation
Classical automation is a sequence of steps where every action triggers the next via explicit rules. The computer never makes a choice; it follows a protocol. Output is deterministic: identical data and conditions give an identical result. Examples: data extraction from PDFs, file format validation, synchronising two databases.
AI, by contrast, produces new outputs from ambiguous inputs. Its output may be useful… or completely wrong. Therefore error rate must be measured, tolerated, and corrected.
These are different beasts. Forcing a single “solution” leads to projects that spin indefinitely in experimentation, delivering nothing.
The decisive criterion: is the input structured or not?
This is the first filter to apply — a useful heuristic, not a law.
- Structured input (well-formed CSV, API with schema, digital form) → a rule engine, a simple orchestration, or a workflow tool is enough — with a predictable result and a marginal cost close to zero.
- Unstructured input (emails, PDF contracts, support tickets, free-form logs) → AI can extract value, but you must accept a margin of error, human validation budget, and handling of edge cases.
The boundary is not always sharp — semi-structured input is often handled by a mix of both. But as a first decision filter, it avoids most scoping mistakes.
Six task families, six different choices
- Process execution (deterministic sequences of steps): classic automation, possibly orchestrated with a human decision layer.
- Structured data processing (ETL, formats, validations): classic automation — scripts and rules, no AI.
- Content generation (texts, reports, first drafts): templates and rules for the fixed parts; AI for first drafts, with systematic human review.
- Semantic analysis (sorting, classifying, extracting from unstructured input): AI for the first pass, human control behind it.
- Real-time customer support: rule-based answers for standard cases; assisted AI with human escalation for the rest.
- Quality control: automated anomaly detection plus human expertise — rules alone are too brittle, AI alone too unreliable.
The risk of AI alone shows here: a bloated machine that reliably produces garbage. Classic automation’s cost spikes when input ambiguity creeps in. The hybrid approach is usually the best option: AI covers the unstructured first pass, then a human or a rule cleans up before final delivery.
Five decisive rules before green-lighting anything
Don’t start implementation until you can answer each question with data.
1. What is the process volume?
Two hours per month won’t justify automation effort. Two hours per day will. The target isn’t the technology; it’s time saved versus cost. If forecasted savings over 24 months are less than total cost (build + run + maintain), drop the project.
2. Is the expected outcome absolutely predictable?
If a simple rule can make the outcome deterministic, choose classical automation. AI does not remove ambiguity; it manages it with an error rate. That rate must be measured from day one.
3. Can you tolerate (and correct) errors?
If a mistake reaches production and remains undetected for six months, do not use AI. Example: an invoice calculator under-estimating VAT. Stricter rules with automated tests, or mandatory human approval before release, are the only safe routes.
4. Do you have a reliable ground truth dataset to validate outputs?
Without a golden dataset, measuring AI error rate is impossible. Improving the model is impossible. The project will stay in POC limbo forever.
5. Will human review be part of the loop?
Some contexts — legal documents, medical data, financial statements — may require reinforced human validation, depending on risk and the regulatory framework. Here AI is an assistant, not a replacement.
Deadly traps that quietly kill AI projects
1. Automating fast what isn’t automatable at all
Some processes resist automation, by design. Example: subjective quality assessment of a service delivery. Here automation is counter-productive. Better to train teams on clear criteria than sell them a mythical tool.
2. Chasing POC “accuracy” without field metrics
A lab-ready proof of concept proves nothing in the wild. Real-world data is messier. AI projects must start with a limited pilot on real inputs, measuring error rate and correction cost before scaling.
3. Underestimating maintenance cost of the AI model
A business rule rarely changes (or changes slowly). Model performance can drift as data, usage, input formats, underlying models or vendors evolve. Plan for ongoing maintenance from the start — it never stops at the initial build.
A three-stage roadmap
- Audit candidate process: volume, input ambiguity, error tolerance, availability of ground truth data.
- Pilot a minimal slice on real data, measuring error rate and human correction cost.
- Decide based on ROI: time saved over 24 months versus total cost. If positive, scale. Otherwise stop.
The only number that matters: return over 24 months
Post-pilot, compute:
- Development + maintenance cost over 24 months
- Time saved by automation/AI (measured, not estimated), valued at your team’s loaded hourly rate
- Cost of correcting errors (detection + repair)
If the result is positive, expand the project. Otherwise shut it down before it becomes a bottomless pit.