The document automation market is seeing massive growth year over year. As it’s often said “You can have accuracy, speed, or low costs, but you only get to choose one.” Is that mantra still true today?
There are automation solutions for everything – home automation, industrial automation, process automation, task automation, driving automation, and even a fully automated robotic kitchen (just don’t get in its way…).
The success of document automation is more than product selection. The way you think about human and machine interaction will determine overall accuracy and the business impact you hope to achieve.
4 Critical Concepts in High-Accuracy Document Automation
- Accuracy is in the Eye of the Beholder
- Human-in-the-Loop Architecture
- Choose Automation Solutions that Promote Transparency
- Avoid Perfection and Promote Human Judgement
1. Accuracy with Document Automation Solutions is All in the Eye of the Beholder
Because document automation is all about improving business outcomes by scaling technology rather than human workers, accuracy has become more of a relative term than a hard-set rule.
When is 63% Document Automation Accuracy Great?
When it represents a significant reduction in manual labor, it’s considered a win.
But there are many use-cases where automation must be as accurate or more accurate than a human worker to be considered a success. Think self-driving cars, or medical diagnoses. Even if humans, on average, make more mistakes than the machine, we don’t really accept the margin of error because public perception is that the machine should be better.
When is 95% Accuracy a Failure?
When it represents optical character recognition (OCR) accuracy.
Anything less than 100% OCR accuracy creates massive error rates. Here’s why:
Say you’re getting 95% accuracy on an invoice and you need to extract 10 independent fields. The overall per-field accuracy is actually 60%, not 95% because .95 to the power of 10 is .60, or 60%. Imagine – just 4% better OCR boosts accuracy to 90%.
You may accept 95% accuracy for full-text search, but not for automated data extraction. Document automation needs 100% OCR accuracy, but it’s impossible using OCR alone. Read more about that here.
For high-accuracy document automation, choose solutions that provide:
- Built-in validation
- Programmable error correction and
- Human review to validate data based on machine-generated confidence scoring or by arbitrary business rules
Define accuracy in terms of real-life results and within acceptable boundaries of risk.
2. The Accuracy of Document Automation Solutions is Dependent on Human-in-the-Loop Architecture
Perhaps, without exception, all enterprise automation solutions are purpose-built programs (machines), designed to do one thing within an explicit set of rules and guardrails. And if you’re thinking “Wait, RPA does more than just one thing,” consider that every bot you deploy is really it’s own program, operating within specifically defined parameters.
Human-in-the-loop architecture is the idea that automated processes are designed around human workers.
Instead of thinking “How can I remove humans,” it is an approach that says, “How can I improve human throughput?”
This way of thinking improves accuracy, speed, and costs. It’s the only way to achieve all three.
Think about the beauty of this way of thinking: Building a smarter system vs. Incorporating more meaningful human interaction. It’s like a single group of soldiers with robotic exoskeletons defeating swarms of enemy bots (hey, at least they always win in the movies, and we like that, right?).
(Or, if you’ve read Atlas Shrugged, the defeating of Blane the Mono.)
Below is a fun, real-life example of the power of human-in-the-loop design. It’s a demo of an automation solution designed to separate out elements from audio tracks. Think – removing an annoying cell phone ringing, or a cough from an orchestra performance.
At first, it seems the entire process should be fully and easily machine-automated, but the demo proves the value of human interaction during the editing process for producing stellar results.
A machine alone would never have performed as well.
3. Choose Document Automation Solutions that Promote Transparency
In a recent conversation with a customer responsible for automation solution design, he said that transparency was one of the most important benefits to him. In his environment, they’ve paired UiPath with Grooper intelligent document processing to transform very tedious verification workflows.
By using subject matter expertise to program a system that mimics human knowledge, they design automated processes that center around human cognition.
Now, instead of doing similar processes 26 different ways, they have one transparent platform for solution design. All inputs and outputs are clearly defined and optimized.
Another core element of transparency with automation solutions is trust.
Complex machine learning algorithms and neural nets are relatively easy to build, but trust erodes at the first sign of failure. Building a layer of transparency into why particular outputs or results are provided by the machine instills the trust needed for success.
When you are sourcing a document automation solution, make transparency a core requirement. You’ll have better visibility into performance improvements while maintaining the highest degree of accuracy possible.
4. Incorporate Human Judgement and Forget About Building the Perfect Algorithm
I asked our Product Manager about creating the perfect algorithms and if the hype around self-learning document automation systems was really just marketing trying to sound fancy.
Here’s what he had to say:
"Our philosophy about document extraction and data integration – including machine learning and classification – is significantly different from the rest of the market, which gives us a dramatically improved ability to classify, separate, and extract data from challenging documents.
Underlying the use of more "traditional" AI models (mostly neural nets) is the assumption that, after a certain threshold of training, computers will be able to make *better* decisions about our data in a way that is essentially foreign to us.
The machine learning frameworks we’ve built automate human understanding of data, using tools that are fundamentally intelligible. Our core philosophy is that a human given a sophisticated tool (that includes ML algorithms) will do better on virtually every document processing problem than a well-trained neural net; this is a model (AI-assisted human judgement) that is finding increasing adoption in fields such as medicine and national security.
Users understand their data better than anyone else, and automating their understanding is both easier and achieves better results than attempting to build a fundamentally unintelligible machine learning model that tries to make better decisions than a subject matter expert could.
We have yet to be proven wrong on this fundamental philosophical orientation.Grooper gives architects and end users a more robust, more complete set of tools for solving complex document processing and data integration problems than any other solution on the market today.
What this means in terms of capabilities is that we get good results from data or document sets that are incredibly challenging – small number of examples, degraded, or unpredictable text, documents where the semantic content of the words can’t aid in classification, documents where the variability within categories is high or the similarity across categories is high.
Our models, once developed, also perform well against documents they haven’t seen – human assisted AI is less likely to overfit than pure-ML based decision systems."
Underlying the use of more "traditional" AI models (mostly neural nets) is the assumption that, after a certain threshold of training, computers will be able to make *better* decisions about our data in a way that is essentially foreign to us.
Improving accuracy with document automation solutions is more than just a technical endeavor. It requires a level of holistic thinking that is centered on agility and overall business effectiveness.
Automation is all about outcomes and working smarter. By being realistic and pragmatic about accuracy, centering design around humans, and striving for transparency above perfection, you will guarantee success with your chosen document automation solution.
