How to build processes with AI to save your team time and headaches

Are we ready for AI?

Document the steps

Before you even start building and AI workflow, you need to do it manually for a little while. Document everything you do, capturing the data sources, noting all the steps you take to complete the process, and also what sort of exceptions and weird cases you’re running into.

Human in the loop

At this stage, your team is building workflows that can do some of the manual work for you, but you still need to review and approve decisions. This is the first step that actually involves AI. The AI is just removing some of the dumber steps like doing a google search or clicking a button, and it gets the information you need in front of you sooner. At this stage, you’ve unlocked real leverage for your subject matter experts. Their time is freed up to review more data faster.

Human on the loop

At this step, you have built the workflow, and you have given the AI enough information to understand your decision making process in handling data, that it can make a choice and close the loop on its own. You need to make sure that the agents or workflows also document their process so that you can audit it.

With a human on the loop, your subject matter experts can take a mathematically representative sample of the data to review. You and your team can work together to determine what error rate you deem acceptable based on your use case. If the error rate from your sample is within an acceptable margin, then you can let the loop continue to run. You can always update your workflows with more exceptions as you run into them.

This is also where you start to move to “over the loop”. As you continue to refine these workflows, you can remove more and more manual intervention.

Human over the loop

Most people conflate “on the loop” and “over the loop”, but I think there’s a distinguishing feature that separates them into 2 different steps.

The final step, as far as I see it, is human over the loop. At this point, the AI processes run in the background without oversight. The process is set up, the evaluations are always reporting an acceptable margin of error, and there are automated triggers to send error messages if the process breaks down. The only time a human needs to look at the process after that, is if something breaks. Your automated error messages will alert a subject matter expert that something has gone wrong, they can assess the error logs, determine where the loop broke, and what needs to be fixed.

Wrapping up

One of the things people get concerned about when these sorts of automations come in is “won’t I just automate away my job?” Over the last month I’ve been talking to a lot of companies working on solving curation with AI. Half of them think the models will be so good a year from now that we won’t need human curation of data anymore. The other half think about it the same way that I do. The models can get better and better, but at the end of the day, they still don’t actually know anything. They are trained on the data that is available, and with all the conflicting information out there, the models can only reach a certain level. At the end of the day, AI is not a replacement for human expertise, but a force multiplier.