Why Human-In-The-Loop Computing Is The Future Of Machine Learning

November 16th, 2015

Via: ComputerWorld:

Now that machine learning is becoming more and more mainstream, some design patterns are starting to emerge. As the CEO of CrowdFlower, I’ve worked with many companies building machine learning algorithms and I’ve noticed a best practice in nearly every successful deployment of machine learning on tough business problems. That practice is called “human-in-the-loop” computing. Here’s how it works:

First, a machine learning model takes a first pass on the data, or every video, image or document that needs labeling. That model also assigns a confidence score, or how sure the algorithm is that it’s making the right judgment. If the confidence score is below a certain value, it sends the data to a human annotator to make a judgment. That new human judgment is used both for the business process and is fed back into the machine learning algorithm to make it smarter. In other words, when the machine isn’t sure what the answer is, it relies on a human, then adds that human judgment to its model.

This simple pattern is at the heart of many well known, real-world use-cases of machine learning. And it solves one of the biggest issues with machine learning, namely: it’s often very easy to get an algorithm to 80 percent accuracy but near impossible to get an algorithm to 99 percent. The best machine learning lets humans handle that 20 percent since 80 percent accuracy is simply not good enough for most real-world applications.

One Response to “Why Human-In-The-Loop Computing Is The Future Of Machine Learning”

  1. Miraculix says:

    Which translates (roughly) to :

    Autonomous machines will NEVER possess actual compassion, making them a perfect fot for most military applications, where following orders from up the hierarchical chain is paramount. Followed immediately by peerless physical performance. Ever more agile robotics and piles of R&D cash pretty much guarantee a major “staffing” shift in combat-related tasks. How soon is still an open question, but I’m guessing sooner than later along the present trajectory.

    Meanwhile, “Ghost in the Shell” style bio-mechanical enhancement is Strangelove’s wet dream come to life. Only the successful welding of man & machine will enable the true “super-soldier” hinted at for nearly a century now everywhere from sci-fi and pulp comics to the pages of the Wall Street Journal. Who knew James Cameron would turn out to be a reliable soothsayer (of sorts)?

    Honestly, I’m glad I probably won’t live long enough to some DARPA-funded high-tech version of his nightmarishly shiny killing machines patrolling earth and sky 24/7.

Leave a Reply

You must be logged in to post a comment.