Beware the “Expert”

The old saw that an expert is “somebody who tells you why something cannot be done” is truer
than ever before. History has shown that the best inventions or solutions rarely come from experts;
they almost always come from outsiders. That is, from people who aren’t domain experts but who
offer a fresh perspective.

When Kaggle runs a competition, it has found that the first responders are experts in a particular
domain who say, “We know this industry, we’ve done this before and we’ll figure it out.” And just as
inevitably, within two weeks, complete newcomers to the field trounce their best results. For
example, the Hewlett Foundation sponsored a 2012 competition to develop an automated scoring
algorithm for student-written essays. Of the 155 teams competing, three were awarded a total of
$100,000 in prize money. What was particularly interesting was the fact that none of the winners had
prior experience with natural language processing (NLP). Nonetheless, they beat the experts, many of
them with decades of experience in NLP under their belts.