An article co-written by McIntire Information Technology Professor Sarah Lebovitz, “The No. 1 Question to Ask When Evaluating AI Tools,” was recently published in the spring issue of MIT Sloan Management Review.
Drawing from her research paper “Is AI Ground Truth Really True? The Dangers of Training and Evaluating AI Tools Based on Experts’ Know-What,” co-authored with Professor Natalia Levina of New York University and Professor Hila Lifshitz-Assaf of Warwick Business School, the article discusses questions used in determining if an AI solution is worth implementing based on evaluating the “ground truth” on which the AI has been trained and validated.
Lebovitz and her colleagues explain that they spent 11 months observing managers in a leading healthcare organization, conducting pilot studies of five AI tools claiming to make highly accurate critical predictions. By analyzing the organization’s product evaluation process, the researchers discovered an effective way to evaluate an AI tool’s quality through its ground truth, information known to be true based on objective, empirical evidence.
According to Lebovitz and her co-authors, “Ground truth refers to the data in training data sets that teaches an algorithm how to arrive at a predicted output,” and is considered to be the “correct” answer to a prediction problem that an AI tool is learning to solve. “This data set then becomes the standard against which developers measure the accuracy of the system’s predictions,” they write, pointing to the surprising lack of conversation about the quality of the labels that serve as the ground truth for model development.
The article then explores questions concerning objective, subjective, and externally verifiable decision contexts of ground truth, weighing them against “the AI developers’ methods for determining the ground truth used to train and validate the algorithm,” and implores managers reviewing AI products to ask, “Is the ground truth really true?”
Read Lebovitz’s article in MIT Sloan Management Review.
Watch Lebovitz’s explanation about ground truth below:
