No data set can represent the entire universe of options. Thus, it is important to identify the target application and audience upfront, and then tailor the training data to that target. Another possible approach could be to train multiple versions of the algorithm, each of which is trained to input a dataset and classify it, then repeat this for all datasets that are available. If the output from classification is the same between models, then the AI models can be combined
Dr. John Halamka, president of Mayo Clinic Platform, speaking at HIMSS in 2021, spoke to certain steps that can start to address some of the bias issues: “The AI algorithms are only as good as the underlying data,” . “And yet, we don’t publish statistics describing how these algorithms are developed.” The solution, he said, is greater transparency – spelling out and sharing via technology the ethnicity, race, gender, education, income and other details that go into an algorithm.
The solution, he said, is greater transparency – spelling out and sharing via technology the ethnicity, race, gender, education, income and other details that go into an algorithm.
As AI systems continue to be used, one tailored design is to update their training dataset so that they are increasingly tailored to their user base. This can introduce unintended consequences. First, as the AI becomes more and more tailored to the user base, this may introduce bias compared to the carefully curated data often used originally for training. Second, the system may become less accurate over time because the oversight used to ensure AI accuracy may no longer be in place in the real world. A good example of this is the Microsoft ChatBot, which was designed to be a friendly companion but, on release, rapidly learned undesirable language and behaviors, and had to be shut down.
There are multiple approaches to eliminate bias in AI, and none are foolproof (Figure.) These range from approaches to formulate an application so that it is relatively free of bias, to collecting data in a relatively unbiased way, to designing mathematical algorithms to minimize bias. One approach is to do external validation of AI models before they are deployed into the real-world and used in a clinical setting. This could be in the form of using large and diverse datasets to test the claims in new studies submitted for publication or regulatory approval and carefully test the models’ performance on the external data set.
There needs to be an element of governance and peer review for all algorithms, as even the most solid and tested algorithm is bound to have unexpected results arise. An algorithm is never done learning – it must be constantly developed and fed more data to improve. Companies should provide answers to key questions, such as ‘How was the algorithm trained? On what basis did it draw this conclusion?’ An algorithm should be interrogated under both common and rare scenarios with varied populations before it’s introduced to real-world situations.
