Shunit Agmon

Clinical research as a corpus contains years’ worth of biases, one of the most prominent being the longstanding underrepresentation of women in clinical trials. Even in obstetrics and gynecology, where the research subjects are only women, there is a bias in the topics chosen for research: they focus more on the reproductive role of women, the successful completion of pregnancy and fetus health, rather than on women’s health.

Machine-learning models trained on clinical trial abstracts may capture and amplify biases in the data. Specifically, word embeddings are models that enable representing words as vectors and are the building block of most natural language processing systems. If word embeddings are trained on clinical trial abstracts, predictive models that use the embeddings will exhibit gender performance gaps.

However, positive trends exist in clinical research, in the increase of women inclusion and in the general quality and practices used. I will present TeDi-BERT, a method to capture these temporal trends through temporal distribution matching on contextual word embeddings (specifically, BERT) and explore its effect on the bias manifested in downstream tasks.
Temporal distribution matching is implemented through an adversarial classifier, trying to distinguish old from new clinical trial abstracts based on their embeddings. The temporal distribution matching acts as a form of domain adaptation from older to more recent clinical trials. I will show an evaluation of our model on two clinical tasks: prediction of unplanned readmission to the intensive care unit, and hospital length of stay prediction.

In both clinical tasks, TeDi-BERT improved performance for female patients, as expected; but it also improved performance for male patients. Our results show that accuracy for one gender does not need to be exchanged for bias reduction, but rather that good science improves clinical results for all. Contextual word embedding models trained to capture temporal trends can help mitigate the effects of bias that changes over time in the training data.