Labels masked by our model actions – Practical solutions and theoretical dreams
Labels masked by our model actions – Practical solutions and theoretical dreams
Sharon is a Principal Data Scientist at PayPal, leading the machine learning modeling guild of the fraud detection group. She develops models for fraud detection, which are at the core of PayPal’s risk system, affecting millions of PayPal users daily. Sharon also co-hosts PayPal’s internal global Data Science Podcast. She holds a BSc and an MSc in Industrial Engineering from Ben Gurion University and Tel Aviv University.
Sharon is a Principal Data Scientist at PayPal, leading the machine learning modeling guild of the fraud detection group. She develops models for fraud detection, which are at the core of PayPal’s risk system, affecting millions of PayPal users daily. Sharon also co-hosts PayPal’s internal global Data Science Podcast. She holds a BSc and an MSc in Industrial Engineering from Ben Gurion University and Tel Aviv University.
After we release the first version of a model to production, our population may change according to the new actions driven by our model. In many cases, these actions affect our label. For example, if we stop a suspicious transaction, we will never know if the transaction was indeed fraudulent as it never took place. However, we want to train a new model version to be able to detect those suspicious transactions as well, so it will replace the existing one. Some format of A/B testing or a control group can be an excellent solution to solve this challenge, however in many cases, and it is not possible due to business, ethical, technical, or other reasons. In this round table, we will discuss the different implications of this issue. We’ll share our practical solutions and theoretical dreams on dealing with this challenge in our day to day, including transforming part of the problem to a regression problem, using self-learning, weak supervision, and other semi-supervised methods, and investigating the trade-offs of each method.
After we release the first version of a model to production, our population may change according to the new actions driven by our model. In many cases, these actions affect our label. For example, if we stop a suspicious transaction, we will never know if the transaction was indeed fraudulent as it never took place. However, we want to train a new model version to be able to detect those suspicious transactions as well, so it will replace the existing one. Some format of A/B testing or a control group can be an excellent solution to solve this challenge, however in many cases, and it is not possible due to business, ethical, technical, or other reasons. In this round table, we will discuss the different implications of this issue. We’ll share our practical solutions and theoretical dreams on dealing with this challenge in our day to day, including transforming part of the problem to a regression problem, using self-learning, weak supervision, and other semi-supervised methods, and investigating the trade-offs of each method.
| 8:45 | Reception |
|---|---|
| 9:30 | Opening words by WiDS TLV ambassadors Or Basson and Noah Eyal Altman |
| 9:40 | Dr. Kira Radinski - Learning to predict the future of healthcare |
| 10:10 | Prof. Yonina Eldar - Model-Based Deep Learning: Applications to Imaging and Communications |
| 10:40 | Break |
| 10:50 | Lightning talks |
| 12:20 | Lunch & Poster session |
|---|---|
| 13:20 | Roundtable session & Poster session |
| 14:05 | Roundtable closure |
| 14:20 | Break |
| 14:30 | Dr. Anna Levant - 3D Metrology: Seeing the Unseen |
| 15:00 | Aviv Ben-Arie - Counterfactual Explanations: The Future of Explainable AI? |
| 15:30 | Closing remarks |
| 15:40 | End |
© 2018-2022 WiDS TLV – Intuit. All rights reserved.
Scotty – By Nir Azoulay
Design: Sharon Geva