Causal Transformer for Estimating Counterfactual Outcomes

Melnychuk, Valentyn; Frauen, Dennis und Feuerriegel, Stefan (2022): Causal Transformer for Estimating Counterfactual Outcomes. ICML 2022, Baltimore, 17.07.2022-23.07.2022. arXiv.

Abstract

Estimating counterfactual outcomes over time from observational data is relevant for many applications (e.g., personalized medicine). Yet, state-of-the-art methods build upon simple long short-term memory (LSTM) networks, thus rendering inferences for complex, long-range dependencies challenging. In this paper, we develop a novel Causal Transformer for estimating counterfactual outcomes over time. Our model is specifically designed to capture complex, long-range dependencies among time-varying confounders. For this, we combine three transformer subnetworks with separate inputs for time-varying covariates, previous treatments, and previous outcomes into a joint network with in-between cross-attentions. We further develop a custom, end-to-end training procedure for our Causal Transformer. Specifically, we propose a novel counterfactual domain confusion loss to address confounding bias: it aims to learn adversarial balanced representations, so that they are predictive of the next outcome but non-predictive of the current treatment assignment. We evaluate our Causal Transformer based on synthetic and real-world datasets, where it achieves superior performance over current baselines. To the best of our knowledge, this is the first work proposing transformer-based architecture for estimating counterfactual outcomes from longitudinal data.

• Dokumententyp: Konferenzbeitrag (Paper)
• Keywords: Machine Learning (cs.LG); Machine Learning (stat.ML); FOS: Computer and information sciences; FOS: Computer and information sciences; Artificial Intelligence; AI, Künstliche Intelligenz; KI
• Fakultät: Betriebswirtschaft > Institute of Artificial Intelligence (AI) in Management
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme
• Sprache: Englisch