Learning a causal model of how an intervention (action) changes the state of a system is crucial for decision-making. When we only have observational (offline) data, learning such a model is challenging due to the presence of confounders - variables that can bias the estimated effects of actions on state transitions. In this work, we leverage the potential outcome causal inference framework to control for the confounding variables and learn a robot motion model for downstream decision-making. This work is accepted by ICRA 2023 [Paper].