Reinforcement Learning

Aznaur Aliev

M.S. Student

We used Reinforcement Learning; but did it work?

Prof. Susan Murphy, Statistics and Computer Science and Radcliffe Alumnae Professor at the Radcliffe Institute, Harvard University

-

B9 L2 H2

Reinforcement Learning provides an attractive suite of online learning methods for personalizing interventions in Digital Behavioral Health. However, after a reinforcement learning algorithm has been run in a clinical study, how do we assess whether personalization occurred? We might find users for whom it appears that the algorithm has indeed learned in which contexts the user is more responsive to a particular intervention. But could this have happened completely by chance? We discuss some first approaches to addressing these questions.

Inference for Longitudinal Data After Adaptive Sampling

Prof. Susan Murphy, Statistics and Computer Science and Radcliffe Alumnae Professor at the Radcliffe Institute, Harvard University

-

B9 L2 H2

In this work, we proved statistical inference for the common Z-estimator based on adaptively sampled data. Adaptive sampling methods, such as reinforcement learning (RL) and bandit algorithms, are increasingly used for the real-time personalization of interventions in digital applications like mobile health and education. As a result, there is a need to be able to use the resulting adaptively collected user data to address a variety of inferential questions, including questions about time-varying causal effects.