Are modern recommendation systems treated like a reinforcement learning problem, with a sum of discounted future rewards, or as strictly single step transactions? Many products do significant offline data analysis on actions taken to inform changes, but it seems under appreciated how much more powerful making policy changes on a live, massively parallel set of independent environments/users is. Offline RL is fundamentally harder than online RL — you have to guard against bootstrapping into an optimistic fantasy untested by reality.