Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning

A new arXiv paper examines permutation surrogate tests, a common tool for estimating directed influence between reinforcement learning agents, by validating them against known ground truth. In two multi-agent settings, a social dilemma and a coordination race, the authors find that non-stationarity in agent behavior undermines these surrogate methods. The study provides diagnostics and corrective approaches to make information-theoretic influence measures more reliable.