A new arXiv paper introduces CoArena, a benchmark that evaluates computer-use and multi-agent systems continuously rather than against a fixed task set. The authors argue static benchmarks age, leak into training data, and stop reflecting real capability. CoArena instead refreshes its tasks in real time to keep evaluation meaningful.
A new arXiv paper argues that when datasets are built by running a detector, heuristic, or model over candidate pools, the resulting label precision depends on the true-positive rate within each pool rather than on detector quality alone. The authors apply Bayes' rule to show how this prevalence effect introduces hidden contamination into detector-defined datasets, a problem they describe as silent. The work suggests dataset builders should account for pool-level prevalence when estimating or reporting precision.
A new study questions whether time-series foundation models can be fairly evaluated using test data collected after their pretraining cutoff. It finds that even a temporally later, contamination-free hold-out does not fully isolate genuine generalization, as familiarity with the underlying data distribution absorbed during pretraining persists. The result suggests the field needs evaluation practices that go beyond simply withholding recent data.