Researchers introduce Tabby, a long-context probabilistic foundation model designed for time series data, built on an encoder-only patch Transformer architecture. The release includes a fully open account of the pretraining process, covering the decisions and components behind the model's construction. The work aims to make time series foundation model development more reproducible and accessible.
A new arXiv preprint introduces OpenAI4S, a system that frames computational research by AI co-scientists as sessions in which code serves as the action taken at each step. The authors argue that long-running studies require persistent computational state and provenance so that workflows remain inspectable, resumable and reproducible.
A new arXiv paper examines how reporting choices, hyperparameter tuning, and dataset characteristics affect offline policy learning results. Drawing on roughly 160,000 training runs, the authors argue that reliable progress requires careful reporting, well-tuned baselines, and evaluation across varied conditions. The work offers practical guidance for making policy-learning benchmarks more reproducible and comparable.
A new arXiv paper argues that deep learning research on automated emotion recognition from electrocardiogram signals is hard to compare because studies differ in preprocessing, training and evaluation setups. The authors present a unified evaluation framework intended to allow fairer, more direct comparison between architectures. The work is a cross-listing on arXiv's machine learning section.
Researchers released vla-eval, an evaluation harness designed to simplify how vision-language-action models are tested across multiple simulation benchmarks. The tool addresses the friction of conflicting dependencies and inconsistent evaluation protocols that arise when benchmarks are combined in a single pipeline. It aims to make VLA evaluation more reproducible and easier to extend with new benchmarks.
A new arXiv survey addresses the lack of a standard definition for the term "agent" in AI research, which the authors say hinders evaluation, comparison, and reproducibility. The paper organizes agenticness along five dimensions and compiles related criteria, metrics, and benchmarks.
A new audit examines 263 publicly released checkpoints that use batch normalization and finds that reported unlearning results shift depending on which version of those statistics is used. Because batch-norm statistics are not produced by gradient updates and are rarely documented in model releases, refitting them on retained data can change the numbers an evaluation relies on. The authors argue that this makes some unlearning verdicts unreliable, since apparent forgetting may reflect checkpoint bookkeeping rather than the removed data genuinely being gone.
Researchers have introduced SEA-LION-Embedding, a set of text embedding models built for Southeast Asian languages and released with open, documented training resources. The work addresses a persistent gap in the field, where leading embedding models cannot be independently reproduced because their training corpora remain private. The release aims to support reliable performance on downstream tasks across the region's many languages.
A research paper on arXiv introduces a method for scoring text against evaluation rubrics using large language models, addressing how black-box judge models can apply identical criteria in inconsistent ways. The approach ties each score to concrete evidence drawn from the evaluated text, making the reasoning behind judgments easier to audit and reproduce. The work is cross-listed under arXiv categories for artificial intelligence, computational linguistics, and machine learning.
A new arXiv paper introduces XAI-Arena, a study of whether large language models can reliably evaluate explanations produced by explainable AI methods. The authors note that current evaluation relies heavily on subjective human judgment, which hurts reproducibility, scalability, and comparability across studies. The work explores automated, LLM-based assessment as a potential alternative to manual expert reviews.
A newly posted arXiv paper asks whether AI agents can identify and repair artifact drift that occurs during network experiments. The work situates this question within the network systems community's early efforts to deploy agentic AI for multi-step tasks in operational and experimental environments. It examines how far current agent capabilities extend in such hands-on settings.
A new research paper introduces an evaluation framework designed to test foundation models across text, image, video, and audio within a single unified pipeline. The work addresses the problem that current modality-specific toolkits rely on incompatible inference engines, prompt conventions, and metric implementations. The system aims to make omni-modal benchmarking composable and reproducible.
An arXiv paper argues that benchmark scores by themselves are not sufficient evidence that AI research agents have made genuine scientific discoveries. The authors introduce the Discovery Certification Protocol, which converts an agent's claimed results into executable recovery and feedback tests that can be independently run. The first of its gates checks that reported results can actually be reproduced from the recorded evidence.