A new arXiv preprint examines whether decoder-only language models built with mixture-of-experts layers can serve as effective first-stage retrievers. The authors note that retrieval quality typically rises with backbone size, but dense models force every query and document through all parameters. The work targets strong retrieval performance while limiting the computational cost per query.
A new arXiv paper proposes a decoding-control method for diffusion vision-language models that uses intermediate answer trajectories to decide how much reasoning a query actually needs. The approach targets a mismatch in which models apply reasoning effort that does not match the difficulty of the task, aiming to make inference more efficient and better calibrated. The work is presented as a preprint and has not yet been peer reviewed.
Multimodal large language models spend significant compute processing large numbers of visual tokens, and effective pruning depends on knowing which tokens actually matter. This paper introduces a learned approach that predicts attention at middle layers, enabling models to identify and drop less relevant visual tokens. The method aims to cut inference costs while maintaining performance across vision-language tasks.