Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts

A new arXiv paper tackles a limitation in Mixture-of-Experts models, which typically activate a fixed number of experts for each input. The authors propose an inference method that remains distribution-consistent when the number of active experts varies dynamically. The approach aims to preserve efficient inference in large foundation models while allowing more flexible expert routing.