papersTODAY 04:00 UTC
SMetric: Session-centric Scheduling Proposed for LLM Agent Serving
A new arXiv paper argues that existing LLM scheduling designs were built around human users and may not suit agentic workloads, where software agents issue the requests. The authors note that agents consume far more tokens per request and exhibit different traffic patterns than people. They propose SMetric, a session-centric scheduling approach intended to balance serving performance under these agent-driven conditions.