Pull Router Cuts Token Costs in Long LLM Conversations via Lazy Memory Loading
A new arXiv paper proposes Pull, a session router that keeps conversation history in an addressable store instead of injecting all prior turns into the context window. Rather than summarizing or truncating old dialogue, the system materializes only the historical state needed at each step, aiming to avoid the quadratic token growth of full-context prompting without permanently losing information. The approach targets multi-hundred-turn, stateful LLM sessions.