arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Izzy Grosof

9 papers · 2025–2026

2025: 22026: 7
2025peak 7 in 20262026

Frequent co-authors

Papers

showing 1–9 of 9 RSS

Tandem Queues with Hybrid Abandonment in Heavy Traffic

Cameron Curtis, Izzy Grosof, Seyed Iravani

cs.PF · · cited 0× · arXiv ↗ · more like this ·

Hybrid-abandonment systems, where jobs abandon from certain stages of service but not others, occur frequently in medical, service, and computing systems. Abandonment affects the heavy-traffic behavior of both the stages with and without abandonment. In contrast to existing heavy-traffic works that focus on cases where there is abandonment at either all stages or no stages, we focus on the heavy-traffic behavior for these hybrid-abandonment systems. We provide the first heavy-traffic characterization of the second queue length in a two-stage tandem queue with abandonment only in the first stage. We characterize the scaled mean of the second stage queue length and prove that the scaled second queue length is asymptotically exponentially distributed and asymptotically independent of the first stage queue length. We derive our results by analyzing Markovian arrival process queues with…

Non-preemptive Datacenter Scheduling via Scaling Cycles

Zhongrui Chen, Heyuan Yao, Izzy Grosof +1 more

Information Systems · ACM SIGMETRICS Performance Evaluation Review · · cited 0× · DOI ↗ · more like this ·

Modern data center servers process multiple jobs in parallel to improve performance. However, each job demands some subset of a server's resources (e.g., CPUs, memory, storage), and a set of jobs can run in parallel only if there are sufficient computational resources to meet each job's needs. Given a stream of arriving jobs, a scheduling policy must choose a set of jobs to run in parallel at every moment in time. While prior work has studied the stability and mean response time of various scheduling policies in this setting, the vast majority of this work assumes that jobs can be preempted at any time with no overhead. In practice, however, datacenter jobs accumulate significant state as they run, making preemption costly or even impossible. In these limited preemption scenarios, little is known about the optimal way to schedule jobs and avoid excessive preemption overhead. We consider…

Optimal Scheduling in Generalized Switch in Heavy Traffic

Runhan Xie, Ziv Scully, Rhonda Righter +1 more

cs.PF · · cited 0× · arXiv ↗ · more like this ·

The generalized switch is a highly flexible queueing model, covering multiclass, multiserver, and multiresource queueing systems as well as a wide variety of stochastic networks. Although many scheduling policies have been developed for this model, they almost entirely address unknown job duration settings. How to optimally use known job durations in the generalized switch has remained open. Moreover, optimizing mean response time remains open in both settings. We introduce the first policy to guarantee heavy-traffic optimal mean response time in the generalized switch, our Smallest Equalizing Bucket (SEB) policy. The key challenge in designing an optimal scheduling policy is that we must simultaneously prioritize small jobs and also minimize resource waste, all while fitting within the generalized switch's service options. SEB overcomes this challenge by grouping jobs into…

Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation

Heyuan Yao, Chutong Gao, Yuan Lyu +2 more

cs.PF · · cited 0× · arXiv ↗ · more like this ·

The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated based on the whole conversation history across all previous turns. The hit ratio, i.e., the average fraction of KV caches accessed directly from existing caches stored in high-bandwidth memory (HBM), is hence a crucial metric that governs system performance. Estimating the hit ratio is a highly nontrivial task due to the complex system dynamics, where the KV cache prefixes grow with turns and some must be evicted due to finite memory capacity. We formulate the system as a multi-turn conversation model under the least-recently-used (LRU) policy. Through a mean-field asymptotic framework, we prove that as the conversation arrival rate and the memory capacity grow proportionally to infinity, the hit ratio converges…

Pareto-Optimal Scheduling in the Half-batch Multiserver-job Model

Ziyuan Wang, Izzy Grosof

cs.PF · · arXiv ↗ · more like this ·

In large-scale computing systems, jobs often demand heterogeneous server allocations: large jobs that occupy a substantial fraction of the servers are of high importance and are thus latency-sensitive, while small jobs fill in the remaining capacity to maintain throughput. To model this dynamic, we introduce the half-batch multiserver-job (MSJ) framework, a queueing model in which large jobs arrive according to a Poisson process and require all servers simultaneously, while small jobs, each needing only one server, are always available. We prove that, in the half-batch MSJ model, the Pareto frontier for large-job mean response time and small-job throughput admits a simple and exact characterization. It is generated by a family of convoy policies, under which the system serves small jobs until $k$ large jobs have arrived and then switches to serving large jobs, together with convex…

Throughput-Optimal Multiresource-Job Scheduling with Continuous Requirement Distribution

Heyuan Yao, Willow Kowalik, Izzy Grosof

cs.PF · · arXiv ↗ · more like this ·

Modern computing systems process jobs with resource requirements such as CPU and memory, which are described by multiresource jobs (MRJ) queueing models. In practice, job resource requirements are spread out over so many values, that it is rare to see the same value twice. This pattern is best modeled by a continuous distribution of requirement values. However, the existing theoretical work on stability or throughput-optimality focuses on queueing models with class-based resource requirements. In class-based models, the number of distinct resource requirements must be small to demonstrate strong empirical performance, making them a poor match for these practical systems. We introduce the first throughput-optimal family of scheduling policies for the continuous MRJ model, with both preemptive and nonpreemptive variants. We further introduce several efficient policy families, which remain…

Markov Modulated JSQ in Heavy Traffic Via the Poisson Equation

Daniela Hurtado-Lange, Izzy Grosof

math.PR · · arXiv ↗ · more like this ·

In parallel-server systems with a single stream of arrivals (a.k.a. load balancing), Join-the-Shortest-Queue (JSQ) is a popular routing algorithm. There is extensive literature studying this system in various asymptotic regimes, but all assume constant parameters (arrival and service rates). We study the JSQ system with Markov-modulated parameters and heterogeneous servers. Our main contributions are: (i) We compute the heavy-traffic distribution of the scaled vector of queue lengths; (ii) We utilize a novel hybrid methodology that combines the Transform Method for queue-lengths analysis and the Poisson equation; (iii) We provide sufficient conditions to ensure state space collapse, showing provable balancing power of JSQ for heterogeneous servers. Unlike other studies involving Markov-modulated queues, these conditions don't depend on the mixing time of the modulating chain and are…

Outperforming Multiserver SRPT at All Loads

Izzy Grosof, Daniela Hurtado-Lange

cs.PF · · arXiv ↗ · more like this ·

A well-designed scheduling policy can unlock significant performance improvements with no additional resources. Multiserver SRPT (SRPT-k) is known to achieve asymptotically optimal mean response time in the heavy-traffic limit, as load approaches capacity. No better policy is known for the M/G/k queue in any regime. We introduce a new policy, SRPT-Except-k+1 & Modified SRPT (SEK-SMOD), which is the first policy to provably achieve lower mean response time than SRPT-k. SEK-SMOD outperforms SRPT-k across all loads and all job size distributions. The key idea behind SEK-SMOD is to prioritize large jobs over small jobs in specific scenarios to improve server utilization, and thereby improve the response time of subsequent jobs in expectation. Our proof is a novel application of hybrid worst-case and stochastic techniques to relative analysis, where we analyze the deviations of our proposed…

Novel Lower Bounds on M/G/k Scheduling

Ziyuan Wang, Izzy Grosof

cs.PF · · arXiv ↗ · more like this ·

In queueing systems, effective scheduling algorithms are essential for optimizing performance. Optimal scheduling for the M/G/k queue has been explored in the heavy traffic limit, but much remains unknown in the intermediate load regime. In this paper, we give the first framework for proving nontrivial lower bounds on the mean response time of the M/G/k system under arbitrary scheduling policies. Our bounds tighten previous naive lower bounds by more than 60\%, yielding significant improvements particularly for moderate loads. Key to our approach is a new variable-speed queue, which more accurately captures the work completion behavior of multiserver systems. To analyze the expected work of this queue, we develop a novel manner of employing the drift method or the BAR approach, by developing test functions via the solutions to a differential equation. We validate our results numerically…