arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Shwartz

23 papers · 1985–2025

1985: 11986: 11987: 01988: 01989: 21990: 01991: 11992: 01993: 21994: 01995: 11996: 11997: 11998: 01999: 02000: 02001: 12002: 02003: 12004: 02005: 22006: 02007: 52008: 02009: 12010: 12011: 02012: 12013: 02014: 02015: 02016: 02017: 02018: 02019: 02020: 02021: 02022: 02023: 02024: 02025: 1
1985peak 5 in 20072025

Frequent co-authors

Papers

showing 1–23 of 23 RSS

Exact finite approximations of average-cost countable Markov Decision Processes

Arie Leizarowitz, Adam Shwartz

math.PR · · cited 836× · arXiv ↗ · more like this ·

For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the original process when restricting to the approximating set. The embedded process can be used as an approximation which, being finite, is more convenient for computation and implementation.

Constrained Discounted Dynamic Programming

Eugene A. Feinberg, Adam Shwartz

Artificial Intelligence · Mathematics of Operations Research · · cited 92× · DOI ↗ · more like this ·

This paper deals with constrained optimization of Markov Decision Processes with a countable state space, compact action sets, continuous transition probabilities, and upper semicontinuous reward functions. The objective is to maximize the expected total discounted reward for one reward function, under several inequality constraints on similar criteria with other reward functions. Suppose a feasible policy exists for a problem with M constraints. We prove two results on the existence and structure of optimal policies. First, we show that there exists a randomized stationary optimal policy which requires at most M actions more than a nonrandomized stationary one. This result is known for several particular cases. Second, we prove that there exists an optimal policy which is (i) stationary (nonrandomized) from some step onward, (ii) randomized Markov before this step, but the total number…

A Branching Random Walk with a Barrier

J. D. Biggins, Boris D. Lubachevsky, Adam Shwartz +1 more

Mathematical Physics · The Annals of Applied Probability · · cited 43× · DOI ↗ · more like this ·

Suppose that a child is likely to be weaker than its parent and a child who is too weak will not reproduce. What is the condition for a family line to survive? Let $b$ denote the mean number of children a viable parent will have; we suppose that this is independent of strength as long as strength is positive. Let $F$ denote the distribution of the change in strength from parent to child, and define $h = \sup_\theta(-\log(\int e^{\theta t} dF(t)))$. We show that the situation is black or white: 1. If $b < e^h, \text{then} P(\text{family line dies}) = 1$. 2. If $b > e^h, \text{then} P(\text{family survives}) > 0$. Define $f(x) := E(\text{number of members in the family} \mid \text{initial strength} x)$. We show that if $b < e^h$, then there exists a positive constant $C$ such that $\lim_{x \rightarrow \infty}e^{- \alpha x}f(x) = C$, where $\alpha$ is the smaller of the (at most) two…

Optimal priority assignment: a time sharing approach

Eitan Altman, Adam Shwartz

Computer Networks and Communications · IEEE Transactions on Automatic Control · · cited 40× · DOI ↗ · more like this ·

Nonstationary time-sharing policies are introduced to obtain optimal controls for new constrained optimization problems. The criteria are expected time averages of sizes of the queues. These policies and their cost are computed through linear programs. The achievable region of the vector of queues' length is characterized. Other applications of time-sharing policies are discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Induced rare events: analysis via large deviations and time reversal

Adam Shwartz, Alan Weiss

Mathematical Physics · Advances in Applied Probability · · cited 29× · DOI ↗ · more like this ·

When a subsystem goes into an infrequent state, how does the remainder of the system behave? We show how to calculate the relevant distributions using the notions of reversed time for Markov processes and large deviations. For ease of exposition, most of the work deals with a specific queueing model due to Flatto, Hahn, and Wright. However, we show how the theorems may be applied to much more general jump-Markov systems. We also show how the tools of time-reversal and large deviations complement each other to yield general theorems. We show that the way a constant coefficient process approaches a rare event is roughly by following the path of another constant coefficient process. We also obtain some properties, including a priori bounds, for the change of measure associated with some large deviations functionals; these are useful for accelerating simulations.

An escape time criterion for queueing networks: Asymptotic risk-sensitive control via differential games

Rami Atar, Paul Dupuis, Adam Shwartz

math.PR · · cited 28× · arXiv ↗ · more like this ·

We consider the problem of risk-sensitive control of a stochastic network. In controlling such a network, an escape time criterion can be useful if one wishes to regulate the occurrence of large buffers and buffer overflow. In this paper a risk-sensitive escape time criterion is formulated, which in comparison to the ordinary escape time criteria penalizes exits which occur on short time intervals more heavily. The properties of the risk-sensitive problem are studied in the large buffer limit, and related to the value of a deterministic differential game with constrained dynamics. We prove that the game has value, and that the value is the (viscosity) solution of a PDE. For a simple network, the value is computed, demonstrating the applicability of the approach.

Efficient routing in heavy traffic under partial sampling of service times

Rami Atar, Adam Shwartz

math.PR · · cited 15× · arXiv ↗ · more like this ·

We consider a queue with renewal arrivals and n exponential servers in the Halfin-Whitt heavy traffic regime, where n and the arrival rate increase without bound, so that a critical loading condition holds. Server k serves at rate $μ_k $, and the empirical distribution of the $μ_k $ is assumed to converge weakly. We show that very little information on the service rates is required for a routing mechanism to perform well. More precisely, we construct a routing mechanism that has access to a single sample from the service time distribution of each of $n$ to the power of $1/2 + ε$ randomly selected servers, but not to the actual values of the service rates, the performance of which is asymptotically as good as the best among mechanisms that have the complete information on $ μ_k $.

On constrained optimization of the Klimov network and related Markov decision processes

Armand M. Makowski, Adam Shwartz

Statistics and Probability · IEEE Transactions on Automatic Control · · cited 14× · DOI ↗ · more like this ·

A constrained version of the server allocation problem for the Klimov network is solved, and it is established that the optimal constrained schedule is obtained by randomizing between two fixed-priority schemes. This generalizes the work of P. Nain and K.W. Ross (1986) in the context of the competing queue problem and also covers the discounted cost case. In order to establish these results, a general framework is developed for optimization under a single constraint in the presence of indexlike policies. This methodology is in principle of wider applicability.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Stochastic Approximation in Hilbert Space: Identification and Optimization of Linear Continuous Parameter Systems

Harold J. Kushner, Adam Shwartz

Computational Theory and Mathematics · SIAM Journal on Control and Optimization · · cited 13× · DOI ↗ · more like this ·

We treat a class of stochastic approximations (with small constant gain $\varepsilon $), with values in a Hilbert space. The problem and algorithm arise, e.g., when one seeks to iteratively identify the transfer function of a linear system (continuous parameter) or to adaptively optimize the transfer function of a stochastic linear system. Weak convergence methods are used to prove convergence of the interpolated sequence as $\varepsilon \to 0$, and to characterize the equations satisfied by the limit. Projected and unprojected cases are dealt with. In one important case, convergence to a constrained optimum is proved when $\varepsilon n \to 0$ as $\varepsilon \to 0$. The normalized error sequence is analyzed. It is shown that the limit (as $\varepsilon \to 0$) of the interpolated normalized error sequence satisfies a linear integral equation driven by a Hilbert space Wiener process.…

Predicting the Impact of Measures Against P2P Networks: Transient Behavior and Phase Transition

Eitan Altman, Philippe Nain, Adam Shwartz +1 more

Computer Networks and Communications · IEEE/ACM Transactions on Networking · · cited 12× · DOI ↗ · more like this ·

The paper has two objectives. The first is to study rigorously the transient behavior of some peer-to-peer (P2P) networks whenever information is replicated and disseminated according to epidemic-like dynamics. The second is to use the insight gained from the previous analysis in order to predict how efficient are measures taken against P2P networks. We first introduce a stochastic model that extends a classical epidemic model and characterize the P2P swarm behavior in presence of free-riding peers. We then study a second model in which a peer initiates a contact with another peer chosen randomly. In both cases, the network is shown to exhibit phase transitions: A small change in the parameters causes a large change in the behavior of the network. We show, in particular, how phase transitions affect measures of content providers against P2P networks that distribute nonauthorized music…

A Strongly Polynomial Algorithm for Controlled Queues

Alexander Zadorojniy, Guy Even, Adam Shwartz

Computational Theory and Mathematics · Mathematics of Operations Research · · cited 10× · DOI ↗ · more like this ·

We consider the problem of computing optimal policies of finite-state finite-action Markov decision processes (MDPs). A reduction to a continuum of constrained MDPs (CMDPs) is presented such that the optimal policies for these CMDPs constitute a path in a graph defined over the deterministic policies. This path contains, in particular, an optimal policy of the original MDP. We present an algorithm based on this new approach that finds this path, and thus an optimal policy. In the general case, this path might be exponentially long in the number of states and actions. We prove that the length of this path is polynomial if the MDP satisfies a coupling property. Thus we obtain a strongly polynomial algorithm for MDP s that satisfies the coupling property. We prove that discrete time versions of controlled M/M/1 queues induce MDP s that satisfy the coupling property. The only previously…

Markov Decision Processes with Slow Scale Periodic Decisions

Michael J. Jacobson, Nahum Shimkin, Adam Shwartz

Artificial Intelligence · Mathematics of Operations Research · · cited 10× · DOI ↗ · more like this ·

We consider a class of discrete time, dynamic decision-making models which we refer to as Periodically Time-Inhomogeneous Markov Decision Processes (PTMDPs). In these models, the decision-making horizon can be partitioned into intervals, called slow scale cycles, of N + 1 epochs. The transition law and reward function are time-homogeneous over the first N epochs of each slow scale cycle, but distinct at the final epoch. The motivation for such models is in applications where decisions of different nature are taken at different time scales, i.e., many “low-level” decisions are made between less frequent “high-level” ones. For the PTMDP model, we consider the problem of optimizing the expected discounted reward when rewards devalue by a discount factor λ at the beginning of each slow scale cycle. When N is large, initially stationary policies (i.s.p.'s) are natural candidates for optimal…

Erasure, capture, and noise errors in controlled multiple-access networks

Adam Shwartz, Maawiya Ould Sidi

Computer Networks and Communications · IEEE Transactions on Communications · · cited 10× · DOI ↗ · more like this ·

An ALOHA-type communication system with many nodes accessing a common receiver through a time-slotted shared radio channel is considered. Due to topological and environmental conditions, the receiver is prone to fail to detect some or all of the packets transmitted in a slot; this phenomenon is called erasure. The receiver may also capture, that is, detect a single transmission out of many. In addition, noise errors may cause the receiver to detect nonexistent collisions. Using the feedback information regarding the detection of zero or one packet or a collision at the receiver, the nodes determine their transmission policy. A class of decentralized multiaccess algorithms that maintain system stability under the above phenomenon is presented, and the maximal throughput they can support is determined. The maximal throughput of these algorithms is insensitive to some noise errors and…

Predicting the Impact of Measures Against P2P Networks on the Transient Behaviors

Eitan Altman, Philippe Nain, Adam Shwartz +1 more

cs.NI · · cited 7× · arXiv ↗ · more like this ·

The paper has two objectives. The first is to study rigorously the transient behavior of some P2P networks whenever information is replicated and disseminated according to epidemic-like dynamics. The second is to use the insight gained from the previous analysis in order to predict how efficient are measures taken against peer-to-peer (P2P) networks. We first introduce a stochastic model which extends a classical epidemic model and characterize the P2P swarm behavior in presence of free riding peers. We then study a second model in which a peer initiates a contact with another peer chosen randomly. In both cases the network is shown to exhibit a phase transition: a small change in the parameters causes a large change in the behavior of the network. We show, in particular, how the phase transition affects measures that content provider networks may take against P2P networks that…

Death and discounting

Adam Shwartz

Artificial Intelligence · IEEE Transactions on Automatic Control · · cited 7× · DOI ↗ · more like this ·

Markov decision processes (MDP) with discounted cost are equivalent to processes with a finite-random duration, and, hence, the discount factor models a (random) time horizon for the life of the process. We elaborate on this idea, but show that an objective function which is a linear combination of several discounted costs (each with a different discount factor) does not, in general, model processes with several time scales, but rather processes with partial information.

Explicit solution for a network control problem in the large deviation regime

Rami Atar, Paul Dupuis, Adam Shwartz

math.PR · · cited 4× · arXiv ↗ · more like this ·

We consider optimal control of a stochastic network,where service is controlled to prevent buffer overflow. We use a risk-sensitive escape time criterion, which in comparison to the ordinary escape time criteria heavily penalizes exits which occur on short time intervals. A limit as the buffer sizes tend to infinity is considered. In [2] we showed that, for a large class of networks, the limit of the normalized cost agrees with the value function of a differential game. The game's value is characterized in [2] as the unique solution to a Hamilton-Jacobi-Bellman Partial Differential Equation (PDE). In the current paper we apply this general theory to the important case of a network of queues in tandem. Our main results are: (i) the construction of an explicit solution to the corresponding PDE, and (ii) drawing out the implications for optimal risk-sensitive and robust regulation of the…

Uniqueness of a constrained variational problem and large deviations of buffer size

Adam Shwartz, Alan Weiss

math.PR · · cited 1× · arXiv ↗ · more like this ·

We show global uniqueness of the solution to a class of constrained variational problems, using scaling properties. This is used to establish the essential uniqueness of solutions of a large deviations problem in multiple dimensions. The result is motivated by models of buffers, and in particular the probability of, and typical path to overflow in the limit of small buffers, which we analyze.

Minding the gender gap in advanced STEM courses: effects of student preparedness and activity level

Ilana Ram, Adam Shwartz, Ido Roll

Safety Research · Instructional Science · · cited 0× · DOI ↗ · more like this ·

Abstract The gender achievement gap in higher STEM education is well documented as STEM course grades often show a bias favoring men, even when controlling for overall student achievements. The current work, which was preregistered prior to data collection, replicates and extends prior work to evaluate several possible contributors to the gender gap in advanced STEM courses. The contributing factors include students’ STEM-preparedness, learning assessment design, and students’ activity level in the course. We analyzed the data of a cohort of 315 undergraduate engineering students in a leading technological university in Israel. A mixed-effect model analysis revealed a gender gap in both high-stakes and low-stakes assessments of learning. We found that the gap first appears in lower-division undergraduate mathematics courses and remains steady through advanced STEM courses, with…

Bit dropping in ATM: A large deviations analysis

Adam Shwartz Electrical, Adam Shwartz, Alan Weiss

Computer Networks and Communications · · cited 0× · OpenAlex ↗ · more like this ·

this paper is to illustrate the ideas behind this technique and to demonstrate its applicability. For more details on the general theory, on applications to the performance of other models and further details on the model under investigation here the reader is referred to [SW]. A sample of recent references to other work using the increasingly popular tool---large deviations---is also available in [SW]

Discrete Time Markov Processes

Adam Shwartz

Statistics and Probability · · cited 0× · OpenAlex ↗ · more like this ·

Introduction Discrete-time Markov processes, or Markov chains, are a powerful tool for modeling and analyzing discrete-time systems whose behavior is influenced by randomness. A Markov chains is probably the simplest object which incorporates both dynamics (that is, notions of &amp;quot;state&amp;quot; and time) and randomness. Let us illustrate the idea through a gambling example. Example 1. A gambler bets one dollar on &amp;quot;red&amp;quot; at every turn of a (fair) game of roulette. Then, at every turn he gains either one dollar (win, with probability 1=2) or (\\Gamma1) (lose). In an ideal game, the gains form a sequence of independent, identically distributed (i.i.d.) random variables. We cannot predict the outcome of each bet, although we do know the odds. Denote by X t the total fortune the gambler has at time t. If we know X s , then we can calculate the distribution of X

Call Admission: A New Approach to Quality of Service

Martin I. Reiman, Adam Shwartz

Computer Networks and Communications · · cited 0× · OpenAlex ↗ · more like this ·

Quality of service guarantees are an important and much discussed aspect of ATM network design. However, there is no standard definition of quality of service. Moreover, some often-used criteria seem quite crude. We consider call admission to a bufferless ATM multiplexer with on/off sources. A new criterion for a guarantee on average cell loss is proposed. We calculate the optimal policy that minimizes blocking subject to the guarantee, when there is only one type of user. The measure of cell-loss we propose is applicable to a wide range of models. It gives rise to a mathematical programming formulation, which we derive explicitly for our case. 1 Introduction Quality of service guarantees are a key issue in the design of modern computer communications networks. In the context of ATM (Asynchronous Transfer Mode) networks, the two most obvious performance issues---from the users&amp;apos…

Throughput-Delay Tradeoff with Impatient Arrivals

Rachel Levi, Adam Shwartz

Political Science and International Relations · · cited 0× · OpenAlex ↗ · more like this ·

Consider a queue where the (single) server is free to choose between two service modes (fast and slow), while arrivals may decline to join the queue if the queue is too long. The server is interested in Throughput---Cost trade-off: using the fast service is expensive, while slow service is free. Can the server guarantee a long run, time average Cost and Throughput that approach a specific, desired region, regardless the behavior of the customers? We characterize explicitly all convex approachable sets, as well as all nonapproachable sets, as follows. There is a polygon so that: Any convex set that intersects two sides of the polygon is approachable, Any set whose closure fails to intersect these sides is not approachable. The mathematical model that describes such systems is a Semi-Markov decision process with two independent decision makers, and vector payoff, where each coordinate is…

On random multiple access with unknown interference probabilities

Tzvi Shukhman, Adam Shwartz

Computer Networks and Communications · · cited 0× · DOI ↗ · more like this ·

We consider the time slotted, packet switched, broadcast type random multiple access channel with infinite number of stations. Due to interference, some successful transmissions are erased, and some empty slots are erroneously interpreted as collisions. Stochastic approximation arguments are employed to derive an adaptive algorithm that performs well when the interference probabilities are unknown.