arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Kenneth Rose

333 papers · 1982–2025

1982: 11983: 11984: 01985: 11986: 01987: 21988: 01989: 11990: 41991: 11992: 21993: 31994: 31995: 31996: 81997: 21998: 61999: 52000: 72001: 112002: 502003: 212004: 72005: 72006: 92007: 122008: 72009: 132010: 172011: 112012: 172013: 202014: 112015: 92016: 122017: 112018: 102019: 42020: 82021: 52022: 32023: 32024: 42025: 1
1982peak 50 in 20022025

Frequent co-authors

Papers

showing 1–25 of 333 RSS

Deterministic annealing for clustering, compression, classification, regression, and related optimization problems

Kenneth Rose

Artificial Intelligence · Proceedings of the IEEE · · cited 879× · DOI ↗ · more like this ·

The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures…

Video coding with optimal inter/intra-mode switching for packet loss resilience

R. Zhang, S.L. Regunathan, Kenneth Rose

Signal Processing · IEEE Journal on Selected Areas in Communications · · cited 633× · DOI ↗ · more like this ·

Resilience to packet loss is a critical requirement in predictive video coding for transmission over packet-switched networks, since the prediction loop propagates errors and causes substantial degradation in video quality. This work proposes an algorithm to optimally estimate the overall distortion of decoder frame reconstruction due to quantization, error propagation, and error concealment. The method recursively computes the total decoder distortion at pixel level precision to accurately account for spatial and temporal error propagation. The accuracy of the estimate is demonstrated via simulation results. The estimate is integrated into a rate-distortion (RD)-based framework for optimal switching between intra-coding and inter-coding modes per macroblock. The cost in computational complexity is modest. The framework is further extended to optimally exploit feedback/acknowledgment…

Statistical mechanics and phase transitions in clustering

Kenneth Rose, E. Gurewitz, Geoffrey Fox

Artificial Intelligence · Physical Review Letters · · cited 463× · DOI ↗ · more like this ·

A new approach to clustering based on statistical physics is presented. The problem is formulated as fuzzy clustering and the association probability distribution is obtained by maximizing the entropy at a given average variance. The corresponding Lagrange multiplier is related to the ``temperature'' and motivates a deterministic annealing process where the free energy is minimized at each temperature. Critical temperatures are derived for phase transitions when existing clusters split. It is a hierarchical clustering estimating the most probable cluster parameters at various average variances.

Vector quantization by deterministic annealing

Kenneth Rose, E. Gurewitz, Geoffrey Fox

Computer Vision and Pattern Recognition · IEEE Transactions on Information Theory · · cited 264× · DOI ↗ · more like this ·

A deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data. The problem is reformulated within a probabilistic framework. No prior knowledge is assumed on the source density, and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion. The corresponding Lagrange multiplier is inversely related to the 'temperature' and is used to control the annealing process. In this process, as the temperature is lowered, the system undergoes a sequence of phase transitions when existing clusters split naturally, without use of heuristics. The resulting codebook is independent of the codebook used to initialize the iterations.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Constrained clustering as an optimization method

Kenneth Rose, E. Gurewitz, Geoffrey Fox

Artificial Intelligence · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 169× · DOI ↗ · more like this ·

A deterministic annealing approach to clustering is derived on the basis of the principle of maximum entropy. This approach is independent of the initial state and produces natural hierarchical clustering solutions by going through a sequence of phase transitions. It is modified for a larger class of optimization problems by adding constraints to the free energy. The concept of constrained clustering is explained, and three examples are are given in which it is used to introduce deterministic annealing. The previous clustering method is improved by adding cluster mass variables and a total mass constraint. The traveling salesman problem is reformulated as constrained clustering, yielding the elastic net (EN) approach to the problem. More insight is gained by identifying a second Lagrange multiplier that is related to the tour length and can also be used to control the annealing process.…

A mapping approach to rate-distortion computation and analysis

Kenneth Rose

Computer Vision and Pattern Recognition · IEEE Transactions on Information Theory · · cited 147× · DOI ↗ · more like this ·

In rate-distortion theory, results are often derived and stated in terms of the optimizing density over the reproduction space. In this paper, the problem is reformulated in terms of the optimal mapping from the unit interval with Lebesgue measure that would induce the desired reproduction probability density. This results in optimality conditions that are "random relatives" of the known Lloyd (1982) optimality conditions for deterministic quantizers. The validity of the mapping approach is assured by fundamental isomorphism theorems for measure spaces. We show that for the squared error distortion, the optimal reproduction random variable is purely discrete at supercritical distortion (where the Shannon (1948) lower bound is not tight). The Gaussian source is thus the only source that produces continuous reproduction variables for the entire range of positive rate. To analyze the…

Philanthropic Foundations: New Scholarship, New Possibilities.

Kenneth Rose, Ellen Condliffe Lagemann

Sociology and Political Science · Journal of American History · · cited 131× · DOI ↗ · more like this ·

foundations themselves have discouraged scholarly writing.Having been the subject of hostile Congressional investigations on at least four occasions, they have been extremely skittish about opening their files to outside scholars."(p.ix) Which is true, but not the only reason why 42,000plus foundations in America that currently have assets of some $326 billion are frequently less than forthcoming sharing information.Other reasons include Internal Revenue Service agents seeking evidence of donor malfeasance, donors

Jointly Optimized Spatial Prediction and Block Transform for Video and Image Coding

Jingning Han, Ankur Saxena, Vinay Melkote +1 more

Signal Processing · IEEE Transactions on Image Processing · · cited 110× · DOI ↗ · more like this ·

This paper proposes a novel approach to jointly optimize spatial prediction and the choice of the subsequent transform in video and image compression. Under the assumption of a separable first-order Gauss-Markov model for the image signal, it is shown that the optimal Karhunen-Loeve Transform, given available partial boundary information, is well approximated by a close relative of the discrete sine transform (DST), with basis vectors that tend to vanish at the known boundary and maximize energy at the unknown boundary. The overall intraframe coding scheme thus switches between this variant of the DST named asymmetric DST (ADST), and traditional discrete cosine transform (DCT), depending on prediction direction and boundary information. The ADST is first compared with DCT in terms of coding gain under ideal model conditions and is demonstrated to provide significantly improved…

The effects of quantization on multilayer neural networks

Günhan Dündar, Kenneth Rose

Artificial Intelligence · IEEE Transactions on Neural Networks · · cited 99× · DOI ↗ · more like this ·

The effect of weight quantization in multilayer neural networks is discussed. A method is derived by which one can predict the performance degradation at the output given the properties of the network and number of bits of quantization. Predictions from this method are evaluated against simulation results. An algorithm to decrease the noise at the output is presented and the results are compared with those above.

A global optimization technique for statistical classifier design

David J. Miller, A.V. Rao, Kenneth Rose +1 more

Artificial Intelligence · IEEE Transactions on Signal Processing · · cited 97× · DOI ↗ · more like this ·

A global optimization method is introduced that minimize the rate of misclassification. We first derive the theoretical basis for the method, on which we base the development of a novel design algorithm and demonstrate its effectiveness and superior performance in the design of practical classifiers for some of the most popular structures currently in use. The method, grounded in ideas from statistical physics and information theory, extends the deterministic annealing approach for optimization, both to incorporate structural constraints on data assignments to classes and to minimize the probability of error as the cost objective. During the design, data are assigned to classes in probability so as to minimize the expected classification error given a specified level of randomness, as measured by Shannon's entropy. The constrained optimization is equivalent to a free-energy…

Toward optimality in scalable predictive coding

Kenneth Rose, S.L. Regunathan

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 76× · DOI ↗ · more like this ·

A method is proposed for efficient scalability in predictive coding, which overcomes known fundamental shortcomings of the prediction loop at enhancement layers. The compression efficiency of an enhancement-layer is substantially improved by casting the design of its prediction module within an estimation-theoretic framework, and thereby exploiting all information available at that layer for the prediction of the signal, and encoding of the prediction error. While the most immediately important application is in video compression, the method is derived in a general setting and is applicable to any scalable predictive coder. Thus, the estimation-theoretic approach is first developed for basic DPCM compression and demonstrates the power of the technique in a simple setting that only involves straightforward prediction, scalar quantization, and entropy coding. Results for the scalable…

Towards jointly optimal spatial prediction and adaptive transform in video/image coding

Jingning Han, Ankur Saxena, Kenneth Rose

Signal Processing · · cited 73× · DOI ↗ · more like this ·

This paper proposes a new approach to combined spatial (Intra) prediction and adaptive transform coding in block-based video and image compression. Context-adaptive spatial prediction from available, previously decoded boundaries of the block, is followed by optimal transform coding of the prediction residual. The derivation of both the prediction and the adaptive transform for the prediction error, assumes a separable first-order Gauss-Markov model for the image signal. The resulting optimal transform is shown to be a close relative of the sine transform with phase and frequencies such that basis vectors tend to vanish at known boundaries and maximize energy at unknown boundaries. The overall scheme switches between the above sine-like transform and discrete cosine transform (per direction, horizontal or vertical) depending on the prediction and boundary information. It is implemented…

Skeleton of <i>Diacodexis</i> , Oldest Known Artiodactyl

Kenneth D. Rose

Geometry and Topology · Science · · cited 70× · DOI ↗ · more like this ·

A nearly complete skeleton of early Eocene Diacodexis, the oldest known member of the mammalian order Artiodactyla, is described. Its slender, elongate limb elements indicate that Diacodexis was highly cursorial and closer in postcranial adaptations to tragulids and other primitive ruminants than to living or extinct nonruminant artiodactyls. Its skeletal specializations call into question the widespread notion that Diacodexis was the ancestor of all later artiodactyls.

VQ-index

Ertem Tuncel, Hakan Ferhatosmanoğlu, Kenneth Rose

Computer Vision and Pattern Recognition · · cited 64× · DOI ↗ · more like this ·

In this paper, we introduce a novel indexing technique based on efficient compression of the feature space for approximate similarity searching in large multimedia databases. Its main novelty is that state-of-the-art tools from the discipline of data compression are adopted to optimize the complexity-performance tradeoff in large data sets. The design procedure optimizes the query access time by jointly accounting for both database distribution and query statistics. We achieve efficient compression by using appropriate vector quantization (VQ) techniques, namely, multi-stage VQ and split-VQ, which are especially suited for limited memory applications. We partition the data set using the accumulated query history, and each partition of data points is separately compressed using a vector quantizer tailored to its distribution. The employed VQ techniques inherently provide a spectrum of…

Advances in Recursive Per-Pixel End-to-End Distortion Estimation for Robust Video Coding in H.264/AVC

Hua Yang, Kenneth Rose

Signal Processing · IEEE Transactions on Circuits and Systems for Video Technology · · cited 62× · DOI ↗ · more like this ·

This paper is focused on expanding the applicability of the recursive optimal per-pixel estimate (ROPE) of end-to-end distortion, to the H.264/AVC standard. One open question involves the complexity of cross-correlation terms that appear in the case of subpixel prediction and other pixel filtering operations. Several efficient model-based solutions are proposed. Another open question involves the largely ignored effects of rounding operations, whose cumulative impact may seriously degrade the estimate accuracy. Two approaches are proposed for rounding error compensation: one appeals to the maximum entropy principle, while a low complexity alternative is based on quantization theoretic approximations. The former effectively estimates the distribution of the decoder reconstruction, thereby significantly broadening the applicability of ROPE to all additive distortion measures. Simulation…

A deterministic annealing approach for parsimonious design of piecewise regression models

A.V. Rao, David J. Miller, Kenneth Rose +1 more

Artificial Intelligence · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 62× · DOI ↗ · more like this ·

A new learning algorithm is proposed for piecewise regression modeling. It employs the technique of deterministic annealing to design space partition regression functions. While the performance of traditional space partition regression functions such as CART and MARS is limited by a simple tree-structured partition and by a hierarchical approach for design, the deterministic annealing algorithm enables the joint optimization of a more powerful piecewise structure based on a Voronoi partition. The new method is demonstrated to achieve consistent performance improvements over regular CART as well as over its extension to allow arbitrary hyperplane boundaries. Comparison tests, on several benchmark data sets from the regression literature, are provided.

Mixture of experts regression modeling by deterministic annealing

A.V. Rao, David J. Miller, Kenneth Rose +1 more

Artificial Intelligence · IEEE Transactions on Signal Processing · · cited 60× · DOI ↗ · more like this ·

We propose a new learning algorithm for regression modeling. The method is especially suitable for optimizing neural network structures that are amenable to a statistical description as mixture models. These include mixture of experts, hierarchical mixture of experts (HME), and normalized radial basis functions (NRBF). Unlike recent maximum likelihood (ML) approaches, we directly minimize the (squared) regression error. We use the probabilistic framework as means to define an optimization method that avoids many shallow local minima on the complex cost surface. Our method is based on deterministic annealing (DA), where the entropy of the system is gradually reduced, with the expected regression cost (energy) minimized at each entropy level. The corresponding Lagrangian is the system's "free-energy", and this annealing process is controlled by variation of the Lagrange multiplier, which…

DCT/DST alternate-transform image coding

Kenneth Rose, A. Heiman, I. Dinstein

Computer Vision and Pattern Recognition · IEEE Transactions on Communications · · cited 58× · DOI ↗ · more like this ·

An image coding method for low bit rates is proposed. It is based on alternate use of the discrete cosine transform (DCT) and the discrete sine transform (DST) on image blocks. This procedure achieves the removal of redundancies in the correlation between neighboring blocks as well as the preservation of continuity across the block boundaries. An outline of the mathematical justification of the method, assuming a certain first-order Gauss-Markov model, is given. The resulting coding method is then adapted to nonstationary real images by locally adapting the model parameters and improving the block classification technique. Simulation results are shown and compared with the performance of related previous methods, namely adaptive DCT and fast Karhunen-Loeve transform (FKLT).< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

3D DRAM Design and Application to 3D Multicore Systems

Hongbin Sun, Jibang Liu, Rakesh Anigundi +4 more

Hardware and Architecture · IEEE Design & Test of Computers · · cited 54× · DOI ↗ · more like this ·

Editor's note:From a system architecture perspective, 3D technology can satisfy the high memory bandwidth demands that future multicore/manycore architectures require. This article presents a 3D DRAM architecture design and the potential for using 3D DRAM stacking for both L2 cache and main memory in 3D multicore architecture.—Yuan Xie, Pennsylvania State University

Design of on-chip error correction systems for multilevel NOR and NAND flash memories

Fei Sun, S. Devarajan, Kenneth Rose +1 more

Computer Networks and Communications · IET Circuits Devices & Systems · · cited 48× · DOI ↗ · more like this ·

The design of on-chip error correction systems for multilevel code-storage NOR flash and data-storage NAND flash memories is concerned. The concept of trellis coded modulation (TCM) has been used to design on-chip error correction system for NOR flash. This is motivated by the non-trivial modulation process in multilevel memory storage and the effectiveness of TCM in integrating coding with modulation to provide better performance at relatively short block length. The effectiveness of TCM-based systems, in terms of error-correcting performance, coding redundancy, silicon cost and operational latency, has been successfully demonstrated. Meanwhile, the potential of using strong Bose–Chaudhiri–Hocquenghem (BCH) codes to improve multilevel data-storage NAND flash memory capacity is investigated. Current multilevel flash memories store 2 bits in each cell. Further storage capacity may be…

Perceptual harmonic cepstral coefficients for speech recognition in noisy environment

Liang Gu, Kenneth Rose

Signal Processing · · cited 47× · DOI ↗ · more like this ·

Perceptual harmonic cepstral coefficients (PHCC) are proposed as features to extract from speech for recognition in noisy environments. A weighting function, which depends on the prominence of the harmonic structure, is applied to the power spectrum to ensure accurate representation of the voiced speech spectral envelope. The harmonics' weighted power spectrum undergoes mel-scaled band-pass filtering, and the log-energy of the filters' output is discrete cosine transformed to produce cepstral coefficients. Lower spectral clipping is applied to the power spectrum, followed by within-filter root-power amplitude compression to reduce amplitude variation without compromise of the gain invariance properties. Experiments show significant recognition gains of PHCC over MFCC, with 23% and 36% error rate reduction for the Mandarin digit database in white and babble noise environments.

Southern Strategies: Southern Women and the Woman Suffrage Question

Kenneth D. Rose, Elna C. Green

Sociology and Political Science · The American Historical Review · · cited 47× · DOI ↗ · more like this ·

The biographies of more than 800 women form the basis for Elna Green's study of the suffrage and the antisuffrage movements in the South. Green's comprehensive analysis highlights the effects that factors such as class background, marital status, educational level, and attitudes about race and gender roles had in inspiring the region's women to work in favor of, or in opposition to, their own enfranchisement. Green sketches the ranks of both movements--which included women and men, black and white--and identifies the ways in which issues of class, race, and gender determined the composition of each side. Coming from a wide array of beliefs and backgrounds, Green argues, southern women approached enfranchisement with an equally varied set of strategies and ideologies. Each camp defined and redefined itself in opposition to the other. But neither was entirely homogeneous: issues such as…

Natural type selection in adaptive lossy compression

Ram Zamir, Kenneth Rose

Artificial Intelligence · IEEE Transactions on Information Theory · · cited 46× · DOI ↗ · more like this ·

Consider approximate (lossy) matching of a source string /spl sim/P, with a random codebook generated from reproduction distribution Q, at a specified distortion d. Previous work determined the minimum coding rate R/sub 1/=R(P, Q, d) for this setting. We observe that for a large word length and with high probability, the matching codeword is typical with a distribution Q/sub 1/ which is different from Q. If a new random codebook is generated /spl sim/Q/sub 1/, then the source string will favor codewords which are typical with a new distribution Q/sub 2/, resulting in a minimum coding rate R/sub 2/=R(P, Q/sub 1/, d), and so on. We show that the sequences of distributions Q/sub 1/, Q/sub 2/,... and rates R/sub 1/, R/sub 2/,..., generated by this procedure, converge to an optimum reproduction distribution Q*, and the rate-distortion function R(P, d), respectively. We also derive a fixed…

The asymptotic closed-loop approach to predictive vector quantizer design with application in video coding

H. Khalil, Kenneth Rose, S.L. Regunathan

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 44× · DOI ↗ · more like this ·

The basic vector quantization (VQ) technique employed in video coding belongs to the category of predictive vector quantization (PVQ), as it involves quantization of the (motion compensated) frame prediction error. It is well known that the design of PVQ suffers from fundamental difficulties, due to the prediction loop, which have an impact on the convergence and the stability of the design procedure. We propose an approach to PVQ design that enjoys the stability of open-loop design while it ensures ultimate optimization of the closed-loop system. The method is derived for general predictive quantization, and we demonstrate it on video compression at low bit rates, where it provides substantial improvement over standard open and closed loop design techniques. Further, the approach outperforms standard DCT-based video coding.

Combined source-channel vector quantization using deterministic annealing

David J. Miller, Kenneth Rose

Computer Vision and Pattern Recognition · IEEE Transactions on Communications · · cited 42× · DOI ↗ · more like this ·

The authors present a new approach to combined source-channel vector quantization. The method, derived within information theory and probability theory, utilizes deterministic annealing to avoid some local minima that trap conventional descent algorithms such as the generalized Lloyd algorithm. The resulting vector quantizers satisfy the necessary conditions for local optimality for the noisy channel case. They tested the method against several versions of the noisy channel generalized Lloyd algorithm on stationary, first order Gauss-Markov sources with a binary symmetric channel. The method outperformed other methods under all test conditions, with the gains over noisy channel GLA growing with the codebook size. The quantizers designed using deterministic annealing are also shown to behave robustly under channel mismatch conditions. As a comparison with a separate source-channel…