arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Porter

107 papers · 1990–2026

1990: 11991: 11992: 21993: 01994: 01995: 21996: 31997: 51998: 31999: 02000: 32001: 22002: 12003: 32004: 52005: 62006: 42007: 72008: 42009: 52010: 42011: 72012: 22013: 22014: 52015: 02016: 32017: 32018: 12019: 32020: 32021: 62022: 12023: 12024: 62025: 22026: 1
1990peak 7 in 20072026

Frequent co-authors

Papers

showing 1–25 of 107 RSS

A history-based test prioritization technique for regression testing in resource constrained environments

Jungmin Kim, Adam Porter

Software · · cited 381× · DOI ↗ · more like this ·

Regression testing is an expensive and frequently executed maintenance process used to revalidate modified software. To improve it, regression test selection (RTS) techniques strive to lower costs without overly reducing effectiveness by carefully selecting a subset of the test suite. Under certain conditions, some can even guarantee that the selected test cases perform no worse than the original test suite.But this ignores certain software development realities such as resource and time constraints that may prevent using RTS techniques as intended (e.g., regression testing must be done overnight, but RTS selection returns two days worth of tests). In practice, testers work around this by prioritizing the test cases and running only those that fit within existing constraints. Unfortunately this generally violates key RTS assumptions, voiding RTS technique guarantees and making…

An empirical study of regression test selection techniques

Todd Graves, Mary Jean Harrold, Jungmin Kim +2 more

Software · ACM Transactions on Software Engineering and Methodology · · cited 356× · DOI ↗ · more like this ·

Regression testing is the process of validating modified software to detect whether new errors have been introduced into previously tested code and to provide confidence that modifications are correct. Since regression testing is an expensive process, researchers have proposed regression test selection techniques as a way to reduce some of this expense. These techniques attempt to reduce costs by selecting and running only a subset of the test cases in a program's existing test suite. Although there have been some analytical and empirical evaluations of individual techniques, to our knowledge only one comparative study, focusing on one aspect of two of these techniques, has been reported in the literature. We conducted an experiment to examine the relative costs and benefits of several regression test selection techniques. The experiment examined five techniques for reusing test cases…

Understanding the sources of variation in software inspections

Adam Porter, Harvey Siy, Audris Mockus +1 more

Information Systems · ACM Transactions on Software Engineering and Methodology · · cited 144× · DOI ↗ · more like this ·

In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other factors must be affecting inspection performance. The nature and extent of these other factors now have to be determined to ensure that they had not biased our earlier results. Also, identifying these other factors might suggest additional ways to improve the efficiency of inspections. Acting on the hypothesis that the “inputs” into the inspection process (reviewers, authors, and code units) were…

Using symbolic evaluation to understand behavior in configurable software systems

Elnatan Reisner, Charles Song, Kin-Keung Ma +2 more

Information Systems · Digital Repository at the University of Maryland (University of Maryland College Park) · · cited 107× · DOI ↗ · more like this ·

Many modern software systems are designed to be highly configurable, which increases flexibility but can make programs hard to test, analyze, and understand. We present an initial empirical study of how configuration options affect program behavior. We conjecture that, at certain levels of abstraction, configuration spaces are far smaller than the worst case, in which every configuration is distinct. We evaluated our conjecture by studying three configurable software systems: vsftpd, ngIRCd, and grep. We used symbolic evaluation to discover how the settings of run-time configuration options affect line, basic block, edge, and condition coverage for our subjects under a given test suite. Our results strongly suggest that for these subject programs, test suites, and configuration options, when abstracted in terms of the four coverage criteria above, configuration spaces are in fact much…

Metric-driven analysis and feedback systems for enabling empirically guided software development

Richard W. Selby, Adam Porter, Doug Schmidt +1 more

Information Systems · International Conference on Software Engineering · · cited 88× · DOI ↗ · more like this ·

Metric-driven analysis and feedback systems enable developers to define empirically guided processes for software development and maintenance. These sys-tems enable developers to achieve S. E.I. process matu-rit y levels 4 and 5, “managed ” and “optimizing ” pro-cesses. We define a set of architectural principles and abstract interfaces for designing metric-driven analysis and feedback systems. A fundamental principle is to make measurement active by integrating measurement and process, which contrasts with the primarily pas-sive use of measurement in the past. A second princi-ple is that these systems should provide capabilities for user-specifiable interpretation of process events, object state changes, and calendar time abstractions in order to allow a representative basis for project analysis. An-other principle is that these systems should provide an extensible integration…

Moving Forward with Combinatorial Interaction Testing

Cemal Yılmaz, Sandro Fouché, Myra B. Cohen +3 more

Software · Computer · · cited 71× · DOI ↗ · more like this ·

Combinatorial interaction testing (CIT) is an effective failure detection method for many types of software systems. This review discusses the current approaches CIT uses in detecting parameter interactions, the difficulties of applying it in practice, recent advances, and opportunities for future research.

Incremental covering array failure characterization in large configuration spaces

Sandro Fouché, Myra B. Cohen, Adam Porter

Software · · cited 63× · DOI ↗ · more like this ·

The increasing complexity of configurable software systems has created a need for more intelligent sampling mechanisms to detect and characterize failure-inducing dependencies between configurations. Prior work - in idealized environments - has shown that test schedules based on a mathematical object, called a covering array, in combination with classification techniques, can meet this need. Applying this approach in practice, however, is tricky because testing time and resource availability are unpredictable, and because failure characteristics can change from release to release. With current approaches developers must set a key covering array parameter (its strength) based on estimated release times and failure characterizations. This will influence the outcome of their results.

An empirical study of regression test application frequency

Jungmin Kim, Adam Porter, Gregg Rothermel

Software · · cited 63× · DOI ↗ · more like this ·

Regression testing is an expensive maintenance process used to revalidate modified software. Regression test selection (RTS) techniques try to lower the cost of regression testing by selecting and running a subset of the existing test cases. Many such techniques have been proposed and initial studies show that they can produce savings. We believe, however, that issues such as the frequency with which testing is done have a strong effect on the behavior of these techniques. Therefore, we conducted an experiment to assess the effects of test application frequency on the costs and benefits of regression test selection techniques. Our results expose essential tradeoffs that should be considered when using these techniques over a series of software releases.

Learning a classifier for false positive error reports emitted by static code analysis tools

Uğur Koç, Parsa Saadatpanah, Jeffrey S. Foster +1 more

Information Systems · · cited 61× · DOI ↗ · more like this ·

The large scale and high complexity of modern software systems make perfectly precise static code analysis (SCA) infeasible. Therefore SCA tools often over-approximate, so not to miss any real problems. This, however, comes at the expense of raising false alarms, which, in practice, reduces the usability of these tools. To partially address this problem, we propose a novel learning process whose goal is to discover program structures that cause a given SCA tool to emit false error reports, and then to use this information to predict whether a new error report is likely to be a false positive as well. To do this, we first preprocess code to isolate the locations that are related to the error report. Then, we apply machine learning techniques to the preprocessed code to discover correlations and to learn a classifier. We evaluated this approach in an initial case study of a widely-used…

Experimental software engineering

Lawrence G. Votta, Adam Porter

Information Systems · · cited 57× · DOI ↗ · more like this ·

Article Free Access Share on Experimental software engineering: a report on the state of the art Authors: Lawrence G. Votta Software & Systems Research Lab, AT&T Bell Laboratories, Naperville, Illinois Software & Systems Research Lab, AT&T Bell Laboratories, Naperville, IllinoisView Profile , Adam Porter University of Maryland, Computer Science Department, College Park, Maryland University of Maryland, Computer Science Department, College Park, MarylandView Profile Authors Info & Claims ICSE '95: Proceedings of the 17th international conference on Software engineeringApril 1995Pages 277–279https://doi.org/10.1145/225014.225040Published:23 April 1995Publication History 29citation568DownloadsMetricsTotal Citations29Total Downloads568Last 12 Months90Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified…

Applying classification techniques to remotely-collected program execution data

Murali Haran, Alan F. Karr, Alessandro Orso +2 more

Computer Networks and Communications · · cited 55× · DOI ↗ · more like this ·

There is an increasing interest in techniques that support measurement and analysis of fielded software systems. One of the main goals of these techniques is to better understand how software actually behaves in the field. In particular, many of these techniques require a way to distinguish, in the field, failing from passing executions. So far, researchers and practitioners have only partially addressed this problem: they have simply assumed that program failure status is either obvious (i.e., the program crashes) or provided by an external source (e.g., the users). In this paper, we propose a technique for automatically classifying execution data, collected in the field, as coming from either passing or failing program runs. (Failing program runs are executions that terminate with a failure, such as a wrong outcome.) We use statistical learning algorithms to build the classification…

Covering arrays for efficient fault characterization in complex configuration spaces

Cemal Yılmaz, Myra B. Cohen, Adam Porter

Software · · cited 52× · DOI ↗ · more like this ·

Testing systems with large configurations spaces that change often is a challenging problem. The cost and complexity of QA explodes because often there isn't just one system, but a multitude of related systems. Bugs may appear in certain configurations, but not in others.The Skoll system and process has been developed to test these types of systems through distributed, continuous quality assurance, leveraging user resources around-the-world, around-the-clock. It has been shown to be effective in automatically characterizing configurations in which failures manifest. The derived information helps developers quickly narrow down the cause of failures which then improves turn around time for fixes. However, this method does not scale well. It requires one to exhaustively test each configuration in the configuration space.In this paper we examine an alternative approach. The idea is to…

Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach

Cemal Yılmaz, Emine Dumlu, Myra B. Cohen +1 more

Software · IEEE Transactions on Software Engineering · · cited 49× · DOI ↗ · more like this ·

The configuration spaces of modern software systems are too large to test exhaustively. Combinatorial interaction testing (CIT) approaches, such as covering arrays, systematically sample the configuration space and test only the selected configurations. The basic justification for CIT approaches is that they can cost-effectively exercise all system behaviors caused by the settings of t or fewer options. We conjecture, however, that in practice some of these behaviors are not actually tested because of unanticipated masking effects - test case failures that perturb system execution so as to prevent some behaviors from being exercised. While prior research has identified this problem, most solutions require knowing the masking effects a priori. In practice this is impractical, if not impossible. In this work, we reduce the harmful consequences of masking effects. First we define a novel…

Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks

Murali Haran, Alan F. Karr, Michael Last +4 more

Computer Networks and Communications · IEEE Transactions on Software Engineering · · cited 47× · DOI ↗ · more like this ·

There is an increasing interest in techniques that support analysis and measurement of fielded software systems. These techniques typically deploy numerous instrumented instances of a software system, collect execution data when the instances run in the field, and analyze the remotely collected data to better understand the system's in-the-field behavior. One common need for these techniques is the ability to distinguish execution outcomes (e.g., to collect only data corresponding to some behavior or to determine how often and under which condition a specific behavior occurs). Most current approaches, however, do not perform any kind of classification of remote executions and either focus on easily observable behaviors (e.g., crashes) or assume that outcomes' classifications are externally provided (e.g., by the users). To address the limitations of existing approaches, we have…

Skoll: A Process and Infrastructure for Distributed Continuous Quality Assurance

Adam Porter, Cemal Yılmaz, Atif M. Memon +2 more

Software · IEEE Transactions on Software Engineering · · cited 45× · DOI ↗ · more like this ·

Software engineers increasingly emphasize agility and flexibility in their designs and development approaches. They increasingly use distributed development teams, rely on component assembly and deployment rather than green field code writing, rapidly evolve the system through incremental development and frequent updating, and use flexible product designs supporting extensive end-user customization. While agility and flexibility have many benefits, they also create an enormous number of potential system configurations built from rapidly changing component implementations. Since today's quality assurance (QA) techniques do not scale to handle highly configurable systems, we are developing and validating novel software QA processes and tools that leverage the extensive computing resources of user and developer communities in a distributed, continuous manner to improve software quality…

CiCUTS: Combining System Execution Modeling Tools with Continuous Integration Environments

James H. Hill, Douglas C. Schmidt, Adam Porter +1 more

Computer Networks and Communications · · cited 44× · DOI ↗ · more like this ·

System execution modeling (SEM) tools provide an effective means to evaluate the quality of service (QoS) of enterprise distributed real-time and embedded (DRE) systems. SEM tools facilitate testing and resolving performance issues throughout the entire development life-cycle, rather than waiting until final system integration. SEM tools have not historically focused on effective testing. New techniques are therefore needed to help bridge the gap between the early integration capabilities of SEM tools and testing so developers can focus on resolving strategic integration and performance issues, as opposed to wrestling with tedious and error-prone low-level testing concerns. This paper provides two contributions to research on using SEM tools to address enterprise DRE system integration challenges. First, we evaluate several approaches for combining continuous integration environments…

The past, present, and future of MOOCs and their relevance to software engineering

B. Dasarathy, Kevin Sullivan, Douglas C. Schmidt +2 more

Computer Science Applications · · cited 42× · DOI ↗ · more like this ·

Massive open online courses (MOOCs) are a recent development in online education aimed at unlimited participation and open access via the Web. They are a potentially disruptive technology, changing how education is delivered and funded around the world. MOOCs are relevant to software researchers and practitioners, not only because they will increasingly receive lifelong education through MOOCs and related technologies, but also because content creation, delivery, and enhancement of MOOCs is evolving into a new form of socially- and cognitively-embedded software development. This paper discusses how education is being enhanced by MOOCs and other digital learning technology.

Feedback driven adaptive combinatorial testing

Emine Dumlu, Cemal Yılmaz, Myra B. Cohen +1 more

Software · Sabanci University · · cited 42× · DOI ↗ · more like this ·

The configuration spaces of modern software systems are too large to test exhaustively. Combinatorial interaction testing (CIT) approaches, such as covering arrays, systematically sample the configuration space and test only the selected configurations. The basic justification for CIT approaches is that they can cost-effectively exercise all system behaviors caused by the settings of t or fewer options. We conjecture, however, that in practice many such behaviors are not actually tested because of masking effects -- failures that perturb execution so as to prevent some behaviors from being exercised. In this work we present a feedback-driven, adaptive, combinatorial testing approach aimed at detecting and working around masking effects. At each iteration we detect potential masking effects, heuristically isolate their likely causes, and then generate new covering arrays that allow…

Specification-based testing of reactive software: tools and experiments

Lalita Jategaonkar Jagadeesan, Adam Porter, Carlos Puchol +2 more

Software · · cited 41× · OpenAlex ↗ · more like this ·

Testing commercial software is expensive and time consuming. Automated testing methods promise to save a great deal of time and money throughout the software industry. One approach that is well-suited for the reactive systems found in telephone switching systems is specification-based testing. We have built a set of tools to automatically test software applications for violations of safety properties expressed in temporal logic. Our testing system automatically constructs finite state machine oracles corresponding to safety properties, builds test harnesses, and integrates them with the application. The test harness then generates inputs automatically to test the application. We describe a study examining the feasibility of this approach for testing industrial applications. To conduct this study we formally modeled an Automatic Protection Switching system (APS), which is an application…

Classifying User Requirements from Online Feedback in Small Dataset Environments using Deep Learning

Rohan Reddy Mekala, Asif Irfan, Eduard C. Groen +2 more

Information Systems · Fraunhofer-Publica (Fraunhofer-Gesellschaft) · · cited 40× · DOI ↗ · more like this ·

An overwhelming number of users access app repositories like App Store/Google Play and social media platforms like Twitter, where they provide feedback on digital experiences. This vast textual corpus comprising user feedback has the potential to unearth detailed insights regarding the users’ opinions on products and services. Various tools have been proposed that employ natural language processing (NLP) and traditional machine learning (ML) based models as an inexpensive mechanism to identify requirements in user feedback. However, they fall short on their classification accuracy over unseen data due to factors like the cost of generating voluminous de-biased labeled datasets and general inefficiency. Recently, Van Vliet et al. [1] achieved state-of-the-art results extracting and classifying requirements from user reviews through traditional crowdsourcing. Based on their reference…

Techniques and processes for improving the quality and performance of open‐source software

Adam Porter, Cemal Yılmaz, Atif M. Memon +3 more

Information Systems · Software Process Improvement and Practice · · cited 39× · DOI ↗ · more like this ·

Abstract Open‐source development processes have emerged as an effective approach to reduce cycle‐time and decrease design, implementation, and quality assurance (QA) costs for certain types of software, particularly systems infrastructure software, such as operating systems (OS), compilers and language processing tools, text and drawing editors, and middleware. This article presents two contributions to the study of open‐source software processes. First, we describe key challenges of open‐source software and illustrate how QA processes—specifically those tailored to open‐source development—help mitigate these challenges better than traditional closed‐source processes do. Second, we summarize results of empirical studies that evaluate how our Skoll distributed continuous quality assurance (DCQA) techniques and processes help to resolve key challenges in developing and validating…

Role-Playing Computer Ethics: Designing and Evaluating the Privacy by Design (PbD) Simulation

Katie Shilton, Donal Heidenblad, Adam Porter +2 more

Safety Research · Science and Engineering Ethics · · cited 38× · DOI ↗ · more like this ·

There is growing consensus that teaching computer ethics is important, but there is little consensus on how to do so. One unmet challenge is increasing the capacity of computing students to make decisions about the ethical challenges embedded in their technical work. This paper reports on the design, testing, and evaluation of an educational simulation to meet this challenge. The privacy by design simulation enables more relevant and effective computer ethics education by letting students experience and make decisions about common ethical challenges encountered in real-world work environments. This paper describes the process of incorporating empirical observations of ethical questions in computing into an online simulation and an in-person board game. We employed the Values at Play framework to transform empirical observations of design into a playable educational experience. First, we…

Main effects screening

Cemal Yılmaz, Arvind S. Krishna, Atif M. Memon +4 more

Computer Networks and Communications · · cited 36× · DOI ↗ · more like this ·

Developers of highly configurable performance-intensive software systems often use a type of in-house performance-oriented "regression testing" to ensure that their modifications have not adversely affected their software's performance across its large configuration space. Unfortunately, time and resource constraints often limit developers to in-house testing of a small number of configurations and unreliable extrapolation from these results to the entire configuration space, which allows many performance bottlenecks and sources of QoS degradation to escape detection until systems are fielded. To improve performance assessment of evolving systems across large configuration spaces, we have developed a distributed continuous quality assurance (DCQA) process called main effects screening that uses in-the-field resources to execute formally designed experiments to help reduce the…