arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Yvan Labiche

110 papers · 2000–2026

2000: 12001: 02002: 22003: 42004: 62005: 122006: 122007: 42008: 42009: 22010: 52011: 72012: 22013: 52014: 62015: 122016: 52017: 62018: 42019: 12020: 32021: 12022: 02023: 32024: 12025: 02026: 2
2000peak 12 in 20052026

Frequent co-authors

Papers

showing 1–25 of 110 RSS

Is mutation an appropriate tool for testing experiments?

James H. Andrews, Landon Brand, Yvan Labiche

Software · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · · cited 787× · DOI ↗ · more like this ·

The empirical assessment of test techniques plays an important role in software testing research. One common practice is to instrument faults, either manually or by using mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. This paper investigates this important question based on a number of programs with comprehensive pools of test cases and known faults. It is concluded that, based on the data available thus far, the use of mutation operators is yielding trustworthy results (generated mutants are similar to real faults). Mutants appear however to be different from hand-seeded faults that seem to be harder to detect than real faults.

Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria

James H. Andrews, Lionel Briand, Yvan Labiche +1 more

Software · IEEE Transactions on Software Engineering · · cited 475× · DOI ↗ · more like this ·

The empirical assessment of test techniques plays an important role in software testing research. One common practice is to seed faults in subject software, either manually or by using a program that generates all possible mutants based on a set of mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults, thus facilitating the statistical analysis of fault detection effectiveness of test suites; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. Focusing on four common control and data flow criteria (block, decision, C-use, and P-use), this paper investigates this important issue based on a middle size industrial program with a comprehensive pool of test cases and known faults. Based on the data available thus far, the results are very consistent across the investigated criteria as…

Toward the Reverse Engineering of UML Sequence Diagrams for Distributed Java Software

Lionel Briand, Yvan Labiche, Johanne Leduc

Artificial Intelligence · IEEE Transactions on Software Engineering · · cited 187× · DOI ↗ · more like this ·

This paper proposes a methodology and instrumentation infrastructure toward the reverse engineering of UML (Unified Modeling Language) sequence diagrams from dynamic analysis. One motivation is, of course, to help people understand the behavior of systems with no (complete) documentation. However, such reverse-engineered dynamic models can also be used for quality assurance purposes. They can, for example, be compared with design sequence diagrams and the conformance of the implementation to the design can thus be verified. Furthermore, discrepancies can also suggest failures in meeting the specifications. Due to size constraints, this paper focuses on the distribution aspects of the methodology we propose. We formally define our approach using metamodels and consistency rules. The instrumentation is based on aspect-oriented programming in order to alleviate the effort overhead usually…

Impact analysis and change management of UML models

Lionel Briand, Yvan Labiche, L. O'Sullivan

Information Systems · · cited 183× · DOI ↗ · more like this ·

The use of Unified Modeling Language (UML) analysis/design models on large projects leads to a large number of interdependent UML diagrams. As software systems evolve, those diagrams undergo changes to, for instance, correct errors or address changes in the requirements. Those changes can in turn lead to subsequent changes to other elements in the UML diagrams. Impact analysis is then defined as the process of identifying the potential consequences (side-effects) of a change, and estimating what needs to be modified to accomplish a change. In this article, we propose a UML model-based approach to impact analysis that can be applied before any implementation of the changes, thus allowing an early decision-making and change planning process. We first verify that the UML diagrams are consistent (consistency check). Then changes between two different versions of a UML model are identified…

The impact of UML documentation on software maintenance: an experimental evaluation

Erik Arisholm, Lionel Briand, S.E. Hove +1 more

Information Systems · IEEE Transactions on Software Engineering · · cited 159× · DOI ↗ · more like this ·

The Unified Modeling Language (UML) is becoming the de facto standard for software analysis and design modeling. However, there is still significant resistance to model-driven development in many software organizations because it is perceived to be expensive and not necessarily cost-effective. Hence, it is important to investigate the benefits obtained from modeling. As a first step in this direction, this paper reports on controlled experiments, spanning two locations, that investigate the impact of UML documentation on software maintenance. Results show that, for complex tasks and past a certain learning curve, the availability of UML documentation may result in significant improvements in the functional correctness of changes as well as the quality of their design. However, there does not seem to be any saving of time. For simpler tasks, the time needed to update the UML…

Assessing and improving state-based class testing: a series of experiments

Lionel Briand, Massimiliano Di Penta, Yvan Labiche

Software · IEEE Transactions on Software Engineering · · cited 131× · DOI ↗ · more like this ·

This work describes an empirical investigation of the cost effectiveness of well-known state-based testing techniques for classes or clusters of classes that exhibit a state-dependent behavior. This is practically relevant as many object-oriented methodologies recommend modeling such components with statecharts which can then be used as a basis for testing. Our results, based on a series of three experiments, show that in most cases state-based techniques are not likely to be sufficient by themselves to catch most of the faults present in the code. Though useful, they need to be complemented with black-box, functional testing. We focus here on a particular technique, Category Partition, as this is the most commonly used and referenced black-box, functional testing technique. Two different oracle strategies have been applied for checking the success of test cases. One is a very precise…

Automating impact analysis and regression test selection based on UML designs

Lionel Briand, Yvan Labiche, G. Soccar

Software · · cited 131× · DOI ↗ · more like this ·

We present a methodology and a tool to support test selection from regression test suites based on change analysis in object-oriented designs. We assume that designs are represented using the Unified Modeling Language (UML) and we propose a formal mapping between design changes and a classification of regression test cases, i.e., three categories: reusable, retestable, and obsolete. We provide evidence of the feasibility of the methodology and its usefulness by using our prototype tool on an industrial case study.

Solving the Class Responsibility Assignment Problem in Object-Oriented Analysis with Multi-Objective Genetic Algorithms

Michael Bowman, Lionel Briand, Yvan Labiche

Information Systems · IEEE Transactions on Software Engineering · · cited 128× · DOI ↗ · more like this ·

In the context of object-oriented analysis and design (OOAD), class responsibility assignment is not an easy skill to acquire. Though there are many methodologies for assigning responsibilities to classes, they all rely on human judgment and decision making. Our objective is to provide decision-making support to reassign methods and attributes to classes in a class diagram. Our solution is based on a multi-objective genetic algorithm (MOGA) and uses class coupling and cohesion measurement for defining fitness functions. Our MOGA takes as input a class diagram to be optimized and suggests possible improvements to it. The choice of a MOGA stems from the fact that there are typically many evaluation criteria that cannot be easily combined into one objective, and several alternative solutions are acceptable for a given OO domain model. Using a carefully selected case study, this paper…

An investigation of graph-based class integration test order strategies

Lionel Briand, Yvan Labiche, Yihong Wang

Information Systems · IEEE Transactions on Software Engineering · · cited 116× · DOI ↗ · more like this ·

The issue of ordering class integration in the context of integration testing has been discussed by a number of researchers. More specifically, strategies have been proposed to generate a test order while minimizing stubbing. Recent papers have addressed the problem of deriving an integration order in the presence of dependency cycles in the class diagram. Such dependencies represent a practical problem as they make any topological ordering of classes impossible. Three main approaches, aimed at "breaking" cycles, have been proposed. The first one was proposed by Tai and Daniels (1999) and is based on assigning a higher-level order according to aggregation and inheritance relationships and a lower-level order according to associations. The second one was proposed by Le Traon et al. (2000) and is based on identifying strongly connected components in the dependency graph. The third one was…

An experimental investigation of formality in UML-based development

Lionel Briand, Yvan Labiche, Massimiliano Di Penta +1 more

Information Systems · IEEE Transactions on Software Engineering · · cited 115× · DOI ↗ · more like this ·

The object constraint language (OCL) was introduced as part of the Unified Modeling Language (UML). Its main purpose is to make UML models more precise and unambiguous by providing a constraint language describing constraints that the UML diagrams alone do not convey, including class invariants, operation contracts, and statechart guard conditions. There is an ongoing debate regarding the usefulness of using OCL in UML-based development, questioning whether the additional effort and formality is worth the benefit. It is argued that natural language may be sufficient, and using OCL may not bring any tangible benefits. This debate is in fact similar to the discussion about the effectiveness of formal methods in software engineering, but in a much more specific context. This paper presents the results of two controlled experiments that investigate the impact of using OCL on three software…

Facilitating the transition from use case models to analysis models

Tao Yue, Lionel Briand, Yvan Labiche

Information Systems · ACM Transactions on Software Engineering and Methodology · · cited 114× · DOI ↗ · more like this ·

Use case modeling, including use case diagrams and use case specifications (UCSs), is commonly applied to structure and document requirements. UCSs are usually structured but unrestricted textual documents complying with a certain use case template. However, because Use Case Models (UCMods) remain essentially textual, ambiguity is inevitably introduced. In this article, we propose a use case modeling approach, called Restricted Use Case Modeling (RUCM), which is composed of a set of well-defined restriction rules and a modified use case template. The goal is two-fold: (1) restrict the way users can document UCSs in order to reduce ambiguity and (2) facilitate the manual derivation of initial analysis models which, when using the Unified Modeling Language (UML), are typically composed of class diagrams, sequence diagrams, and possibly other types of diagrams. Though the proposed…

Stress testing real-time systems with genetic algorithms

Lionel Briand, Yvan Labiche, Marwa Shousha

Hardware and Architecture · · cited 108× · DOI ↗ · more like this ·

Reactive real-time systems have to react to external events within time constraints: Triggered tasks must execute within deadlines. The goal of this article is to automate, based on the system task architecture, the derivation of test cases that maximize the chances of critical deadline misses within the system. We refer to that testing activity as stress testing. We have developed a method based on genetic algorithms and implemented it in a tool. Case studies were run and results show that the tool may actually help testers identify test cases that will likely stress the system to such an extent that some tasks may miss deadlines.

aToucan

Tao Yue, Lionel Briand, Yvan Labiche

Information Systems · ACM Transactions on Software Engineering and Methodology · · cited 98× · DOI ↗ · more like this ·

The transition from an informal requirements specification in natural language to a structured, precise specification is an important challenge in practice. It is particularly so for object-oriented methods, defined in the context of the OMG's Model Driven Architecture (MDA), where a key step is to transition from a use case model to an analysis model. However, providing automated support for this transition is challenging, mostly because, in practice, requirements are expressed in natural language and are much less structured than other kinds of development artifacts. Such an automated transformation would enable at least the generation of an initial, likely incomplete, analysis model and enable automated traceability from requirements to code, through various intermediate models. In this article, we propose a method and a tool called aToucan, building on existing work, to…

Using genetic algorithms and coupling measures to devise optimal integration test orders

Lionel Briand, J. H. Feng, Yvan Labiche

Information Systems · · cited 97× · DOI ↗ · more like this ·

We present here an improved strategy to devise optimal integration test orders in object-oriented systems. Our goal is to minimize the complexity of stubbing during integration testing as this has been shown to be a major source of expenditure. Our strategy to do so is based on the combined use of inter-class coupling measurement and genetic algorithms. The former is used to assess the complexity of stubs and the latter is used to minimize complex cost functions based on coupling measurement. Using a precisely defined procedure, we investigate this approach in a case study involving a real system. Results are very encouraging as the approach clearly helps obtaining systematic and optimal results.

Coverage‐based regression test case selection, minimization and prioritization: a case study on an industrial system

Daniel Di Nardo, Nadia Alshahwan, Lionel Briand +1 more

Software · Software Testing Verification and Reliability · · cited 95× · DOI ↗ · more like this ·

Summary This paper presents a case study of coverage‐based regression testing techniques on a real world industrial system with real regression faults. The study evaluates four common prioritization techniques, a test selection technique, a test suite minimization technique and a hybrid approach that combines selection and minimization. The study also examines the effects of using various coverage criteria on the effectiveness of the studied approaches. The results show that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information in prioritization techniques does not significantly enhance fault detection rates. The results show that test selection does not provide significant savings in execution cost (<2%), which might be…

Using Machine Learning to Support Debugging with Tarantula

Lionel Briand, Yvan Labiche, Xuetao Liu

Software · · cited 93× · DOI ↗ · more like this ·

Using a specific machine learning technique, this paper proposes a way to identify suspicious statements during debugging. The technique is based on principles similar to Tarantula but addresses its main flaw: its difficulty to deal with the presence of multiple faults as it assumes that failing test cases execute the same fault(s). The improvement we present in this paper results from the use of C4.5 decision trees to identify various failure conditions based on information regarding the test cases' inputs and outputs. Failing test cases executing under similar conditions are then assumed to fail due to the same fault(s). Statements are then considered suspicious if they are covered by a large proportion of failing test cases that execute under similar conditions. We report on a case study that demonstrates improvement over the original Tarantula technique in terms of statement…

Using simulation to empirically investigate test coverage criteria based on statechart

Lionel Briand, Yvan Labiche, Y. Wang

Software · International Conference on Software Engineering · · cited 81× · DOI ↗ · more like this ·

A number of testing strategies have been proposed using state machines and statecharts as test models in order to derive test sequences and validate classes or class clusters. Though such criteria have the advantage of being systematic, little is known on how cost effective they are and how they compare to each other. This article presents a precise simulation and analysis procedure to analyze the cost-effectiveness of statechart-based testing techniques. We then investigate, using this procedure, the cost and fault detection effectiveness of adequate test sets for the most referenced coverage criteria for statecharts on three different representative case studies. Through the analysis of common results and differences across studies, we attempt to draw more general conclusions regarding the costs and benefits of using the criteria under investigation.

Investigating the use of analysis contracts to improve the testability of object‐oriented code

Lionel Briand, Yvan Labiche, Hao Sun

Software · Software Practice and Experience · · cited 75× · DOI ↗ · more like this ·

Abstract A number of activities involved in testing software are known to be difficult and time consuming. Among them is the definition and coding of test oracles and the isolation of faults once failures have been detected. Through a thorough and rigorous empirical study, we investigate how the instrumentation of contracts could address both issues. Contracts are known to be a useful technique in specifying the precondition and postcondition of operations and class invariants, thus making the definition of object‐oriented analysis or design elements more precise. It is one of the reasons the Object Constraint Language (OCL) was made part of the Unified Modeling Language. Our aim in this paper is to reuse and instrument contracts to ease testing. A thorough case study is run where we define OCL contracts, instrument them using a commercial tool and assess the benefits and limitations of…

Assessing, Comparing, and Combining State Machine-Based Testing and Structural Testing: A Series of Experiments

Samar Mouchawrab, Lionel Briand, Yvan Labiche +1 more

Software · IEEE Transactions on Software Engineering · · cited 72× · DOI ↗ · more like this ·

A large number of research works have addressed the importance of models in software engineering. However, the adoption of model-based techniques in software organizations is limited since these models are perceived to be expensive and not necessarily cost-effective. Focusing on model-based testing, this paper reports on a series of controlled experiments. It investigates the impact of state machine testing on fault detection in class clusters and its cost when compared with structural testing. Based on previous work showing this is a good compromise in terms of cost and effectiveness, this paper focuses on a specific state-based technique: the round-trip paths coverage criterion. Round-trip paths testing is compared to structural testing, and it is investigated whether they are complementary. Results show that even when a state machine models the behavior of the cluster under test as…

Towards the reverse engineering of UML sequence diagrams

Lionel Briand, Yvan Labiche, Miao Yu

Information Systems · · cited 68× · DOI ↗ · more like this ·

The objective of the work reported here is to defineand assess a method to reverse engineer UML sequencediagrams from execution traces. We do so based onformal transformation rules and we reverse engineerdiagrams that show all relevant technical information,including conditions, iterations of messages, and specificobject identities and types being involved in theinteractions. We present the fundamental principles ofour methodology, illustrate it with examples, and validateit through a case study.

Automatic Evaluation of Intrusion Detection Systems

Frédéric Massicotte, François Gagnon, Yvan Labiche +2 more

Computer Networks and Communications · Annual Computer Security Applications Conference/Proceedings of the annual Computer Security Applications Conference · · cited 60× · DOI ↗ · more like this ·

An intrusion detection system (IDS) is a crucial element of a network security posture. Although there are many IDS products available, it is rather difficult to find information about their accuracy. Only a few organizations evaluate these products. Furthermore, the data used to test and evaluate these IDS is usually proprietary. Thus, the research community cannot easily evaluate the next generation of IDS. Toward this end, DARPA provided in 1998, 1999 and 2000 an intrusion detection evaluation data set. However, no new data set has been released by DARPA since 2000, in part because of the cumbersomeness of the task. In this paper, we propose a strategy to address certain aspects of generating a publicly available documented data set for testing and evaluating intrusion detection systems. We also present a tool that automatically analyzes and evaluates IDS using our proposed data set

Coverage-Based Test Case Prioritisation: An Industrial Case Study

Daniel Di Nardo, Nadia Alshahwan, Lionel Briand +1 more

Software · UCL Discovery (University College London) · · cited 58× · DOI ↗ · more like this ·

This paper presents an industrial case study of coverage-based prioritisation techniques on a real world system with real regression faults. The study evaluates four common and different test case prioritisation techniques and examines the effects of using various coverage criteria on the fault detection rates of the prioritised test suites. The results show that prioritisation techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information does not significantly enhance fault detection rates.

A case study using the round-trip strategy for state-based class testing

Giuliano Antoniol, Lionel Briand, Massimiliano Di Penta +1 more

Software · PolyPublie (École Polytechnique de Montréal) · · cited 55× · DOI ↗ · more like this ·

A number of strategies have been proposed for state-based class testing. An important proposal made by Chow (1978), that was subsequently adapted by Binder (1999), consists in deriving test sequences covering all round-trip paths in a finite state machine (FSMs). Based on a number of (rather strong) assumptions, and for traditional FSMs, it can be demonstrated that all operation and transfer errors in the implementation can be uncovered. Through experimentation, this paper investigates this strategy when used in the context of UML statecharts. Based on a set of mutation operators proposed for object-oriented code we seed a significant number of faults in an implementation of a specific container class. We then investigate the effectiveness of four test teams at uncovering faults, based on the round-trip path strategy, and analyze the faults that seem to be difficult to detect. Our main…

An Orchestrated Survey of Available Algorithms and Tools for Combinatorial Testing

Sunint Kaur Khalsa, Yvan Labiche

Software · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · · cited 54× · DOI ↗ · more like this ·

For functional testing based on the input domain of a functionality, parameters and their values are identified and a test suite is generated using a criterion exercising combinations of those parameters and values. Since software systems are large, resulting in large numbers of parameters and values, a technique based on combinatorics called Combinatorial Testing (CT) is used to automate the process of creating those combinations. CT is typically performed with the help of combinatorial objects called Covering Arrays. The goal of the present work is to determine available algorithms/tools for generating a combinatorial test suite. We tried to be as complete as possible by using a precise protocol for selecting papers describing those algorithms/tools. The 75 algorithms/tools we identified are then categorized on the basis of different comparison criteria, including: the test suite…

Traffic-aware stress testing of distributed systems based on UML models

Vahid Garousi, Lionel Briand, Yvan Labiche

Computer Networks and Communications · · cited 54× · DOI ↗ · more like this ·

A stress test methodology aimed at increasing chances of discovering faults related to network traffic in distributed systems is presented. The technique uses the UML 2.0 model of the distributed system under test, augmented with timing information, and is based on an analysis of the control flow in sequence diagrams. It yields stress test requirements that are made of specific control flow paths along with time values indicating when to trigger them. Different variants of our stress testing technique already exist (they stress different aspects of a distributed system) and we focus here on one variant that is designed to identify and to stress test the system at the instant when data traffic on a network is maximal. Using a real-world distributed system specification, we design and implement a prototype distributed system and describe, for that particular system, how the stress test…