Author

Bernhard Sendhoff

110 papers · 1997–2026

1997peak 12 in 20052026

Frequent co-authors

Papers

showing 1–25 of 110 RSS

A Reference Vector Guided Evolutionary Algorithm for Many-Objective Optimization

Ran Cheng, Yaochu Jin, Markus Olhofer +1 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · 19 Jan 2016 · cited 1,706× · DOI ↗ · more like this ·

In evolutionary multiobjective optimization, maintaining a good balance between convergence and diversity is particularly crucial to the performance of the evolutionary algorithms (EAs). In addition, it becomes increasingly important to incorporate user preferences because it will be less likely to achieve a representative subset of the Pareto-optimal solutions using a limited population size as the number of objectives increases. This paper proposes a reference vector-guided EA for many-objective optimization. The reference vectors can be used not only to decompose the original multiobjective optimization problem into a number of single-objective subproblems, but also to elucidate user preferences to target a preferred subset of the whole Pareto front (PF). In the proposed algorithm, a scalarization approach, termed angle-penalized distance, is adopted to balance convergence and…

A framework for evolutionary optimization with approximate fitness functions

Yaochu Jin, Markus Olhofer, Bernhard Sendhoff

Artificial Intelligence · IEEE Transactions on Evolutionary Computation · October 2002 · cited 660× · DOI ↗ · more like this ·

It is not unusual that an approximate model is needed for fitness evaluation in evolutionary computation. In this case, the convergence properties of the evolutionary algorithm are unclear due to the approximation error of the model. In this paper, extensive empirical studies are carried out to investigate the convergence properties of an evolution strategy using an approximate fitness function on two benchmark problems. It is found that incorrect convergence will occur if the approximate model has false optima. To address this problem, individual- and generation-based evolution control are introduced and the resulting effects on the convergence properties are presented. A framework for managing approximate models in generation-based evolution control is proposed. This framework is well suited for parallel evolutionary optimization, which is able to guarantee the correct convergence of…

Generalizing Surrogate-Assisted Evolutionary Computation

Dudy Lim, Yaochu Jin, Yew-Soon Ong +1 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · 11 Dec 2009 · cited 482× · DOI ↗ · more like this ·

Using surrogate models in evolutionary search provides an efficient means of handling today's complex applications plagued with increasing high-computational needs. Recent surrogate-assisted evolutionary frameworks have relied on the use of a variety of different modeling approaches to approximate the complex problem landscape. From these recent studies, one main research issue is with the choice of modeling scheme used, which has been found to affect the performance of evolutionary search significantly. Given that theoretical knowledge available for making a decision on an approximation model <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> is very much limited, this paper describes a generalization of surrogate-assisted evolutionary frameworks for optimization of problems with objectives and constraints that are computationally…

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Yaochu Jin, Bernhard Sendhoff

Computational Theory and Mathematics · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · 23 Apr 2008 · cited 457× · DOI ↗ · more like this ·

Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection…

Test Problems for Large-Scale Multiobjective and Many-Objective Optimization

Ran Cheng, Yaochu Jin, Markus Olhofer +1 more

Computational Theory and Mathematics · IEEE Transactions on Cybernetics · 26 Aug 2016 · cited 413× · DOI ↗ · more like this ·

The interests in multiobjective and many-objective optimization have been rapidly increasing in the evolutionary computation community. However, most studies on multiobjective and many-objective optimization are limited to small-scale problems, despite the fact that many real-world multiobjective and many-objective optimization problems may involve a large number of decision variables. As has been evident in the history of evolutionary optimization, the development of evolutionary algorithms (EAs) for solving a particular type of optimization problems has undergone a co-evolution with the development of test problems. To promote the research on large-scale multiobjective and many-objective optimization, we propose a set of generic test problems based on design principles widely used in the literature of multiobjective and many-objective optimization. In order for the test problems to be…

A Multiobjective Evolutionary Algorithm Using Gaussian Process-Based Inverse Modeling

Ran Cheng, Yaochu Jin, Kaname Narukawa +1 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · 23 Jan 2015 · cited 375× · DOI ↗ · more like this ·

To approximate the Pareto front, most existing multiobjective evolutionary algorithms store the nondominated solutions found so far in the population or in an external archive during the search. Such algorithms often require a high degree of diversity of the stored solutions and only a limited number of solutions can be achieved. By contrast, model-based algorithms can alleviate the requirement on solution diversity and in principle, as many solutions as needed can be generated. This paper proposes a new model-based method for representing and searching nondominated solutions. The main idea is to construct Gaussian process-based inverse models that map all found nondominated solutions from the objective space to the decision space. These inverse models are then used to create offspring by sampling the objective space. To facilitate inverse modeling, the multivariate inverse function is…

Combining Model-based and Genetics-based Offspring Generation for Multi-objective Optimization Using a Convergence Criterion

Aimin Zhou, Yaochu Jin, Qingfu Zhang +2 more

Computational Theory and Mathematics · View · 22 Sep 2006 · cited 303× · DOI ↗ · more like this ·

In our previous work [1], it has been shown that the performance of multi-objective evolutionary algorithms can be greatly enhanced if the regularity in the distribution of Pareto-optimal solutions is used. This paper suggests a new hybrid multi-objective evolutionary algorithm by introducing a convergence based criterion to determine when the model-based method and when the genetics-based method should be used to generate offspring in each generation. The basic idea is that the genetics-based method, i. e., crossover and mutation, should be used when the population is far away from the Pareto front and no obvious regularity in population distribution can be observed. When the population moves towards the Pareto front, the distribution of the individuals will show increasing regularity and in this case, the model-based method should be used to generate offspring. The proposed hybrid…

A critical survey of performance indices for multi-objective optimisation

Tatsuya Okabe, Yaochu Jin, Bernhard Sendhoff

Computational Theory and Mathematics · View · 9 Jul 2004 · cited 270× · DOI ↗ · more like this ·

A large number of methods for solving multiobjective optimisation (MOO) problems have been developed. To compare these methods rigorously, or to measure the performance of a particular MOO algorithm quantitatively, a variety of performance indices (PIs) have been proposed. We provide an overview of the various PIs and attempts to categorise them into a certain number of classes according to their properties. Comparative studies have been conducted using a group of artificial solution sets and a group of solution sets obtained by various MOO solvers to show the advantages and disadvantages of the PIs. The comparative studies show that many PIs may be misleading in that they fail to truly reflect the quality of solution sets. Thus, it may not be a good practice to evaluate the performance of MOO solvers based on PIs only.

Dynamic Weighted Aggregation for evolutionary multi-objective optimization: why does it work and how?

Yaochu Jin, Markus Olhofer, Bernhard Sendhoff

Computational Theory and Mathematics · 7 Jul 2001 · cited 214× · OpenAlex ↗ · more like this ·

Evolutionary Dynamic Weighted Aggregation (EDWA) has shown to be both effective and computationally efficient [1] for multiobjective optimization (MOO). Besides, it was also found empirically and surprisingly that EDWA was able to deal with multiobjective optimization problems with a concave Pareto front, which has proved to be beyond the capability of the Conventional Weighted Aggregation (CWA) methods [2]. In this paper, a theory on why CWA fails for multi-objective problems with a concave Pareto front is provided schematically. According to this theory, it can easily be explained why EDWA has worked well for both convex and concave multi-objective problems. Simulation examples are conducted on various test functions to support our theory. It is concluded that EDWA is an effective and efficient method for solving multi-objective optimization problems.

On generating FC/sup 3/ fuzzy rule systems from data using evolution strategies

Yaochu Jin, W. von Seelen, Bernhard Sendhoff

Artificial Intelligence · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · 1999 · cited 184× · DOI ↗ · more like this ·

Sophisticated fuzzy rule systems are supposed to be flexible, complete, consistent and compact (FC(3)). Flexibility, and consistency are essential for fuzzy systems to exhibit an excellent performance and to have a clear physical meaning, while compactness is crucial when the number of the input variables increases. However, the completeness and consistency conditions are often violated if a fuzzy system is generated from data collected from real world applications. A systematic design paradigm is proposed using evolution strategies. The structure of the fuzzy rules, which determines the compactness of the fuzzy systems, is evolved along with the parameters of the fuzzy systems. Special attention has been paid to the completeness and consistency of the rule base. The completeness is guaranteed by checking the completeness of the fuzzy partitioning of input variables and the completeness…

On evolutionary optimization with approximate fitness functions

Yaochu Jin, Markus Olhofer, Bernhard Sendhoff

Computational Theory and Mathematics · View · 10 Jul 2000 · cited 160× · OpenAlex ↗ · more like this ·

The evaluation of the quality of solutions is usually very time-consuming in design optimization. Therefore, time-efficient approximate models can be particularly beneficial for the evaluation when evolutionary algorithms are applied. In this paper, the convergence property of an evolution strategy (ES) with neural network based fitness evaluations is investigated. It is found that the evolutionary algorithm will converge incorrectly if the approximate model has false optima. To address this problem, two strategies to control the evolution process are introduced. In addition, methods to eliminate false minima in neural network training are proposed. The effectiveness of the methods are shown with simulation studies on the Ackley function and the Rosenbrock function.

Neural network regularization and ensembling using multi-objective evolutionary algorithms

Yaochu Jin, Tatsuya Okabe, Bernhard Sendhoff

Computational Theory and Mathematics · View · 28 Sep 2004 · cited 142× · DOI ↗ · more like this ·

Regularization is an essential technique to improve generalization of neural networks. Traditionally, regularization is conducted by including an additional term in the cost function of a learning algorithm. One main drawback of these regularization techniques is that a hyperparameter that determines to which extension the regularization influences the learning algorithm must be determined beforehand. This paper addresses the neural network regularization problem from a multi-objective optimization point of view. During the optimization, both structure and parameters of the neural network will be optimized. A slightly modified version of two multi-objective optimization algorithms, the dynamic weighted aggregation (DWA) method and the elitist non-dominated sorting genetic algorithm (NSGA-II) are used and compared. An evolutionary multi-objective approach to neural network regularization…

Simplify Your Covariance Matrix Adaptation Evolution Strategy

Hans-Georg Beyer, Bernhard Sendhoff

Artificial Intelligence · IEEE Transactions on Evolutionary Computation · 10 Mar 2017 · cited 111× · DOI ↗ · more like this ·

The standard covariance matrix adaptation evolution strategy (CMA-ES) comprises two evolution paths, one for the learning of the mutation strength and one for the rank-1 update of the covariance matrix. In this paper, it is shown that one can approximately transform this algorithm in such a manner that one of the evolution paths and the covariance matrix itself disappear. That is, the covariance update and the covariance matrix square root operations are no longer needed in this novel so-called matrix adaptation (MA) ES. The MA-ES performs nearly as well as the original CMA-ES. This is shown by empirical investigations considering the evolution dynamics and the empirical expected runtime on a set of standard test functions. Furthermore, it is shown that the MA-ES can be used as a search engine in a bi-population (BiPop) ES. The resulting BiPop-MA-ES is benchmarked using the BBOB…

A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation

Dudy Lim, Yew-Soon Ong, Yaochu Jin +1 more

Computational Theory and Mathematics · Surrey Research Insight Open Access (The University of Surrey) · 7 Jul 2007 · cited 104× · DOI ↗ · more like this ·

Surrogate-Assisted Memetic Algorithm (SAMA) is a hybrid evolutionary algorithm, particularly a memetic algorithm that employs surrogate models in the optimization search. Since most of the objective function evaluations in SAMA are approximated, the search performance of SAMA is likely to be affected by the characteristics of the models used. In this paper, we study the search performance of using different meta modeling techniques, ensembles, and multi-surrogates in SAMA. In particular, we consider the SAMA-TRF, a SAMA model management framework that incorporates a trust region scheme for interleaving use of exact objective function with computationally cheap local meta models during local searches. Four different metamodels, namely Gaussian Process (GP), Radial Basis Function (RBF), Polynomial Regression (PR), and Extreme Learning Machine (ELM) neural network are used in the study.…

Managing approximate models in evolutionary aerodynamic design optimization

Yaochu Jin, Markus Olhofer, Bernhard Sendhoff

Computational Theory and Mathematics · View · 13 Nov 2002 · cited 81× · DOI ↗ · more like this ·

Approximate models have to be used in evolutionary optimization when the original fitness function is computationally very expensive. Unfortunately, the convergence property of the evolutionary algorithm is unclear when an approximate model is used for fitness evaluation because approximation errors are involved in the model. What is worse, the approximate model may introduce false optima that lead the evolutionary algorithm to a wrong solution. To address this problem, individual and generation based evolution control are introduced to ensure that the evolutionary algorithm using approximate fitness functions will converge correctly. A framework for managing approximate models in generation-based evolution control is proposed. This framework is well suited for parallel evolutionary optimization in which evaluation of the fitness function is time-consuming. Simulations on two benchmark…

A condition for the genotype-phenotype mapping: Causality

Bernhard Sendhoff, Martin Kreutz, Werner von Seelen

adap-org · 4 Nov 1997 · cited 78× · arXiv ↗ · more like this ·

The appropriate choice of the genotype-phenotype mapping in combination with the mutation operator is important for a successful evolutionary search process. We suggest a measure to quantify the quality of this combination by addressing the question whether the relation among distances is carried over from one space to the other. Search processes which do not destroy the neighbourhood structure are termed strongly causal. We apply the proposed measure to parameter and structure optimisation problems in order to assess the combination (mapping, mutation operator) and at the same time to be able to propose improved settings.

Evolution by Adapting Surrogates

Minh Nghia Le, Yew-Soon Ong, Stefan Menzel +2 more

Computational Theory and Mathematics · Evolutionary Computation · 7 May 2012 · cited 72× · DOI ↗ · more like this ·

To deal with complex optimization problems plagued with computationally expensive fitness functions, the use of surrogates to replace the original functions within the evolutionary framework is becoming a common practice. However, the appropriate datacentric approximation methodology to use for the construction of surrogate model would depend largely on the nature of the problem of interest, which varies from fitness landscape and state of the evolutionary search, to the characteristics of search algorithm used. This has given rise to the plethora of surrogate-assisted evolutionary frameworks proposed in the literature with ad hoc approximation/surrogate modeling methodologies considered. Since prior knowledge on the suitability of the data centric approximation methodology to use in surrogate-assisted evolutionary optimization is typically unavailable beforehand, this paper presents a…

Comparing neural networks and Kriging for fitness approximation in evolutionary optimization

Lars Willmes, Thomas Bäck, Yaochu Jin +1 more

Artificial Intelligence · Surrey Research Insight Open Access (The University of Surrey) · 2003 · cited 65× · DOI ↗ · more like this ·

Neural networks and Kriging method are compared for constructing fitness approximation models in evolutionary optimization algorithms. The two models are applied in an identical framework to the optimization of a number of well known test functions. In addition, two different ways of training the approximators are evaluated: in one setting the models are built off-line using data from previous optimization runs and in the other setting the models are built online from the data available from the current optimization.

Adaptive encoding for aerodynamic shape optimization using evolution strategies

Markus Olhofer, Yaochu Jin, Bernhard Sendhoff

Computational Theory and Mathematics · View · 13 Nov 2002 · cited 62× · DOI ↗ · more like this ·

The evaluation of fluid dynamic properties of various different structures is a computationally very demanding process. This is of particular importance when population based evolutionary algorithms are used for the optimization of aerodynamic structures like wings or turbine blades. Besides choosing algorithms which only need few generations or function evaluations, it is important to reduce the number of object parameters as much as possible. This is usually done by restricting the optimization to certain attributes of the design which are seen as important. By doing so, the freedom for the optimization is restricted to areas of the design space where good solutions are expected. This can be problematic especially if the properties of the design and their interactions are not known sufficiently well like for example for transonic flow conditions. In order to be able to combine the…

Connectedness, regularity and the success of local search in evolutionary multi-objective optimization

Yaochu Jin, Bernhard Sendhoff

Computational Theory and Mathematics · Surrey Research Insight Open Access (The University of Surrey) · 9 Jul 2004 · cited 57× · DOI ↗ · more like this ·

Local search techniques have proved to be very efficient in evolutionary multi-objective optimization (MOO). However, the reasons behind the success of local search in MOO have not yet been well discussed. This paper attempts to investigate empirically the main factors that may have contributed significantly to the success of local search in MOO. It is found that for many widely used test problems, the Pareto optimal solutions are connected both in objective space and parameter space. Besides, the Pareto-optimal solutions often distribute so regularly in parameter space that they can be defined by piecewise linear functions. By constructing an approximate model using the solutions produced by an optimizer, the quality of the non-dominated solution set can be further improved. The evolutionary dynamic weighted aggregation (EDWA) method has been adopted as a local search technique in…

Fitness Approximation In Evolutionary Computation - a Survey

Yaochu Jin, Bernhard Sendhoff

Artificial Intelligence · View · 9 Jul 2002 · cited 53× · DOI ↗ · more like this ·

Fitness evaluation is a basic operation in evolutionary computation. However, an explicit fitness function is not always available in real-world applications. In many cases, it is necessary to estimate the fitness function by constructing an approximate model. In this paper, a short survey on fitness approximation in evolutionary computation is given. Main issues like approximation models, model management schemes, learning methods as well as data sampling techniques are presented. Finally, interesting open problems are discussed.

Evolution Strategies for Robust Optimization

Hans-Georg Beyer, Bernhard Sendhoff

Artificial Intelligence · 22 Sep 2006 · cited 51× · DOI ↗ · more like this ·

In this paper, we propose two evolutionary strategies for the optimization of problems with actuator noise as encountered in robust optimization, where the design or objective parameters are subject to noise: the ROSAES and the ROCSAES. Both algorithms use a control rule for increasing the population size when the residual error to the optimizer state has been reached. Theoretical analysis has previously shown that the residual error depends among other factors on the population size and on the variance of the noise. Furthermore, ROSAES exploits the similarity of the mutation term in evolutionary strategies and the additive noise term in the case of actuator noise. The population variance is controlled to guarantee that the realized noise level is adjusted correctly. Simulations are carried out on test functions and the results are analyzed with respect to the performance and the…

Co-evolutionary Modular Neural Networks for Automatic Problem Decomposition

Vaibhav Khare, Xin Yao, Bernhard Sendhoff +2 more

Artificial Intelligence · Surrey Research Insight Open Access (The University of Surrey) · 13 Dec 2005 · cited 46× · DOI ↗ · more like this ·

Decomposing a complex computational problem into sub-problems, which are computationally simpler to solve individually and which can be combined to produce a solution to the full problem, can efficiently lead to compact and general solutions. Modular neural networks represent one of the ways in which this divide-and-conquer strategy can be implemented. Here we present a co-evolutionary model which is used to design and optimize modular neural networks with task-specific modules. The model consists of two populations. The first population consists of a pool of modules and the second population synthesizes complete systems by drawing elements from the pool of modules. Modules represent a part of the solution, which co-operates with others in the module population to form a complete solution. With the help of two artificial supervised learning tasks created by mixing two sub-tasks we…

Robust Optimization Over Time: Problem Difficulties and Benchmark Problems

Haobo Fu, Bernhard Sendhoff, Ke Tang +1 more

Artificial Intelligence · IEEE Transactions on Evolutionary Computation · 4 Dec 2014 · cited 40× · DOI ↗ · more like this ·

The focus of most research in evolutionary dynamic optimization has been tracking moving optimum (TMO). Yet, TMO does not capture all the characteristics of real-world dynamic optimization problems (DOPs), especially in situations where a solution's future fitness has to be considered. To account for a solution's future fitness explicitly, we propose to find robust solutions to DOPs, which are formulated as the robust optimization over time (ROOT) problem. In this paper we analyze two robustness definitions in ROOT and then develop two types of benchmark problems for the two robustness definitions in ROOT, respectively. The two types of benchmark problems are motivated by the inappropriateness of existing DOP benchmarks for the study of ROOT. Additionally, we evaluate four representative methods from the literature on our proposed ROOT benchmarks, in order to gain a better understanding…

Functions with noise-induced multimodality: a test for evolutionary robust Optimization-properties and performance analysis

Hans-Georg Beyer, Bernhard Sendhoff

Artificial Intelligence · IEEE Transactions on Evolutionary Computation · October 2006 · cited 35× · DOI ↗ · more like this ·

This paper proposes and analyzes a class of test functions for evolutionary robust optimization, the "functions with noise-induced multimodality" (FNIMs). After a motivational introduction gleaned from a real-world optimization problem, the robust optimizer properties of this test class are investigated with respect to different robustness measures. The steady-state behavior of evolution strategies on FNIMs will be investigated empirically. Being based on the empirical results, a subclass of FNIMs is identified which is amenable to an asymptotical performance analysis. The results of this analysis will be used to derive recommendations for the choice of strategy-specific parameters such as population size and truncation ratio