Derivative-Free Optimization (DFO)

Wider research context/theoretical framework. Optimization is one of the basic techniques for improving existing products or procedures to better match desired goals encoded in an objective function, under restrictions encoded into the constraints. A growing number of applications in science and engineering have objectives without well-defined functional description, e.g., when function values arise from expensive measurements or complex simulations. This fueled a resurgence of interest in derivative-free optimization algorithms since these specifically work with functions and constraints for which no derivative information is available. Our group has long-term expertise in derivative-free optimization, contributing both to deterministic and stochastic algorithms with theoretical convergence guarantees and to easily parallelizable stochastic algorithms based on the heuristic adaptation of populations of good points.

Hypotheses/research questions/objectives. The goal of this project is the extension of the techniques used in our recent unconstrained black box solvers STBBO and VSBBON to the case of bound constraints, linear constraints, and nonlinear inequality and equality constraints. We aim at a robust and efficient derivativefree solver suite that copes with expensive, often noisy function values and high dimensions, the bottleneck of current techniques. We exploit the available problem structure, in particular sparsity and local effective low-dimensionality.

Approach/methods. We shall integrate many known techniques for constraint handling into our recent (as yet unconstrained) solvers, selecting the best from many possible variants. To facilitate this we design a strategy language for specifying algorithmic variants on a high level. This will enable us to efficiently design combination strategies and permit the automatization of tuning them. To facilitate tuning and comparison of our algorithms with those of others, we plan to extend our recent test environment to the constrained case, implement a self-tuning environment that applies the new solvers to optimize their parameters and their available choices.

Level of originality/innovation. (i) The systematic integration of efficient heuristic techniques into provably convergent schemes while preserving the latter"s theoretical complexity guarantees is new, (ii) Software for handling nonlinear equality constraints is virtually nonexistent, (iii) Very little prior work by others exists on sparsity handling and low rank methods in derivative-free optimization, (iv) We will do extensive benchmarking and tuning of all current state-of-the-art solvers including those to be developed, thereby presenting each solver in its best light.

Primary researcher involved. Mr. Morteza Kimiaei nearly finished his Ph.D. work on bound-constrained and derivative-free optimization, and is ideally equipped to work on this project.