DFO-LS Optimization Methods

The Derivative-Free Optimizer for Least-Squares (DFO-LS) [1] is an algorithm especially constructed to handle objective functions formulated as least-squares problems.

Configuration

Requirements and optional keys for setting up the configuration file to use DFO-LS are given below.

Codes Blocks

No special configuration is needed in any portion of the codes blocks to use DFO-LS.

Options

The following required and optional keys can be used within the options: block:

  • software [str (required)]: dfols

  • method [str (required)]: dfols

  • components [int (required)]: Number of residuals in the objective function (see also objective function setup).

  • software_options [dict (optional)]: Pass options directly to the user_params dictionary in the dfols instance. See [2] for a list of available options.

Objective Function

The objective function must return a tuple of type (float, iter[floats]) where the first term is the sum of residuals \(\sum_i^n{r_i(x)^2}\) and the iterable contains a vector of the residuals \((r_i(x), ..., r_n(x))\).

Example snippets (Using NumPy to form the residual vectors):

 1   import numpy as np
 2   # As passed to options.objective_function:
 3   def obj_f(J):
 4       # Objective function is always passed
 5       # to the rsopt job dictionary `J`
 6
 7       # ... Code to calculate `residuals`
 8
 9       r = np.array(residuals)
10       r_sum = np.sum(r**2)
11
12       return r_sum, r
13
14   # Example if using code block type `python`
15   # without options.objective_function:
16   def my_function(x, y):
17       # Assuming user defined `parameters` (x, y)
18       # in the configuration file
19
20       x0, y0 = 5, 5
21
22       r = np.array([x - x0, y - y0])
23       r_sum = np.sum(r**2)
24
25       return r_sum, r

Example Options Block

options:
 software: dfols
 method: dfols
 components: 214
 # Run until tolerance is reached or exit_criteria is hit
 software_options: {'user_params': {'model.rel_tol': 1e-4}}
 exit_criteria:
   # If model.rel_tol isn't reached after 400 simulations then rsopt will terminate
   sim_max: 400
 # objective_function can be optional if using python in codes
 objective_function: [objective.py, obj_f]

See rsopt/examples/python_chwirut_example for an example using DFO-LS