Source code¶
pySOT.adaptive_sampling module¶
Module:  adaptive_sampling 

Author:  David Eriksson <dme65@cornell.edu>, David Bindel <bindel@cornell.edu> 

class
pySOT.adaptive_sampling.
CandidateDDS
(data, numcand=None, weights=None)¶ An implementation of the DDS candidate points method
Only a few candidate points are generated and the candidate point with the lowest value predicted by the surrogate model is selected. The DDS method only perturbs a subset of the dimensions when perturbing the best solution. The probability for a dimension to be perturbed decreases after each evaluation and is capped in order to guarantee global convergence.
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateDDS_CONT
(data, numcand=None, weights=None)¶ CandidateDDS where only the the continuous variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateDDS_INT
(data, numcand=None, weights=None)¶ CandidateDDS where only the the integer variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateDYCORS
(data, numcand=None, weights=None)¶ An implementation of the DYCORS method
The DYCORS method only perturbs a subset of the dimensions when perturbing the best solution. The probability for a dimension to be perturbed decreases after each evaluation and is capped in order to guarantee global convergence.
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 minprob – Smallest allowed perturbation probability
 n0 – Evaluations spent when the initial phase ended
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateDYCORS_CONT
(data, numcand=None, weights=None)¶ CandidateDYCORS where only the the continuous variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateDYCORS_INT
(data, numcand=None, weights=None)¶ CandidateDYCORS where only the the integer variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateSRBF
(data, numcand=None, weights=None)¶ An implementation of Stochastic RBF
This is an implementation of the candidate points method that is proposed in the first SRBF paper. Candidate points are generated by making normally distributed perturbations with standard deviation sigma around the best solution. The candidate point that minimizes a specified merit function is selected as the next point to evaluate.
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateSRBF_CONT
(data, numcand=None, weights=None)¶ CandidateSRBF where only the the continuous variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateSRBF_INT
(data, numcand=None, weights=None)¶ CandidateSRBF where only the the integer variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateUniform
(data, numcand=None, weights=None)¶ Create Candidate points by sampling uniformly in the domain
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateUniform_CONT
(data, numcand=None, weights=None)¶ CandidateUniform where only the the continuous variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
CandidateUniform_INT
(data, numcand=None, weights=None)¶ CandidateUniform where only the the integer variables are perturbed
Parameters:  data (Object) – Optimization problem object
 numcand (int) – Number of candidate points to be used. Default is min([5000, 100*data.dim])
 weights (list of numpy.array) – Weights used for the merit function, to balance exploration vs exploitation
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 weights – Weights used for the merit function
 proposed_points – List of points proposed to the optimization algorithm
 dmerit – Minimum distance between the points and the proposed points
 xcand – Candidate points
 fhvals – Predicted values by the surrogate model
 next_weight – Index of the next weight to be used
 numcand – Number of candidate points
 budget – Remaining evaluation budget
 probfun – Function that computes the perturbation probability of a given iteration
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb, the others are fixed
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
GeneticAlgorithm
(data)¶ Genetic algorithm for minimizing the surrogate model
Parameters: data (Object) – Optimization problem object
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 proposed_points – List of points proposed to the optimization algorithm
 budget – Remaining evaluation budget
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=None)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far (Ignored)
 sigma (float) – Current sampling radius w.r.t the unit box (Ignored)
 subset (numpy.array) – Coordinates to perturb (Ignored)
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points (Ignored)
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
MultiSampling
(strategy_list, cycle)¶ Maintains a list of adaptive sampling methods
A collection of adaptive sampling methods and weights so that the user can use multiple adaptive sampling methods for the same optimization problem. This object keeps an internal list of proposed points in order to be able to compute the minimum distance from a point to all proposed evaluations. This list has to be reset each time the optimization algorithm restarts
Parameters:  strategy_list (list) – List of adaptive sampling methods to use
 cycle (list) – List of integers that specifies the sampling order, e.g., [0, 0, 1] uses method1, method1, method2, method1, method1, method2, ...
Raises: ValueError – If cycle is incorrect
Variables:  sampling_strategies – List of adaptive sampling methods to use
 cycle – List that specifies the sampling order
 nstrats – Number of adaptive sampling strategies
 current_strat – The next adaptive sampling strategy to be used
 proposed_points – List of points proposed to the optimization algorithm
 data – Optimization problem object
 fhat – Response surface object
 budget – Remaining evaluation budget
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Todo
Get rid of the proposed_points object and replace it by something that is controlled by the strategy.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=<function candidate_merit_weighted_distance>)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far
 sigma (float) – Current sampling radius w.r.t the unit box
 subset (numpy.array) – Coordinates to perturb
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
Todo
Change the merit function from being hardcoded

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool

class
pySOT.adaptive_sampling.
MultiStartGradient
(data, method='LBFGSB', num_restarts=30)¶ A MultiStart Gradient method for minimizing the surrogate model
A wrapper around the scipy.optimize implementation of boxconstrained gradient based minimization.
Parameters:  data (Object) – Optimization problem object
 method (string) –
Optimization method to use. The options are
 LBFGSB
 QuasiNewton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS)
 TNC
 Truncated Newton algorithm
 num_restarts (int) – Number of restarts for the multistart gradient
Raises: ValueError – If number of candidate points is incorrect or if the weights aren’t a list in [0, 1]
Variables:  data – Optimization problem object
 fhat – Response surface object
 xrange – Variable ranges, xup  xlow
 dtol – Smallest allowed distance between evaluated points 1e3 * sqrt(dim)
 bounds – n x 2 matrix with lower and upper bound constraints
 proposed_points – List of points proposed to the optimization algorithm
 budget – Remaining evaluation budget
Note
This object needs to be initialized with the init method. This is done when the initial phase has finished.
Note
SLSQP is supposed to work with bound constraints but for some reason it sometimes violates the constraints anyway.

init
(start_sample, fhat, budget)¶ Initialize the sampling method after the initial phase
This initializes the list of sampling methods after the initial phase has finished and the experimental design has been evaluated. The user provides the points in the experimental design, the surrogate model, and the remaining evaluation budget.
Parameters:  start_sample (numpy.array) – Points in the experimental design
 fhat (Object) – Surrogate model
 budget (int) – Evaluation budget

make_points
(npts, xbest, sigma, subset=None, proj_fun=None, merit=None)¶ Proposes npts new points to evaluate
Parameters:  npts (int) – Number of points to select
 xbest (numpy.array) – Best solution found so far (Ignored)
 sigma (float) – Current sampling radius w.r.t the unit box (Ignored)
 subset (numpy.array) – Coordinates to perturb (Ignored)
 proj_fun (Object) – Routine for projecting infeasible points onto the feasible region
 merit (Object) – Merit function for selecting candidate points (Ignored)
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array

remove_point
(x)¶ Remove x from proposed_points
This removes x from the list of proposed points in the case where the optimization strategy decides to not evaluate x.
Parameters: x (numpy.array) – Point to be removed Returns: True if points was removed, False otherwise Type: bool
pySOT.ensemble_surrogate module¶
Module:  ensemble_surrogate 

Author:  David Eriksson <dme65@cornell.edu> 

class
pySOT.ensemble_surrogate.
EnsembleSurrogate
(model_list, maxp=100)¶ Compute and evaluate an ensemble of interpolants.
Maintains a list of surrogates and decides how to weights them by using DempsterShafer theory to assign pignistic probabilities based on statistics computed using LOOCV.
Parameters:  model_list (list) – List of surrogate models
 maxp (int) – Maximum number of points
Variables:  nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 rhs – Right hand side for interpolation system
 x – Interpolation points
 fx – Values at interpolation points
 dim – Number of dimensions
 model_list – List of surrogate models
 weights – Weight for each surrogate model
 surrogate_list – List of internal surrogate models for LOOCV

add_point
(xx, fx)¶ Add a new function evaluation
This function also updates the list of LOOCV surrogate models by cleverly just adding one point to n of the models. The scheme in which new models are built is illustrated below:
2 1 1,2
2,3 1,3 1,2 1,2,3
2,3,4 1,3,4 1,2,4 1,2,3 1,2,3,4
2,3,4,5 1,3,4,5 1,2,4,5 1,2,3,5 1,2,3,4 1,2,3,4,5
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

compute_weights
()¶ Compute mode weights
Given n observations we use n surrogates built with n1 of the points in order to predict the value at the removed point. Based on these n predictions we calculate three different statistics:
 Correlation coefficient with true function values
 Root mean square deviation
 Mean absolute error
Based on these three statistics we compute the model weights by applying DempsterShafer theory to first compute the pignistic probabilities, which are taken as model weights.
Returns: Model weights Return type: numpy.array

deriv
(x, d=None)¶ Evaluate the derivative of the ensemble surrogate at the point x
Parameters: x (numpy.array) – Point for which we want to compute the RBF gradient Returns: Derivative of the ensemble surrogate at x Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the ensemble surrogate the point xx
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Value of the ensemble surrogate at x
Return type: float

evals
(x, ds=None)¶ Evaluate the ensemble surrogate at the points xx
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (numpy.array) – Distances between the centers and the points x, of size npts x ncenters
Returns: Values of the ensemble surrogate at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the ensemble surrogate.
pySOT.experimental_design module¶
Module:  experimental_design 

Author:  David Eriksson <dme65@cornell.edu> Yi Shen <ys623@cornell.edu> 

class
pySOT.experimental_design.
BoxBehnken
(dim)¶ BoxBehnken experimental design
The BoxBehnken experimental design consists of the midpoint of the edges plus a center point of the unit hypercube
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 npts – Number of desired sampling points (2^dim)

generate_points
()¶ Generate a matrix with the initial sample points, scaled to the unit hypercube
Returns: BoxBehnken design in the unit cube of size npts x dim Return type: numpy.array

class
pySOT.experimental_design.
LatinHypercube
(dim, npts, criterion='c')¶ Latin Hypercube experimental design
Parameters:  dim (int) – Number of dimensions
 npts (int) – Number of desired sampling points
 criterion (string) –
Sampling criterion
 “center” or “c”
 center the points within the sampling intervals
 “maximin” or “m”
 maximize the minimum distance between points, but place the point in a randomized location within its interval
 “centermaximin” or “cm”
 same as “maximin”, but centered within the intervals
 “correlation” or “corr”
 minimize the maximum correlation coefficient
Variables:  dim – Number of dimensions
 npts – Number of desired sampling points
 criterion – A string that specifies how to sample

generate_points
()¶ Generate a matrix with the initial sample points, scaled to the unit hypercube
Returns: Latin hypercube design in the unit cube of size npts x dim Return type: numpy.array

class
pySOT.experimental_design.
SymmetricLatinHypercube
(dim, npts)¶ Symmetric Latin Hypercube experimental design
Parameters:  dim (int) – Number of dimensions
 npts (int) – Number of desired sampling points
Variables:  dim – Number of dimensions
 npts – Number of desired sampling points

generate_points
()¶ Generate a matrix with the initial sample points, scaled to the unit hypercube
Returns: Symmetric Latin hypercube design in the unit cube of size npts x dim that is of full rank Return type: numpy.array Raises: ValueError – Unable to find an SLHD of rank at least dim + 1

class
pySOT.experimental_design.
TwoFactorial
(dim)¶ Twofactorial experimental design
The twofactorial experimental design consists of the corners of the unit hypercube, and hence \(2^{dim}\) points.
Parameters: dim (int) – Number of dimensions
Raises: ValueError – If dim >= 15
Variables:  dim – Number of dimensions
 npts – Number of desired sampling points (2^dim)

generate_points
()¶ Generate a matrix with the initial sample points, scaled to the unit hypercube
Returns: Fullfactorial design in the unit cube of size (2^dim) x dim Return type: numpy.array
pySOT.heuristic_methods module¶
Module:  heuristic_methods 

Author:  David Eriksson <dme65@cornell.edu> 

class
pySOT.heuristic_methods.
GeneticAlgorithm
(function, dim, xlow, xup, intvar=None, popsize=100, ngen=100, start='SLHD', projfun=None)¶ Genetic algorithm
This is an implementation of the realvalued Genetic algorithm that is useful for optimizing on a surrogate model, but it can also be used on its own. The mutations are normally distributed perturbations, the selection mechanism is a tournament selection, and the crossover oepration is the standard linear combination taken at a randomly generated cutting point.
The number of evaluations are popsize x ngen
Parameters:  function (Object) – Function that can be used to evaluate the entire population. It needs to take an input of size nindividuals x nvariables and return a numpy.array of length nindividuals
 dim (int) – Number of dimensions
 xlow (numpy.array) – Lower variable bounds, of length dim
 xup (numpy.array) – Lower variable bounds, of length dim
 intvar (list) – List of indices with the integer valued variables (e.g., [0, 1, 5])
 popsize (int) – Population size
 ngen (int) – Number of generations
 start (string) – Method for generating the initial population
 proj_fun (Object) – Function that can project ONE infeasible individual onto the feasible region
Variables:  nvariables – Number of variables (dimensions) of the objective function
 nindividuals – population size
 lower_boundary – lower bounds for the optimization problem
 upper_boundary – upper bounds for the optimization problem
 integer_variables – List of variables that are integer valued
 start – Method for generating the initial population
 sigma – Perturbation radius. Each pertubation is N(0, sigma)
 p_mutation – Mutation probability (1/dim)
 tournament_size – Size of the tournament (5)
 p_cross – Crossover probability (0.9)
 ngenerations – Number of generations
 function – Object that can be used to evaluate the objective function
 projfun – Function that can be used to project an individual onto the feasible region

optimize
()¶ Method used to run the Genetic algorithm
Returns: Returns the best individual and its function value Return type: numpy.array, float
pySOT.gp_regression module¶
Module:  gp_regression 

Author:  David Eriksson <dme65@cornell.edu> 

class
pySOT.gp_regression.
GPRegression
(maxp=100, gp=None)¶ Compute and evaluate a GP
Gaussian Process Regression object.
Depends on scitkitlearn==0.18.1.
 More details:
 http://scikitlearn.org/stable/modules/generated/sklearn.gaussian_process.GaussianProcessRegressor.html
Parameters:  maxp (int) – Initial capacity
 gp (GaussianProcessRegressor) – GP object (can be None)
Variables:  nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 x – Interpolation points
 fx – Function evaluations of interpolation points
 gp – Object of type GaussianProcessRegressor
 dim – Number of dimensions
 model – MARS interpolation model

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the GP regression object at a point x
Parameters:  x (numpy.array) – Point for which we want to compute the GP regression gradient
 ds (None) – Not used
Returns: Derivative of the GP regression object at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the GP regression object at the point x
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Value of the GP regression obejct at x
Return type: float

evals
(x, ds=None)¶ Evaluate the GP regression object at the points x
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (None) – Not used
Returns: Values of the GP regression object at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the interpolation.
pySOT.mars_interpolant module¶
Module:  mars_interpolant 

Author:  Yi Shen <ys623@cornell.edu> 

class
pySOT.mars_interpolant.
MARSInterpolant
(maxp=100)¶ Compute and evaluate a MARS interpolant
MARS builds a model of the form
\[\hat{f}(x) = \sum_{i=1}^{k} c_i B_i(x).\]The model is a weighted sum of basis functions \(B_i(x)\). Each basis function \(B_i(x)\) takes one of the following three forms:
 a constant 1.
 a hinge function of the form \(\max(0, x  const)\) or \(\max(0, const  x)\). MARS automatically selects variables and values of those variables for knots of the hinge functions.
 a product of two or more hinge functions. These basis functions c an model interaction between two or more variables.
Parameters: maxp (int) – Initial capacity
Variables:  nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 x – Interpolation points
 fx – Function evaluations of interpolation points
 dim – Number of dimensions
 model – MARS interpolation model

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the derivative of the MARS interpolant at a point x
Parameters:  x (numpy.array) – Point for which we want to compute the MARS gradient
 ds (None) – Not used
Returns: Derivative of the MARS interpolant at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the MARS interpolant at the point x
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Value of the MARS interpolant at x
Return type: float

evals
(x, ds=None)¶ Evaluate the MARS interpolant at the points x
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (None) – Not used
Returns: Values of the MARS interpolant at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the interpolation.
pySOT.merit_functions module¶
Module:  merit_functions 

Author:  David Eriksson <dme65@cornell.edu>, David Bindel <bindel@cornell.edu> 

pySOT.merit_functions.
candidate_merit_weighted_distance
(cand, npts=1)¶ Weighted distance merit function for the candidate points based methods
Parameters:  cand (Object) – Candidate point object
 npts (int) – Number of points selected for evaluation
Returns: Points selected for evaluation, of size npts x dim
Return type: numpy.array
pySOT.poly_regression module¶
Module:  poly_regression 

Author:  David Bindel <bindel@cornell.edu> 

class
pySOT.poly_regression.
PolyRegression
(bounds, basisp, maxp=100)¶ Compute and evaluate a polynomial regression surface.
Parameters:  bounds (numpy.array) – a (dims, 2) array of lower and upper bounds in each coordinate
 basisp (numpy.array) – a (nbasis, dims) array, where the ith basis function is prod_j L_basisp(i,j)(x_j), L_k = the degree k Legendre polynomial
 maxp (int) – Initial point capacity
Variables:  nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 x – Interpolation points
 fx – Function evaluations of interpolation points
 bounds – Upper and lower bounds, one row per dimension
 dim – Number of dimensions
 basisp – Multiindices representing terms in a tensor poly basis Each row is a list of dim indices indicating a polynomial degree in the associated dimension.
 updated – True if the RBF coefficients are up to date

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx – Point to add
 fx – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the derivative of the regression surface at a point x
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Derivative of the polynomial at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the regression surface at point xx
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Prediction at the point x
Return type: float

evals
(x, ds=None)¶ Evaluate the regression surface at points x
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (None) – Not used
Returns: Prediction at the points x
Return type: float

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the object.

pySOT.poly_regression.
basis_HC
(n, d)¶ Generate list of shape functions for HC poly space.
Parameters:  n (int) – Dimension of the space
 d (int) – Degree bound
Returns: An Nbyn matrix with S(i,j) = degree of variable j in shape i
Return type: numpy.array

pySOT.poly_regression.
basis_SM
(n, d)¶ Generate list of shape functions for SM poly space.
Parameters:  n (int) – Dimension of the space
 d (int) – Degree bound
Returns: An Nbyn matrix with S(i,j) = degree of variable j in shape i
Return type: numpy.array

pySOT.poly_regression.
basis_TD
(n, d)¶ Generate list of shape functions for TD poly space.
Parameters:  n (int) – Dimension of the space
 d (int) – Degree bound
Returns: An Nbyn matrix with S(i,j) = degree of variable j in shape i
Return type: numpy.array

pySOT.poly_regression.
basis_TP
(n, d)¶ Generate list of shape functions for TP poly space.
Parameters:  n (int) – Dimension of the space
 d (int) – Degree bound
Returns: An Nbyn matrix with S(i,j) = degree of variable j in shape i There are N = n^d shapes.
Return type: numpy.array

pySOT.poly_regression.
basis_base
(n, testf)¶ Generate list of shape functions for a subset of a TP poly space.
Parameters:  n (int) – Dimension of the space
 testf (Object) – Return True if a given multiindex is in range
Returns: An Nbyn matrix with S(i,j) = degree of variable j in shape i
Return type: numpy.array

pySOT.poly_regression.
dlegendre
(x, d)¶ Evaluate Legendre polynomial derivatives at all coordinates in x.
Parameters:  x (numpy.array) – Array of coordinates
 d (int) – Max degree of polynomials
Returns: x.shapebyd arrays of Legendre polynomial values and derivatives
Return type: numpy.array

pySOT.poly_regression.
legendre
(x, d)¶ Evaluate Legendre polynomials at all coordinates in x.
Parameters:  x (numpy.array) – Array of coordinates
 d (int) – Max degree of polynomials
Returns: A x.shapebyd array of Legendre polynomial values
Return type: numpy.array

pySOT.poly_regression.
test_legendre1
()¶

pySOT.poly_regression.
test_legendre2
()¶

pySOT.poly_regression.
test_poly
()¶
pySOT.kernels module¶
Module:  kernels 

Author:  David Eriksson <dme65@cornell.edu>, 

class
pySOT.kernels.
CubicKernel
¶ Cubic RBF kernel
This is a basic class for the Cubic RBF kernel: \(\varphi(r) = r^3\) which is conditionally positive definite of order 2.

deriv
(dists)¶ evaluates the derivative of the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(3 \ x_i  x_j \^2\) Return type: numpy.array

eval
(dists)¶ evaluates the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\x_i  x_j \^3\) Return type: numpy.array

order
()¶ returns the order of the Cubic RBF kernel
Returns: 2 Return type: int

phi_zero
()¶ returns the value of \(\varphi(0)\) for Cubic RBF kernel
Returns: 0 Return type: float


class
pySOT.kernels.
LinearKernel
¶ Linear RBF kernel
This is a basic class for the Linear RBF kernel: \(\varphi(r) = r\) which is conditionally positive definite of order 1.

deriv
(dists)¶ evaluates the derivative of the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is 1 Return type: numpy.array

eval
(dists)¶ evaluates the Linear kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\x_i  x_j \\) Return type: numpy.array

order
()¶ returns the order of the Linear RBF kernel
Returns: 1 Return type: int

phi_zero
()¶ returns the value of \(\varphi(0)\) for Linear RBF kernel
Returns: 0 Return type: float


class
pySOT.kernels.
TPSKernel
¶ Thinplate spline RBF kernel
This is a basic class for the TPS RBF kernel: \(\varphi(r) = r^2 \log(r)\) which is conditionally positive definite of order 2.

deriv
(dists)¶ evaluates the derivative of the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\x_i  x_j \(1 + 2 \log (\x_i  x_j \) )\) Return type: numpy.array

eval
(dists)¶ evaluates the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\x_i  x_j \^2 \log (\x_i  x_j \)\) Return type: numpy.array

order
()¶ returns the order of the TPS RBF kernel
Returns: 2 Return type: int

phi_zero
()¶ returns the value of \(\varphi(0)\) for TPS RBF kernel
Returns: 0 Return type: float

pySOT.tails module¶
Module:  tails 

Author:  David Eriksson <dme65@cornell.edu>, 

class
pySOT.tails.
ConstantTail
¶ Constant polynomial tail
This is a standard linear polynomial in ddimension, built from the basis \(\{1\}\).

degree
()¶ returns the degree of the constant polynomial tail
Returns: 0 Return type: int

deriv
(x)¶ evaluates the gradient of the linear polynomial tail for one point
Parameters: x (numpy.array) – Point to evaluate, of length dim Returns: A numpy.array of size dim x dim_tail(dim) Return type: numpy.array

dim_tail
(dim)¶ returns the dimensionality of the constant polynomial space for a given dimension
Parameters: dim (int) – Number of dimensions of the Cartesian space Returns: 1 Return type: int

eval
(X)¶ evaluates the constant polynomial tail for a set of points
Parameters: X (numpy.array) – Points to evaluate, of size npts x dim Returns: A numpy.array of size npts x dim_tail(dim) Return type: numpy.array


class
pySOT.tails.
LinearTail
¶ Linear polynomial tail
This is a standard linear polynomial in ddimension, built from the basis \(\{1,x_1,x_2,\ldots,x_d\}\).

degree
()¶ returns the degree of the linear polynomial tail
Returns: 1 Return type: int

deriv
(x)¶ evaluates the gradient of the linear polynomial tail for one point
Parameters: x (numpy.array) – Point to evaluate, of length dim Returns: A numpy.array of size dim x dim_tail(dim) Return type: numpy.array

dim_tail
(dim)¶ returns the dimensionality of the linear polynomial space for a given dimension
Parameters: dim (int) – Number of dimensions of the Cartesian space Returns: 1 + dim Return type: int

eval
(X)¶ evaluates the linear polynomial tail for a set of points
Parameters: X (numpy.array) – Points to evaluate, of size npts x dim Returns: A numpy.array of size npts x dim_tail(dim) Return type: numpy.array

pySOT.rbf module¶
Module:  rbf 

Author:  David Eriksson <dme65@cornell.edu>, David Bindel <bindel@cornell.edu> 

class
pySOT.rbf.
RBFInterpolant
(kernel=<class 'pySOT.kernels.CubicKernel'>, tail=<class 'pySOT.tails.LinearTail'>, maxp=500, eta=1e08)¶ Compute and evaluate RBF interpolant.
Manages an expansion of the form
\[f(x) = \sum_j c_j \phi(\xx_j\) + \sum_j \lambda_j p_j(x)\]where the functions \(p_j(x)\) are lowdegree polynomials. The fitting equations are
\[\begin{split}\begin{bmatrix} \eta I & P^T \\ P & \Phi+\eta I \end{bmatrix} \begin{bmatrix} \lambda \\ c \end{bmatrix} = \begin{bmatrix} 0 \\ f \end{bmatrix}\end{split}\]where \(P_{ij} = p_j(x_i)\) and \(\Phi_{ij}=\phi(\x_ix_j\)\). The regularization parameter \(\eta\) allows us to avoid problems with potential poor conditioning of the system. The regularization parameter can either be fixed or estimated via LOOCV. Specify eta=’adapt’ for estimation.
Parameters:  kernel (Kernel) – RBF kernel object
 tail (Tail) – RBF polynomial tail object
 maxp (int) – Initial point capacity
 eta (float or 'adapt') – Regularization parameter
Variables:  kernel – RBF kernel
 tail – RBF tail
 eta – Regularization parameter
 ntail – Number of tail functions
 nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 A – Interpolation system matrix
 LU – LUfactorization of the RBF system
 piv – pivot vector for the LUfactorization
 rhs – Right hand side for interpolation system
 x – Interpolation points
 fx – Values at interpolation points
 c – Expansion coefficients
 dim – Number of dimensions
 ntail – Number of tail functions
 updated – True if the RBF coefficients are up to date

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

coeffs
()¶ Compute the expansion coefficients
Returns: Expansion coefficients Return type: numpy.array

deriv
(x, ds=None)¶ Evaluate the derivative of the RBF interpolant at a point x
Parameters:  x (numpy.array) – Point for which we want to compute the RBF gradient
 ds (numpy.array) – Distances between the centers and the point x
Returns: Derivative of the RBF interpolant at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the RBF interpolant at the point x
Parameters: x (numpy.array) – Point where to evaluate Returns: Value of the RBF interpolant at x Return type: float

evals
(x, ds=None)¶ Evaluate the RBF interpolant at the points x
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (numpy.array) – Distances between the centers and the points x, of size npts x ncenters
Returns: Values of the rbf interpolant at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the RBF interpolant

transform_fx
(fx)¶ Replace f with transformed function values for the fitting
Parameters: fx (numpy.array) – Transformed function values
pySOT.rs_wrappers module¶
Module:  rs_wrappers 

Author:  David Bindel <bindel@cornell.edu> 

class
pySOT.rs_wrappers.
RSCapped
(model, transformation=None)¶ Cap adapter for response surfaces.
This adapter takes an existing response surface and replaces it with a modified version in which the function values are replaced according to some transformation. A very common transformation is to replace all values above the median by the median in order to reduce the influence of large function values.
Parameters:  model (Object) – Original response surface object
 transformation (Object) – Function value transformation object. Median capping is used if no object (or None) is provided
Variables:  transformation – Object used to transform the function values.
 model – original response surface object
 fvalues – Function values
 nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 updated – True if the surface is updated

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the derivative of the capped interpolant at a point x
Parameters:  x (numpy.array) – Point for which we want to compute the RBF gradient
 ds (numpy.array) – Distances between the centers and the point x
Returns: Derivative of the capped interpolant at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the capped interpolant at the point x
Parameters: x (numpy.array) – Point where to evaluate Returns: Value of the RBF interpolant at x Return type: float

evals
(x, ds=None)¶ Evaluate the capped interpolant at the points x
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (numpy.array) – Distances between the centers and the points x, of size npts x ncenters
Returns: Values of the capped interpolant at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the capped response surface

class
pySOT.rs_wrappers.
RSPenalty
(model, evals, derivs)¶ Penalty adapter for response surfaces.
This adapter can be used for approximating an objective function plus a penalty function. The response surface is fitted only to the objective function and the penalty is added on after.
Parameters:  model (Object) – Original response surface object
 evals (Object) – Object that takes the response surface and the points and adds up the response surface value and the penalty function value
 devals (Object) – Object that takes the response surface and the points and adds up the response surface derivative and the penalty function derivative
Variables:  eval_method – Object that takes the response surface and the points and adds up the response surface value and the penalty function value
 deval_method – Object that takes the response surface and the points and adds up the response surface derivative and the penalty function derivative
 model – original response surface object
 fvalues – Function values
 nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 updated – True if the surface is updated

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the derivative of the penalty adapter at x
Parameters:  x (numpy.array) – Point for which we want to compute the gradient
 ds (None) – Not used
Returns: Derivative of the interpolant at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the penalty adapter interpolant at the point xx
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Value of the interpolant at x
Return type: float

evals
(x, ds=None)¶ Evaluate the penalty adapter at the points xx
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (None) – Not used
Returns: Values of the interpolant at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the capped response surface

class
pySOT.rs_wrappers.
RSUnitbox
(model, data)¶ Unit box adapter for response surfaces
This adapter takes an existing response surface and replaces it with a modified version where the domain is rescaled to the unit box. This is useful for response surfaces that are sensitive to scaling, such as radial basis functions.
Parameters:  model (Object) – Original response surface object
 data (Object) – Optimization problem object
Variables:  data – Optimization problem object
 model – original response surface object
 fvalues – Function values
 nump – Current number of points
 maxp – Initial maximum number of points (can grow)
 updated – True if the surface is updated

add_point
(xx, fx)¶ Add a new function evaluation
Parameters:  xx (numpy.array) – Point to add
 fx (float) – The function value of the point to add

deriv
(x, ds=None)¶ Evaluate the derivative of the rbf interpolant at x
Parameters:  x (numpy.array) – Point for which we want to compute the MARS gradient
 ds (None) – Not used
Returns: Derivative of the MARS interpolant at x
Return type: numpy.array

eval
(x, ds=None)¶ Evaluate the response surface at the point xx
Parameters:  x (numpy.array) – Point where to evaluate
 ds (None) – Not used
Returns: Value of the interpolant at x
Return type: float

evals
(x, ds=None)¶ Evaluate the capped rbf interpolant at the points xx
Parameters:  x (numpy.array) – Points where to evaluate, of size npts x dim
 ds (None) – Not used
Returns: Values of the MARS interpolant at x, of length npts
Return type: numpy.array

get_fx
()¶ Get the list of function values for the data points.
Returns: List of function values Return type: numpy.array

get_x
()¶ Get the list of data points
Returns: List of data points Return type: numpy.array

reset
()¶ Reset the capped response surface
pySOT.sot_sync_strategies module¶
Module:  sot_sync_strategies 

Author:  David Bindel <bindel@cornell.edu>, David Eriksson <dme65@cornell.edu> 

class
pySOT.sot_sync_strategies.
SyncStrategyNoConstraints
(worker_id, data, response_surface, maxeval, nsamples, exp_design=None, sampling_method=None, extra=None, extra_vals=None)¶ Parallel synchronous optimization strategy without nonbound constraints.
This class implements the parallel synchronous SRBF strategy described by Regis and Shoemaker. After the initial experimental design (which is embarrassingly parallel), the optimization proceeds in phases. During each phase, we allow nsamples simultaneous function evaluations. We insist that these evaluations run to completion – if one fails for whatever reason, we will resubmit it. Samples are drawn randomly from around the current best point, and are sorted according to a merit function based on distance to other sample points and predicted function values according to the response surface. After several successive significant improvements, we increase the sampling radius; after several failures to improve the function value, we decrease the sampling radius. We restart once the sampling radius decreases below a threshold.
Parameters:  worker_id (int) – Start ID in a multistart setting
 data (Object) – Problem parameter data structure
 response_surface (Object) – Surrogate model object
 maxeval (int) – Stopping criterion. If positive, this is an evaluation budget. If negative, this is a time budget in seconds.
 nsamples (int) – Number of simultaneous fevals allowed
 exp_design (Object) – Experimental design
 sampling_method (Object) – Sampling method for finding points to evaluate
 extra (numpy.array) – Points to be added to the experimental design
 extra_vals (numpy.array) – Values of the points in extra (if known). Use nan for values that are not known.

adjust_step
()¶ Adjust the sampling radius sigma.
After succtol successful steps, we cut the sampling radius; after failtol failed steps, we double the sampling radius.

check_common
()¶ Checks that the inputs are correct

check_input
()¶ Checks that the inputs are correct

log_completion
(record)¶ Record a completed evaluation to the log.
Parameters: record (Object) – Record of the function evaluation

on_complete
(record)¶ Handle completed function evaluation.
When a function evaluation is completed we need to ask the constraint handler if the function value should be modified which is the case for say a penalty method. We also need to print the information to the logfile, update the best value found so far and notify the GUI that an evaluation has completed.
Parameters: record (Object) – Evaluation record

on_reply_accept
(proposal)¶

proj_fun
(x)¶ Projects a set of points onto the feasible region
Parameters: x (numpy.array) – Points, of size npts x dim Returns: Projected points Return type: numpy.array

propose_action
()¶ Propose an action

sample_adapt
()¶ Generate and queue samples from the search strategy

sample_initial
()¶ Generate and queue an initial experimental design.

start_batch
()¶ Generate and queue a new batch of points

class
pySOT.sot_sync_strategies.
SyncStrategyPenalty
(worker_id, data, response_surface, maxeval, nsamples, exp_design=None, sampling_method=None, extra=None, penalty=1000000.0)¶ Parallel synchronous optimization strategy with nonbound constraints.
This is an extension of SyncStrategyNoConstraints that also works with bound constraints. We currently only allow inequality constraints, since the candidate based methods don’t work well with equality constraints. We also assume that the constraints are cheap to evaluate, i.e., so that it is easy to check if a given point is feasible. More strategies that can handle expensive constraints will be added.
We use a penalty method in the sense that we try to minimize:
\[f(x) + \mu \sum_j (\max(0, g_j(x))^2\]where \(g_j(x) \leq 0\) are cheap inequality constraints. As a measure of promising function values we let all infeasible points have the value of the feasible candidate point with the worst function value, since large penalties makes it impossible to distinguish between feasible points.
When it comes to the value of \(\mu\), just choose a very large value.
Parameters:  worker_id (int) – Start ID in a multistart setting
 data (Object) – Problem parameter data structure
 response_surface (Object) – Surrogate model object
 maxeval (int) – Function evaluation budget
 nsamples (int) – Number of simultaneous fevals allowed
 exp_design (Object) – Experimental design
 sampling_method (Object) – Sampling method for finding points to evaluate
 extra (numpy.array) – Points to be added to the experimental design
 penalty (float) – Penalty for violating constraints

check_input
()¶ Checks that the inputs are correct

on_complete
(record)¶ Handle completed function evaluation.
When a function evaluation is completed we need to ask the constraint handler if the function value should be modified which is the case for say a penalty method. We also need to print the information to the logfile, update the best value found so far and notify the GUI that an evaluation has completed.
Parameters: record (Object) – Evaluation record

penalty_fun
(xx)¶ Computes the penalty for constraint violation
Parameters: xx (numpy.array) – Points to compute the penalty for Returns: Penalty for constraint violations Return type: numpy.array

class
pySOT.sot_sync_strategies.
SyncStrategyProjection
(worker_id, data, response_surface, maxeval, nsamples, exp_design=None, sampling_method=None, extra=None, proj_fun=None)¶ Parallel synchronous optimization strategy with nonbound constraints. It uses a supplied method to project proposed points onto the feasible region in order to always evaluate feasible points which is useful in situations where it is easy to project onto the feasible region and where the objective function is nonsensical for infeasible points.
This is an extension of SyncStrategyNoConstraints that also works with bound constraints.
Parameters:  worker_id (int) – Start ID in a multistart setting
 data (Object) – Problem parameter data structure
 response_surface (Object) – Surrogate model object
 maxeval (int) – Function evaluation budget
 nsamples (int) – Number of simultaneous fevals allowed
 exp_design (Object) – Experimental design
 sampling_method (Object) – Sampling method for finding points to evaluate
 extra (numpy.array) – Points to be added to the experimental design
 proj_fun (Object) – Function that projects one point onto the feasible region

check_input
()¶ Checks that the inputs are correct

proj_fun
(x)¶ Projects a set of points onto the feasible region
Parameters: x (numpy.array) – Points, of size npts x dim Returns: Projected points Return type: numpy.array
pySOT.test_problems module¶
Module:  test_problems 

Author:  David Eriksson <dme65@cornell.edu>, David Bindel <bindel@cornell.edu> 

class
pySOT.test_problems.
Ackley
(dim=10)¶ Ackley function
\[f(x_1,\ldots,x_n) = 20\exp\left( 0.2 \sqrt{\frac{1}{n} \sum_{j=1}^n x_j^2} \right) \exp \left( \frac{1}{n} \sum{j=1}^n \cos(2 \pi x_j) \right) + 20  e\]subject to
\[15 \leq x_i \leq 20\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information:
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Ackley function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Exponential
(dim=10)¶ Exponential function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^n e^{jx_j}  \sum_{j=1} e^{5.12 j}\]subject to
\[5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Exponential function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Griewank
(dim=10)¶ Griewank function
\[f(x_1,\ldots,x_n) = 1 + \frac{1}{4000} \sum_{j=1}^n x_j^2  \prod_{j=1}^n \cos \left( \frac{x_i}{\sqrt{i}} \right)\]subject to
\[512 \leq x_i \leq 512\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Griewank function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Hartman3
(dim=3)¶ Hartman 3 function
Details: http://www.sfu.ca/~ssurjano/hart3.html
Global optimum: \(f(0.114614,0.555649,0.852547)=3.86278\)
Parameters: dim (int) – Number of dimensions (has to be = 3)
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Hartman 3 function at x
Parameters: x – Data point Returns: Value at x

class
pySOT.test_problems.
Hartman6
(dim=6)¶ Hartman 6 function
Details: http://www.sfu.ca/~ssurjano/hart6.html
Global optimum: \(f(0.20169,0.150011,0.476874,0.275332,0.311652,0.6573)=3.32237\)
Parameters: dim (int) – Number of dimensions (has to be = 6)
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Hartman 3 function at x
Parameters: x – Data point Returns: Value at x

class
pySOT.test_problems.
Keane
(dim=10)¶ Keane’s “bump” function
\[f(x_1,\ldots,x_n) = \left \frac{\sum_{j=1}^n \cos^4(x_j)  2 \prod_{j=1}^n \cos^2(x_j)}{\sqrt{\sum_{j=1}^n jx_j^2}} \right\]subject to
\[0 \leq x_i \leq 5\]\[0.75  \prod_{j=1}^n x_j < 0\]\[\sum_{j=1}^n x_j  7.5n < 0\]Global optimum: 0.835 for large n
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

deriv_ineq_constraints
(x)¶ Evaluate the derivative of the Keane inequality constraints at x
Parameters: x (numpy.array) – Data points, of size npts x dim Returns: Derivative at the constraints, of size npts x nconstraints x ndims Return type: float

eval_ineq_constraints
(x)¶ Evaluate the Keane inequality constraints at x
Parameters: x (numpy.array) – Data points, of size npts x dim Returns: Value at the constraints, of size npts x nconstraints Return type: float

objfunction
(x)¶ Evaluate the Keane function at a point x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Levy
(dim=10)¶ Ackley function
Details: https://www.sfu.ca/~ssurjano/levy.html
Global optimum: \(f(1,1,...,1)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Levy function at x
Parameters: x – Data point Returns: Value at x

class
pySOT.test_problems.
LinearMI
(dim=5)¶ This is a linear mixed integer problem with nonbound constraints
There are 5 variables, the first 3 are discrete and the last 2 are continuous.
Global optimum: \(f(1,0,0,0,0) = 1\)
Parameters: dim (int) – Number of dimensions (has to be 5)
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

eval_ineq_constraints
(x)¶ Evaluate the LinearMI inequality constraints at x
Parameters: x (numpy.array) – Data points, of size npts x dim Returns: Value at the constraints, of size npts x nconstraints Return type: float

objfunction
(x)¶ Evaluate the LinearMI function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Michalewicz
(dim=10)¶ Michalewicz function
\[f(x_1,\ldots,x_n) = \sum_{i=1}^n \sin(x_i) \sin^{20} \left( \frac{ix_i^2}{\pi} \right)\]subject to
\[0 \leq x_i \leq \pi\]Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Michalewicz function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Quartic
(dim=10)¶ Quartic function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^n j x_j^4 + random[0,1)\]subject to
\[1.28 \leq x_i \leq 1.28\]Global optimum: \(f(0,0,...,0)=0+noise\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Quartic function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Rastrigin
(dim=10)¶ Rastrigin function
\[f(x_1,\ldots,x_n)=10n\sum_{i=1}^n (x_i^2  10 \cos(2 \pi x_i))\]subject to
\[5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Rastrigin function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Rosenbrock
(dim=10)¶ Rosenbrock function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^{n1} \left( 100(x_j^2x_{j+1})^2 + (1x_j)^2 \right)\]subject to
\[2.048 \leq x_i \leq 2.048\]Global optimum: \(f(1,1,...,1)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Rosenbrock function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
SchafferF7
(dim=10)¶ SchafferF7 function
\[f(x_1,\ldots,x_n) = \left[\frac{1}{n1}\sqrt{s_i} \cdot (\sin(50.0s_i^{\frac{1}{5}})+1)\right]^2\]where
\[s_i = \sqrt{x_i^2 + x_{i+1}^2}\]subject to
\[100 \leq x_i \leq 100\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the SchafferF7 function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Schwefel
(dim=10)¶ Schwefel function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^{n} \left( x_j \sin(\sqrt{x_j}) \right) + 418.982997 n\]subject to
\[512 \leq x_i \leq 512\]Global optimum: \(f(420.968746,420.968746,...,420.968746)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Schwefel function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Sphere
(dim=10)¶ Sphere function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^n x_j^2\]subject to
\[5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Sphere function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
StyblinskiTang
(dim=10)¶ StyblinskiTang function
\[f(x_1,\ldots,x_n) = \frac{1}{2} \sum_{j=1}^n \left(x_j^4 16x_j^2 +5x_j \right)\]subject to
\[5 \leq x_i \leq 5\]Global optimum: \(f(2.903534,2.903534,...,2.903534)= 39.16599 \cdot n\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the StyblinskiTang function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float

class
pySOT.test_problems.
Whitley
(dim=10)¶ Quartic function
\[f(x_1,\ldots,x_n) = \sum_{i=1}^n \sum_{j=1}^n \left( \frac{(100(x_i^2x_j)^2+(1x_j)^2)^2}{4000}  \cos(100(x_i^2x_j)^2 + (1x_j)^2 ) + 1 \right)\]subject to
\[10.24 \leq x_i \leq 10.24\]Global optimum: \(f(1,1,...,1)=0\)
Parameters: dim (int) – Number of dimensions
Variables:  dim – Number of dimensions
 xlow – Lower bound constraints
 xup – Upper bound constraints
 info – Problem information
 min – Global optimum
 integer – Integer variables
 continuous – Continuous variables

objfunction
(x)¶ Evaluate the Whitley function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
pySOT.utils module¶
Module:  utils 

Author:  David Eriksson <dme65@cornell.edu> 

pySOT.utils.
check_opt_prob
(obj)¶ Routine for checking that an implementation of the optimization problem follows the standard. This method checks everything, but can’t make sure that the objective function and constraint methods return values of the correct type since this would involve actually evaluating the objective function which isn’t feasible when the evaluations are expensive. If some test fails, an exception is raised through assert.
Parameters: obj (Object) – Optimization problem Raises: AttributeError – If object doesn’t follow the pySOT standard

pySOT.utils.
from_unit_box
(x, data)¶ Maps a set of points from the unit box to the original domain
Parameters:  x (numpy.array) – Points to be mapped from the unit box, of size npts x dim
 data (Object) – Optimization problem, needs to have attributes xlow and xup
Returns: Points mapped to the original domain
Return type: numpy.array

pySOT.utils.
progress_plot
(controller, title='', interactive=False)¶ Makes a progress plot from a POAP controller
This method depends on matplotlib and will terminate if matplotlib.pyplot is unavailable.
Parameters:  controller (Object) – POAP controller object
 title (string) – Title of the plot
 interactive (bool) – True if the plot should be interactive

pySOT.utils.
round_vars
(data, x)¶ Round integer variables to closest integer that is still in the domain
Parameters:  data (Object) – Optimization problem object
 x (numpy.array) – Set of points, of size npts x dim
Returns: The set of points with the integer variables rounded to the closest integer in the domain
Return type: numpy.array

pySOT.utils.
to_unit_box
(x, data)¶ Maps a set of points to the unit box
Parameters:  x (numpy.array) – Points to be mapped to the unit box, of size npts x dim
 data (Object) – Optimization problem, needs to have attributes xlow and xup
Returns: Points mapped to the unit box
Return type: numpy.array

pySOT.utils.
unit_rescale
(xx)¶ Shift and rescale elements of a vector to the unit interval
Parameters: xx (numpy.array) – Vector that should be rescaled to the unit interval Returns: Vector scaled to the unit interval Return type: numpy.array