Constraint Error

[qup20]

I would like to ask a question. I used the new version and adding the constraint has no effect, it still fetches the points that are not accepted, why is that? Also, the new version of logger will prompt me: TypeError: Object of type bool_ is not JSON serializable.
Thank you for your answer!

[bwheelz36]

Hey, could you please add a minimum example that reproduces these errors so we can debug?

[till-m]

The current constrained optimization is designed for constraint functions that are expensive to evaluate (meaning it is not easy to say whether a point fulfills the constraint without explicitly evaluating the constraint function, and evaluating is computationally expensive). In practice, this means that you will sample a lot of points during modelling that do not fulfill the constraints (even by design), because we simply don't know whether a point fulfills the constrain before sampling. On the other hand, we want to make use of the information gained by every evaluation of the constraint function, since this is such a costly operation and we want to perform as few of these as possible. I hope this clears up the first question to some degree. If you want to read more about the method we are using to model constraints, please read Gardner et al, 2014 or feel free to ask a question here.
Nonetheless, calling .max on the optimizer should give you the point maximizing the function out of all points that fulfill the constraints.

That being said, if your constraint function is not hard to evaluate, there may be better ways of incorporating your constraints into the optimization. For this, please provide a minimum example.

[till-m]

To reproduce the logging bug:

import numpy as np
from bayes_opt import BayesianOptimization
from scipy.optimize import NonlinearConstraint
from bayes_opt.logger import JSONLogger
from bayes_opt.event import Events

logger = JSONLogger(path="./logs.json")

def target_function(x, y):
    # Gardner is looking for the minimum, but this packages looks for maxima, thus the sign switch
    return np.cos(2*x)*np.cos(y) + np.sin(x)

def constraint_function(x, y):
    return np.cos(x) * np.cos(y) - np.sin(x) * np.sin(y)

constraint_limit = 0.5

constraint = NonlinearConstraint(constraint_function, -np.inf, constraint_limit)

# Bounded region of parameter space
pbounds = {'x': (0, 6), 'y': (0, 6)}

optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds,
    random_state=1,
)
optimizer.subscribe(Events.OPTIMIZATION_STEP, logger)
optimizer.maximize(
    init_points=2,
    n_iter=10,
)

I'll look into this.

@qup20 thank you very much for reporting this bug. Quick sidenote, it's generally better to keep issues focused on a single topic, so please open separate issues for different bugs in the future.

[qup20]

The current constrained optimization is designed for constraint functions that are expensive to evaluate (meaning it is not easy to say whether a point fulfills the constraint without explicitly evaluating the constraint function, and evaluating is computationally expensive). In practice, this means that you will sample a lot of points during modelling that do not fulfill the constraints (even by design), because we simply don't know whether a point fulfills the constrain before sampling. On the other hand, we want to make use of the information gained by every evaluation of the constraint function, since this is such a costly operation and we want to perform as few of these as possible. I hope this clears up the first question to some degree. If you want to read more about the method we are using to model constraints, please read Gardner et al, 2014 or feel free to ask a question here. Nonetheless, calling .max on the optimizer should give you the point maximizing the function out of all points that fulfill the constraints.

That being said, if your constraint function is not hard to evaluate, there may be better ways of incorporating your constraints into the optimization. For this, please provide a minimum example.

---

Thank you for your answer! I think I understand what you mean. My task is to optimize the arrangement of some rectangulars, so I should constain theses rectangles do not overlap. At first I constrained the 9 rectangles not to overlap, I guess this constraint should be too strong to sample the points that meet the condition. Now I lower the constraint to 3 rectangles without overlap, and it is able to get more points that are within the constraint. But I am very sad, I encountered a new problem, when my program samples several times, it will report an error that the input is empty. My procedure is basically the same as the minimal example you gave, with only two differences:

import numpy as np
from bayes_opt import BayesianOptimization
from scipy.optimize import NonlinearConstraint


def target_function(x, y):
    # My target_function is not easy to say, I need to call ANSYS for finite element calculation to get the result(float).
    return new_function(x,y)

def constraint_function(x, y):
    #Here my constraint is to determine whether the 3 rectangles overlap by the shape center coordinates and side lengths, I omitted it.
    if overlap:
        return 1
    return -1

# If I set constraint_limit=0 it will prompt an error, so I set it to [0]
constraint_limit = [0]
constraint = NonlinearConstraint(constraint_function, -np.inf, constraint_limit)


pbounds = {'x': (0, 6), 'y': (0, 6)}

optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds
    random_state=1,
)

optimizer.maximize(
    init_points=2,
    n_iter=10,
)

When my program samples several times, it will report this error

Can you help me solve this issue?
Thank you very much!

[qup20]

To reproduce the logging bug:

import numpy as np
from bayes_opt import BayesianOptimization
from scipy.optimize import NonlinearConstraint
from bayes_opt.logger import JSONLogger
from bayes_opt.event import Events

logger = JSONLogger(path="./logs.json")

def target_function(x, y):
    # Gardner is looking for the minimum, but this packages looks for maxima, thus the sign switch
    return np.cos(2*x)*np.cos(y) + np.sin(x)

def constraint_function(x, y):
    return np.cos(x) * np.cos(y) - np.sin(x) * np.sin(y)

constraint_limit = 0.5

constraint = NonlinearConstraint(constraint_function, -np.inf, constraint_limit)

# Bounded region of parameter space
pbounds = {'x': (0, 6), 'y': (0, 6)}

optimizer = BayesianOptimization(
    f=target_function,
    constraint=constraint,
    pbounds=pbounds
    random_state=1,
)
optimizer.subscribe(Events.OPTIMIZATION_STEP, logger)
optimizer.maximize(
    init_points=2,
    n_iter=10,
)

I'll look into this.

@qup20 thank you very much for reporting this bug. Quick sidenote, it's generally better to keep issues focused on a single topic, so please open separate issues for different bugs in the future.

---

Thank you for your answer!I am writing my program according to this code format you provided, but he still prompts an error.
This error is prompted when the first sample point are calculated to return the function value, but in the old version it was correct , I didn't add constraints and I didn't change my targrt_function.

I've just joined GitHub, I'll take note of your reminder in the future, thank you very much! @till-m

[till-m]

Hi @qup20,

first of all, I hope my reminder didn't come off as harsh, it wasn't inteded to be. People like you reporting bugs are invaluable to open source software projects, so I very much appreciate that you opened an issue!

Regarding the logging, I've identified the problem and will try to provide a fix, potentially later today. For now, if that's possible for you, just use the default logger instead of the JSONLogger.

Now, regarding your constraints:
First of all, these types of constraints are not something that the internal model is designed for, since your constraint is discrete, in a sense: Either there is an overlap, or there isn't. The type of constraints the Gardner-model is designed for are continuous functions $c$, the value of which should fall between certain bounds, i.e. $$\lambda_{\text{lower}} \leq c(x,y) \leq \lambda_{\text{upper}}.$$
Additionally, these functions $c$ are hard to evaluate (i.e., on the order of the target function), meaning that we cannot afford to discard information from sampled points.

The good news is, that, at least intuitively, this doesn't sound like a particularly computationally expensive constraint. In your case, I would thus use @bwheelz36's approach of dealing with constraints, which involves a surrogate target function that returns a low value for infeasible regions. Consider something like this:

import numpy as np
from bayes_opt import BayesianOptimization


def target_function(x, y):
    return -x ** 2 - (y - 1) ** 2 + 1

def surrogate_target_function(x, y):
    if x**2 + y**2 < 3:
        return target_function(x, y)
    else:
        # this should be a value that is small enough to make the region unattractive for the optimizer
        # in particular it should be smaller than the lowest sensible value your target function could return.
        return -20

pbounds = {'x': (0, 6), 'y': (0, 6)}

optimizer = BayesianOptimization(
    f=surrogate_target_function,
    pbounds=pbounds,
    random_state=1,
)

optimizer.maximize(
    init_points=2,
    n_iter=50,
)

Regarding the recent bug, please make sure your minimal, reproducible examples are complete to so that all the relevant information is there and that the code is executable. See here for more info on how to do this. Right now, the target function is not particularly relevant, so it was good that you didn't include it, but the exact way that the constraint function is calculated is important, so it should be included. Please make sure that your example is actually runnable. Since I can't run your code, it is very hard for me to figure out why you're encountering an error.

If I set constraint_limit=0 it will prompt an error, so I set it to [0]

This is strange and shouldn't happen. Not sure what the problem is here. If you can provide a minimal example for this (or even just the stacktrace) I'd be very grateful.

I hope this is helpful to some degree.

[till-m]

The logging bug should be fixed in the linked PR. @qup20 to use this before the PR is merged, you can uninstall this package and reinstall the specific commit:

pip uninstall bayesian-optimization; pip install git+https://github.com/till-m/BayesianOptimization.git@67a7a497bc645a5bd9d69313f7d91d5707e51ebe

[qup20]

You are great! @till-m I am so grateful to you！The logger error has been solved following your hint. For your suggestion to set the proxy function to return a smaller value, I have tried it before but the optimization did not work well. But you reminded me that Garden-Model handles continuous functions and I modified my constraint to calculate the overlapping area of the rectangle and this result is somehow continuous. Anyway, with your help, all my problems are now solved, thanks again! Now I can happily tweak my parameters to get a desired optimization result, and I wish you all the best!