Hybrid continuou and discrete BO

[mohamed82008]

Hi, thanks for this great package. Do you have hybrid BO with discrete and continuous variables planned? I recently came across https://arxiv.org/pdf/2106.04682.pdf and it discusses an interesting approach so I thought I would ask here. Thanks.

[jbrea]

That looks like a cool paper, thanks! I think this is one direction to go with this package. Other directions include 1) make it work better in higher dimensions (there is quite some recent literature on scaling BO) 2) have better default options (e.g. using some of the heuristics used by the python BO package dragonfly).

Unfortunately I don't have much time in the next few months to add substantial features to this package. But PRs are highly appreciated :)

[mohamed82008]

Great directions! I think these would make great GSoC projects. Also, I see that you depend on NLopt. Is it possible to use something like Nonconvex.jl for the optimization?

[jbrea]

Yes. Replacing the optimiser should be a simple thing. At the time I wrote the optimisation code I only considered Optim and NLopt and found the latter preferable. Are there compelling reasons to switch to Nonconvex.jl?

[mohamed82008]

Nonconvex makes it easy to switch between many optimization packages. Also it will make it easier to incorporate inequality and equality constraints into the optimization. Nonconvex also uses Zygote by default so that might be more efficient in some cases. If you are ok with the direction, I will be happy to open a PR.

[jbrea]

Yes, that sounds great. Thanks!

[LeviManring]

Any update on this aspect? https://arxiv.org/abs/1805.03463 also has some interesting ideas on this front

[jbrea]

I don't have currently the time to work on this but I wrote a GSoC proposal.

@LeviManring I think the method in the paper you mention performs typically worse than HyBO (the paper mentioned by @mohamed82008; see its supplementary section).