box constraints

[brandondube]

Most of the optimizers in optym would be "content" with a box constraint post-processor that just clips the suggested x to lie inside lk <= xk <= hk (low, x, high for parameter num k). L-BFGS-B would not because the hessian approximation would break, but it has its own constraints anyway.

There is also pymanopt, which implements these constraints "properly" as well as things like the trust region algorithm, which suggests just np.clip() is not the perfect approach

https://pymanopt.org/

I do want to have trust-region, interior-point, and damped least squares at some point. I suppose the implementation of the constraint need not match between all of them, but if TR, IP, and DLS constraints work in an L-BFGS-B-y way and everything else is different, that's not super optimal.

Probably, I should research how to implement the manifold implied by box constraints and do everything in a pymanopt-y way.

@Jashcraf thoughts? Would be useful for APLC/PAPLCs.

Also in https://vanderbei.princeton.edu/tex/talks/Drexel16/Drexel_small.pdf contrast is written as a constraint and the objective is to maximize the area under the mask (= throughput) with an additional constrain on the spatial gradient of the mask function. There is no explicit mask = 0 or 1 so I presume what Vanderbei did was design in continuous and then clip after. Thonks