Brought branch up-to-date with master

.github/workflows/python_testing.yml

@@ -10,22 +10,18 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: ["2.7", "3.6", "3.7", "3.8", "3.9"]
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
       run: |
-        python -m pip install --upgrade pip
-        python -m pip install nose
-        pip install -r requirements.txt
-    - name: Build package
-      run: |
-        python setup.py build
+        python -m pip install .
+        pip install pytest
     - name: Run unit tests
       run: |
-        nosetests -v
+        python -m pytest --pyargs dfols

.github/workflows/upload_pypi.yml

@@ -0,0 +1,50 @@
+name: Package and upload to PyPI
+
+# Publish when a (published) GitHub Release is created
+on:
+  release:
+    types:
+      - published
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.x"
+
+      - name: Install dependencies
+        run: pip install build
+
+      - name: Build binary wheel and source tarball
+        run: python -m build
+
+      - name: Store the distribution packages
+        uses: actions/upload-artifact@v3
+        with:
+          name: python-package-distributions
+          path: dist/
+
+  publish-to-pypi:
+    name: Publish to PyPI
+    needs:
+      - build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/DFO-LS
+    permissions:
+      id-token: write
+    steps:
+      - name: Download all the dists
+        uses: actions/download-artifact@v3
+        with:
+          name: python-package-distributions
+          path: dist/
+      - name: Publish distribution to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

README.rst

@@ -41,19 +41,19 @@ Cartis, C., Fiala, J., Marteau, B. and Roberts, L., `Improving the Flexibility a
 
 If you use DFO-LS for problems with constraints, including bound constraints, please also cite:
 
-Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://arxiv.org/abs/2111.05443>`_, *arXiv preprint arXiv:2111.05443*, (2021).
+Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579.
 
 Requirements
 ------------
 DFO-LS requires the following software to be installed:
 
-* Python 2.7 or Python 3 (http://www.python.org/)
+* Python 3.9 or higher (http://www.python.org/)
 
 Additionally, the following python packages should be installed (these will be installed automatically if using *pip*, see `Installation using pip`_):
 
-* NumPy 1.11 or higher (http://www.numpy.org/)
-* SciPy 0.18 or higher (http://www.scipy.org/)
-* Pandas 0.17 or higher (http://pandas.pydata.org/)
+* NumPy (http://www.numpy.org/)
+* SciPy version 1.11 or higher (http://www.scipy.org/)
+* Pandas (http://pandas.pydata.org/)
 
 **Optional package:** DFO-LS versions 1.2 and higher also support the `trustregion <https://github.com/lindonroberts/trust-region>`_ package for fast trust-region subproblem solutions. To install this, make sure you have a Fortran compiler (e.g. `gfortran <https://gcc.gnu.org/wiki/GFortran>`_) and NumPy installed, then run :code:`pip install trustregion`. You do not have to have trustregion installed for DFO-LS to work, and it is not installed by default.
 
@@ -118,7 +118,7 @@ If you do not have root privileges or you want to install DFO-LS for your privat
 
 instead.
 
-To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing :code:`setup.py`) and rerun the installation using :code:`pip`, as above:
+To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing :code:`pyproject.toml`) and rerun the installation using :code:`pip`, as above:
 
  .. code-block:: bash
 
@@ -127,11 +127,12 @@ To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.
 
 Testing
 -------
-If you installed DFO-LS manually, you can test your installation by running:
+If you installed DFO-LS manually, you can test your installation using the pytest package:
 
  .. code-block:: bash
 
-    $ python setup.py test
+    $ pip install pytest
+    $ python -m pytest --pyargs dfols
 
 Alternatively, the HTML documentation provides some simple examples of how to run DFO-LS.
 
@@ -151,7 +152,7 @@ If DFO-LS was installed manually you have to remove the installed files by hand
 
 Bugs
 ----
-Please report any bugs using GitHub's issue tracker.
+Please report any bugs using `GitHub's issue tracker <https://github.com/numericalalgorithmsgroup/dfols/issues>`_.
 
 License
 -------

dfols/__init__.py

@@ -38,10 +38,10 @@
 # Ensure compatibility with Python 2
 from __future__ import absolute_import, division, print_function, unicode_literals
 
-from .version import __version__
-__all__ = ['__version__']
+# DFO-LS version
+__version__ = '1.4.1'
 
 # Main solver & exit flags
 from .solver import *
-__all__ += ['solve']
+__all__ = ['solve']
 

dfols/controller.py

@@ -41,7 +41,7 @@
 
 __all__ = ['Controller', 'ExitInformation', 'EXIT_SLOW_WARNING', 'EXIT_MAXFUN_WARNING', 'EXIT_SUCCESS',
            'EXIT_INPUT_ERROR', 'EXIT_TR_INCREASE_ERROR', 'EXIT_LINALG_ERROR', 'EXIT_FALSE_SUCCESS_WARNING',
-           'EXIT_AUTO_DETECT_RESTART_WARNING']
+           'EXIT_AUTO_DETECT_RESTART_WARNING', 'EXIT_EVAL_ERROR']
 
 module_logger = logging.getLogger(__name__) 
 
@@ -54,6 +54,7 @@
 EXIT_INPUT_ERROR = -1  # error, bad inputs
 EXIT_TR_INCREASE_ERROR = -2  # error, trust region step increased model value
 EXIT_LINALG_ERROR = -3  # error, linalg error (singular matrix encountered)
+EXIT_EVAL_ERROR = -4  # error, objective evaluation error (e.g. nan result received)
 
 
 class ExitInformation(object):
@@ -83,11 +84,13 @@ def message(self, with_stem=True):
             return "Error (linear algebra): " + self.msg
         elif self.flag == EXIT_FALSE_SUCCESS_WARNING:
             return "Warning (max false good steps): " + self.msg
+        elif self.flag == EXIT_EVAL_ERROR:
+            return "Error (function evaluation): " + self.msg
         else:
             return "Unknown exit flag: " + self.msg
 
     def able_to_do_restart(self):
-        if self.flag in [EXIT_TR_INCREASE_ERROR, EXIT_TR_INCREASE_WARNING, EXIT_LINALG_ERROR, EXIT_SLOW_WARNING, EXIT_AUTO_DETECT_RESTART_WARNING]:
+        if self.flag in [EXIT_TR_INCREASE_ERROR, EXIT_TR_INCREASE_WARNING, EXIT_LINALG_ERROR, EXIT_SLOW_WARNING, EXIT_AUTO_DETECT_RESTART_WARNING, EXIT_EVAL_ERROR]:
             return True
         elif self.flag in [EXIT_MAXFUN_WARNING, EXIT_INPUT_ERROR]:
             return False

dfols/solver.py

@@ -552,6 +552,17 @@ def solve_main(objfun, x0, argsf, xl, xu, projections, npt, rhobeg, rhoend, maxf
             ##print("x from xnew again", x)
             number_of_samples = max(nsamples(control.delta, control.rho, current_iter, nruns_so_far), 1)
             rvec_list, obj_list, num_samples_run, exit_info = control.evaluate_objective(x, number_of_samples, params)
+            if np.any(np.isnan(rvec_list)):
+                # Just exit without saving the current point
+                # We should be able to do a hard restart though, because it's unlikely 
+                # that we will get the same trust-region step after expanding the radius/re-initialising
+                module_logger.warning("NaN encountered in evaluation of trust-region step")
+                if params("interpolation.throw_error_on_nans"):
+                    raise np.linalg.LinAlgError("NaN encountered in objective evaluations")
+
+                exit_info = ExitInformation(EXIT_EVAL_ERROR, "NaN received from objective function evaluation")
+                nruns_so_far += 1
+                break  # quit
             if exit_info is not None:
                 if num_samples_run > 0:
                     control.model.save_point(x, np.mean(rvec_list[:num_samples_run, :], axis=0), num_samples_run,

dfols/tests/test_solver.py

@@ -170,7 +170,7 @@ def runTest(self):
         self.assertTrue(array_compare(soln.resid, objfun(soln.x), thresh=1e-10), "Wrong resid")
         print(soln.jacobian)
         print(jac(soln.x))
-        self.assertTrue(array_compare(soln.jacobian, jac(soln.x), thresh=1e-2), "Wrong Jacobian")
+        self.assertTrue(array_compare(soln.jacobian, jac(soln.x), thresh=5e-2), "Wrong Jacobian")
         self.assertTrue(abs(soln.obj) < 1e-10, "Wrong fmin")
 
 

docs/advanced.rst

@@ -2,7 +2,7 @@ Advanced Usage
 ==============
 This section describes different optional user parameters available in DFO-LS.
 
-In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2021]_.
+In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2022]_.
 
 The default values, used if no override is given, in some cases vary depending on whether :code:`objfun` has stochastic noise; that is, whether evaluating :code:`objfun(x)` several times at the same :code:`x` gives the same result or not. Whether or not this is the case is determined by the :code:`objfun_has_noise` input to :code:`dfols.solve()` (and not by inspecting :code:`objfun`, for instance).
 
@@ -119,5 +119,5 @@ References
 .. [CFMR2018]   
    Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] 
 
-.. [HR2021]   
-   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://arxiv.org/abs/2111.05443>`_, *arXiv preprint arXiv:2111.05443*, (2021).
+.. [HR2022]   
+   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].

docs/build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: f48a4434832128cee1c37b97e0d08ba7
+config: 35e7e5a3a73f5accc3f3cb99e4006184
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/build/html/_sources/advanced.rst.txt

@@ -2,7 +2,7 @@ Advanced Usage
 ==============
 This section describes different optional user parameters available in DFO-LS.
 
-In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2021]_.
+In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2022]_.
 
 The default values, used if no override is given, in some cases vary depending on whether :code:`objfun` has stochastic noise; that is, whether evaluating :code:`objfun(x)` several times at the same :code:`x` gives the same result or not. Whether or not this is the case is determined by the :code:`objfun_has_noise` input to :code:`dfols.solve()` (and not by inspecting :code:`objfun`, for instance).
 
@@ -119,5 +119,5 @@ References
 .. [CFMR2018]   
    Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] 
 
-.. [HR2021]   
-   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://arxiv.org/abs/2111.05443>`_, *arXiv preprint arXiv:2111.05443*, (2021).
+.. [HR2022]   
+   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].

docs/build/html/_sources/contributors.rst.txt

@@ -3,7 +3,7 @@ Contributors
 
 Main author
 -----------
-* Lindon Roberts (Australian National University)
+* Lindon Roberts (University of Sydney)
 
 Contributors
 ------------

docs/build/html/_sources/history.rst.txt

@@ -51,3 +51,13 @@ Version 1.3.0 (8 Nov 2021)
 * Handle finitely many arbitrary convex constraints in addition to simple bound constraints.
 * Add module-level logging for more informative log outputs.
 * Only new functionality is added, so there is no change to the solver for unconstrained/bound-constrained problems.
+
+Version 1.4.0 (29 Jan 2024)
+---------------------------
+* Require newer SciPy version (at least 1.11) as a dependency - avoids occasional undetermined behavior but no changes to the DFO-LS algorithm.
+* Gracefully handle NaN objective value from evaluation at a new trial point (trust-region step). 
+
+Version 1.4.1 (11 Apr 2024)
+---------------------------
+* Migrate package setup to pyproject.toml (required for Python version 3.12)
+* Drop support for Python 2.7 and <=3.8 in line with SciPy >=1.11 dependency (introduced v1.4.0)

docs/build/html/_sources/index.rst.txt

@@ -9,7 +9,7 @@ DFO-LS: Derivative-Free Optimizer for Least-Squares Minimization
 
 **Date:** |today|
 
-**Author:** `Lindon Roberts <lindon.roberts@anu.edu.au>`_
+**Author:** `Lindon Roberts <lindon.roberts@sydney.edu.au>`_
 
 DFO-LS is a flexible package for finding local solutions to nonlinear least-squares minimization problems (with optional constraints), without requiring any derivatives of the objective. DFO-LS stands for Derivative-Free Optimizer for Least-Squares.
 
@@ -26,7 +26,7 @@ The constraint set :math:`C` is the intersection of multiple convex sets provide
 Full details of the DFO-LS algorithm are given in our papers: 
 
 * C. Cartis, J. Fiala, B. Marteau and L. Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] . 
-* Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://arxiv.org/abs/2111.05443>`_, *arXiv preprint arXiv:2111.05443*, (2021).
+* Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
 
 DFO-LS is a more flexible version of `DFO-GN <https://github.com/numericalalgorithmsgroup/dfogn>`_.
 

docs/build/html/_sources/info.rst.txt

@@ -86,13 +86,13 @@ At each step, we compute a trial step :math:`s_k` designed to make our approxima
 
 In DFO-LS, we construct our approximation :math:`m_k(s)` by interpolating a linear approximation for each residual :math:`r_i(x)` at several points close to :math:`x_k`. To make sure our interpolated model is accurate, we need to regularly check that the points are well-spaced, and move them if they aren't (i.e. improve the geometry of our interpolation points).
 
-A complete description of the DFO-LS algorithm is given in our papers [CFMR2018]_ and [HR2021]_.
+A complete description of the DFO-LS algorithm is given in our papers [CFMR2018]_ and [HR2022]_.
 
 References
 ----------
 
 .. [CFMR2018]   
    Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] 
 
-.. [HR2021]   
-   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://arxiv.org/abs/2111.05443>`_, *arXiv preprint arXiv:2111.05443*, (2021).
+.. [HR2022]   
+   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].