Restructure code base and ensure tests still pass (#5)

* Restructure code base and ensure tests still pass * Move epicsdemo into ophyd_async.epics * Prefix some modules with _ and clearly define public API in __init__ files * Temporarily remove support for python 3.8 due to cython issue in p4p * Fix docs CI job * Remove python 3.8 check in init.py * Add extra tests to pass patch * Fix linting * Increasing tolerance for patch changes due to codecov flakiness r.e. moving files * Rename to ophyd-async
bluesky · Sep 14, 2023 · e4d16b7 · e4d16b7
1 parent 967206f
commit e4d16b7
Show file tree

Hide file tree

Showing 74 changed files with 588 additions and 553 deletions.
diff --git a/.codecov.yml b/.codecov.yml
@@ -9,6 +9,6 @@ coverage:
     patch:
       default:
         target: auto
-        threshold: 0.1%
+        threshold: 1%
 github_checks:
   annotations: false
diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile
@@ -3,7 +3,7 @@
 # The devcontainer should use the build target and run as root with podman
 # or docker with user namespaces.
 #
-FROM python:3.8 as build
+FROM python:3.9 as build
 
 ARG PIP_OPTIONS
 
@@ -24,7 +24,7 @@ WORKDIR /context
 # install python package into /venv
 RUN pip install ${PIP_OPTIONS}
 
-FROM python:3.8-slim as runtime
+FROM python:3.9-slim as runtime
 
 # Add apt-get system dependecies for runtime here if needed
 

diff --git a/.github/workflows/code.yml b/.github/workflows/code.yml
@@ -29,12 +29,12 @@ jobs:
       fail-fast: false
       matrix:
         os: ["ubuntu-latest"] # can add windows-latest, macos-latest
-        python: ["3.9", "3.10", "3.11"]
+        python: ["3.10", "3.11"]
         install: ["-e .[dev]"]
         # Make one version be non-editable to test both paths of version code
         include:
           - os: "ubuntu-latest"
-            python: "3.8"
+            python: "3.9"
             install: ".[dev]"
 
     runs-on: ${{ matrix.os }}

diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -46,7 +46,7 @@ jobs:
         if: github.event_name == 'push' && github.actor != 'dependabot[bot]'
         # We pin to the SHA, not the tag, for security reasons.
         # https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions
-        uses: peaceiris/actions-gh-pages@64b46b4226a4a12da2239ba3ea5aa73e3163c75b # v3.9.1
+        uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847 # v3.9.3
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_dir: .github/pages

diff --git a/.gitignore b/.gitignore
@@ -84,4 +84,4 @@ docs/*/generated
 docs/savefig
 
 # generated version number
-ophyd/_version.py
+ophyd_async/_version.py
diff --git a/docs/conf.py b/docs/conf.py
@@ -4,6 +4,7 @@
 # list see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
+import os
 import sys
 from pathlib import Path
 from subprocess import check_output
@@ -13,9 +14,11 @@
 import ophyd_async
 
 # -- General configuration ------------------------------------------------
+# Source code dir relative to this file
+sys.path.insert(0, os.path.abspath("../../src"))
 
 # General information about the project.
-project = "ophyd_async"
+project = "ophyd-async"
 copyright = "2014, Brookhaven National Lab"
 
 # The full version, including alpha/beta/rc tags.
@@ -208,14 +211,17 @@
 # every member listed in __all__ and no others. Default is True
 autosummary_ignore_module_all = False
 
+# Turn on sphinx.ext.autosummary
+autosummary_generate = True
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates"]
+
 # Look for signatures in the first line of the docstring (used for C functions)
 autodoc_docstring_signature = True
 
 # numpydoc config
 numpydoc_show_class_members = False
 
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ["_templates"]
-
 # Where to put Ipython savefigs
 ipython_savefig_dir = "../build/savefig"
diff --git a/docs/developer/explanations/decisions/0002-switched-to-pip-skeleton.rst b/docs/developer/explanations/decisions/0002-switched-to-pip-skeleton.rst
@@ -1,4 +1,4 @@
-2. Adopt ophyd_async for project structure
+2. Adopt ophyd-async for project structure
 ==========================================
 
 Date: 2022-02-18
@@ -11,7 +11,7 @@ Accepted
 Context
 -------
 
-We should use the following `pip-skeleton <https://github.com/bluesky/ophyd_async>`_.
+We should use the following `pip-skeleton <https://github.com/bluesky/ophyd-async>`_.
 The skeleton will ensure consistency in developer
 environments and package management.
 
@@ -23,7 +23,7 @@ We have switched to using the skeleton.
 Consequences
 ------------
 
-This module will use a fixed set of tools as developed in ophyd_async
+This module will use a fixed set of tools as developed in ophyd-async
 and can pull from this skeleton to update the packaging to the latest techniques.
 
 As such, the developer environment may have changed, the following could be

diff --git a/docs/developer/explanations/decisions/0004-repository-structure.rst b/docs/developer/explanations/decisions/0004-repository-structure.rst
@@ -1,7 +1,13 @@
+.. role:: bash(code)
+   :language: bash
+
+.. role:: python(code)
+   :language: python
+
 4. Repository Structure
 =======================
 
-Date: 2023-08-30
+Date: 2023-09-07
 
 Status
 ------
@@ -53,29 +59,48 @@ During this process, the folder structure should incrementally be changed to
     │   └── ophyd_async
     │       ├── core
     │       │   ├── __init__.py
-    │       │   ├── backends
+    │       │   ├── _device
     │       │   │   ├── __init__.py
-    │       │   │   ├── _aioca.py
-    │       │   │   └── _p4p.py
-    │       │   ├── devices
-    │       │   ├── signals
-    │       │   ├── epicsdemo
+    │       │   │   ├── _backend
+    │       │   │   │   ├── __init__.py
+    │       │   │   │   ├── signal_backend.py
+    │       │   │   │   └── sim.py
+    │       │   │   ├── _signal
+    │       │   │   │   ├── __init__.py
+    │       │   │   │   └── signal.py
+    │       │   │   ├── device_collector.py
+    │       │   │   ├── device_vector.py
+    │       │   │   └── ...
     │       │   ├── async_status.py
-    │       │   ├── device_collector.py
     │       │   └── utils.py
-    │       └── devices
-    │           ├── epics
-    │           └── tango
+    │       ├── epics
+    │       │   ├── _backend
+    │       │   │   ├── __init__.py
+    │       │   │   ├── _p4p.py
+    │       │   │   └── _aioca.py
+    │       │   ├── areadetector
+    │       │   │   ├── __init__.py
+    │       │   │   ├── ad_driver.py
+    │       │   │   └── ...
+    │       │   ├── signal
+    │       │   │   └── ...
+    │       │   ├── motion
+    │       │   │   ├── __init__.py
+    │       │   │   └── motor.py
+    │       │   └── demo
+    │       │       └── ...
+    │       └── panda
+    │           └── ...
     ├── tests
     │   ├── core
     │   │   └── ...
-    │   └── devices
+    │   └── epics
     └── ...
 
-The `__init__.py` files of each submodule (core, devices.epics and devices.tango) will
+The :bash:`__init__.py` files of each submodule (core, epics, panda and eventually tango) will
 be modified such that end users experience little disruption to how they use Ophyd Async.
-For such users, `from ophyd.v2.core import ...` can be replaced with 
-`from ophyd_async.core import ...`.
+For such users, :python:`from ophyd.v2.core import ...` can be replaced with 
+:python:`from ophyd_async.core import ...`.
 
 
 Consequences

diff --git a/docs/developer/how-to/pin-requirements.rst b/docs/developer/how-to/pin-requirements.rst
@@ -46,7 +46,7 @@ of the dependencies and sub-dependencies with pinned versions.
 You can download any of these files by clicking on them. It is best to use
 the one that ran with the lowest Python version as this is more likely to
 be compatible with all the versions of Python in the test matrix.
-i.e. ``requirements-test-ubuntu-latest-3.8.txt`` in this example.
+i.e. ``requirements-test-ubuntu-latest-3.9.txt`` in this example.
 
 Applying the lock file
 ----------------------

diff --git a/docs/developer/tutorials/dev-install.rst b/docs/developer/tutorials/dev-install.rst
@@ -16,7 +16,7 @@ Install dependencies
 --------------------
 
 You can choose to either develop on the host machine using a `venv` (which
-requires python 3.8 or later) or to run in a container under `VSCode
+requires python 3.9 or later) or to run in a container under `VSCode
 <https://code.visualstudio.com/>`_
 
 .. tab-set::

diff --git a/docs/examples/ad_demo.py b/docs/examples/ad_demo.py
@@ -9,7 +9,7 @@
 from bluesky.utils import ProgressBarManager, register_transform
 from ophyd.v2.core import DeviceCollector
 
-from ophyd_async.devices import areadetector
+from ophyd_async.epics import areadetector
 
 # Create a run engine, with plotting, progressbar and transform
 RE = RunEngine({}, call_returns_result=True)
@@ -25,9 +25,7 @@
 
 # Create v2 devices
 with DeviceCollector():
-    det1 = areadetector.MySingleTriggerSim(pv_prefix)
-    det2 = areadetector.MyHDFWritingSim(pv_prefix)
-    det3 = areadetector.MyHDFFlyerSim(pv_prefix)
+    det3 = areadetector.HDFStreamerDet(pv_prefix)
 
 
 # And a plan

diff --git a/docs/index.rst b/docs/index.rst
@@ -14,13 +14,13 @@ The documentation is split into 2 sections:
         :link: user/index
         :link-type: doc
 
-        The User Guide contains documentation on how to install and use ophyd_async.
+        The User Guide contains documentation on how to install and use ophyd-async.
 
     .. grid-item-card:: :material-regular:`code;4em`
         :link: developer/index
         :link-type: doc
 
-        The Developer Guide contains documentation on how to develop and contribute changes back to ophyd_async.
+        The Developer Guide contains documentation on how to develop and contribute changes back to ophyd-async.
 
 .. toctree::
     :hidden:

diff --git a/docs/user/examples/epics_demo.py b/docs/user/examples/epics_demo.py
@@ -7,8 +7,8 @@
 from bluesky.utils import ProgressBarManager, register_transform
 from ophyd import Component, Device, EpicsSignal, EpicsSignalRO
 
-from ophyd_async.core import epicsdemo
-from ophyd_async.core.device_collector import DeviceCollector
+from ophyd_async.core import DeviceCollector
+from ophyd_async.epics import demo
 
 # Create a run engine, with plotting, progressbar and transform
 RE = RunEngine({}, call_returns_result=True)
@@ -19,7 +19,7 @@
 register_transform("RE", prefix="<")
 
 # Start IOC with demo pvs in subprocess
-pv_prefix = epicsdemo.start_ioc_subprocess()
+pv_prefix = demo.start_ioc_subprocess()
 
 
 # Create ophyd devices
@@ -32,5 +32,5 @@ class OldSensor(Device):
 
 # Create ophyd-async devices
 with DeviceCollector():
-    det = epicsdemo.Sensor(pv_prefix)
-    samp = epicsdemo.SampleStage(pv_prefix)
+    det = demo.Sensor(pv_prefix)
+    samp = demo.SampleStage(pv_prefix)
diff --git a/docs/user/how-to/make-a-simple-device.rst b/docs/user/how-to/make-a-simple-device.rst
@@ -6,14 +6,14 @@
 Make a Simple Device
 ====================
 
-.. currentmodule:: ophyd_async.core.core
+.. currentmodule:: ophyd_async.core
 
 To make a simple device, you need to subclass from the
 `StandardReadable` class, create some `Signal` instances, and optionally implement
 other suitable Bluesky `Protocols <hardware_interface>` like
 :class:`~bluesky.protocols.Movable`.
 
-The rest of this guide will show examples from ``src/ophyd_async/core/epicsdemo/__init__.py``
+The rest of this guide will show examples from ``src/ophyd_async/epics/demo/__init__.py``
 
 Readable
 --------
@@ -22,7 +22,7 @@ For a simple :class:`~bluesky.protocols.Readable` object like a `Sensor`, you ne
 define some signals, then tell the superclass which signals should contribute to
 ``read()`` and ``read_configuration()``:
 
-.. literalinclude:: ../../../src/ophyd_async/core/epicsdemo/__init__.py
+.. literalinclude:: ../../../src/ophyd_async/epics/demo/__init__.py
    :pyobject: Sensor
 
 First some Signals are constructed and stored on the Device. Each one is passed
@@ -53,7 +53,7 @@ Movable
 For a more complicated device like a `Mover`, you can still use `StandardReadable`
 and implement some addition protocols:
 
-.. literalinclude:: ../../../src/ophyd_async/core/epicsdemo/__init__.py
+.. literalinclude:: ../../../src/ophyd_async/epics/demo/__init__.py
    :pyobject: Mover
 
 The ``set()`` method implements :class:`~bluesky.protocols.Movable`. This
@@ -70,7 +70,7 @@ Assembly
 
 Compound assemblies can be used to group Devices into larger logical Devices:
 
-.. literalinclude:: ../../../src/ophyd_async/core/epicsdemo/__init__.py
+.. literalinclude:: ../../../src/ophyd_async/epics/demo/__init__.py
    :pyobject: SampleStage
 
 This applies prefixes on construction:

diff --git a/docs/user/reference/api.rst b/docs/user/reference/api.rst
@@ -6,9 +6,9 @@
 API
 ===
 
-.. automodule:: ophyd_async.core
+.. automodule:: ophyd_async
 
-    ``ophyd_async.core``
+    ``ophyd_async``
     -----------------------------------
 
 This is the internal API reference for ophyd_async
@@ -24,10 +24,6 @@ This is the internal API reference for ophyd_async
     :template: custom-module-template.rst
     :recursive:
 
-    ophyd_async.core.backends
-    ophyd_async.core.devices
-    ophyd_async.core.epicsdemo
-    ophyd_async.core.signals
-    ophyd_async.core.async_status
-    ophyd_async.core.device_collector
-    ophyd_async.core.utils
+    core
+    epics
+    panda
diff --git a/docs/user/tutorials/installation.rst b/docs/user/tutorials/installation.rst
@@ -9,7 +9,7 @@ Installation
 Check your version of python
 ----------------------------
 
-You will need python 3.8 or later. You can check your version of python by
+You will need python 3.9 or later. You can check your version of python by
 typing into a terminal::
 
     $ python3 --version
@@ -30,7 +30,7 @@ Installing the library
 
 You can now use ``pip`` to install the library and its dependencies::
 
-    $ python3 -m pip install ophyd_async
+    $ python3 -m pip install ophyd-async
 
 If you require a feature that is not currently released you can also install
 from github::
@@ -40,4 +40,4 @@ from github::
 The library should now be installed and the commandline interface on your path.
 You can check the version that has been installed by typing::
 
-    $ ophyd_async --version
+    $ ophyd-async --version