BUG: test in crowsetta/tests/test_data.py failed

[shaupert]

Before submitting a bug, please make sure the issue hasn't been already addressed by searching through the past issues

Describe the bug
While running tests with Pytest I found that 1 of the tests in crowsetta/tests/test_data.py failed on my colab session.
It looks like the problem comes from : fixture 'tmp_path' not found

To Reproduce
I cloned the repo on my colab session and run the test with the following command pytest crowsetta/tests/test_data.py

Screenshots
Errors are reproduced in text below

============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-3.6.4, py-1.11.0, pluggy-0.7.1
rootdir: /content, inifile:
plugins: typeguard-2.7.1
collected 43 items                                                             

crowsetta/tests/test_data.py .E......................................... [100%]

==================================== ERRORS ====================================
___________ ERROR at setup of test_extract_data_files_user_specified ___________
file /content/crowsetta/tests/test_data.py, line 51
  def test_extract_data_files_user_specified(tmp_path):
E       fixture 'tmp_path' not found
>       available fixtures: a_bboxes_list, a_das_csv_path, a_kaggle_phn_path, a_kaggle_wrd_path, a_nist_phn_path, a_nist_wrd_path, a_notmat_path, a_phn_path, a_raven_txt_file, a_seq, a_simple_csv_path, a_textgrid_path, a_transcript_path, a_wrd_path, add_crowsetta, an_audtxt_path, audtxt_paths, birdsong_rec_root, birdsong_rec_wav_path, birdsong_rec_xml_file, birdsongrec_as_generic_seq_csv, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, csv_missing_fields_in_header, csv_root, csv_with_invalid_fields_in_header, csv_with_no_onset_or_offset_column, csv_with_onset_s_but_no_offset_s, csv_with_onset_sample_but_no_offset_sample, das_csv_paths, default_user_data_dir, different_list_of_segments, different_seq, doctest_namespace, example_user_format_annotation_file, example_user_format_as_generic_seq_csv, example_user_format_root, example_user_format_script, kaggle_phn_paths, list_of_segments, monkeypatch, notmat_as_generic_seq_csv, notmat_paths, notmats_root, pytestconfig, raven_dataset_annot_col, raven_root, raven_txt_file_with_no_rows, raven_txt_files, record_property, record_xml_attribute, record_xml_property, recwarn, same_seq, simple_csv_paths, test_data_root, textgrid_paths, timit_kaggle_root, timit_nist_root, timit_phn_as_generic_seq_csv, tmpdir, tmpdir_factory, yarden_annot_mat
>       use 'pytest --fixtures [testpath]' for help on them.

/content/crowsetta/tests/test_data.py:51
====================== 42 passed, 1 error in 1.14 seconds ======================

Desktop (please complete the following information):

• Operating System: Linux
• Version : 4.0.0.post2

Additional context
I created this issue while performing the pyOpenSci/software-submission#68.

[NickleDave]

Hi @shaupert could you please try running tests with the code in the main branch, which is ahead of the releases on PyPI and conda? Using a development environment set up as described here: https://crowsetta.readthedocs.io/en/latest/development/contributors.html#setting-up-a-development-environment

I think the source of the issue is the same as @rhine3 had in #218 and #220. Because I added vignettes + related features, but did not release a new version since I knew I'd be making even more changes during the review! Partly my fault for not making that clearer. And partly because pyOpenSci guides don't have explicit language about this right now as @rhine3 points out

[shaupert]

Hi @NickleDave .
So I tried to figure out how to install brew, pipx and nox on my Ubuntu machine.
Everything worked well following all the provided instructions.
Finally, after cloning crowsetta, I tried to run the command : nox -s dev
But I got the following error for numba:

Collecting numba>=0.51.0
  Using cached numba-0.56.4.tar.gz (2.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-j6nwlu35/numba_19281ba753714f67950a0aed649beb71/setup.py", line 51, in <module>
          _guard_py_ver()
        File "/tmp/pip-install-j6nwlu35/numba_19281ba753714f67950a0aed649beb71/setup.py", line 48, in _guard_py_ver
          raise RuntimeError(msg.format(cur_py, min_py, max_py))
      RuntimeError: Cannot install on Python version 3.11.2; only versions >=3.7,<3.11 are supported.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0 -> 23.0.1
[notice] To update, run: python -m pip install --upgrade pip
nox > Command /home/haupert/Downloads/crowsetta/.venv/bin/python -m pip install -e '.[dev,test,doc]' failed with exit code 1
nox > Session dev failed.

After downgrading my version of Python to 3.9.16, I still have the same error and could no go further as I don't know how to use nox.

You have to know that I'm using conda and I created a virtual environment dedicated to crowsetta and lauching the different tools from this virtual env.

I also tried again to run the test directly on a colab session after cloning crowsetta (!git clone https://github.com/vocalpy/crowsetta.git) and install crowsetta from the clone (!pip install /content/crowsetta) and still have the same issue as before with tmp_path not found.

I'm afraid, I can not help you more on that issue. I did my best. I don't think the problem is big, I'm sure you will be able to fix it.

Sylvain

[NickleDave]

Thank you @shaupert for taking the time to test this.

I think what's going on with nox -s dev is that by default nox uses whichever version of Python that pipx installs it with.
Then nox tries to install numba in a way that won't work on your machine.

It's a bit confusing but under the hood nox uses virtualenv to run different versions of Python; it's not related to your system version. (I had to figure this out from GitHub issues, it's not well documented.)

I did just push a commit that tells nox to use Python 3.10 instead for the dev session--it should work now, I think. You're on Mac? I was able to set up on a Mac yesterday.

I don't think the tests will ever work for a package installed off of an index, since the distribution package doesn't include (1) pytest as a dependency or (2) the test data, which would make the size of the distribution much bigger.
The error you're getting is because pytest can't find one of its own built in fixtures! I guess you might fix that if you update the pytest version but you'd still need the test data.

Like you said, it's not the end of the world if you can't run the tests for the review. They're running on CI and passing so we know they must be working somewhere 🙂 Just telling you since I thought you might be interested in what's going on here, or at least what I think is going on 🤯

We did have some discussion for the PyOS packaging guide about whether people should include tests. Since PyPI serves as a kind of "source" for other package managers (e.g. Debian) then it can be good if packages include tests. But IIUC we would really run up the PyPI bills if we uploaded a ton of giant packages, not to mention increasing our carbon footprint. Anyway, not relevant for the review here. I'll go ahead and close this.

[shaupert]

Just to let you know that I'm on Ubuntu 22.04.02.
Thanks for your answer :)

[NickleDave]

Just to let you know that I'm on Ubuntu 22.04.02.

Thanks! No offense meant by suggesting you're a Mac user 😼

Looks like numba for Python 3.11 on any platform is still a WIP although they are getting close: numba/numba#8304 (comment)