Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"The handle is invalid" error when trying to use a std stream saved before teh call to pytest.main #12876

Open
4 tasks done
ikappaki opened this issue Oct 10, 2024 · 2 comments
Labels
plugin: capture related to the capture builtin plugin type: docs documentation improvement, missing or needing clarification

Comments

@ikappaki
Copy link

Hi,

it seems there's an issue when trying to use a standard system output stream that was saved before calling pytest.main on MS-Windows

E   OSError: [WinError 6] The handle is invalid

To reproduce on MS-Windows 11:
1.Create a script that calls pytest.main but saves sys.stdout in another module before doing so. A test file will then attempt to write to this saved stdout stream

save.py

STDOUT = None

issue_test.py

import save
import sys

save.STDOUT.write(":save.STDOUT\n")

def test_addition():
    assert 1 + 1 == 2

ptissue.py

import pytest
import save
import sys

if __name__ == "__main__":
    save.STDOUT = sys.stdout
    pytest.main(sys.argv[1:])
  1. Run the script, you will encounter the error
>  python .\ptissue.py
=============================================================================================================== test session starts ================================================================================================================
platform win32 -- Python 3.11.4, pytest-8.3.3, pluggy-1.5.0
collected 0 items / 1 error

====================================================================================================================== ERRORS ======================================================================================================================
__________________________________________________________________________________________________________ ERROR collecting issue_test.py __________________________________________________________________________________________________________
issue_test.py:4: in <module>
    save.STDOUT.write(":save.STDOUT\n")
E   OSError: [WinError 6] The handle is invalid
============================================================================================================= short test summary info ==============================================================================================================
ERROR issue_test.py - OSError: [WinError 6] The handle is invalid
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================================================= 1 error in 0.08s =================================================================================================================
:save.STDOUT

The error does not occur if you pass the -s option:

> python .\ptissue.py -s
=============================================================================================================== test session starts ================================================================================================================
platform win32 -- Python 3.11.4, pytest-8.3.3, pluggy-1.5.0
collecting ... :save.STDOUT
collected 1 item

issue_test.py .

================================================================================================================ 1 passed in 0.01s =================================================================================================================

Thanks

> pip list
Package    Version
---------- -------
colorama   0.4.6
iniconfig  2.0.0
packaging  24.1
pip        23.1.2
pluggy     1.5.0
pytest     8.3.3
setuptools 65.5.0

> pytest --version
pytest 8.3.3
  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible
@RonnyPfannschmidt
Copy link
Member

That's expected to enable fdcapture

@ikappaki
Copy link
Author

ikappaki commented Oct 10, 2024

That's expected to enable fdcapture

Thanks, is this expected behavior documented anywhere by chance for reference purposes?

chrisrink10 pushed a commit to basilisp-lang/basilisp that referenced this issue Oct 14, 2024
Hi,

can you please review patch to address the `basilisp test` failure on
MS-Windows. It fixes #1080.

The problem arises because pytest's mechanism for capturing standard
streams during tests breaks when those streams are saved before
pytest.main is called and then used in the test. This happens when
running `basilisp test` from the CLI:

In `basilisp.init`, `sys.stdout` and `sys.stderr` are saved to the
`*out*` and `*err*` dynamic variables before `pytest.main` is called. If
a test prints anything, it tries to write to these original streams,
causing failures due to an issue I just reported in
pytest-dev/pytest#12876.

This patch rebinds `*out*` and `*err*` to the streams set by `pytest`
and restores them when `pytest` finishes (via the `configure`
functions). I've tested it with a case that prints to `*out*` and
`*err*` at the top level, inside a test, and inside a fixture, and it
works as expected.
```clojure
(ns tests.issuetests.issue-test
  (:require [basilisp.test :refer [deftest are is testing use-fixtures]])
  (:import sys))

(println :top-out)
(binding [*out* *err*]
  (println :top-err))

(defn fixture-gen-test []
  (println :fix-gen-out)
  (yield)
  (binding [*out* *err*]
    (println :fix-gen-err)))

(use-fixtures :each fixture-gen-test)

(deftest print_test
  (println :test-out)
  (is (= 5 5))
  (binding [*out* *err*]
    (println :test-err)))

```

```powershell
> basilisp test
=============================================================================================================== test session starts ================================================================================================================
platform win32 -- Python 3.11.4, pytest-8.4.0.dev100+g410c51011, pluggy-1.5.0
rootdir: C:\clj\issues\issue-bas-test-macro-print
configfile: pyproject.toml
plugins: basilisp-0.2.4
collecting ... :top-out
:top-err
collected 1 item

tests\issuetests\issue_test.lpy :fix-gen-out
:test-out
:test-err
.:fix-gen-err
```

I’d also like to discuss `Basilisp` integration tests, which I think the
above testing falls under, though there’s currently no support for such
tests in `Basilisp`.

Ideally I think, I would like to create a `Basilisp` project in a
temporary directory, add the above test, install the fix alongside
`pytest`, activate the environment, and then run `basilisp test`.

Do you have any suggestions how to extend the `Basilisp` test suite to
implement this? or shall I open a new ticket for this discussion? I was
thinking of having a new `--integration` pytest option for these tests,
something I have been experimenting with in the basilisp-blender
package.

Thanks

---------

Co-authored-by: ikappaki <ikappaki@users.noreply.github.com>
@Zac-HD Zac-HD added type: docs documentation improvement, missing or needing clarification plugin: capture related to the capture builtin plugin labels Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: capture related to the capture builtin plugin type: docs documentation improvement, missing or needing clarification
Projects
None yet
Development

No branches or pull requests

3 participants