New tests not run

[sverhoeven]

When I have a failing tests in a previous run and add a test then then ci only runs the failing tests. It does not run the new test.
I expected new tests and previously failed tests to be run.

See https://github.com/sverhoeven/pytest-lastfailed-test/actions

[sjvrijn]

Well, that is the expected behavior of the --last-failed flag for pytest:

$ pytest -h
[...]
  --lf, --last-failed   Rerun only the tests that failed at the last run (or all if none failed)
[...]

There is the --new-first flag which runs new test(file)s first. This blogpost (pytest v5.3.5) suggests they can be run together, but it doesn't seem to work on my machine with pytest v7.3.1. (Trying to install pytest 5.3.5 gives me TypeError: required field "lineno" missing from alias, so those dependencies are not sufficiently fixed either)

If there is a way to make --new-first work together with --last-failed, I do think it's a nice idea to include it, perhaps with an option to disable. If that's not easy to include, I'm inclined to keep the current behavior.

In either case, making the documentation more explicit about which tests are run in which case sounds worthwhile.