Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Models that fail integration tests but defy isolation #1085

Open
ablaom opened this issue Jan 16, 2024 · 0 comments
Open

Models that fail integration tests but defy isolation #1085

ablaom opened this issue Jan 16, 2024 · 0 comments
Labels
invalid This doesn't seem right

Comments

@ablaom
Copy link
Member

ablaom commented Jan 16, 2024

Integration tests (contained in the new file /test/integration.tl) fail for two models in a very subtle way I have been unable to nail down after several days of trying. The models are:

  • AdaBoostStumpClassifier (interface package: MLJDecisionTreeInterface)
  • COFDetector (interface package :OutlierDetectorNeighbors)

If I run ] test MLJ the models fail, but if I manually execute the /test/integration.tl script, the models pass. I have carefully compared the two environment manifests that appear to be operating in the two cases and they are identical. I can see no trace of some RNG-related difference.

For now I have excluded the models from integration tests, with a reference to this issue in the test code.

Note that to force integration tests to run, you must set the env variable MLJ_TEST_INTEGRATION="true".

edit

Adding to this list:

  • TSVDTransform

Passes tests locally but fails in CI.

@ablaom ablaom added the invalid This doesn't seem right label Jan 16, 2024
@ablaom ablaom changed the title Two models fail integration tests but defy isolation Models that fail integration tests but defy isolation Jun 5, 2024
@github-project-automation github-project-automation bot moved this to priority low / involved in General Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
Status: priority low / involved
Development

No branches or pull requests

1 participant