Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using Nimble expectations in Swift Testing #1145

Open
younata opened this issue Jun 12, 2024 · 0 comments
Open

Allow using Nimble expectations in Swift Testing #1145

younata opened this issue Jun 12, 2024 · 0 comments

Comments

@younata
Copy link
Member

younata commented Jun 12, 2024

Swift Testing is the biggest thing from Apple for testing since Xcode 4 integrated test results in the editor. Nimble's expectations should be supported from Swift Testing.

Unfortunately, there are a couple hurdles preventing us from doing that.

First is actually registering failures with Swift Testing. We can get a basic integration using the Issue.record static method. However, if we had the ability to directly create and record issues, then we could improve our integration dramatically. I filed swiftlang/swift-testing#474 to record on their end providing that API.

Second, and much more of a blocker, is actually detecting whether the test runner is Swift Testing or XCTest. This is necessary to correctly report errors to either Swift Testing or XCTest.

In the lab I had with the Testing engineers yesterday, they mentioned that I could use Test.current != nil to detect positively if Swift Testing is being used. However, Test.current is implemented as a task local value, which means that we can't actually guarantee that Test.current != nil means that we're using the Swift Testing test runner. For example, if you use Task.detached from a test, and then check Test.current inside that detached task, you will get back nil.

Another approach might be to utilize our XCTest test tracking, and if the current test case is nil, then fall back to report to Swift Testing.

A third approach might be to try reporting to both? We'll likely have to adapt NimbleXCTestHandler to gracefully fail if there isn't an XCTestCase to report to, and similarly engineer the future NimbleSwiftTestingHandler to gracefully fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant