Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent Snapshot Test Failures Due to Minor Rendering Differences #926

Open
iOS-Kaizen opened this issue Nov 4, 2024 · 0 comments
Open

Comments

@iOS-Kaizen
Copy link

iOS-Kaizen commented Nov 4, 2024

Describe the bug
We are experiencing intermittent failures with snapshot tests due to minor rendering inconsistencies. The tests pass most of the time, but roughly 10% of the time, they fail due to slight visual differences, such as thin red lines or minor artifacts around elements in the snapshot. This issue only occurs in our CI pipeline on Bitrise and not on local runs

To Reproduce
Run the snapshot tests on Bitrise in our CI pipeline.
Observe that the tests occasionally fail due to minor visual discrepancies (around 1 in 10 runs).
When tests fail, the snapshot comparison shows thin red lines or subtle changes in layout that are not significant to the functionality but are flagged as differences.
Update I faced it locally too.

Expected behavior
Snapshot tests should pass consistently as the rendered component visually remains the same.

Screenshots
Screenshot 2024-11-04 at 2 26 10 PM
Screenshot 2024-11-04 at 2 08 31 PM

Screenshot 2024-11-04 at 2 26 59 PM Screenshot 2024-10-24 at 12 17 20 PM

Environment

  • swift-snapshot-testing version [1.16.1]
  • Xcode [15.4]
  • Swift [5]
  • OS: [iOS 17.5]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant