Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] MultiClusterSpecIT test {scoring.QstrWithFieldAndScoringSortedEval} failing #117751

Open
elasticsearchmachine opened this issue Nov 29, 2024 · 5 comments
Open

[CI] MultiClusterSpecIT test {scoring.QstrWithFieldAndScoringSortedEval} failing #117751

elasticsearchmachine opened this issue Nov 29, 2024 · 5 comments
Assignees
@tteofili
Labels
:Analytics/ES|QL AKA ESQL needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Nov 29, 2024 •
edited
Loading

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:esql:qa:server:multi-clusters:v8.18.0#bwcTest" -Dtests.class="org.elasticsearch.xpack.esql.ccq.MultiClusterSpecIT" -Dtests.method="test {scoring.QstrWithFieldAndScoringSortedEval}" -Dtests.seed=A7D0F80D73A91D07 -Dtests.bwc=true -Dtests.locale=ki-KE -Dtests.timezone=Etc/GMT+2 -Druntime.java=23

Applicable branches:
8.x

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Data mismatch:
row 1 column 0:a list containing
row 1 column 0:0: expected "7140" but was "2714"
row 1 column 1:a list containing
row 1 column 1:0: expected "The Lord of the Rings Poster Collection: Six Paintings by Alan Lee (No. 1)" but was "Return of the King Being the Third Part of The Lord of the Rings"
row 2 column 0:a list containing
row 2 column 0:0: expected "2714" but was "7140"
row 2 column 1:a list containing
row 2 column 1:0: expected "Return of the King Being the Third Part of The Lord of the Rings" but was "The Lord of the Rings Poster Collection: Six Paintings by Alan Lee (No. 1)"

Issue Reasons:

  • [8.x] 5 failures in test test {scoring.QstrWithFieldAndScoringSortedEval} (14.3% fail rate in 35 executions)
  • [8.x] 2 failures in step openjdk17_8.18.0_java-matrix-bwc (66.7% fail rate in 3 executions)
  • [8.x] 3 failures in pipeline elasticsearch-periodic (100.0% fail rate in 3 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Analytics/ES|QL AKA ESQL >test-failure Triaged test failures from CI labels Nov 29, 2024
elasticsearchmachine added a commit that referenced this issue Nov 29, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.esql.ccq.MultiClusterSpecIT test {scorin…
b7c38a1 
…g.QstrWithFieldAndScoringSortedEval} #117751
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 4 failures in test test {scoring.QstrWithFieldAndScoringSortedEval} (6.3% fail rate in 63 executions)
  • [main] 3 failures in step 9.0.0_bwc-snapshots (5.5% fail rate in 55 executions)
  • [main] 2 failures in pipeline elasticsearch-pull-request (4.5% fail rate in 44 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) needs:risk Requires assignment of a risk label (low, medium, blocker) labels Nov 29, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-analytical-engine (Team:Analytics)

@alex-spies alex-spies added the :Search Relevance/Ranking Scoring, rescoring, rank evaluation. label Nov 29, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Nov 29, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@alex-spies
Copy link
Contributor

Heya, we've encountered a bunch of failures also on Serverless: https://gradle-enterprise.elastic.co/s/vwjcwfars26h6/tests/overview?outcome=FAILED

@tteofili tteofili self-assigned this Nov 29, 2024
@tteofili tteofili mentioned this issue Nov 29, 2024
Merged
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch 8.x

Mute Reasons:

  • [8.x] 5 failures in test test {scoring.QstrWithFieldAndScoringSortedEval} (14.3% fail rate in 35 executions)
  • [8.x] 2 failures in step openjdk17_8.18.0_java-matrix-bwc (66.7% fail rate in 3 executions)
  • [8.x] 3 failures in pipeline elasticsearch-periodic (100.0% fail rate in 3 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Nov 30, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.esql.ccq.MultiClusterSpecIT test {scorin…
1f08d64 
…g.QstrWithFieldAndScoringSortedEval} #117751
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tteofili tteofili

Labels
:Analytics/ES|QL AKA ESQL needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tteofili @elasticsearchmachine @alex-spies