Skip to content

Commit

Permalink
✅(jest) Make tests run concurrently (#5378)
Browse files Browse the repository at this point in the history
**Description**

<!-- Please provide a short description and potentially linked issues
justifying the need for this PR -->

They used to be running in a concurrent fashion in the past. But we
stopped doing it due to a recent regression on the stability of our CI
with Node 23.

The regression being still noticeable even without it and because of the
cost of not doing the runs in a concurrent fashion, we move back to
concurrent execution.

<!-- * Your PR is fixing a bug or regression? Check for existing issues
related to this bug and link them -->
<!-- * Your PR is adding a new feature? Make sure there is a related
issue or discussion attached to it -->

<!-- You can provide any additional context to help into understanding
what's this PR is attempting to solve: reproduction of a bug, code
snippets... -->

**Checklist** — _Don't delete this checklist and make sure you do the
following before opening the PR_

- [x] The name of my PR follows [gitmoji](https://gitmoji.dev/)
specification
- [x] My PR references one of several related issues (if any)
- [x] New features or breaking changes must come with an associated
Issue or Discussion
- [x] My PR does not add any new dependency without an associated Issue
or Discussion
- [x] My PR includes bumps details, please run `yarn bump` and flag the
impacts properly
- [x] My PR adds relevant tests and they would have failed without my PR
(when applicable)

<!-- More about contributing at
https://github.com/dubzzz/fast-check/blob/main/CONTRIBUTING.md -->

**Advanced**

<!-- How to fill the advanced section is detailed below! -->

- [x] Category: ✅ Add or update tests
- [x] Impacts: Faster CI

<!-- [Category] Please use one of the categories below, it will help us
into better understanding the urgency of the PR -->
<!-- * ✨ Introduce new features -->
<!-- * 📝 Add or update documentation -->
<!-- * ✅ Add or update tests -->
<!-- * 🐛 Fix a bug -->
<!-- * 🏷️ Add or update types -->
<!-- * ⚡️ Improve performance -->
<!-- * _Other(s):_ ... -->

<!-- [Impacts] Please provide a comma separated list of the potential
impacts that might be introduced by this change -->
<!-- * Generated values: Can your change impact any of the existing
generators in terms of generated values, if so which ones? when? -->
<!-- * Shrink values: Can your change impact any of the existing
generators in terms of shrink values, if so which ones? when? -->
<!-- * Performance: Can it require some typings changes on user side?
Please give more details -->
<!-- * Typings: Is there a potential performance impact? In which cases?
-->
  • Loading branch information
dubzzz authored Oct 29, 2024
1 parent 0d6eda8 commit 12a5678
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 29 deletions.
58 changes: 29 additions & 29 deletions packages/jest/test/jest-fast-check.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ describe.each<DescribeOptions>([
])('$specName', ({ runnerName, useWorkers, testRunner }) => {
const options = { useWorkers, testRunner };

it('should pass on successful no prop mode', async () => {
it.concurrent('should pass on successful no prop mode', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner('successful no prop', () => {
Expand All @@ -74,7 +74,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] successful no prop/);
});

it('should fail on failing no prop mode', async () => {
it.concurrent('should fail on failing no prop mode', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner('failing no prop', () => {
Expand All @@ -91,7 +91,7 @@ describe.each<DescribeOptions>([
});

if (useWorkers) {
it('should fail on property blocking the main thread', async () => {
it.concurrent('should fail on property blocking the main thread', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()], { timeout: 500 })('property block main thread', () => {
Expand All @@ -108,7 +108,7 @@ describe.each<DescribeOptions>([
});
}

it('should pass on truthy synchronous property', async () => {
it.concurrent('should pass on truthy synchronous property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.string(), fc.string(), fc.string()])('property pass sync', (a, b, c) => {
Expand All @@ -124,7 +124,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass sync \(with seed=-?\d+\)/);
});

it('should pass on truthy asynchronous property', async () => {
it.concurrent('should pass on truthy asynchronous property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.string(), fc.string(), fc.string()])('property pass async', async (a, b, c) => {
Expand All @@ -141,7 +141,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass async \(with seed=-?\d+\)/);
});

it('should fail on falsy synchronous property', async () => {
it.concurrent('should fail on falsy synchronous property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()])('property fail sync', (a) => {
Expand All @@ -158,7 +158,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail sync \(with seed=-?\d+\)/);
});

it('should fail on falsy asynchronous property', async () => {
it.concurrent('should fail on falsy asynchronous property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()])('property fail async', async (a) => {
Expand All @@ -176,7 +176,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail async \(with seed=-?\d+\)/);
});

it('should pass on truthy record-based property', async () => {
it.concurrent('should pass on truthy record-based property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop({ a: fc.string(), b: fc.string(), c: fc.string() })('property pass record', ({ a, b, c }) => {
Expand All @@ -195,7 +195,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass record \(with seed=-?\d+\)/);
});

it('should fail on falsy record-based property', async () => {
it.concurrent('should fail on falsy record-based property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop({ a: fc.string(), b: fc.string(), c: fc.string() })('property fail record', ({ a, b, c }) => {
Expand All @@ -212,7 +212,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail record \(with seed=-?\d+\)/);
});

it('should fail on falsy record-based property with seed', async () => {
it.concurrent('should fail on falsy record-based property with seed', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop({ a: fc.string(), b: fc.string(), c: fc.string() }, { seed: 4869 })(
Expand All @@ -230,7 +230,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail record seeded \(with seed=4869\)/);
});

it('should fail with locally requested seed', async () => {
it.concurrent('should fail with locally requested seed', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.constant(null)], { seed: 4242 })('property fail with locally requested seed', (_unused) => false);
Expand All @@ -245,7 +245,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail with locally requested seed \(with seed=4242\)/);
});

it('should fail with globally requested seed', async () => {
it.concurrent('should fail with globally requested seed', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
fc.configureGlobal({ seed: 4848 });
Expand All @@ -261,7 +261,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property fail with globally requested seed \(with seed=4848\)/);
});

it('should fail with seed requested at jest level', async () => {
it.concurrent('should fail with seed requested at jest level', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.constant(null)])('property fail with globally requested seed', (_unused) => false);
Expand All @@ -277,7 +277,7 @@ describe.each<DescribeOptions>([
});

describe('.skip', () => {
it('should never be executed', async () => {
it.concurrent('should never be executed', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.skip.prop([fc.constant(null)])('property never executed', (_unused) => false);
Expand All @@ -294,7 +294,7 @@ describe.each<DescribeOptions>([

if (testRunner === undefined) {
describe('.failing', () => {
it('should fail on successful no prop mode', async () => {
it.concurrent('should fail on successful no prop mode', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.failing('successful no prop', () => {
Expand All @@ -310,7 +310,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] successful no prop/);
});

it('should pass on failing no prop mode', async () => {
it.concurrent('should pass on failing no prop mode', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.failing('failing no prop', () => {
Expand All @@ -326,7 +326,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] failing no prop/);
});

it('should pass because failing', async () => {
it.concurrent('should pass because failing', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.failing.prop([fc.constant(null)])('property pass because failing', async (_unused) => false);
Expand All @@ -340,7 +340,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass because failing \(with seed=-?\d+\)/);
});

it('should fail because passing', async () => {
it.concurrent('should fail because passing', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.failing.prop([fc.constant(null)])('property fail because passing', async (_unused) => true);
Expand All @@ -357,7 +357,7 @@ describe.each<DescribeOptions>([
}

describe('.concurrent', () => {
it('should pass on truthy property', async () => {
it.concurrent('should pass on truthy property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.concurrent.prop([fc.constant(null)])('property pass on truthy property', (_unused) => true);
Expand All @@ -371,7 +371,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass on truthy property \(with seed=-?\d+\)/);
});

it('should fail on falsy property', async () => {
it.concurrent('should fail on falsy property', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.concurrent.prop([fc.constant(null)])('property fail on falsy property', (_unused) => false);
Expand All @@ -388,7 +388,7 @@ describe.each<DescribeOptions>([

if (testRunner === undefined) {
describe('.failing', () => {
it('should pass because failing', async () => {
it.concurrent('should pass because failing', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.concurrent.failing.prop([fc.constant(null)])(
Expand All @@ -405,7 +405,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[√✓] property pass because failing \(with seed=-?\d+\)/);
});

it('should fail because passing', async () => {
it.concurrent('should fail because passing', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.concurrent.failing.prop([fc.constant(null)])(
Expand All @@ -426,7 +426,7 @@ describe.each<DescribeOptions>([
});

describe('timeout', () => {
it('should fail as test takes longer than global Jest timeout', async () => {
it.concurrent('should fail as test takes longer than global Jest timeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()])('property takes longer than global Jest timeout', async () => {
Expand All @@ -443,7 +443,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property takes longer than global Jest timeout/);
});

it('should fail as test takes longer than Jest local timeout', async () => {
it.concurrent('should fail as test takes longer than Jest local timeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()])(
Expand All @@ -464,7 +464,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property takes longer than Jest local timeout/);
});

it('should fail as test takes longer than Jest config timeout', async () => {
it.concurrent('should fail as test takes longer than Jest config timeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, { ...options, testTimeoutConfig: 1000 }, () => {
runner.prop([fc.nat()])('property takes longer than Jest config timeout', async () => {
Expand All @@ -481,7 +481,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property takes longer than Jest config timeout/);
});

it('should fail as test takes longer than Jest setTimeout', async () => {
it.concurrent('should fail as test takes longer than Jest setTimeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
if (typeof jest !== 'undefined') {
Expand All @@ -501,7 +501,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property takes longer than Jest setTimeout/);
});

it('should fail as test takes longer than Jest CLI timeout', async () => {
it.concurrent('should fail as test takes longer than Jest CLI timeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
runner.prop([fc.nat()])('property takes longer than Jest CLI timeout', async () => {
Expand All @@ -518,7 +518,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property takes longer than Jest CLI timeout/);
});

it('should fail but favor local Jest timeout over Jest setTimeout', async () => {
it.concurrent('should fail but favor local Jest timeout over Jest setTimeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
if (typeof jest !== 'undefined') {
Expand All @@ -542,7 +542,7 @@ describe.each<DescribeOptions>([
expect(out).toMatch(/[×✕] property favor local Jest timeout over Jest setTimeout/);
});

it('should fail but favor Jest setTimeout over Jest CLI timeout', async () => {
it.concurrent('should fail but favor Jest setTimeout over Jest CLI timeout', async () => {
// Arrange
const specDirectory = await writeToFile(runnerName, options, () => {
if (typeof jest !== 'undefined') {
Expand Down
3 changes: 3 additions & 0 deletions packages/jest/vitest.config.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
import { defineConfig } from 'vitest/config';

const major = Number(process.versions.node.split('.')[0]);

export default defineConfig({
test: {
testTimeout: 60000, // 60s
include: ['**/test/*.{test,spec}.?(c|m)[jt]s?(x)'],
retry: major === 23 ? 5 : undefined,
},
});

0 comments on commit 12a5678

Please sign in to comment.