Skip to content

Commit

Permalink
Speed up algorithm processing, update deps, update docs (#1)
Browse files Browse the repository at this point in the history
- Speed up AllPairs algorithm processing time
- Optimize AllPairs algorithm resources usage
- Update dependencies
- Update documentation
  • Loading branch information
pavelicii authored Nov 4, 2023
1 parent 406cbc8 commit 97b6cba
Show file tree
Hide file tree
Showing 18 changed files with 571 additions and 346 deletions.
21 changes: 10 additions & 11 deletions .github/workflows/build-checkstyle.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,22 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2.4.0
uses: actions/checkout@v4

- name: Validate Gradle Wrapper
uses: gradle/wrapper-validation-action@v1

- name: Setup JDK 8
uses: actions/setup-java@v2.5.0
uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 8

- name: Validate Gradle Wrapper
uses: gradle/wrapper-validation-action@v1.0.4
- name: Setup Gradle
uses: gradle/gradle-build-action@v2

- name: Run Build
uses: gradle/gradle-build-action@v2.1.3
with:
arguments: build
- name: Run Build and Tests
run: ./gradlew clean build

- name: Run Checkstyle
uses: gradle/gradle-build-action@v2.1.3
with:
arguments: checkstyle
run: ./gradlew checkstyle
22 changes: 12 additions & 10 deletions .github/workflows/tag-release-publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,22 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2.4.0
uses: actions/checkout@v4

- name: Validate Gradle Wrapper
uses: gradle/wrapper-validation-action@v1

- name: Setup JDK 8
uses: actions/setup-java@v2.5.0
uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 8

- name: Run Build
uses: gradle/gradle-build-action@v2.1.3
with:
arguments: build
- name: Setup Gradle
uses: gradle/gradle-build-action@v2

- name: Run Build and Tests
run: ./gradlew clean build

- name: Get version from build.gradle.kts
run: |
Expand All @@ -39,15 +43,13 @@ jobs:
run: git tag -m ${{ env.tag }} ${{ env.tag }} && git push origin ${{ env.tag }}

- name: Release to GitHub
uses: softprops/action-gh-release@v0.1.14
uses: softprops/action-gh-release@v1
with:
tag_name: ${{ env.tag }}
files: build/libs/allpairs4j-${{ env.version }}.jar

- name: Publish to Sonatype
uses: gradle/gradle-build-action@v2.1.3
with:
arguments: publish
run: ./gradlew publish
env:
ORG_GRADLE_PROJECT_sonatypeUsername: ${{ secrets.OSSRH_USERNAME }}
ORG_GRADLE_PROJECT_sonatypePassword: ${{ secrets.OSSRH_PASSWORD }}
Expand Down
2 changes: 1 addition & 1 deletion NOTICE
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AllPairs4J
Copyright 2022 Pavel Nazimok - @pavelicii
Copyright 2023 Pavel Nazimok - @pavelicii

This product is derived from the work by MetaCommunications Inc.
It can be obtained at:
Expand Down
100 changes: 59 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,29 @@
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/pavelicii/allpairs4j/build-checkstyle.yaml?branch=master&logo=GitHub)](https://github.com/pavelicii/allpairs4j/actions/workflows/build-checkstyle.yaml)
[![Maven Central](https://img.shields.io/maven-central/v/io.github.pavelicii/allpairs4j)](https://search.maven.org/artifact/io.github.pavelicii/allpairs4j)

AllPairs4J is an open source Java library for generation of minimal set of test combinations.
AllPairs4J is an open source Java library for generation of minimal set of test combinations.

AllPairs4J is a Java port of [allpairspy](https://github.com/thombashi/allpairspy) project
(with some [improvements and bugfixes](https://github.com/thombashi/allpairspy/pull/10))
developed by MetaCommunications Engineering and Tsuyoshi Hombashi.
## Pairwise Testing • [pairwise.org](https://www.pairwise.org/)

## Pairwise Testing

In computer science, _all-pairs testing_ or _pairwise testing_ is a combinatorial method of software testing that, for
each pair of input parameters to a system, tests all possible discrete combinations of those parameters. For example,
if you want to create a test suite for browser testing, the domain can be described by the following parameters:
Assuming you want to create test cases for web browser testing, the domain can be described by the following parameters:

```text
Browser: Chrome, Firefox, Safari, Edge
OS: Windows, Linux, macOS
RAM: 1024, 2048, 4096, 8192, 16384
Drive: HDD, SSD
Screen: 1024x768, 1366x768, 1680x1050, 1920x1080, 2560x1440, 3840x2160
```

There are hundreds of possible combinations of these values. Instead of spending unreasonable amount of time testing
them all, pairwise suggests testing all possible pairs of values. For example, `{Chrome, Windows}` is one pair,
`{4096, SSD}` is another; together they represent a test case that covers many pairs: `{Chrome, Windows, 4096, SSD}`.
In the end, you have good coverage while the number of test cases remains manageable.

With AllPairs4J, you are also able to add limitations on the domain to restrict generation of certain pairs.
For example, specify that `Safari` can only be paired with `macOS`, and `Edge` can only be paired with `Windows`.
There are hundreds of possible combinations of these values. However, usually most faults are caused by interactions
of at most two factors, therefore testing all pairs is an effective alternative to exhaustive testing.
For example, `{Chrome, Windows}` is one pair, `{4096, SSD}` is another; together they can represent a test case
that also covers many other pairs: `{Chrome, Windows, 4096, SSD, 2560x1440}`. In the end, you have good
coverage while the number of test cases remains manageable.

For more info on pairwise testing see https://www.pairwise.org/.
With AllPairs4J, you are also able to add **constraints** - limitations on the domain to restrict generation of certain
pairs. For example, specify that `Safari` can only be paired with `macOS`, and `Edge` can only be paired with `Windows`.
Or you can go beyond pairs.

## Features

Expand All @@ -50,7 +45,7 @@ Java 8 or higher.

```groovy
dependencies {
implementation 'io.github.pavelicii:allpairs4j:1.0.0'
implementation("io.github.pavelicii:allpairs4j:1.0.1")
}
```

Expand All @@ -60,7 +55,7 @@ dependencies {
<dependency>
<groupId>io.github.pavelicii</groupId>
<artifactId>allpairs4j</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
</dependency>
```

Expand Down Expand Up @@ -109,10 +104,11 @@ System.out.println(allPairs);

</details>

### Generate Filtered Pairwise Combinations
### Constraints: Generate Filtered Pairwise Combinations

To limit generated pairs, you need to describe Constraints. Each Constraint is a Predicate that is tested against
each generated Case. If it evaluates to `true`, the Case is deleted from the result.
To filter out unwanted combinations, you need to describe Constraints. Each potential test Case is tested against them.
If any test evaluates to `true`, the Case under test won't be present in the result and the algorithm will search
for another Case, so that in the end all possible pairs are covered (considering all the Constraints).

For example, let's create limitations so that:
* `Browser=Safari` can only be paired with `OS=macOS`
Expand All @@ -132,10 +128,7 @@ AllPairs allPairs = new AllPairs.AllPairsBuilder()
.withConstraint(c -> (int) c.get("RAM") < 4000)
.build();

int index = 1;
for (Case c : allPairs) {
System.out.printf("%3d: %s%n", index++, c);
}
System.out.println(allPairs);
```

#### Output:
Expand All @@ -158,6 +151,31 @@ for (Case c : allPairs) {

</details>

#### Constraints tips:

Try to simplify constraints as much as possible. Too complicated constraints might cause longer algorithm processing
time, especially on a large input of Parameters.

For example, for the following input:

```
Browser: Chrome
OS: Linux, macOS
Drive: HDD, SSD
```

Consider two different constraints:

```java
// filter out 'Chrome-Linux-HDD' combination:
complicatedConstraint = c -> c.get("Browser").equals("Chrome") && c.get("OS").equals("Linux") && c.get("Drive").equals("HDD")
// filter out 'Linux-HDD' pair:
simplifiedConstraint = c -> c.get("OS").equals("Linux") && c.get("Drive").equals("HDD")
```

It is better to use `simplifiedConstraint`, because the usage of `complicatedConstraint` implies there might be pairs
including non-`Chrome` browsers, while in fact there is only one possible browser.

### Generate Triplewise Combinations

You can specify test combination size to go beyond pairs.
Expand All @@ -174,7 +192,7 @@ AllPairs allPairs = new AllPairs.AllPairsBuilder()
new Parameter("Drive", "HDD", "SSD")))
.build();

System.out.println(allPairs.toString());
System.out.println(allPairs);
```

#### Output:
Expand Down Expand Up @@ -234,30 +252,30 @@ System.out.println(allPairs.toString());

</details>

### Configuration summary
### Configuration Summary

#### Sample code:
#### Builder:

```java
AllPairs allPairs = new AllPairs.AllPairsBuilder()
.withParameters( List<Parameter> ) // required
.withParameter( Parameter ) // alternative way to specify Parameters one by one
.withConstraints( List<Predicate<Case>> ) // not required, default is no Constraints
.withConstraint( Predicate<Case> ) // alternative way to specify Constraints one by one
.withTestCombinationSize( int ) // not required, default is 2
.printEachCaseDuringGeneration() // not required, useful for debug
.withParameter( Parameter ) // specifies 1 Parameter
.withParameters( List<Parameter> ) // alternative way to specify multiple Parameters as List
.withConstraint( Predicate<ConstrainableCase> ) // specifies 1 Constraint, default is no Constraints
.withConstraints( List<Predicate<ConstrainableCase>> ) // alternative way to specify multiple Constraints as List
.withTestCombinationSize( int ) // specifies test combination size, default is 2 (pair)
.printEachCaseDuringGeneration() // prints Cases during generation, useful for debug
.build();

List<Case> generatedCases = allPairs.getGeneratedCases();
List<Case> generatedCases = allPairs.getGeneratedCases(); // work with resulting List of Cases
for (Case c : allPairs) { ... } // or use Iterator
```

#### Data types:

* **Parameter**: Named `List<Object>` storing all input values.
* **Parameter**: named `List<Object>` storing all input values
* **Case**: `Map<String, Object>` storing one generated test case,
where `key` is mapped to the `Parameter` name, `value` is mapped to one of the `Parameter` values.
* **Predicate\<Case\>**: constraint to test against `Case`. When evaluates to `true`, the `Case` is considered invalid
and is filtered out form the resulting set.
where `key` is mapped to the `Parameter` name, `value` is mapped to one of the `Parameter` values
* **Predicate\<ConstrainableCase\>**: constraint

## Contributing

Expand All @@ -266,4 +284,4 @@ Pull requests are welcome. For major changes, please open an issue first to disc
Please make sure to update tests as appropriate.

[SemVer](https://semver.org/) is used for versioning. For the versions available,
see the [releases on this repository](https://github.com/pavelicii/allpairs4j/releases).
see the [releases](https://github.com/pavelicii/allpairs4j/releases) on this repository.
19 changes: 13 additions & 6 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ plugins {
}

group = "io.github.pavelicii"
version = "1.0.0"
version = "1.0.1"

repositories {
mavenCentral()
Expand All @@ -21,7 +21,7 @@ java {
}

checkstyle {
toolVersion = "9.3"
toolVersion = "9.3" // Latest version compatible with Java 8
sourceSets = listOf() // Don't check anything with Checkstyle during 'check' task
}
tasks.register("checkstyle", Checkstyle::class) {
Expand All @@ -30,13 +30,20 @@ tasks.register("checkstyle", Checkstyle::class) {
}

dependencies {
testImplementation("org.junit.jupiter:junit-jupiter-api:5.8.2")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.8.2")
testImplementation("org.assertj:assertj-core:3.22.0")
testImplementation("org.junit.jupiter:junit-jupiter:5.10.0")
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
testImplementation("org.assertj:assertj-core:3.24.2")
}

tasks.test {
useJUnitPlatform()
testLogging {
events("passed", "skipped", "failed")
}
}

tasks.withType<Javadoc> {
(options as StandardJavadocDocletOptions).addStringOption("Xdoclint:none", "-quiet")
}

publishing {
Expand All @@ -59,7 +66,7 @@ publishing {
licenses {
license {
name.set("The Apache License, Version 2.0")
url.set("http://www.apache.org/licenses/LICENSE-2.0.txt")
url.set("https://www.apache.org/licenses/LICENSE-2.0.txt")
}
}
developers {
Expand Down
Loading

0 comments on commit 97b6cba

Please sign in to comment.