Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: merge bazel-super-formatter repo here #14

Merged
merged 14 commits into from
Oct 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
docs/*.md linguist-generated=true
docs/linting.md linguist-generated=false
docs/formatting.md linguist-generated=false
5 changes: 3 additions & 2 deletions .github/workflows/release_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@ cat << EOF
\`\`\`starlark
bazel_dep(name = "aspect_rules_eslint", version = "${TAG:1}")

# Depending on which linters you setup, you may need more dependencies.
# See example/MODULE.bazel in the rules_lint repository.
# Next, follow the install instructions for
# - linting: https://github.com/aspect-build/rules_lint/blob/${TAG}/docs/linting.md
# - formatting: https://github.com/aspect-build/rules_lint/blob/${TAG}/docs/formatting.md
\`\`\`
EOF

2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,5 @@ bazel-*
.idea/
.ijwb/
node_modules
# alex experiment...
protobuf/
2 changes: 1 addition & 1 deletion MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ bazel_dep(name = "aspect_bazel_lib", version = "1.36.0")

# Needed in the root because we use js_lib_helpers in our aspect impl
bazel_dep(name = "aspect_rules_js", version = "1.32.6")
bazel_dep(name = "bazel_skylib", version = "1.4.1")
bazel_dep(name = "bazel_skylib", version = "1.4.2")
bazel_dep(name = "platforms", version = "0.0.7")

# Needed in the root because we dereference ProtoInfo in our aspect impl
Expand Down
188 changes: 119 additions & 69 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,120 +1,170 @@
# Run linters under Bazel
# Run linters and formatters under Bazel

> It is currently EXPERIMENTAL and pre-release. No support is promised.
> There may be breaking changes, or we may archive and abandon the repository.

This ruleset integrates linting as a first-class concept under Bazel.
This ruleset integrates linting and formatting as first-class concepts under Bazel.

Features:

- **No changes needed to rulesets**. Works with the Bazel rules you already use.
- **No changes needed to BUILD files**. You don't need to add lint wrapper macros, and lint doesn't appear in `bazel query` output.
Instead, users can lint their existing `*_library` targets.
- Lint results can be **presented in various ways**, see "Usage" below.
- **Can format files not known to Bazel**. Formatting just runs directly on the file tree.
No need to create `sh_library` targets for your shell scripts, for example.
- Honors the same configuration files you use for these tools outside Bazel (e.g. in the editor)

This project is inspired by the design for [Tricorder].
This is how Googlers get their static analysis results in code review (Critique).
https://github.com/google/shipshape is an old, abandoned attempt to open-source Tricorder.

Note: we believe that Formatting is **NOT** Linting.
We have a separate project for formatting, see <https://github.com/aspect-build/bazel-super-formatter#design>
It is also inspired by <https://github.com/github/super-linter>.

[aspect cli]: https://docs.aspect.build/v/cli
[tricorder]: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43322.pdf
[reviewdog]: https://github.com/reviewdog/reviewdog

## Usage

### 1. Warnings in the terminal with `bazel lint`
## Design

Formatting and Linting work a bit differently.

| Formatter | Linter |
| ----------------------------------------------------------------- | ------------------------------------------------------ |
| Only one per language, since they could conflict with each other. | Many per language is fine; results compose. |
| Invariant: program's behavior is never changed. | Suggested fixes may change behavior. |
| Developer has no choices. Always blindly accept result. | Fix may be manual, or select from multiple auto-fixes. |
| Changes must be applied. | Violations can be suppressed. |
| Operates on a single file at a time. | Can require the dependency graph. |
| Can always format just changed files / regions | New violations might be introduced in unchanged files. |
| Fast enough to put in a pre-commit workflow. | Some are slow. |

This leads to some minor differences in how they are used in rules_lint.

## Available tools

| Language | Formatter | Linter(s) |
| ------------------------- | --------------------- | ---------- |
| Python | [Black] | [flake8] |
| Java | [google-java-format] | [pmd] |
| Kotlin | [ktfmt] | |
| JavaScript/TypeScript/TSX | [Prettier] | [ESLint] |
| CSS/HTML | [Prettier] | |
| JSON | [Prettier] | |
| Markdown | [Prettier] | |
| Bash | [prettier-plugin-sh] | |
| SQL | [prettier-plugin-sql] | |
| Starlark (Bazel) | [Buildifier] | |
| Swift | [SwiftFormat] (1) | |
| Go | [gofmt] | |
| Protocol Buffers | [buf] | [buf lint] |
| Terraform | [terraform] fmt | |
| Jsonnet | [jsonnetfmt] | |
| Scala | [scalafmt] | |

[prettier]: https://prettier.io
[google-java-format]: https://github.com/google/google-java-format
[black]: https://pypi.org/project/black/
[flake8]: https://flake8.pycqa.org/en/latest/index.html
[pmd]: https://docs.pmd-code.org/latest/index.html
[buf lint]: https://buf.build/docs/lint/overview
[eslint]: https://eslint.org/
[swiftformat]: https://github.com/nicklockwood/SwiftFormat
[terraform]: https://github.com/hashicorp/terraform
[buf]: https://docs.buf.build/format/usage
[ktfmt]: https://github.com/facebook/ktfmt
[buildifier]: https://github.com/keith/buildifier-prebuilt
[prettier-plugin-sh]: https://github.com/un-ts/prettier
[prettier-plugin-sql]: https://github.com/un-ts/prettier
[gofmt]: https://pkg.go.dev/cmd/gofmt
[jsonnetfmt]: https://github.com/google/go-jsonnet
[scalafmt]: https://scalameta.org/scalafmt

1. Non-hermetic: requires that a swift toolchain is installed on the machine.
See https://github.com/bazelbuild/rules_swift#1-install-swift

To add a linter, please follow the steps in [lint/README.md](./lint/README.md) and then send us a PR.
Thanks!!

This ruleset provides an Aspect CLI plugin, so it can register the missing 'lint' command.
> We'll add documentation on adding formatters as well.

Users just type `bazel lint //path/to:targets`.
## Installation

Reports are then written to the terminal.
rules_lint currently only works with bzlmod under Bazel 6+.
This is because we accumulate dependencies which are difficult to express
in a WORKSPACE file.
We might add support for WORKSPACE in the future.

[![asciicast](https://asciinema.org/a/xQWU1Wc1JINOubeguDDQbBqcq.svg)](https://asciinema.org/a/xQWU1Wc1JINOubeguDDQbBqcq)
Follow instructions from the release you wish to use:
<https://github.com/aspect-build/rules_lint/releases>

### 2. Warnings in the terminal with a wrapper
## Usage

You can use vanilla Bazel rather than Aspect CLI.
### Format

Placing a couple commands in a shell script, Makefile, or similar wrapper.
To format files, run the target you create when you install rules_lint.

See the `example/lint.sh` file as an example.
For example, `bazel run tools:format` will re-format all files:

[![asciicast](https://asciinema.org/a/gUUuQTCGIu85YMl6zz2GJIgD8.svg)](https://asciinema.org/a/gUUuQTCGIu85YMl6zz2GJIgD8)
[![asciicast](https://asciinema.org/a/vGTpzD0obvhILEcSxYAVrlpqT.svg)](https://asciinema.org/a/vGTpzD0obvhILEcSxYAVrlpqT)

### 3. Errors during `bazel build`
See [./docs/formatting.md] for more ways to use the formatter, such as a pre-commit hook or a CI check.

By adding `--aspects_parameters=fail_on_violation=true` to the command-line, we pass a parameter
to our linter aspects that cause them to honor the exit code of the lint tool.
### Lint

This makes the build fail when any lint violations are present.
To lint code, we recommend using the Aspect CLI to get the missing `lint` command.

### 4. Failures during `bazel test`
For example, running `bazel lint //src:all` prints lint warnings to the terminal for all targets in the `//src` package:

We haven't implemented this yet, follow https://github.com/aspect-build/rules_lint/issues/11
[![asciicast](https://asciinema.org/a/xQWU1Wc1JINOubeguDDQbBqcq.svg)](https://asciinema.org/a/xQWU1Wc1JINOubeguDDQbBqcq)

### 5. Code review comments
See [./docs/linting.md] for more ways to use the linter, such as running as a test target, or presenting results as code review comments.

You can wire the reports from bazel-out to a tool like [reviewdog].
### Ignoring files

We're working on a demo with https://aspect.build/workflows that automatically runs `bazel lint` as
part of your CI and reports the results (including suggested fixes) to your GitHub Code Review thread.
The linters only visit files that are part of the Bazel dependency graph (listed as `srcs` to some library target).

## Available linters
The formatter honors the `.gitignore` file.
Otherwise use the affordance provided by the tool, for example `.prettierignore` for files to be ignored by Prettier.

- Protocol Buffers
- `buf lint`: <https://buf.build/docs/lint/overview>
- JavaScript
- `eslint`: https://eslint.org/
- Python
- `flake8`: <https://flake8.pycqa.org/en/latest/index.html>
- Java
- `pmd`: <https://docs.pmd-code.org/latest/index.html>
Sometimes engineers want to ignore a file with a certain extension because the content isn't actually valid syntax for the corresponding language.
For example, you might write a template for YAML and name it `my-template.yaml` even though it needs to have some interpolated values inserted before it's syntactically valid.
We recommend instead fixing the file extension. In this example, `my.yaml.tmpl` or `my-template.yaml_` might be better.

### Adding a linter
### Using with your editor

Please follow the steps in [lint/README.md](./lint/README.md) and then send us a PR.
Thanks!!
We believe that existing editor plugins should just work as-is. They may download or bundle their own
copy of the tools, which can lead to some version skew in lint/format rules.

## Installation
For formatting, we believe it's a waste of time to configure these in the editor, because developers
should just rely on formatting happening when they commit and not care what the code looks like before that point.
But we're not trying to stop anyone, either!

rules_lint currently only works with bzlmod under Bazel 6+.
This is because we accumulate dependencies which are difficult to express
in a WORKSPACE file.
We might add support for WORKSPACE in the future.
You could probably configure the editor to always run the same Bazel command, any time a file is changed.
Instructions to do this are out-of-scope for this repo, particularly since they have to be formulated and updated for so many editors.

Follow instructions from the release you wish to use:
<https://github.com/aspect-build/rules_lint/releases>
### Using a formatter from a BUILD rule

Next, you must declare your linters as Bazel aspects.
Generally, you should just allow code generators to make messy files.
You can exclude them from formatting by changing the file extension,
adding a suppression comment at the top (following the formatter's docs)
or adding to the formatter's ignore file (e.g. `.prettierignore`).

> This is needed because Bazel only allows aspect attributes of type
> `bool`, `int` or `string`.
> We want to follow the documentation from
> https://bazel.build/extending/aspects#aspect_definition_2:
> Aspects are also allowed to have private attributes of types label or label_list. Private label attributes can be used to specify dependencies on tools or libraries that are needed for actions generated by aspects.
However there are some valid cases where you really want to run a formatter as a build step.
You can just reach into the external repository where we've installed them.

We suggest creating a `lint.bzl` file in whatever package contains most of your
custom Bazel configuration.
This `lint.bzl` should contain linter aspect declarations.
See the `docs/` folder for "aspect factory functions" that declare your linters.
For example, to run Prettier:

Finally, reference those linters as `//path/to:lint.bzl%mylinter`
in the lint runner.
```starlark
load("@aspect_rules_format_npm//:prettier/package_json.bzl", prettier = "bin")

If you use the Aspect CLI, then include a block like the following in `.aspect/cli/config.yaml`:
prettier.prettier_binary(name = "prettier")

js_run_binary(
name = "fmt",
srcs = ["raw_file.md"],
args = ["raw_file.md"],
chdir = package_name(),
stdout = "formatted_file.md",
tool = "prettier",
)
```
plugins:
- name: lint-plugin
properties:
lint_aspects:
- //:lint.bzl%eslint
- //:lint.bzl%buf
```

If you don't use Aspect CLI, you can put these in some other wrapper like a shell script that runs the linter aspects over the requested targets.
See the `lint.sh` script in the `example/` folder.
5 changes: 5 additions & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,9 @@ stardoc_with_diff_test(
bzl_library_target = "//lint:pmd",
)

stardoc_with_diff_test(
name = "format",
bzl_library_target = "//format:defs",
)

update_docs(name = "update")
62 changes: 62 additions & 0 deletions docs/format.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading