Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modules/nixos: add cgroups and exporter #1466

Merged
merged 2 commits into from
Nov 21, 2024
Merged

modules/nixos: add cgroups and exporter #1466

merged 2 commits into from
Nov 21, 2024

Conversation

zowoq
Copy link
Contributor

@zowoq zowoq commented Sep 22, 2024

No description provided.

flake.nix Outdated
@@ -16,6 +16,8 @@
buildbot-nix.inputs.nixpkgs.follows = "nixpkgs";
buildbot-nix.inputs.treefmt-nix.follows = "treefmt-nix";
buildbot-nix.url = "github:qowoz/buildbot-nix/more-skipped-build-workers";
cgroup-exporter.inputs.nixpkgs.follows = "nixpkgs";
cgroup-exporter.url = "github:arianvp/cgroup-exporter";
Copy link
Contributor Author

@zowoq zowoq Sep 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need to send a PR upstream for aarch64-linux support.

Done. arianvp/cgroup-exporter@f09e049

flake.nix Outdated
@@ -16,6 +16,8 @@
buildbot-nix.inputs.nixpkgs.follows = "nixpkgs";
buildbot-nix.inputs.treefmt-nix.follows = "treefmt-nix";
buildbot-nix.url = "github:qowoz/buildbot-nix/more-skipped-build-workers";
cgroup-exporter.inputs.nixpkgs.follows = "nixpkgs";
cgroup-exporter.url = "github:arianvp/cgroup-exporter";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I went with the cgroup-exporter suggestion without checking if telegraf supported it.

Looks like the exporter collects almost everything and doesn't currently support any config, telegraf doesn't collect anything by default, requires config.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to collect everything? We mainly do alert monitoring rather than looking at performance metrics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the plugin and some metrics had permission issues (telegraf user?) or were broken, exporter seems to work okay (or least doesn't seem to be missing anything notable).

Probably don't want to collect everything but might be easier to start with the exporter and try switching to the plugin later once we have a better idea of what we do want to collect rather than needing to configure it from the start.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We mainly do alert monitoring rather than looking at performance metrics.

Maybe I should bring back the grafana dashboard? I removed it as no one was using it but maybe it would more useful now? IIRC people on matrix were talking about checking the load on the build boxes, dashboard might be a bit better than people running top.

@zowoq
Copy link
Contributor Author

zowoq commented Nov 21, 2024

I'll keep this as a separate module for now, after it as been used by CI for a while I'll enable it on the community builder and leave the nixpkgs-update host for last when we are confident that it isn't causing any issues with builds.

On the CI builders, I want to see how this works for a couple of weeks before increasing max-jobs again or trying to adjust other settings.

I'll leave the uid-range system feature for another PR, I'm looking at settings extra-platforms, gcc-arch system-features and refactoring the CI / remote builder config.

@zowoq zowoq marked this pull request as ready for review November 21, 2024 01:37
@zowoq zowoq requested review from zimbatm and Mic92 November 21, 2024 01:37
@zowoq zowoq added this pull request to the merge queue Nov 21, 2024
Merged via the queue into master with commit b85d4b0 Nov 21, 2024
3 checks passed
@zowoq zowoq deleted the cgroups branch November 21, 2024 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants