SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts #443

msherif1234 · 2024-10-29T12:14:10Z

Description

prepare ebpf agent to use bpfman to manage netobserv ebpf programs

Dependencies

netobserv/network-observability-operator#829

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

openshift-ci-robot · 2024-10-29T12:14:16Z

openshift-ci · 2024-10-29T12:14:16Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from msherif1234. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2024-10-29T12:27:37Z

Codecov Report

Attention: Patch coverage is 1.42857% with 207 lines in your changes missing coverage. Please review.

Project coverage is 29.23%. Comparing base (626d526) to head (9bd714a).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/tracer/tracer.go	0.00%	189 Missing ⚠️
pkg/agent/packets_agent.go	0.00%	13 Missing ⚠️
pkg/agent/agent.go	37.50%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #443      +/-   ##
==========================================
- Coverage   29.71%   29.23%   -0.48%     
==========================================
  Files          50       50              
  Lines        4877     4959      +82     
==========================================
+ Hits         1449     1450       +1     
- Misses       3322     3403      +81     
  Partials      106      106

Flag	Coverage Δ
unittests	`29.23% <1.42%> (-0.48%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
pkg/agent/config.go	`8.33% <ø> (ø)`
pkg/agent/agent.go	`33.74% <37.50%> (-0.01%)`	⬇️
pkg/agent/packets_agent.go	`0.00% <0.00%> (ø)`
pkg/tracer/tracer.go	`0.00% <0.00%> (ø)`

openshift-ci-robot · 2024-10-29T16:47:40Z

msherif1234 · 2024-10-30T19:43:24Z

/ok-to-test

github-actions · 2024-10-30T19:45:30Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:af27118

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=af27118 make set-agent-image

msherif1234 · 2024-10-30T22:07:33Z

/ok-to-test

github-actions · 2024-10-30T22:09:33Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:dc5079c

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=dc5079c make set-agent-image

msherif1234 · 2024-10-31T11:43:42Z

/ok-to-test

github-actions · 2024-10-31T11:45:50Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:b04ad60

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=b04ad60 make set-agent-image

msherif1234 · 2024-10-31T14:20:03Z

/ok-to-test

github-actions · 2024-10-31T14:22:02Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:4b53105

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=4b53105 make set-agent-image

github-actions · 2024-11-06T14:49:27Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:469ccb5

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=469ccb5 make set-agent-image

msherif1234 · 2024-11-11T15:49:56Z

/ok-to-test

github-actions · 2024-11-11T15:52:08Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:194fdb1

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=194fdb1 make set-agent-image

msherif1234 · 2024-11-11T15:54:03Z

/ok-to-test

github-actions · 2024-11-11T15:56:15Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:4ab660f

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=4ab660f make set-agent-image

msherif1234 · 2024-11-11T17:16:15Z

/ok-to-test

github-actions · 2024-11-11T17:18:25Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:3ca4ab3

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=3ca4ab3 make set-agent-image

stleerh · 2024-11-12T00:32:01Z

e2e/cluster/base/04-agent.yml

@@ -40,4 +40,4 @@ spec:
        - name: bpf-kernel-debug
          hostPath:
            path: /sys/kernel/debug
-            type: Directory
+            type: Directory


Was there a reason to change this file and remove the linefeed char?

no reason I was just doing some changes there but reverted once found a way to work in none privileged mode will revert the whitespace changes

stleerh · 2024-11-12T00:32:11Z

e2e/kafka/manifests/30-agent.yml

@@ -38,4 +38,4 @@ spec:
        - name: bpf-kernel-debug
          hostPath:
            path: /sys/kernel/debug
-            type: Directory
+            type: Directory


stleerh · 2024-11-12T00:49:59Z

hack/build-bytecode-images-multi.sh

+\"kfree_skb\":\"tracepoint\",\
+\"rh_network_events_monitoring\":\"kprobe\"\
+}"
+


This could be a lot more readable if you remove all backslashes and use a single quote around the entire string like this:

PROGRAMS='{ "tcx_ingress_flow_parse":"tcx", "tcx_egress_flow_parse":"tcx", "tc_ingress_flow_parse":"tc", "tc_egress_flow_parse":"tc", "tcx_ingress_pca_parse":"tcx", "tcx_egress_pca_parse":"tcx", "tc_ingress_pca_parse":"tc", "tc_egress_pca_parse":"tc", "tcp_rcv_fentry":"fentry", "tcp_rcv_kprobe":"kprobe", "kfree_skb":"tracepoint", "rh_network_events_monitoring":"kprobe" }'

Note: Each new line does add a space and the only way to get rid of that is to put it all on the same line, which is undesirable. Most likely, the spaces won't matter if you quote the variable like "$PROGRAM".

stleerh · 2024-11-12T00:51:27Z

hack/build-bytecode-images-multi.sh

+\"dns_flows\":\"hash\",\
+\"global_counters\":\"per_cpu_array\",\
+\"filter_map\":\"lpm_trie\"\
+}"


Similar to above.

MAPS='{ "direct_flows":"ringbuf", "aggregated_flows":"per_cpu_hash", "packets_record":"perf_event_array", "dns_flows":"hash", "global_counters":"per_cpu_array", "filter_map":"lpm_trie" }'

stleerh · 2024-11-12T00:59:37Z

pkg/tracer/tracer.go

+	if !cfg.UseEbpfManager {
+		if err := rlimit.RemoveMemlock(); err != nil {
+			log.WithError(err).
+				Warn("can't remove mem lock. The agent could not be able to start eBPF programs")


I know this was existing code. I think the message is suppose to say:
The agent might not be able to start the eBPF programs.

The agent will not be able to start the eBPF programs. WDYT?

Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>

msherif1234 · 2024-11-12T12:42:45Z

/ok-to-test

github-actions · 2024-11-12T12:45:14Z

New image:
quay.io/netobserv/netobserv-ebpf-agent:cb970b1

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=cb970b1 make set-agent-image

openshift-merge-robot · 2024-12-04T09:03:55Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci bot added the do-not-merge/work-in-progress label Oct 29, 2024

openshift-ci-robot added the jira/valid-reference label Oct 29, 2024

msherif1234 force-pushed the bpfman_intg branch 2 times, most recently from eed3217 to 7b5953d Compare October 30, 2024 11:21

msherif1234 marked this pull request as draft October 30, 2024 12:58

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 30, 2024

msherif1234 force-pushed the bpfman_intg branch from 39beda6 to ad1d559 Compare October 30, 2024 21:40

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 30, 2024

msherif1234 force-pushed the bpfman_intg branch from ad1d559 to 5fe4499 Compare October 30, 2024 21:50

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 30, 2024

msherif1234 force-pushed the bpfman_intg branch from 5fe4499 to 3ff24fb Compare October 31, 2024 11:21

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 31, 2024

msherif1234 force-pushed the bpfman_intg branch from 3ff24fb to d62af9a Compare October 31, 2024 11:43

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 31, 2024

msherif1234 force-pushed the bpfman_intg branch from d62af9a to 194fef5 Compare October 31, 2024 12:23

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 31, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Oct 31, 2024

msherif1234 force-pushed the bpfman_intg branch from 194fef5 to 98d2749 Compare October 31, 2024 16:16

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 6, 2024

msherif1234 force-pushed the bpfman_intg branch from 93faa21 to 92c66f1 Compare November 11, 2024 15:49

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

msherif1234 force-pushed the bpfman_intg branch from 92c66f1 to 1d7134f Compare November 11, 2024 15:52

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

msherif1234 marked this pull request as ready for review November 11, 2024 16:11

msherif1234 changed the title ~~WIP: SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts~~ SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts Nov 11, 2024

openshift-ci bot removed the do-not-merge/work-in-progress label Nov 11, 2024

msherif1234 force-pushed the bpfman_intg branch from 1d7134f to 1b6d926 Compare November 11, 2024 17:12

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

msherif1234 force-pushed the bpfman_intg branch from 1b6d926 to 83ebdfb Compare November 11, 2024 17:15

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 11, 2024

stleerh reviewed Nov 12, 2024

View reviewed changes

SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts

9bd714a

Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>

msherif1234 force-pushed the bpfman_intg branch from 83ebdfb to 9bd714a Compare November 12, 2024 12:01

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 12, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 12, 2024

openshift-merge-robot added the needs-rebase label Dec 4, 2024

SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts #443

Are you sure you want to change the base?

SDN-5393: eBPF agent intg with bpfman for ebpf progs life cycle mgmts #443

Conversation

msherif1234 commented Oct 29, 2024 • edited Loading

Description

Dependencies

Checklist

openshift-ci-robot commented Oct 29, 2024 • edited by openshift-ci bot Loading

Description

Dependencies

Checklist

openshift-ci bot commented Oct 29, 2024

codecov bot commented Oct 29, 2024 • edited Loading

Codecov Report

openshift-ci-robot commented Oct 29, 2024 • edited by openshift-ci bot Loading

Description

Dependencies

Checklist

msherif1234 commented Oct 30, 2024

github-actions bot commented Oct 30, 2024

msherif1234 commented Oct 30, 2024

github-actions bot commented Oct 30, 2024

msherif1234 commented Oct 31, 2024

github-actions bot commented Oct 31, 2024

msherif1234 commented Oct 31, 2024

github-actions bot commented Oct 31, 2024

github-actions bot commented Nov 6, 2024

msherif1234 commented Nov 11, 2024

github-actions bot commented Nov 11, 2024

msherif1234 commented Nov 11, 2024

github-actions bot commented Nov 11, 2024

msherif1234 commented Nov 11, 2024

github-actions bot commented Nov 11, 2024

stleerh Nov 12, 2024

Choose a reason for hiding this comment

msherif1234 Nov 12, 2024

Choose a reason for hiding this comment

stleerh Nov 12, 2024

Choose a reason for hiding this comment

msherif1234 Nov 12, 2024

Choose a reason for hiding this comment

stleerh Nov 12, 2024 • edited Loading

Choose a reason for hiding this comment

stleerh Nov 12, 2024

Choose a reason for hiding this comment

stleerh Nov 12, 2024

Choose a reason for hiding this comment

msherif1234 Nov 12, 2024

Choose a reason for hiding this comment

msherif1234 commented Nov 12, 2024

github-actions bot commented Nov 12, 2024

openshift-merge-robot commented Dec 4, 2024

msherif1234 commented Oct 29, 2024 •

edited

Loading

openshift-ci-robot commented Oct 29, 2024 •

edited by openshift-ci bot

Loading

codecov bot commented Oct 29, 2024 •

edited

Loading

openshift-ci-robot commented Oct 29, 2024 •

edited by openshift-ci bot

Loading

stleerh Nov 12, 2024 •

edited

Loading