Skip to content

Commit

Permalink
feat(inputs.procstat): add tcp related metrics
Browse files Browse the repository at this point in the history
For a given process, return the number of connections in each state:
SYN_SENT, SYN_RECV, ESTABLISHED, etc.

Also added a new measurement, procstat_tcp, that returns the endpoints
where the process is connected to and where it is listening.
If it is listening in "[::]" or "0.0.0.0", resolve with the local IPs.
  • Loading branch information
adrianlzt committed Aug 22, 2023
1 parent 63e2674 commit 4d1d52c
Show file tree
Hide file tree
Showing 13 changed files with 1,284 additions and 58 deletions.
1 change: 1 addition & 0 deletions docs/LICENSE_OF_DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ following works:
- github.com/eapache/queue [MIT License](https://github.com/eapache/queue/blob/master/LICENSE)
- github.com/eclipse/paho.golang [Eclipse Public License - v 2.0](https://github.com/eclipse/paho.golang/blob/master/LICENSE)
- github.com/eclipse/paho.mqtt.golang [Eclipse Public License - v 2.0](https://github.com/eclipse/paho.mqtt.golang/blob/master/LICENSE)
- github.com/elastic/gosigar [Apache License 2.0](https://github.com/elastic/gosigar/blob/master/LICENSE)
- github.com/emicklei/go-restful [MIT License](https://github.com/emicklei/go-restful/blob/v3/LICENSE)
- github.com/fatih/color [MIT License](https://github.com/fatih/color/blob/master/LICENSE.md)
- github.com/form3tech-oss/jwt-go [MIT License](https://github.com/form3tech-oss/jwt-go/blob/master/LICENSE)
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,8 @@ require (
modernc.org/sqlite v1.24.0
)

require github.com/elastic/gosigar v0.14.2

require (
cloud.google.com/go v0.110.4 // indirect
cloud.google.com/go/compute v1.20.1 // indirect
Expand Down
3 changes: 3 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -494,6 +494,8 @@ github.com/eclipse/paho.golang v0.11.0 h1:6Avu5dkkCfcB61/y1vx+XrPQ0oAl4TPYtY0uw3
github.com/eclipse/paho.golang v0.11.0/go.mod h1:rhrV37IEwauUyx8FHrvmXOKo+QRKng5ncoN1vJiJMcs=
github.com/eclipse/paho.mqtt.golang v1.4.2 h1:66wOzfUHSSI1zamx7jR6yMEI5EuHnT1G6rNA5PM12m4=
github.com/eclipse/paho.mqtt.golang v1.4.2/go.mod h1:JGt0RsEwEX+Xa/agj90YJ9d9DH2b7upDZMK9HRbFvCA=
github.com/elastic/gosigar v0.14.2 h1:Dg80n8cr90OZ7x+bAax/QjoW/XqTI11RmA79ZwIm9/4=
github.com/elastic/gosigar v0.14.2/go.mod h1:iXRIGg2tLnu7LBdpqzyQfGDEidKCfWcCMS0WKyPWoMs=
github.com/emicklei/go-restful/v3 v3.10.1 h1:rc42Y5YTp7Am7CS630D7JmhRjq4UlEUuEKfrDac4bSQ=
github.com/emicklei/go-restful/v3 v3.10.1/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
Expand Down Expand Up @@ -1779,6 +1781,7 @@ golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20180810173357-98c5dad5d1a0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
Expand Down
44 changes: 44 additions & 0 deletions plugins/inputs/procstat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
## - "mem": to enable collection of memory percentage used
## - "mem_percent": to enable collection of procs' limits
## - "limits": to enable collection of procs' limits
## - "tcp_stats": tcp_* and upd_socket metrics
## - "connections_endpoints": new metric procstat_tcp with connections and listeners endpoints
## Default value:
# metrics_include = [
# "threads",
Expand Down Expand Up @@ -198,11 +200,53 @@ the `win_perf_counters` input plugin as a more mature alternative.
- running (int)
- result_code (int, success = 0, lookup_error = 1)

If ``tcp_stats`` enabled, added fields:

- procstat
- fields:
- tcp_close (int)
- tcp_close_wait (int)
- tcp_closing (int)
- tcp_established (int)
- tcp_fin_wait1 (int)
- tcp_fin_wait2 (int)
- tcp_last_ack (int)
- tcp_listen (int)
- tcp_none (int)
- tcp_syn_recv (int)
- tcp_syn_sent (int)

If ``connections_endpoints`` enabled, added fields:

- procstat_tcp
- tags:
- pid (when `pid_tag` is true)
- cmdline (when 'cmdline_tag' is true)
- process_name
- pidfile (when defined)
- exe (when defined)
- pattern (when defined)
- user (when selected)
- systemd_unit (when defined)
- cgroup (when defined)
- fields:
- conn (string)
- listen (string)

To gather connection info, if Telegraf is not run as root, it needs the
following capabilities:

```text
sudo setcap "CAP_DAC_READ_SEARCH,CAP_SYS_PTRACE+ep" telegraf
```

*NOTE: Resource limit > 2147483647 will be reported as 2147483647.*

## Example Output

```text
procstat_lookup,host=prash-laptop,pattern=influxd,pid_finder=pgrep,result=success pid_count=1i,running=1i,result_code=0i 1582089700000000000
procstat,host=prash-laptop,pattern=influxd,process_name=influxd,user=root involuntary_context_switches=151496i,child_minor_faults=1061i,child_major_faults=8i,cpu_time_user=2564.81,cpu_time_idle=0,cpu_time_irq=0,cpu_time_guest=0,pid=32025i,major_faults=8609i,created_at=1580107536000000000i,voluntary_context_switches=1058996i,cpu_time_system=616.98,cpu_time_steal=0,cpu_time_guest_nice=0,memory_swap=0i,memory_locked=0i,memory_usage=1.7797634601593018,num_threads=18i,cpu_time_nice=0,cpu_time_iowait=0,cpu_time_soft_irq=0,memory_rss=148643840i,memory_vms=1435688960i,memory_data=0i,memory_stack=0i,minor_faults=1856550i 1582089700000000000
procstat,host=laptop,pattern=httpd,process_name=httpd,user=root child_major_faults=0i,child_minor_faults=70i,cpu_time=0i,cpu_time_guest=0,cpu_time_guest_nice=0,cpu_time_idle=0,cpu_time_iowait=0,cpu_time_irq=0,cpu_time_nice=0,cpu_time_soft_irq=0,cpu_time_steal=0,cpu_time_system=0.01,cpu_time_user=0.02,cpu_usage=0,created_at=1611738400000000000i,involuntary_context_switches=15i,listen=1i,major_faults=0i,memory_data=999424i,memory_locked=0i,memory_rss=4677632i,memory_stack=135168i,memory_swap=0i,memory_usage=0.013990458101034164,memory_vms=6078464i,minor_faults=1636i,nice_priority=20i,num_fds=8i,num_threads=1i,pid=1738811i,read_bytes=0i,read_count=4397i,realtime_priority=0i,rlimit_cpu_time_hard=2147483647i,rlimit_cpu_time_soft=2147483647i,rlimit_file_locks_hard=2147483647i,rlimit_file_locks_soft=2147483647i,rlimit_memory_data_hard=2147483647i,rlimit_memory_data_soft=2147483647i,rlimit_memory_locked_hard=65536i,rlimit_memory_locked_soft=65536i,rlimit_memory_rss_hard=2147483647i,rlimit_memory_rss_soft=2147483647i,rlimit_memory_stack_hard=2147483647i,rlimit_memory_stack_soft=8388608i,rlimit_memory_vms_hard=2147483647i,rlimit_memory_vms_soft=2147483647i,rlimit_nice_priority_hard=0i,rlimit_nice_priority_soft=0i,rlimit_num_fds_hard=1048576i,rlimit_num_fds_soft=1048576i,rlimit_realtime_priority_hard=0i,rlimit_realtime_priority_soft=0i,rlimit_signals_pending_hard=127473i,rlimit_signals_pending_soft=127473i,signals_pending=0i,tcp_close=0i,tcp_close_wait=0i,tcp_closing=0i,tcp_established=0i,tcp_fin_wait1=0i,tcp_fin_wait2=0i,tcp_last_ack=0i,tcp_listen=1i,tcp_syn_recv=0i,tcp_syn_sent=0i,voluntary_context_switches=169i,write_bytes=53248i,write_count=10i 1611738522000000000
procstat_tcp,host=laptop,pattern=httpd,process_name=httpd,user=root conn="",listen="192.168.1.35:80,192.168.1.48:80,[da01:beef:234:3830:aeda:f001:a00c:0091]:80,[aa01:beef:234:3830:e8e:0000:000a:6b0f]:80" 1611738522000000000
```
36 changes: 36 additions & 0 deletions plugins/inputs/procstat/connections.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package procstat

import (
"fmt"
"net"
)

const (
// dockerMACPrefix https://macaddress.io/faq/how-to-recognise-a-docker-container-by-its-mac-address
dockerMACPrefix = "02:42"
//nolint:lll // avoid splitting the link
// virtualBoxMACPrefix https://github.com/mdaniel/virtualbox-org-svn-vbox-trunk/blob/2d259f948bc352ee400f9fd41c4a08710cd9138a/src/VBox/HostDrivers/VBoxNetAdp/VBoxNetAdp.c#L93
virtualBoxMACPrefix = "0a:00:27"
// hardwareAddrLength is the number of bytes of a MAC address
hardwareAddrLength = 6
)

// errPIDNotFound is the error generated when the pid does not have network info
var errPIDNotFound = fmt.Errorf("pid not found")

// inodeInfo represents information of a proc associated with an inode
type inodeInfo struct {
pid uint32
}

// networkInfo implements networkInfo using the netlink calls and parsing /proc to map sockets to PIDs
type networkInfo struct {
// tcp contains the connection info for each pid
tcp map[uint32][]connInfo
// listenPorts is a map with the listen ports in the host, used to ignore inbound connections
listenPorts map[uint32]interface{}
// publicIPs list of IPs considered "public" (used to connect to other hosts)
publicIPs []net.IP
// privateIPs list of IPs considered "private" (loopback, virtual interfaces, point2point, etc)
privateIPs []net.IP
}
35 changes: 35 additions & 0 deletions plugins/inputs/procstat/connections_fallback.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
//go:build !linux
// +build !linux

package procstat

import (
"fmt"
"net"
)

type connInfo struct{}

func (n *NetworkInfo) IsAListenPort(port uint32) bool {
return false
}

func (n *NetworkInfo) Fetch() error {
return nil
}

func (n *NetworkInfo) GetConnectionsByPid(pid uint32) (conn []connInfo, err error) {
return conn, fmt.Errorf("platform not supported")
}

func (n *NetworkInfo) GetPublicIPs() []net.IP {
return []net.IP{}
}

func (n *NetworkInfo) GetPrivateIPs() []net.IP {
return []net.IP{}
}

func (n *NetworkInfo) IsPidListeningInAddr(pid uint32, ip net.IP, port uint32) bool {
return false
}
Loading

0 comments on commit 4d1d52c

Please sign in to comment.