Agent Gets Unhealthy on updating endpoint policy

[ghost]

Kibana/Elasticsearch Stack version
Version: 8.16
Build: 77295
Commit: 812401bd0fbcec6f00763b5c048b2fdeea8319bb

Preconditions

• Test-Signing need to be enabled on windows endpoint
• Download the artifact from here https://snapshots.elastic.co/8.16.0-9e5932e8/downloads/beats/elastic-agent/elastic-agent-8.16.0-SNAPSHOT-windows-x86_64.zip

Steps

• Install Elastic Agent with Elastic Defend integration on windows endpoint
• Navigate to Security > Endpoint Page
• Do any change in policy configuration
• Observed after above steps agent and endpoint both went into unhealthy state.

Additional Details

• System integration error system_error.zip

• Elastic Defend error

• Agent Diagnostics elastic-agent-diagnostics-2024-08-13T05-40-11Z-00.zip

Expected Result

• Agent should remain on health status on 8.16

Screen-Shot

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

Secondary review for this ticket is Done.

[pierrehilbert]

Hello,
Is the agent running as unprivileged in your tests?

[amolnater-qasource]

Hi @pierrehilbert

Thank you for looking into this.

No, the agent is not running unprivileged. The issue is specifically observed on Windows agent even on removing Elastic Defend integration.

UPDATE:
The issue is reproduced on Linux on updating Endpoint from Prevention to Detection.

Linux agent logs:
elastic-agent-diagnostics-2024-08-13T07-28-30Z-00.zip

Please let us know if anything else is required from our end.

Thanks!

[michalpristas]

it's complaining about PID being 0, this should not happen and can mean process exited and structure was updated with default value for int(0)(probably not)

[ycombinator]

This looks like a duplicate of elastic/beats#40542.

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.16.0 BC1 kibana cloud environment and we had below observations:

Observations:

• System Integration errors are fixed for Windows agent.
• Further, on making any changes to the Elastic Defend integration, agent gets unhealthy. The issue for this is also reported here: [Fleet]: On making changes to elastic defend Endpoints get unhealthy. #5754

Logs:
elastic-agent-diagnostics-2024-10-21T11-46-04Z-00.zip

Screenshot:

Build details:
VERSION: 8.16.0
BUILD: 79314
COMMIT: 5575428dd3aef69366cddb4ccf07a2a26d30ce48
Artifact Link: https://staging.elastic.co/8.16.0-e8d5928a/downloads/beats/elastic-agent/elastic-agent-8.16.0-windows-x86_64.zip

Hence, we are marking this issue as QA:Validated.

Thanks!