"authentication failed: missing or improperly formatted Authorization header: expected 'Authorization: Bearer secret_token' or 'Authorization: ApiKey base64(API key ID:API key)

[khteh]

Describe the bug

I have APM server and fleet 8.9.0 running in my k8s cluster and seen the following error today:

[my-apm-server-7f75cffcf7-bmdm5 apm-server] {"log.level":"error","@timestamp":"2023-08-08T05:35:10.283Z","log.logger":"request","log.origin":{"file.name":"middleware/log_middleware.go","file.line":58},"message":"authentication failed: missing or improperly formatted Authorization header: expected 'Authorization: Bearer secret_token' or 'Authorization: ApiKey base64(API key ID:API key)'","service.name":"apm-server","url.original":"/intake/v2/events","http.request.method":"POST","user_agent.original":"apm-agent-nodejs/3.45.0 (buyerportal-apm 2.0.0)","source.address":"10.0.93.34","http.request.id":"e43db8cb-93b1-46e1-9851-522268ec10d0","event.duration":89766,"http.response.status_code":401,"error.message":"authentication failed: missing or improperly formatted Authorization header: expected 'Authorization: Bearer secret_token' or 'Authorization: ApiKey base64(API key ID:API key)'","ecs.version":"1.6.0"}

To Reproduce

Steps to reproduce the behavior:

1. Use this config '...'
2. Then call '....'
3. Then do '....'
4. See error

Expected behavior

Environment (please complete the following information)

• OS: [e.g. Linux] Ubuntu 23.04
• Node.js version: v20.3.1
• APM Server version: 8.9.0
• Agent version: 3.48.0

How are you starting the agent? (please tick one of the boxes)

• Calling agent.start() directly (e.g. require('elastic-apm-node').start(...))
• Requiring elastic-apm-node/start from within the source code
• Starting node with -r elastic-apm-node/start

Additional context

Agent config options:

Click to expand

var apm = require('elastic-apm-node').start({
  // Override service name from package.json
  // Allowed characters: a-z, A-Z, 0-9, -, _, and space
  serviceName: 'my-apm-server',
  secretToken: '',
  // Set custom APM Server URL (default: http://127.0.0.1:8200)
  serverUrl: 'https://my-apm-http:8200',
  // Only activate the agent if it's running in production
  active: process.env.NODE_ENV === 'production',
  verifyServerCert: false,
  ignoreUrls: [
    /^\/health\//i,
    /^\/[._]+next\//i,
    /^\/robots.txt/
  ],  
  logLevel: 'trace'
})

package.json dependencies:

Click to expand

replace this line with your dependencies section from package.json

[david-luna]

Hi @khteh

I guess this issue comes from the conversation in discuss.elastic.com. If I understood right you managed to configure ECK with the assistance of my colleagues and now the final step is to have the nodejs agent properly configured which is why you created #3598 .

So let's focus on #3598 for now and review this one after the former is being fixed. Is that okay?

[khteh]

It is fixed. I see the data in kibana.

[david-luna]

That's good news. I'll proceed to close this issue then