Webhooks and Nginx

[clriis]

I have previously been using webhooks on onedrive with Lighttpd as a proxy and has been working brilliantly.
Now I'm trying with Nginx and it's giving me a headache.
I have setup Nginx with the config like documented, using this snippet:

server {
	listen 80;
	location /webhooks/onedrive {
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Original-Request-URI $request_uri;
		proxy_read_timeout 300s;
		proxy_connect_timeout 75s;
		proxy_buffering off;
		proxy_http_version 1.1;
		proxy_pass http://127.0.0.1:8888;
	}
}

The server starts OK and is HTTPS certified.

But I get this error:

ERROR: Microsoft OneDrive API returned an error with the following message:
  Error Message:    HTTP request returned status code 400 ()
  Error Reason:     Subscription validation request failed. Notification endpoint must respond with 200 OK to validation request.

When I run onedrive as a service (systemctl start onedrive@.service) I would expect the port 127.0.0.1:8888 to be listening but I get nothing when running lsof -i :8888 and also I cannot telnet it.
Is that not a correct assumption?

[abraunegg]

@clriis
A couple of questions:

Error Message: HTTP request returned status code 400 ()

What client 'version' are you using?

It is rare for this feature to not work ... but this output line specifically has me asking some questions. Your answer to this question will be rather revealing.

When I run onedrive as a service (systemctl start onedrive@.service) I would expect the port 127.0.0.1:8888 to be listening but I get nothing when running lsof -i :8888 and also I cannot telnet it.
Is that not a correct assumption?

This issue sounds similar to #2447 where the fix was:

location /path/to/webhook {
    proxy_http_version 1.1;
    proxy_pass http://<ip>:8888;
}

So what should 'proxy_pass' be configured with? This directive specifies the proxied server to which requests should be forwarded. In this case, requests matching /webhooks/onedrive will be forwarded to http://127.0.0.1:8888.

The IP address used in the proxy_pass directive should be the IP address of the backend server that will handle the proxied requests.

In documentation configuration, 127.0.0.1 is the loopback address, meaning the backend server (onedrive) is running on the same machine as the nginx server.

Does your 'onedrive' application configuration match your nginx configuration? Please read: https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md#webhook_listening_host-and-webhook_listening_port

Are there any local / host level firewall configuration getting in the way?

[clriis]

Onedrive client version: onedrive v2.4.25-1+np2+4.1
Nginx and Onedrive is on the same box, a headless RPi 4 with 'bookworm'.
UFW is disabled. There is some firewall in my router but ports 80 and 443 are forwarded to the box.
As you can see the config I am using is copied from the documentation, which includes:

location /webhooks/onedrive {
                proxy_http_version 1.1;
		proxy_pass http://127.0.0.1:8888;

[abraunegg]

OK ..

So .. right now I do not have an environment to 100% test anything related to webhooks, however when I enable webhooks using the default configuration in the development context I am working in yields the following:

ERROR: Cannot create or renew subscription.

ERROR: Microsoft OneDrive API returned an error with the following message:
  Error Message:    HTTP request returned status code 400 ()
  Error Reason:     NotificationUrl scheme='http' is not supported.
  Error Code:       ValidationError
  Error Timestamp:  2024-06-21T21:01:12
  API Request ID:   65a5776b-7346-49ad-9457-4204b2e5ace6
  Calling Function: logSubscriptionError()

netstat -patn shows the port available:

...
tcp        0      0 0.0.0.0:8888            0.0.0.0:*               LISTEN      130104/./onedrive   
...

So .. my thoughts here are that:

webhook_public_url = "http://my.public.ip.address:8888/"

needs to be

webhook_public_url = "https://my.public.ip.address:8888/"

as potentially Microsoft have made a change to the API so that the webhook URL must be HTTPS and no longer a HTTP URL.

Now you mentioned everything is correctly setup ... so things to check:

• Dont use systemd for the moment - use --monitor --verbose manually till you sort this out
• Double check your 'webhook_public_url' - is it specifying the correct https URL ?
• Externally, can you hit the URL that you specified .. does it respond ?
• When running you should see the client operational using netstat on the port you have selected .. if not .. then thats the first step to work out - potentially stop any client from running, maybe even reboot your system. Is there another process trying to operate on that port ?

[clriis]

Right.

How can I test if the onedrive client is listening on port 8888, when it EXITS after the MS error message? (I do believe it does listen though, because when started with system.d it 'survives' for a little while and I have been able to identify it with netstat).

Anyway, I got a bit further, and the Error msg I get now is 401:

Configuration file successfully loaded
Configuring Global Azure AD Endpoints
Initializing the Synchronization Engine ...
Initializing monitor ...
OneDrive monitor interval (seconds): 300
Initializing subscription for updates ...
Webhook: handled validation request

ERROR: Microsoft OneDrive API returned an error with the following message:
  Error Message:    HTTP request returned status code 401 ()
  Error Reason:     Operation: Create; Exception: [Status Code: Unauthorized; Reason: Unauthenticated]
  Error Timestamp:  2024-06-22T13:49:20
  API Request ID:   1101a948-ceeb-4de9-b78e-62dd55088a9d
ERROR: Unable to initialize subscriptions for updates. Please fix this issue.

I was "upgraded" to the 401 message when I moved the proxy snippet to the SSL (443) section in the Nginx config (I have included the full Nginx configuration at the bottom of this message)

In the onedrive client config file, my url line is:
webhook_public_url = "https://my.public.ip.address/webhooks/onedrive"
If I remove the folders and add the port, I get error 400 from Microsoft.

My URL is reachable from outside through HTTPS and is responding

There is no other app/service using port 8888

I have renewed the OneDrive token with Microsoft

I can reach the ^/webhook/onedrive if I set up a temporary python http daemon to listen on 8888:
curl -I https://my.public.ip.address/webhooks/onedrive

---

My Nginx config:

server {
    listen 80;
    server_name my.public.ip.address;

    # Redirect HTTP to HTTPS
    location / {
        return 301 https://$host$request_uri;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/html;
        try_files $uri =404;
    }
}

server {
    listen 443 ssl;
    server_name my.public.ip.address;

    ssl_certificate /etc/letsencrypt/live/my.public.ip.address/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/my.public.ip.address/privkey.pem;

    location / {
        root /var/www/html;
        index index.html index.htm;
    }

    # Proxy for /webhooks/onedrive
    location /webhooks/onedrive {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Original-Request-URI $request_uri;
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
        proxy_buffering off;
        proxy_http_version 1.1;
        proxy_pass http://127.0.0.1:8888;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/html;
        try_files $uri =404;
    }
}

[abraunegg]

@clriis

How can I test if the onedrive client is listening on port 8888, when it EXITS after the MS error message?

Whilst that is the behavior of v2.4.x client for 'webhooks' , it is not the behavior of the v2.5.x client. Unfortunately that behavior of the v2.4.x client will not be changed. All real development/change stopped on v2.4.x 18 months ago with all effort going towards v2.5.x.

My suggestion here is to look at compiling manually v2.5.0 RC2 or #2728 which is my development branch for what will become RC3, to help you diagnose further:

• OneDrive Client Version v2.5.0-rc2 is now available for testing #2710
• 2724+updated logging test - For testing purposes only #2728

Once you compile one of the v2.5.x client versions, the 'webhook listener' remains operational with the port running .. to allow you to debug further.

Once compiled, my suggestion is to also run the client manually (ensuring any systemd service is stopped / disabled) using: ./onedrive -m -v --debug-https as this will show you what the client is sending / receiving at the HTTP layer without going into debug verbosity. This verbosity will help you track down where in your configuration things may be failing.

Dont install v2.5.x via make install unless you want to post doing other tests to meet your acceptance criteria that the new client works in the manner you are expecting. If you do go down that path, you will need to manually uninstall any v2.4.x client package as well. If you do not install the v2.5.x client, and leave the v2.4.x client package intact - once things are working 'webhook' wise with v2.5.x , they should also operate correctly with v2.4.x without issue.

From your firewall, you need to port forward your external IP TCP/80 and TCP 443 - which you said you already have done and have working - just double check this that things are being correctly reverse NAT'd at the firewall and that all the correct connection tracking is in place.

When your nginx configuration (from above) is added to how things should be setup is visualised, this is diagram should reflect this:

Please evaluate your environment in reference to this

[abraunegg]

@clriis
Circling back on this, please can you review the updated documentation for v2.5.0 RC3.

https://github.com/abraunegg/onedrive/blob/onedrive-v2.5.0-release-candidate-3/docs/webhooks.md