dep updates/fix updating streams/close #965/env option to disable mus…

…t-taple

Signed-off-by: Zoey <zoey@z0ey.de>

close #965 by merging https://github.com/NginxProxyManager/nginx-proxy-manager/pull/4166/files and adding multi lang support

Co-Authored-By: Remco Kersten <55450744+kerstenremco@users.noreply.github.com>

Dockerfile

@@ -71,15 +71,12 @@ RUN apk upgrade --no-cache -a && \
     sed -i "s|APPSEC_PROCESS_TIMEOUT=.*|APPSEC_PROCESS_TIMEOUT=10000|g" /src/crowdsec-nginx-bouncer/lua-mod/config_example.conf
 
 
-FROM zoeyvid/nginx-quic:351-python
+FROM zoeyvid/nginx-quic:352-python
 SHELL ["/bin/ash", "-eo", "pipefail", "-c"]
 
-# until https://github.com/certbot/certbot/issues/9967 is closed
-ENV PYTHONWARNINGS=ignore
-
 COPY                                  rootfs                                                     /
-COPY --from=zoeyvid/certbot-docker:60 /usr/local                                                 /usr/local
-COPY --from=zoeyvid/curl-quic:423     /usr/local/bin/curl                                        /usr/local/bin/curl
+COPY --from=zoeyvid/certbot-docker:64 /usr/local                                                 /usr/local
+COPY --from=zoeyvid/curl-quic:426     /usr/local/bin/curl                                        /usr/local/bin/curl
 
 COPY --from=strip-backend             /app                                                       /app
 COPY --from=frontend                  /app/dist                                                  /html/frontend
@@ -125,6 +122,8 @@ ENV NODE_ENV=production \
     DB_SQLITE_FILE=/data/etc/npm/database.sqlite
 
 ENV ACME_SERVER="https://acme-v02.api.letsencrypt.org/directory" \
+    ACME_MUST_STAPLE=true \
+    ACME_SERVER_TLS_VERIFY=true \
     PUID=0 \
     PGID=0 \
     NIBEP=48693 \

README.md

@@ -4,36 +4,23 @@ This project comes as a pre-built docker image that enables you to easily forwar
 running at home or otherwise, including free TLS, without having to know too much about Nginx or Certbot.
 
 - [Quick Setup](#quick-setup)
-<!---
-- [Screenshots](https://nginxproxymanager.com/screenshots)
---->
-
-<!---
-**Note: To fix [this issue](https://github.com/SpiderLabs/ModSecurity/issues/2848), instead of running `nginx -s reload`, this fork stops nginx and starts it again. This can result in a 502 error when you update your hosts. See https://github.com/ZoeyVid/NPMplus/issues/296 and https://github.com/ZoeyVid/NPMplus/issues/283.** <br>
---->
-**Note: Reloading the NPMplus UI can cause a 502 error. See https://github.com/ZoeyVid/NPMplus/issues/241.** <br>
+
 **Note: NO armv7, route53 and aws cloudfront ip ranges support.** <br>
-**Note: add `net.ipv4.ip_unprivileged_port_start=0` at the end of `/etc/sysctl.conf` to support PUID/PGID in network mode host.** <br>
-**Note: If you don't use network mode host, which I don't recommend, don't forget to expose port 443 on tcp AND udp (http3/quic needs udp).** <br>
-**Note: If you don't use network mode host, which I don't recommend, don't forget to enable IPv6 in Docker, see [here](https://github.com/nextcloud/all-in-one/blob/main/docker-ipv6-support.md), you only need to follow step one and two before deploying NPMplus!** <br>
-**Note: Don't forget to open Port 80 (tcp) and 443 (tcp AND udp, http3/quic needs udp) in your firewall (because of network mode host, you also need to open this ports in ufw, if you use ufw).** <br>
-**Note: ModSecurity overblocking (403 Error)? Please see `/opt/npm/etc/modsecurity`, if you also use CRS please see [here](https://coreruleset.org/docs/concepts/false_positives_tuning).** <br>
 **Note: Other Databases like MariaDB may work, but are unsupported.** <br>
+**Note: watchtower does NOT update NPMplus, you need to do it yourself (it will only pull the image, but not update the container itself).** <br>
 **Note: access.log/stream.log, logrotate and goaccess are NOT enabled by default bceuase of GDPR, you can enable them in the compose.yaml.** <br>
 
+**Note: add `net.ipv4.ip_unprivileged_port_start=0` at the end of `/etc/sysctl.conf` to support PUID/PGID in network mode host.** <br>
+**Note: Don't forget to open Port 80 (tcp) and 443 (tcp AND udp, http3/quic needs udp) in your firewall (because of network mode host, you also need to open this ports in ufw, if you use ufw).** <br>
+**Note: If you don't use network mode host, which I don't recommend, don't forget to also expose port 443/udp (http3/quic needs udp), to enable IPv6 in Docker see step 1 and 2 [here](https://github.com/nextcloud/all-in-one/blob/main/docker-ipv6-support.md).** <br>
+
 
 ## Project Goal
 I created this project to fill a personal need to provide users with an easy way to accomplish reverse
 proxying hosts with TLS termination and it had to be so easy that a monkey could do it. This goal hasn't changed.
 While there might be advanced options they are optional and the project should be as simple as possible
 so that the barrier for entry here is low.
 
-<!---
-### Sponsor the original creator (not us):
-<a href="https://www.buymeacoffee.com/jc21" target="_blank"><img src="http://public.jc21.com/github/by-me-a-coffee.png" alt="Buy Me A Coffee" style="height: 51px !important;width: 217px !important;" ></a>
---->
-
-
 ## Features
 
 - Beautiful and Secure Admin Interface based on [Tabler](https://tabler.github.io)
@@ -49,52 +36,52 @@ so that the barrier for entry here is low.
 - Supports HTTP/3 (QUIC) protocol.
 - Supports CrowdSec IPS. Please see [here](https://github.com/ZoeyVid/NPMplus#crowdsec) to enable it.
 - goaccess included, see compose.yaml to enable, runs by default on https://<ip>:91 (nginx config from [here](https://github.com/xavier-hernandez/goaccess-for-nginxproxymanager/blob/main/resources/nginx/nginx.conf))
-- Supports ModSecurity, with coreruleset as an option. You can configure ModSecurity/coreruleset by editing the files in the `/opt/npm/etc/modsecurity` folder.
-  - If the core ruleset blocks valid requests, please check the `/opt/npm/etc/modsecurity/crs-setup.conf` file.
+- Supports ModSecurity, with coreruleset as an option. You can configure ModSecurity/coreruleset by editing the files in the `/opt/npm/etc/modsecurity` folder (no support from me, you need to write the rules yourself - for CRS I can try to help you).
+  - ModSecurity by default blocks uploads of big files, you need to edit its config to fix this, but it can use a lot of resources to scan big files by ModSecurity
+  - ModSecurity overblocking (403 Error) with CRS? Please see [here](https://coreruleset.org/docs/concepts/false_positives_tuning) and edit the `/opt/npm/etc/modsecurity/crs-setup.conf` file.
   - Try to whitelist the Content-Type you are sending (for example, `application/activity+json` for Mastodon and `application/dns-message` for DoH).
   - Try to whitelist the HTTP request method you are using (for example, `PUT` is blocked by default, which also affects NPM).
-<!---
-  - Note: To fix [this issue](https://github.com/SpiderLabs/ModSecurity/issues/2848), instead of running `nginx -s reload`, this fork stops nginx and starts it again. This will result in a 502 error when you update your hosts. See https://github.com/ZoeyVid/NPMplus/issues/296 and https://github.com/ZoeyVid/NPMplus/issues/283.
---->
+  - CRS plugins are supported, you can find a guide in this readme
 - Darkmode button in the footer for comfortable viewing (CSS done by [@theraw](https://github.com/theraw))
 - Fixes proxy to https origin when the origin only accepts TLSv1.3
-- Only enables TLSv1.2 and TLSv1.3 protocols
-- Faster creation of TLS certificates can be achieved by eliminating unnecessary Nginx reloads and configuration creations.
-- Uses OCSP Stapling for enhanced security
+- Only enables TLSv1.2 and TLSv1.3 protocols, also ML-KEM support
+- Faster creation of TLS certificates is achieved by eliminating unnecessary nginx reloads and configuration creations.
+- Uses OCSP Stapling for enhanced security (manual certs not supported)
 - Resolved dnspod plugin issue
   - To migrate manually, delete all dnspod certs and recreate them OR change the credentials file as per the template given [here](https://github.com/ZoeyVid/NPMplus/blob/develop/global/certbot-dns-plugins.js)
 - Smaller docker image with alpine-based distribution
 - Admin backend interface runs with https
 - Default page also runs with https
+- option to change default TLS cert
 - Uses [fancyindex](https://gitHub.com/Naereen/Nginx-Fancyindex-Theme) if used as webserver
 - Exposes INTERNAL backend api only to localhost
 - Basic security headers are added if you enable HSTS (HSTS has always subdomains and preload enabled)
 - access.log is disabled by default, unified and moved to `/opt/npm/nginx/access.log`
 - Error Log written to console
 - `Server` response header hidden
 - PHP 8.2/8.3 optional, with option to add extensions; available packages can added using envs in the compose file
-- Allows different acme servers/certbot config file (/opt/npm/tls/certbot/config.ini)
+- Allows different acme servers using env
 - Supports up to 99 domains per cert
 - Brotli compression can be enabled
 - HTTP/2 always enabled with fixed upload
-- Allows infinite upload size
+- Allows infinite upload size (may be limited if you use ModSecurity)
 - Automatic database vacuum (only sqlite)
-- Automatic cleaning of old certbot certs (set FULLCLEAN to true)
+- Automatic cleaning of old invalid certbot certs (set CLEAN to true)
 - Password reset (only sqlite) using `docker exec -it npmplus password-reset.js USER_EMAIL PASSWORD`
 - Supports TLS for MariaDB/MySQL; set `DB_MYSQL_TLS` env to true. Self-signed certificates can be uploaded to `/opt/npm/etc/npm/ca.crt` and `DB_MYSQL_CA` set to `/data/etc/npm/ca.crt` (not tested, unsupported)
-- Supports PUID/PGID in network mode host; add `net.ipv4.ip_unprivileged_port_start=0` at the end of `/etc/sysctl.conf`
-- Option to set IP bindings for multiple instances in network mode host
-- Option to change backend port
-- multi lang support, if you want to add an language, see this commit as an example: https://github.com/ZoeyVid/NPMplus/commit/a026b42329f66b89fe1fbe5e6034df5d3fc2e11f
-- See the composefile for all available options
-- If you want to redirect all HTTP traffic to HTTPS, you can use the `compose.override.yaml` file.
+- multi lang support, if you want to add an language, see this commit as an example: https://github.com/ZoeyVid/NPMplus/commit/a026b42329f66b89fe1fbe5e6034df5d3fc2e11f (implementation based on [@lateautumn233](https://github.com/lateautumn233) fork)
+- See the compose file for all available options
+- many env options optimized for network_mode host (ports/ip bindings)
+- allow port range in streams
+- fixed smaller issues/bugs
+- other small changes/improvements
 
 ## migration
 - **NOTE: migrating back to the original is not possible**, so make first a **backup** before migration, so you can use the backup to switch back
 - please delete all dnspod certs and recreate them after migration OR you manually change the credentialsfile (see [here](https://github.com/ZoeyVid/npmplus/blob/develop/global/certbot-dns-plugins.json) for the template)
 - stop nginx-proxy-manager download the latest compose.yaml, adjust your paths (of /etc/letsencrypt and /data) to the ones you used with nginx-proxy-manager and adjust the env of the compose file how you like it and then deploy it
 - you can now remove the /etc/letsencrypt mount, since it was moved to /data while migration and redeploy the compose file
-- since this fork has dependency on `network_mode: host`, please don't forget to open port 80/tcp, 443/tcp and 443/udp (and maybe 81/tcp) in your firewall
+- since this fork uses  `network_mode: host` by default (and all guides are written for this mode), please don't forget to open port 80/tcp, 443/tcp and 443/udp (and maybe 81/tcp) in your firewall
 - since many buttons changed, please edit every host you have and click save. (Please also resave it, if all buttons/values are fine, to update the host config to fully fit the NPMplus template)
 - maybe setup crowdsec (see below)
 - please report all (migration) issues you may have
@@ -123,8 +110,10 @@ Password: iArhP1j7p1P6TA92FA2FMbbUGYqwcYzxC4AVEe12Wbi94FY9gNN62aKyF1shrvG4NycjjX
 Immediately after logging in with this default user you will be asked to modify your details and change your password.
 
 # Crowdsec
-1. Install crowdsec using this compose file: https://github.com/ZoeyVid/NPMplus/blob/develop/compose.crowdsec.yaml and enable LOGROTATE in the NPMplus compose file
-2. open `/opt/crowdsec/conf/acquis.d/npmplus.yaml` and fill it with:
+Note: Using Immich behind NPMplus with enabled appsec causes issues, see here: [#1241](https://github.com/ZoeyVid/NPMplus/discussions/1241)
+1. set LOGROTATE to `true` in your `compose.yaml` and redeploy
+2. Install crowdsec using this compose file: https://github.com/ZoeyVid/NPMplus/blob/develop/compose.crowdsec.yaml and set the timezone for it
+3. open `/opt/crowdsec/conf/acquis.d/npmplus.yaml` and fill it with:
 ```yaml
 filenames:
   - /opt/npm/nginx/access.log
@@ -150,13 +139,12 @@ source: appsec
 labels:
   type: appsec
 ```
-3. make sure to use `network_mode: host` in your compose file
-4. run `docker exec crowdsec cscli bouncers add npmplus -o raw` and save the output
-5. open `/opt/npm/etc/crowdsec/crowdsec.conf`
-6. set `ENABLED` to `true`
-7. use the output of step 5 as `API_KEY`
-8. save the file
-9. set LOGROTATE to `true` in your `compose.yaml`
+4. make sure to use `network_mode: host` in your compose file
+5. run `docker exec crowdsec cscli bouncers add npmplus -o raw` and save the output
+6. open `/opt/npm/etc/crowdsec/crowdsec.conf`
+7. set `ENABLED` to `true`
+8. use the output of step 5 as `API_KEY`
+9. save the file
 10. redeploy the `compose.yaml`
 
 # coreruleset plugins
@@ -206,16 +194,10 @@ if you need to run scripts before NPMplus launches put them under: `/opt/npm/etc
 you need to create this folder yourself - **NOTE:** I won't help you creating those patches/scripts if you need them you also need to know how to create them
 
 ## Contributing
-All are welcome to create pull requests for this project, against the `develop` branch.
-CI is used in this project. All PR's must pass before being considered. After passing,
-docker builds for PR's are available on ghcr for manual verifications.
-
-## Contributors/Sponsor upstream NPM
-Special thanks to [all of our contributors](https://github.com/NginxProxyManager/nginx-proxy-manager/graphs/contributors).
-If you want to sponsor them, please see [here](https://github.com/NginxProxyManager/nginx-proxy-manager/blob/master/README.md).
+All are welcome to create pull requests for this project.
 
 # Please report Bugs first to this fork before reporting them to the upstream Repository
-## Getting Support
-1. [Found a bug?](https://github.com/ZoeyVid/NPMplus/issues)
-2. [Discussions](https://github.com/ZoeyVid/NPMplus/discussions)
-3. [Reddit](https://reddit.com/r/NPMplus)
+## Getting Help
+1. [Support/Questions](https://github.com/ZoeyVid/NPMplus/discussions)
+2. [Reddit](https://reddit.com/r/NPMplus)
+3. [Bugs](https://github.com/ZoeyVid/NPMplus/issues)

backend/internal/certificate.js

@@ -16,7 +16,7 @@ const internalAuditLog = require('./audit-log');
 const internalNginx = require('./nginx');
 
 const certbotCommand = 'certbot';
-const certbotArgs = ['--logs-dir', '/tmp/certbot-log', '--work-dir', '/tmp/certbot-work', '--config-dir', '/data/tls/certbot', '--config', '/etc/certbot.ini', '--agree-tos', '--non-interactive', '--no-eff-email', '--register-unsafely-without-email'];
+const certbotArgs = ['--logs-dir', '/tmp/certbot-log', '--work-dir', '/tmp/certbot-work', '--config-dir', '/data/tls/certbot', '--config', '/etc/certbot.ini', '--agree-tos', '--non-interactive', '--no-eff-email', '--register-unsafely-without-email', process.env.ACME_MUST_STAPLE == 'false' ? '' : '--must-staple', process.env.ACME_SERVER_TLS_VERIFY == 'false' ? '--no-verify-ssl' : ''];
 
 function omissions() {
 	return ['is_deleted', 'owner.is_deleted'];

backend/internal/token.js

@@ -65,15 +65,15 @@ module.exports = {
 											};
 										});
 									} else {
-										throw new error.AuthError('Invalid password');
+										throw new error.AuthError('Invalid email or password');
 									}
 								});
 							} else {
-								throw new error.AuthError('No password auth for user');
+								throw new error.AuthError('Invalid email or password');
 							}
 						});
 				} else {
-					throw new error.AuthError('No relevant user found');
+					throw new error.AuthError('Invalid email or password');
 				}
 			});
 	},

backend/package.json

@@ -18,10 +18,10 @@
     "gravatar": "1.8.2",
     "jsonwebtoken": "9.0.2",
     "knex": "3.1.0",
-    "liquidjs": "10.18.0",
+    "liquidjs": "10.19.0",
     "lodash": "4.17.21",
     "moment": "2.30.1",
-    "mysql2": "3.11.3",
+    "mysql2": "3.11.4",
     "node-rsa": "1.1.1",
     "objection": "3.1.5",
     "path": "0.12.7",
@@ -31,11 +31,11 @@
   "license": "MIT",
   "devDependencies": {
     "@apidevtools/swagger-parser": "10.1.0",
-    "@eslint/js": "9.14.0",
-    "eslint": "9.14.0",
+    "@eslint/js": "9.15.0",
+    "eslint": "9.15.0",
     "eslint-config-prettier": "9.1.0",
     "eslint-plugin-prettier": "5.2.1",
-    "globals": "15.11.0",
+    "globals": "15.12.0",
     "prettier": "3.3.3"
   },
   "scripts": {

backend/schema/paths/nginx/streams/streamID/put.json

@@ -29,56 +29,23 @@
 					"additionalProperties": false,
 					"minProperties": 1,
 					"properties": {
-						"domain_names": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/domain_names"
+						"incoming_port": {
+							"$ref": "../../../../components/stream-object.json#/properties/incoming_port"
 						},
-						"forward_scheme": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/forward_scheme"
+						"forwarding_host": {
+							"$ref": "../../../../components/stream-object.json#/properties/forwarding_host"
 						},
-						"forward_host": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/forward_host"
+						"forwarding_port": {
+							"$ref": "../../../../components/stream-object.json#/properties/forwarding_port"
 						},
-						"forward_port": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/forward_port"
+						"tcp_forwarding": {
+							"$ref": "../../../../components/stream-object.json#/properties/tcp_forwarding"
 						},
-						"certificate_id": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/certificate_id"
-						},
-						"ssl_forced": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/ssl_forced"
-						},
-						"hsts_enabled": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/hsts_enabled"
-						},
-						"hsts_subdomains": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/hsts_subdomains"
-						},
-						"http2_support": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/http2_support"
-						},
-						"block_exploits": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/block_exploits"
-						},
-						"caching_enabled": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/caching_enabled"
-						},
-						"allow_websocket_upgrade": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/allow_websocket_upgrade"
-						},
-						"access_list_id": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/access_list_id"
-						},
-						"advanced_config": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/advanced_config"
-						},
-						"enabled": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/enabled"
+						"udp_forwarding": {
+							"$ref": "../../../../components/stream-object.json#/properties/udp_forwarding"
 						},
 						"meta": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/meta"
-						},
-						"locations": {
-							"$ref": "../../../../components/proxy-host-object.json#/properties/locations"
+							"$ref": "../../../../components/stream-object.json#/properties/meta"
 						}
 					}
 				}

compose.crowdsec.yaml

@@ -8,7 +8,7 @@ services:
       - "127.0.0.1:7422:7422"
       - "127.0.0.1:8080:8080"
     environment:
-      - "TZ=Europe/Berlin"
+      - "TZ=your-timezone"
       - "COLLECTIONS=ZoeyVid/npmplus"
     volumes:
       - "/opt/crowdsec/conf:/etc/crowdsec"

compose.geoip.yaml

@@ -5,7 +5,7 @@ services:
     restart: always
     network_mode: bridge
     environment:
-      - "TZ=Europe/Berlin"
+      - "TZ=your-timezone"
       - "GEOIPUPDATE_EDITION_IDS=GeoLite2-Country GeoLite2-City GeoLite2-ASN"
       - "GEOIPUPDATE_ACCOUNT_ID=<your-account-id>"
       - "GEOIPUPDATE_LICENSE_KEY=<your-license-key>"