[Feature] Add Support for robots.txt File

[Draxter]

Currently, Heimdall does not have a built-in way to stop web crawlers or specify indexing rules. Adding support for a robots.txt file will allow administrators to:

• Prevent unauthorised or unwanted web crawlers from accessing certain parts of the app.
• Specify guidelines for search engine crawlers to improve SEO and reduce resource load.

While I understand this can be added manually into Heamdall's http root, it would be useful to have a default robots.txt file built-in with an example content like:

User-agent: *
Disallow:

[RobLoach]

Doesn't this already exist? https://github.com/linuxserver/Heimdall/blob/2.x/public/robots.txt

[Draxter]

My bad, was running an older version!