Disallow Amazonbot

since it crawls multiple times the same profile and sometimes tries the edit path.
rubymonsters · Nov 4, 2024 · 924326b · 924326b
1 parent 11dc39b
commit 924326b
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/public/robots.txt b/public/robots.txt
@@ -4,6 +4,9 @@
 User-Agent: *
 Disallow: /topics
 
+User-Agent: Amazonbot
+Disallow: /
+
 User-Agent: Seobility
 Disallow: /