From 71f707bfe86f2f0a90182bc92033963aef32ce15 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 22 Mar 2018 10:55:02 +0100 Subject: [PATCH] Add the /help page in robots.txt The /help page has docs which we don't want to be crawled as we prefer the docs website instead. Related https://gitlab.com/gitlab-org/gitlab-ce/issues/44433 --- public/robots.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/public/robots.txt b/public/robots.txt index 123272a9834..1f9d42f4adc 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -20,6 +20,7 @@ Disallow: /projects/new Disallow: /groups/new Disallow: /groups/*/edit Disallow: /users +Disallow: /help # Global snippets User-Agent: *