Add the /help page in robots.txt
The /help page has docs which we don't want to be crawled as we prefer the docs website instead. Related https://gitlab.com/gitlab-org/gitlab-ce/issues/44433
This commit is contained in:
parent
5790caa7dc
commit
71f707bfe8
|
@ -20,6 +20,7 @@ Disallow: /projects/new
|
||||||
Disallow: /groups/new
|
Disallow: /groups/new
|
||||||
Disallow: /groups/*/edit
|
Disallow: /groups/*/edit
|
||||||
Disallow: /users
|
Disallow: /users
|
||||||
|
Disallow: /help
|
||||||
|
|
||||||
# Global snippets
|
# Global snippets
|
||||||
User-Agent: *
|
User-Agent: *
|
||||||
|
|
Loading…
Reference in New Issue