kotovalexarian-likes-github/rails--rails
1
0
Fork 0
mirror of https://github.com/rails/rails.git synced 2022-11-09 12:12:34 -05:00
Code Releases Activity

Merge pull request #18485 from andreynering/robots.txt

Browse source 
Adding 'Search Engines Indexing' Section [ci skip]
This commit is contained in:
Zachary Scott 2015-02-16 10:08:01 -08:00
commit d9f8b58d98
2 changed files with 22 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
guides/CHANGELOG.md
View file

@ -6,4 +6,8 @@
*Andrey Nering*
* New section in Configuring: Search Engines Indexing
*Andrey Nering*
Please check [4-2-stable](https://github.com/rails/rails/blob/4-2-stable/guides/CHANGELOG.md) for previous changes.

18
guides/source/configuring.md
View file

@ -1049,3 +1049,21 @@ These configuration points are then available through the configuration object:
Rails.configuration.x.super_debugger # => true
Rails.configuration.x.super_debugger.not_set # => nil
```
Search Engines Indexing
-----------------------
Sometimes, you may want to prevent some pages of your application be visible on search sites like Google,
Bing, Yahoo or Duck Duck Go. The robots that index these sites will first analyse the
`https://your-site.com/robots.txt` file to know what pages it is allowed to index.
Rails creates this file for you on `/public` folder. By default, it allows search engines to index all
pages of your application. If you want to block indexing on all pages of you application, use this:
```
User-agent: *
Disallow: /
```
To block just specific pages, it's necessary to use a more complex syntax. Learn it on the
[official documentation](http://www.robotstxt.org/robotstxt.html).