Resolve "Un-vendor CI templates"

doc/ci/README.md

@@ -132,5 +132,3 @@ your whole GitLab instance as well as in each project.
 - [New CI job permissions model](../user/project/new_ci_build_permissions_model.md)
   Read about what changed in GitLab 8.12 and how that affects your jobs.
   There's a new way to access your Git submodules and LFS objects in jobs.
-
-[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml

doc/ci/caching/index.md

@@ -178,8 +178,8 @@ runs of jobs for things like dependencies and commonly used libraries
 so they don't have to be re-fetched from the public internet.
 
 NOTE: **Note:**
-For more examples, check the [GitLab CI Yml](https://gitlab.com/gitlab-org/gitlab-ci-yml)
-project.
+For more examples, check out our [GitLab CI/CD
+templates](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates).
 
 ### Caching Nodejs dependencies
 
@@ -190,7 +190,7 @@ Nodejs modules are installed in `node_modules/` and are cached per-branch:
 
 ```yaml
 #
-# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Nodejs.gitlab-ci.yml
+# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml
 #
 image: node:latest
 
@@ -217,7 +217,7 @@ are cached per-branch:
 
 ```yaml
 #
-# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/PHP.gitlab-ci.yml
+# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/PHP.gitlab-ci.yml
 #
 image: php:7.2
 
@@ -246,7 +246,7 @@ pip's cache is defined under `.cache/pip/` and both are cached per-branch:
 
 ```yaml
 #
-# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Python.gitlab-ci.yml
+# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml
 #
 image: python:latest
 
@@ -286,7 +286,7 @@ jobs inherit it. Gems are installed in `vendor/ruby/` and are cached per-branch:
 
 ```yaml
 #
-# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Ruby.gitlab-ci.yml
+# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml
 #
 image: ruby:2.5
 

doc/ci/examples/README.md

@@ -4,8 +4,8 @@ comments: false
 
 # GitLab CI/CD Examples
 
-A collection of `.gitlab-ci.yml` template files is maintained at the [GitLab CI/CD YAML project][gitlab-ci-templates]. When you create a new file via the UI,
-GitLab will give you the option to choose one of the templates existent on this project.
+A collection of [`.gitlab-ci.yml` template files][gitlab-ci-templates] is maintained in GitLab. When you create a new file via the UI,
+GitLab will give you the option to choose one of these templates.
 If your favorite programming language or framework are missing we would love your
 help by sending a merge request with a new `.gitlab-ci.yml` to this project.
 
@@ -87,4 +87,4 @@ language users and GitLab by sending a merge request with a guide for that langu
 You may want to apply for the [GitLab Community Writers Program](https://about.gitlab.com/community-writers/)
 to get paid for writing complete articles for GitLab.
 
-[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml
+[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates

doc/ci/examples/browser_performance.md

@@ -110,4 +110,4 @@ performance:
       - sitespeed-results/
 ```
 
-A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml).
+A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml).

doc/development/testing_guide/review_apps.md

@@ -74,7 +74,7 @@ on making Review Apps automatically deployed by each pipeline, both in CE and EE
 [review-apps-ee]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps-ee?project=gitlab-review-apps
 [review-apps.sh]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/review-apps.sh
 [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb
-[Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml
+[Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml
 [gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html
 
 ---

doc/topics/autodevops/index.md

@@ -145,7 +145,7 @@ When using Auto DevOps, you may want to deploy different environments to
 different Kubernetes clusters. This is possible due to the 1:1 connection that
 [exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters).
 
-In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml)
+In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml)
 (used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know:
 
 - `review/` (every environment starting with `review/`)
@@ -832,6 +832,6 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/
 [review-app]: ../../ci/review_apps/index.md
 [container-registry]: ../../user/project/container_registry.md
 [postgresql]: https://www.postgresql.org/
-[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml
+[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml
 [ee]: https://about.gitlab.com/pricing/
 [ce-19507]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19507

lib/gitlab/template/gitlab_ci_yml_template.rb

@@ -20,7 +20,7 @@ module Gitlab
         end
 
         def base_dir
-          Rails.root.join('vendor/gitlab-ci-yml')
+          Rails.root.join('lib/gitlab/ci/templates')
         end
 
         def finder(project = nil)

lib/tasks/gitlab/update_templates.rake

@@ -99,10 +99,6 @@ namespace :gitlab do
       "https://github.com/github/gitignore.git",
       /(\.{1,2}|LICENSE|Global|\.gitignore)\z/
     ),
-    Template.new(
-      "https://gitlab.com/gitlab-org/gitlab-ci-yml.git",
-      /(\.{1,2}|LICENSE|CONTRIBUTING.md|Pages|autodeploy|\.gitlab-ci.yml)\z/
-    ),
     Template.new(
       "https://gitlab.com/gitlab-org/Dockerfile.git",
       /(\.{1,2}|LICENSE|CONTRIBUTING.md|\.Dockerfile)\z/

spec/lib/gitlab/ci/external/file/local_spec.rb

@@ -8,7 +8,7 @@ describe Gitlab::Ci::External::File::Local do
 
   describe '#valid?' do
     context 'when is a valid local path' do
-      let(:location) { '/vendor/gitlab-ci-yml/existent-file.yml' }
+      let(:location) { '/lib/gitlab/ci/templates/existent-file.yml' }
 
       before do
         allow_any_instance_of(described_class).to receive(:fetch_local_content).and_return("image: 'ruby2:2'")
@@ -20,7 +20,7 @@ describe Gitlab::Ci::External::File::Local do
     end
 
     context 'when is not a valid local path' do
-      let(:location) { '/vendor/gitlab-ci-yml/non-existent-file.yml' }
+      let(:location) { '/lib/gitlab/ci/templates/non-existent-file.yml' }
 
       it 'should return false' do
         expect(local_file.valid?).to be_falsy
@@ -48,7 +48,7 @@ describe Gitlab::Ci::External::File::Local do
             - bundle install --jobs $(nproc)  "${FLAGS[@]}"
         HEREDOC
       end
-      let(:location) { '/vendor/gitlab-ci-yml/existent-file.yml' }
+      let(:location) { '/lib/gitlab/ci/templates/existent-file.yml' }
 
       before do
         allow_any_instance_of(described_class).to receive(:fetch_local_content).and_return(local_file_content)
@@ -60,7 +60,7 @@ describe Gitlab::Ci::External::File::Local do
     end
 
     context 'with an invalid file' do
-      let(:location) { '/vendor/gitlab-ci-yml/non-existent-file.yml' }
+      let(:location) { '/lib/gitlab/ci/templates/non-existent-file.yml' }
 
       it 'should be nil' do
         expect(local_file.content).to be_nil
@@ -69,7 +69,7 @@ describe Gitlab::Ci::External::File::Local do
   end
 
   describe '#error_message' do
-    let(:location) { '/vendor/gitlab-ci-yml/non-existent-file.yml' }
+    let(:location) { '/lib/gitlab/ci/templates/non-existent-file.yml' }
 
     it 'should return an error message' do
       expect(local_file.error_message).to eq("Local file '#{location}' is not valid.")

spec/lib/gitlab/ci/external/mapper_spec.rb

@@ -17,7 +17,7 @@ describe Gitlab::Ci::External::Mapper do
       context 'when the string is a local file' do
         let(:values) do
           {
-            include: '/vendor/gitlab-ci-yml/non-existent-file.yml',
+            include: '/lib/gitlab/ci/templates/non-existent-file.yml',
             image: 'ruby:2.2'
           }
         end
@@ -61,7 +61,7 @@ describe Gitlab::Ci::External::Mapper do
           include:
           [
             remote_url,
-            '/vendor/gitlab-ci-yml/template.yml'
+            '/lib/gitlab/ci/templates/template.yml'
           ],
           image: 'ruby:2.2'
         }

spec/lib/gitlab/ci/external/processor_spec.rb

@@ -16,12 +16,12 @@ describe Gitlab::Ci::External::Processor do
     end
 
     context 'when an invalid local file is defined' do
-      let(:values) { { include: '/vendor/gitlab-ci-yml/non-existent-file.yml', image: 'ruby:2.2' } }
+      let(:values) { { include: '/lib/gitlab/ci/templates/non-existent-file.yml', image: 'ruby:2.2' } }
 
       it 'should raise an error' do
         expect { processor.perform }.to raise_error(
           described_class::FileError,
-          "Local file '/vendor/gitlab-ci-yml/non-existent-file.yml' is not valid."
+          "Local file '/lib/gitlab/ci/templates/non-existent-file.yml' is not valid."
         )
       end
     end
@@ -79,7 +79,7 @@ describe Gitlab::Ci::External::Processor do
     end
 
     context 'with a valid local external file is defined' do
-      let(:values) { { include: '/vendor/gitlab-ci-yml/template.yml', image: 'ruby:2.2' } }
+      let(:values) { { include: '/lib/gitlab/ci/templates/template.yml', image: 'ruby:2.2' } }
       let(:local_file_content) do
         <<-HEREDOC
         before_script:
@@ -145,7 +145,7 @@ describe Gitlab::Ci::External::Processor do
     end
 
     context 'when external files are defined but not valid' do
-      let(:values) { { include: '/vendor/gitlab-ci-yml/template.yml', image: 'ruby:2.2' } }
+      let(:values) { { include: '/lib/gitlab/ci/templates/template.yml', image: 'ruby:2.2' } }
 
       let(:local_file_content) { 'invalid content file ////' }
 

spec/lib/gitlab/ci/templates/templates_spec.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe "CI YML Templates" do
+  Gitlab::Template::GitlabCiYmlTemplate.all.each do |template|
+    it "#{template.name} should be valid" do
+      expect { Gitlab::Ci::YamlProcessor.new(template.content) }.not_to raise_error
+    end
+  end
+end

spec/lib/gitlab/ci/yaml_processor_spec.rb

@@ -1371,18 +1371,6 @@ module Gitlab
         end
       end
 
-      describe "Validate configuration templates" do
-        templates = Dir.glob("#{Rails.root.join('vendor/gitlab-ci-yml')}/**/*.gitlab-ci.yml")
-
-        templates.each do |file|
-          it "does not return errors for #{file}" do
-            file = File.read(file)
-
-            expect { Gitlab::Ci::YamlProcessor.new(file) }.not_to raise_error
-          end
-        end
-      end
-
       describe "#validation_message" do
         subject { Gitlab::Ci::YamlProcessor.validation_message(content) }
 

spec/lib/gitlab/template/gitlab_ci_yml_template_spec.rb

@@ -40,7 +40,7 @@ describe Gitlab::Template::GitlabCiYmlTemplate do
 
   describe '#content' do
     it 'loads the full file' do
-      gitignore = subject.new(Rails.root.join('vendor/gitlab-ci-yml/Ruby.gitlab-ci.yml'))
+      gitignore = subject.new(Rails.root.join('lib/gitlab/ci/templates/Ruby.gitlab-ci.yml'))
 
       expect(gitignore.name).to eq 'Ruby'
       expect(gitignore.content).to start_with('#')

vendor/gitlab-ci-yml/.gitlab-ci.yml

@@ -1,4 +0,0 @@
-image: ruby:2.4-alpine
-
-test:
-  script: ./verify_templates.rb

vendor/gitlab-ci-yml/CONTRIBUTING.md

@@ -1,46 +0,0 @@
-## Developer Certificate of Origin + License
-
-By contributing to GitLab B.V., You accept and agree to the following terms and
-conditions for Your present and future Contributions submitted to GitLab B.V.
-Except for the license granted herein to GitLab B.V. and recipients of software
-distributed by GitLab B.V., You reserve all right, title, and interest in and to
-Your Contributions. All Contributions are subject to the following DCO + License
-terms.
-
-[DCO + License](https://gitlab.com/gitlab-org/dco/blob/master/README.md)
-
-_This notice should stay as the first item in the CONTRIBUTING.md file._
-
-## Code of conduct
-
-As contributors and maintainers of this project, we pledge to respect all people
-who contribute through reporting issues, posting feature requests, updating
-documentation, submitting pull requests or patches, and other activities.
-
-We are committed to making participation in this project a harassment-free
-experience for everyone, regardless of level of experience, gender, gender
-identity and expression, sexual orientation, disability, personal appearance,
-body size, race, ethnicity, age, or religion.
-
-Examples of unacceptable behavior by participants include the use of sexual
-language or imagery, derogatory comments or personal attacks, trolling, public
-or private harassment, insults, or other unprofessional conduct.
-
-Project maintainers have the right and responsibility to remove, edit, or reject
-comments, commits, code, wiki edits, issues, and other contributions that are
-not aligned to this Code of Conduct. Project maintainers who do not follow the
-Code of Conduct may be removed from the project team.
-
-This code of conduct applies both within project spaces and in public spaces
-when an individual is representing the project or its community.
-
-Instances of abusive, harassing, or otherwise unacceptable behavior can be
-reported by emailing contact@gitlab.com.
-
-This Code of Conduct is adapted from the [Contributor Covenant][contributor-covenant], version 1.1.0,
-available at [http://contributor-covenant.org/version/1/1/0/](http://contributor-covenant.org/version/1/1/0/).
-
-[contributor-covenant]: http://contributor-covenant.org
-[individual-agreement]: https://docs.gitlab.com/ee/legal/individual_contributor_license_agreement.html
-[corporate-agreement]: https://docs.gitlab.com/ee/legal/corporate_contributor_license_agreement.html
-

vendor/gitlab-ci-yml/LICENSE

@@ -1,25 +0,0 @@
-Copyright (c) 2011-2017 GitLab B.V.
-
-With regard to the GitLab Software:
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
-For all third party components incorporated into the GitLab Software, those 
-components are licensed under the original license provided by the owner of the 
-applicable component.