13 KiB
Serverless
Introduced in GitLab 11.5.
Serverless is currently in alpha.
Run serverless workloads on Kubernetes using Knative.
Overview
Knative extends Kubernetes to provide a set of middleware components that are useful to build modern, source-centric, container-based applications. Knative brings some significant benefits out of the box through its main components:
- Build: Source-to-container build orchestration.
- Eventing: Management and delivery of events.
- Serving: Request-driven compute that can scale to zero.
For more information on Knative, visit the Knative docs repo.
With GitLab serverless, you can deploy both functions-as-a-service (FaaS) and serverless applications.
Requirements
To run Knative on Gitlab, you will need:
- Kubernetes Cluster: An RBAC-enabled Kubernetes cluster is required to deploy Knative. The simplest way to get started is to add a cluster using GitLab's GKE integration.
- Helm Tiller: Helm is a package manager for Kubernetes and is required to install Knative.
- Domain Name: Knative will provide its own load balancer using Istio. It will provide an external IP address for all the applications served by Knative. You will be prompted to enter a wildcard domain where your applications will be served. Configure your DNS server to use the external IP address for that domain.
.gitlab-ci.yml
: GitLab uses Kaniko to build the application and the TriggerMesh CLI to simplify the deployment of knative services and functions.serverless.yml
(for functions only): When using serverless to deploy functions, theserverless.yml
file will contain the information for all the functions being hosted in the repository as well as a reference to the runtime being used.Dockerfile
(for applications only: Knative requires aDockerfile
in order to build your application. It should be included at the root of your project's repo and expose port8080
.
Installing Knative via GitLab's Kubernetes integration
NOTE: Note: The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. RBAC must be enabled.
-
Add a Kubernetes cluster and install Helm.
-
Once Helm has been successfully installed, on the Knative app section, enter the domain to be used with your application and click "Install".
-
After the Knative installation has finished, you can wait for the IP address to be displayed in the Knative IP Address field or retrieve the Istio Ingress IP address by running the following command:
kubectl get svc --namespace=istio-system knative-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} '
Output:
35.161.143.124 my-machine-name:~ my-user$
-
The ingress is now available at this address and will route incoming requests to the proper service based on the DNS name in the request. To support this, a wildcard DNS A record should be created for the desired domain name. For example, if your Knative base domain is
knative.info
then you need to create an A record with domain*.knative.info
pointing the ip address of the ingress.
Deploying Functions
Introduced in GitLab 11.6.
Using functions is useful for dealing with independent events without needing to maintain a complex unified infrastructure. This allows you to focus on a single task that can be executed/scaled automatically and independently.
Currently the following runtimes are offered:
- node.js
- kaniko
You can find all the files referenced in this doc in the functions example project.
Follow these steps to deploy a function using the node.js runtime to your Knative instance:
-
Create a directory that will house the function. In this example we will create a directory called
echo
at the root of the project. -
Create the file that will contain the function code. In this example our file is called
echo.js
and is located inside theecho
directory.If your project is public, skip to step no. 4.
-
If your project is private you will need to Create a GitLab deploy token. This will enable the
tm
cli to be used as a deployment step and access the container registry.- Go to the project you want to create the function for.
- Go to Settings > Repository.
- Click on "Expand" on Deploy Tokens section.
- Enter
gitlab-deploy-token
as the name. - Check the
read_registry
scope - Click on Create deploy token.
- Save the deploy token somewhere safe. Once you leave or refresh the page, you won't be able to access it again.
-
.gitlab-ci.yml
: This template allows to define the stage, environment, and image to be used for your functions. It must be included at the root of your repository:stages: - deploy functions: stage: deploy environment: test image: gcr.io/triggermesh/tm:v0.0.9 script: - tm -n "$KUBE_NAMESPACE" set registry-auth gitlab-registry --registry "$CI_REGISTRY" --username "$CI_REGISTRY_USER" --password "$CI_JOB_TOKEN" --push - tm -n "$KUBE_NAMESPACE" set registry-auth gitlab-registry --registry "$CI_REGISTRY" --username "$CI_DEPLOY_USER" --password "$CI_DEPLOY_PASSWORD" --pull - tm -n "$KUBE_NAMESPACE" deploy --wait
The
gitlab-ci.yml
template creates aDeploy
stage with afunctions
job that invokes thetm
CLI with the required parameters. -
serverless.yml
: This file contains the metadata for your functions, such as name, runtime, and environment. It must be included at the root of your repository. The following is a sampleecho
function which shows the required structure for the file. You can find the relevant files for this project in the functions example project.service: my-functions description: "Deploying functions from GitLab using Knative" provider: name: triggermesh registry-secret: gitlab-registry environment: FOO: BAR functions: echo: handler: echo runtime: https://gitlab.com/triggermesh/runtimes/raw/master/nodejs.yaml description: "echo function using node.js runtime" buildargs: - DIRECTORY=echo environment: FUNCTION: echo
The serverless.yml
file is referencing both an echo
directory (under buildargs
) and an echo
file (under handler
)
which is a reference to echo.js
in the repository. Additionally, it
contains three sections with distinct parameters:
service
Parameter | Description |
---|---|
service |
Name for the Knative service which will serve the function. |
description |
A short description of the service . |
provider
Parameter | Description |
---|---|
name |
Indicates which provider is used to execute the serverless.yml file. In this case, the TriggerMesh tm CLI. |
registry-secret |
Indicates which registry will be used to store docker images. The sample function is using the GitLab Registry (gitlab-registry ). A different registry host may be specified using registry key in the provider object. If changing the default, update the permission and the secret value on the gitlab-ci.yml file |
environment |
Includes the environment variables to be passed as part of function execution for all functions in the file, where FOO is the variable name and BAR are he variable contents. You may replace this with you own variables. |
functions
In the serverless.yml
example above, the function name is echo
and the subsequent lines contain the function attributes.
Parameter | Description |
---|---|
handler |
The function's file name. In the example above, both the function name and the handler are the same. |
runtime |
The runtime to be used to execute the function. |
description |
A short description of the function. |
buildargs |
Pointer to the function file in the repo. In the sample the function is located in the echo directory. |
environment |
Sets an environment variable for the specific function only. |
After the gitlab-ci.yml
template has been added and the serverless.yml
file has been
created, pushing a commit to your project will result in a
CI pipeline being executed which will deploy each function as a Knative service.
Once the deploy stage has finished, additional details for the function will
appear under Operations > Serverless.
This page contains all functions available for the project, the description for accessing the function, and, if available, the function's runtime information. The details are derived from the Knative installation inside each of the project's Kubernetes cluster.
The function details can be retrieved directly from Knative on the cluster:
kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev
The sample function can now be triggered from any HTTP client using a simple POST
call.
Deploying Serverless applications
Introduced in GitLab 11.5.
NOTE: Note: You can reference the sample Knative Ruby App to get started.
Add the following .gitlab-ci.yml
to the root of your repository
(you may skip this step if using the sample Knative Ruby App mentioned above):
stages:
- build
- deploy
build:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
only:
- master
script:
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE
deploy:
stage: deploy
image: gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5
only:
- master
environment: production
script:
- echo "$CI_REGISTRY_IMAGE"
- tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait
Deploy the application with Knative
With all the pieces in place, the next time a CI pipeline runs, the Knative application will be deployed. Navigate to CI/CD > Pipelines and click the most recent pipeline.
Obtain the URL for the Knative deployment
Visit Operations >> Serverless to find the URL for your deployment (listed under the "Domain" column).
Alternatively, use the CI/CD deployment job output to obtain the deployment URL. Once all the stages of the pipeline finish, click the deploy stage.
The output will look like this:
Running with gitlab-runner 11.5.0~beta.844.g96d88322 (96d88322)
on docker-auto-scale 72989761
Using Docker executor with image gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Pulling docker image gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Using docker image sha256:6b3f6590a9b30bd7aafb9573f047d930c70066e43955b4beb18a1eee175f6de1 for gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Running on runner-72989761-project-4342902-concurrent-0 via runner-72989761-stg-srm-1541795796-27929c96...
Cloning repository...
Cloning into '/builds/danielgruesso/knative'...
Checking out 8671ad20 as master...
Skipping Git submodules setup
$ echo "$CI_REGISTRY_IMAGE"
registry.staging.gitlab.com/danielgruesso/knative
$ tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait
Deployment started. Run "tm -n knative-4342902 describe service knative" to see the details
Waiting for ready state.......
Service domain: knative.knative-4342902.knative.info
Job succeeded
The second to last line, labeled Service domain contains the URL for the deployment. Copy and paste the domain into your browser to see the app live.