How to Configure Jenkins with Google Cloud Platform (GCP)
We are going to guide you through this post on how to configure Jenkins with Google Cloud Platform (GCP).
In this article we will cover the basics on configuring Jenkins to use GCP to create agents on-demand.
We will start by covering the requirements to fulfil this setup and then move to how to configure a GCP service account. After that comes the Jenkins configuration which I will start by the plugin installation, then the creation of the service account credentials and the cloud configuration.
At the end there’s a small test job to verify the configuration.
Table of Contents
Requirements for setting up Jenkins with Google Cloud Platform
The following resources are required in order to set this up:
- Account on a Jenkins instance*: With administrator permissions.
- Account on Google Cloud Platform: Capable of using Compute Engine and create service accounts.
- Optional: gcloud command-line tool.
*This instance can be in any cloud or in on-premise. For readers with less experience we recommend this article which describe the process of creating a brand new Jenkins instance.
Creating a Google Cloud Platform service account
This first thing to do is to create a service account in GCP. This account will be used by Jenkins to communicate with GCP, whenever is necessary to create an agent.
This account will have the following roles:
- instanceAdmin
- networkAdmin
- ServiceAccountUser
Using gcloud command-line tool or cloud shell this is easily achieved by running the following commands.
In this example we are going to use a cloud shell and the steps are as follows:
Create the service account:
gcloud iam service-accounts create jenkins-gce
Assign the required roles to the service account:
export PROJECT=$(gcloud info --format='value(config.project)')
export SA_EMAIL=$(gcloud iam service-accounts list --filter="name:jenkins-gce" \
--format='value(email)')
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
--role roles/compute.instanceAdmin $PROJECT
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
--role roles/compute.networkAdmin $PROJECT
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
--role roles/iam.serviceAccountUser $PROJECT
gcloud projects get-iam-policy $PROJECT
You should see:
- members:
- serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
role: roles/compute.instanceAdmin
- members:
- serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
role: roles/compute.networkAdmin
- members:
- serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
- user:<user>
role: roles/iam.serviceAccountUser
Grab the JSON service account key:
gcloud iam service-accounts keys create --iam-account $SA_EMAIL jenkins-gce.json
If you are using cloud shell, use the following command to download the file:
cloudshell download jenkins-gce.json
Using this service account Jenkins will be able to manage all the resources required to create agents on-demand.
Installing Google Compute Engine (GCE) plugin
The plugin required for this setup is called Google Compute Engine which is responsible to communicate with GCP, create the resources and configure the agent on those resources.
Go to Manage Jenkins > Manage Plugins > Available and look for “Google Compute Engine” as we show you in the following image:
Service account credentials setup
In order for Jenkins to authenticate against GCP it’s required to add the service account key to the Credentials section in Jenkins.
Go to Manage Jenkins > Manage Credentials > Global > Add Credentials of kind “Google Service Account from private key”.
Set your project name and upload the json file previously downloaded.
Jenkins Cloud Setup
Now that we have all the requirements for Jenkins to communicate with GCP and create resources, we need to configure how the agents are created and of what type.
Go to Manage Jenkins > Manage Nodes and Clouds > Configure Clouds > Add a new cloud > Google Compute Engine and cover the fields with your information.
The Service Account Credentials should be the one created in the previous step and you can use an instance template or configure the instance directly in Jenkins.
Note: The instance requires java8 installed and on the default path. Create a VM, install Java8 and create a new OS image in your GCP project. Packer can also be used to achieve this. More info here.
This is an example of a Jenkins Cloud setup:
Example in Configuration as Code, replace PROJECT and REGION with the ones you are using:
clouds:
- computeEngine:
cloudName: "GCP"
configurations:
- bootDiskAutoDelete: true
bootDiskSizeGb: 10
bootDiskSizeGbStr: "10"
bootDiskType: "https://www.googleapis.com/compute/v1/projects/$PROJECT/zones/$REGION/diskTypes/pd-balanced"
description: "CICD"
externalAddress: true
javaExecPath: "java"
labelSet:
- name: "linux"
labelString: "linux"
labels: "linux"
launchTimeoutSeconds: 300
launchTimeoutSecondsStr: "300"
mode: EXCLUSIVE
namePrefix: "cicd"
numExecutors: 1
numExecutorsStr: "1"
region: "https://www.googleapis.com/compute/v1/projects/$PROJECT/regions/europe-west1"
retentionTimeMinutes: 6
retentionTimeMinutesStr: "6"
runAsUser: "jenkins"
serviceAccountEmail: "jenkins-gce@$PROJECT.iam.gserviceaccount.com"
template: "https://www.googleapis.com/compute/v1/projects/$PROJECT/global/instanceTemplates/jenkins-template-1"
zone: "https://www.googleapis.com/compute/v1/projects/$PROJECT/zones/$REGION"
credentialsId: "$PROJECT"
instanceId: ""
projectId: "$PROJECT"
Setting up a test job
We are going to configure an small test job to verify that Jenkins is capable of creating agents on demand in GCP as well as configure the agents properly.
Create a new Freestyle Project and called “Test GCP agents”, for example:
Restrict where the job can be run and select the label for GCP agents (gcp in this example):
In build section select “Execute shell” and add this small script to check the agent IP and hostname.
#!/bin/bash
echo "This is my IP"
curl -s ifconfig.co
echo "This is my hostname"
hostname -f
It is as shown in the image:
Run the job and verify the output:
You should be able to see the new VM in the GCP console and the job will output information about the instance which is also available on the console.
With this test we verify that everything works correctly.
Final remarks
Configuring Jenkins to use agents on demand has a lot of advantages, both on the technical and financial side.
Different type of agents can be created depending on the job requirements and, since the agents are ephemeral, the cost is reduced to the minimal required to run the job.
The configuration is simple, the complex task of create new agents is abstracted and can be achieved by a simple click or trigger.
Do you want to tell us something?
CONTACT US!