I have been on a three months boot camp on She Code Africa Cloud School- DevOps/SRE, and this has been an awesome opportunity for me to skill up on my cloud skills. There is no better way to give back, than to write on some of the things I learnt on my journey in this wonderful boot camp.

In this article, I will be writing in details, the process on how to create a GKE (Google Kubernetes Engine) clusters using Terraform and GitHub Actions CICD.

Prerequisites

• GitHub account

• GCP account with billing enabled

• Google SDK CLI installed in your local terminal

• Terraform for IAC

Before I dive Into process, I will like to give some brief definitions.

What is DevOPs?

DevOps is the combination of two words "development" and "Operations" which is a set of practice that works to automate and integrate the processes between software development and IT teams, so that they can build, test and release software faster and more reliably.

GitHub

GitHub is a code hosting web-based platform, used for version control and collaborations. Git makes the process of working with people simple and also easy to work on projects together from anywhere and different locations. In this article I will use GitHub for version control and also for CICD. Follow the link below to sign up on GitHub, if you do not have an account already.

(GitHub)

GitHub Actions

GitHub Actions is CICD tool, which is used to automate, customize and execute software development workflows right in your repository. The best part is that GitHub Actions is fully integrated into GitHub and therefore no need to look for external site, as everything about the workflow, from repository to pipelines can be managed in the same place as all other repository related features. GitHub actions enables you to create custom software development lifecycle workflows directly in your GitHub repository, great! right?

Terraform for IAC

Terraform is an open source infrastructure as code (IAC) tool developed by HashiCorp. Terraform allows provisioning of a wide variety of infrastructure resources through a collection of plugins. Configuration files are used to describe to Terraform the components needed to run applications. Terraform can be used to provision resources by using the simple commands "terraform init" , "terraform Plan" and "terraform apply" . To install Terraform, follow this link for instructions on different machine types installation.

GCP account with billing enabled

The cloud I will be creating the Kubernetes engine on will be GCP. If you wish to follow along in this tutorial, you can sign up for a free months-tier with $300.00 provided by Google to play around and learn new things in the cloud. You will not be billed until the $300.00 finishes. You can decide to upgrade to to a paid account or delete your billing account after.
follow this link to sign up GCP free tier

Create a project and make sure to enable billing for the project.

Google SDK CLI installed in your local terminal

Once the project is created, It is time to install and configure the Google Cloud SDK command line tool in your local terminal. This SDK provides the tools used to interact with the Google Cloud Platform REST API, and also the ability to create and manage GCP resources from a command-line interface.
To install, follow this link for instructions on how to to install and manage SDK on different machines: Google SDK .

After installing Google SDK On the local terminal server, mine is (Ubuntu) Linux distro, initialize Google Cloud by running the command:

$ gcloud init

It is worthy to note that GCP (Google Cloud Platform) permissions and roles are managed by GCP Identity and Access Management Cloud (IAM). In order for Terraform to be able to communicate with GCP to provision resources, some advanced permissions needs to be authorized, and also a service account needs to created with the proper Google Cloud API permissions. These Google Cloud API Permissions needs to be enabled by running the following commands:

$ gcloud services enable servicenetworking.googleapis.com &&
$ gcloud services enable cloudresourcemanager.googleapis.com &&
$ gcloud services enable container.googleapis.com &&
$ gcloud services enable compute.googleapis.com

Created a Google service account by running the following command:

$ gcloud iam service-accounts create <service-account name>

For example:

$ gcloud iam service-accounts create terraform

In the command above, terraform is the name of the service account I created. Make sure to change that to your preferred name without the brackets.

Next, grant the required roles below for the service account, so that Terraform can create GKE clusters and any other resources needed.

$ gcloud projects add-iam-policy-binding <project_name> --member serviceAccount:<service_account_name>@<project_name>.iam.gserviceaccount.com --role roles/iam.serviceAccountUser
$ gcloud projects add-iam-policy-binding <project_name> --member serviceAccount:<service_account_name>@<project_name>.iam.gserviceaccount.com --role roles/resourcemanager.projectIamAdmin
$ gcloud projects add-iam-policy-binding <project_name> --member serviceAccount:<service_account_name>@<project_name>.iam.gserviceaccount.com --role roles/container.admin
$ gcloud projects add-iam-policy-binding <project_name> --member serviceAccount:<service_account_name>@<project_name>.iam.gserviceaccount.com --role roles/compute.admin

Note:- Project_name: this is the project Id created in the beginning.

Creat and downloaded a .json key for the service account with the command below. This key will be used for authentication for terraform and the CICD platform (GitHub Actions)

gcloud iam service-accounts keys create <service-key.json> --iam-account=<service_account_name>@<project_name>.iam.gserviceaccount.com

For example see the command with parameters below:

$ gcloud iam service-accounts keys create tf-key.json --iam-account=terraform@terraform-310215.iam.gserviceaccount.com

Note:

1. Make sure to change the names in bracket into your preferred name (Do not add the brackets to the command)

2. Remember the file were this key is downloaded into in your directory folder, this will be used later.

Terraform stores the state of infrastructure in a local state file, and the best practice is to store Terraform state in a remote storage. For this process, I configured Terraform to store the state in a Google Cloud storage Bucket, which I created by executing the following command in my terminal still:

gsutil mb -p <project_name> -c regional -l <location> gs://<bucket_name>/

1. Project_name: this is the project Id I created in the beginning.

2. Location: You can choose any region and zone. The complete list of available locations can be found on the Google Cloud Documentation.

3. Bucket_name: Chose a name that is globally unique to you.

Example of command used to create bucket with parameters bellow:

$ gsutil mb -p terraform-310215 -c regional -l us-central1-a gs://tf-bucketulo/

Activate object versioning on the bucket created, this is to allow for state recovery in the case of accidental deletions or any human error:

gsutil versioning set on gs://<bucket_name>/

Grant read/write permissions on the bucket to the service account, by running the following command:

$ gsutil iam ch serviceAccount:<service_account_name>@<project_name>.iam.gserviceaccount.com:legacyBucketWriter gs://<bucket_name>/

Terraform configuration files

Now moving on to the creation of terraform configuration files. First we need to create a GitHub repository for code versioning. Go to GitHub and create a repository. If you do not know how to create a repository follow this tutorial link:-

How to create a GitHub repo

After the creation of the repo, return back to your local terminal and clone into the repository you just created.

git clone https://github.com/ulomau/3tier-web-deployment-with-devops-tool.git

If you do not know how to clone a repo, please follow this GitHub doc on how to clone a repository. How to clone a GitHub repo

Change directory in your terminal into the repository created with the following command: example:-

$ cd 3tier-web-deployment-with-devops-tool

Next, create the following files for the Terraform configuration files by running this command if in Ubuntu

$ touch backend.tf main.tf providers.tf variables.tf

This is how the files arrangement will look like:

- 
 backend.tf

- 
 main.tf

- 
 providers.tf

- 
 variables.tf

This is a brief description of each of the file:

1. backend.tf: this is the backend where Terraform state will be stored

2. main.tf : this is for resources configurations.

3. provider.tf : This will contain the providers we will be using, like Google Cloud and GKE

4. variables.tf: this is were the variables will be defined.

Open up the backend.tf file with an editor, eg nano editor. Run the command

$ nano backend.tf

Add the following code bellow into it.

terraform {
  backend "gcs" {
    bucket = "bucket name"
    prefix = "terraform/state"
  }
}

To let GCP authenticate Terraform, we need to export a new shell variable called GOOGLE_APPLICATION_CREDENTIALS into the environment. This uses the service account key created above for the authentication. Run the command:

$ export GOOGLE_APPLICATION_CREDENTIALS=~/<name of the file were the service key is saved>

to confirm that Terraform is able to authenticate with Google Cloud to create the initial state, run the following command:

$ terraform init

If you see this line below among the output then Terraform has been initialized

Terraform has been successfully initialized!

Next, open up main.tf file with an editor and add the following code below into it

resource "google_container_cluster" "default" {
  name        = var.name
  project     = var.project
  description = "Demo GKE Cluster"
  location    = var.location

  remove_default_node_pool = true
  initial_node_count       = var.initial_node_count

  master_auth {
    username = ""
    password = ""

    client_certificate_config {
      issue_client_certificate = false
    }
  }
}

resource "google_container_node_pool" "default" {
  name       = "${var.name}-node-pool"
  project    = var.project
  location   = var.location
  cluster    = google_container_cluster.default.name
  node_count = 1

  node_config {
    preemptible  = false
    machine_type = var.machine_type

    metadata = {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}

open up providers.tf and add the following code below into it and save ( ctrl x then y and save)

provider "google" {
  project     = "terraform-310215"
  region      = "us-central1"
  zone        = "us-central1-a"
}
provider "google-beta" {
  project     = "terraform-310215"
  region      = "us-central1"
  zone        = "us-central1-a"
}

Open up variable.tf and ad the following code and save it.

variable "name" {
  default = "tf-cluster"
}

variable "project" {
  default = "terraform-310215"
}

variable "location" {
  default = "us-central1"
}

variable "initial_node_count" {
  default = 1
}

variable "machine_type" {
  default = "n1-standard-2"
}

Note: make sure to use the name, location, machine type of your choice.

Initialize Terraform again

$ terraform init

We have to commit the codes into our GitHub repository, but let us add some of the files we do not wish to be published on GitHub into gitignore files. One of the files to be added is GCP service account key which is a secret key and shouldn't be exposed.

Run the following command:

$ echo ".terraform"  "the j.son key file name">> .gitignore

Note: replace the "the j.son key file name" with the name of the file you saved your key in.

Also run the command to rewrites the .tf files to a canonical format.

$ terraform fmt

We can proceed and run "terraform plan" and "terraform apply" to create the GKE on GCP, but because we want to use GitHub Actions for CICD we will push the codes to our GitHub repo.

Run the following command to commit the codes to GitHub repo:

git add --all
git commit -m "terraform files"
git push -u origin master

GitHub

Head over to GitHub, open the repository that we just committed codes to. Click on settings to add the GCP service key value as an environment and a secret. Before the key can be added, it needs to be altered a bit in order to be stored in the variable. Because we need to remove all the newline characters from the .json key value. To do that, use the command below:

First open the file with vim editor vi j.son-key

press :

Add the following 
%s;\n; ;g
Press enter.

press : again

type wq!

Save the value of this credential (.json key) in GitHub Secret (Settings → Secrets → New Secret ) and name it GOOGLE_CREDENTIALS.

Also save this credential in GitHub Secret (Settings → environment → New environment ) name it GOOGLE_CREDENTIALS. Next, click on Actions, click on Workflows made for your repository "Suggested" , delete the content in it and add the following code below:

name: 'Terraform CI'
on: 
   push
          branches:
          - main:
    pull_request: 
jobs:
  terraform:
    name: 'Terraform'
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout'
        uses: actions/checkout@v2
      - name: 'Terraform Setup'
        uses: hashicorp/setup-terraform@v1
      - name: 'Terraform Init'
        run: terraform init
        env
             GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
      - name: 'Terraform Format'
        run: terraform fmt 
        env
            GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
      - name: 'Terraform Plan'
        run: terraform plan
        env
            GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
      - name: Terraform Apply
      if: github.ref == 'refs/heads/main' && github.event_name == 'push'
      run: terraform apply -auto-approve
      env
           GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}

This is a yaml configuration file that define all the actions to be done in the workflow of GitHub pipeline. simply give it a name of your choice and then click on the start commit button on the top right of the page.

If everything goes well this will trigger a build for the creation of GKE on GCP using the Terraform .tf Files in our GitHub repo.

Head over to GCP to confirm that a GKE cluster has been created. Hurrah! you have just created a GKE using Terraform as IAC and GitHub Actions as CICD tool.

(Hurrah!)

Conclusion

DevOps provides a more reliable and faster way for software development lifecycle. In this article, I covered how DevOps tool, Terraform which is an IAC and GitHub Actions a CICD pipeline platform enables a flexible, native continuous and integration provisioning of resources on GCP.

What next?

Thank you for taking out time to read my article. I will be coming your way soon with an article titled "Creating a Google cloud VPC with private and public subnets using Bash scripting for automation" .

I also have lined up articles to publish from the plenty things I learnt from She Code Africa Cloud school boot camp.

Meaning of abbreviations used in this article

• GCP : Google Cloud Platform

• GKE : Google Kubernetes Engine

• CICD: Continuous Integration Continuous Deployment

• IAC: Infrastructure as a code

• IAM : Identity and Access Management

• Repo : Repository

• CLI : Command Line Interface

• VPC : Virtual Private Cloud

• YAML : YAML Ain't Markup Language