DiscordTelegramTwitter
Search…
⌃K
Links

GCP (GKE)

​
These instructions will walk you through spinning up a GKE cluster and a CloudSQL database in GCP using Terraform. Afterwards, we will use helm and kubectl to deploy a Graph Node (+ other helper services) on top of them.

Prerequisites

  • A GCP project: https://cloud.google.com/resource-manager/docs/creating-managing-projects
  • Terraform: https://learn.hashicorp.com/tutorials/terraform/install-cli?in=terraform/gcp-get-started
  • GCloud CLI: https://cloud.google.com/sdk/docs/install
  • Kubectl: https://kubernetes.io/docs/tasks/tools/
  • Helm: https://helm.sh/docs/intro/install/
  • A Klaytn API Endpoint
This GCP deployment guide, completed with examples can be found on the Klaytn Indexing repo published and deployed by Bware Labs

Setup

  • Configure your GCloud CLI (and implicitly Terraform): https://cloud.google.com/sdk/docs/initializing
    $ gcloud auth application-default login
    $ gcloud config set project <PROJECT_ID>
  • Confirm gcloud is configured correctly:
    $ gcloud config list
    Additionally, you should have credentials stored under $HOME/.config/gcloud/application_default_credentials.json
  • Create the resources in GCP using Terraform:
    $ terraform init
    $ terraform apply --auto-approve -var="project=<YOUR_PROJECT_ID>"
  • Verify that the new resources have been created:
    • From CLI:
    gcloud container clusters list
    gcloud sql instances list
    • From UI:
      • GKE Cluster: https://console.cloud.google.com/kubernetes/list/overview?referrer=search&project=<PROJECT_ID>
      • CloudSQL Database: https://console.cloud.google.com/sql/instances?referrer=search&project=<PROJECT_ID>
  • Configure kubectl:
$ gcloud components install gke-gcloud-auth-plugin
$ gcloud container clusters get-credentials graph-indexer --region us-central1 --project <PROJECT_ID>
  • Confirm kubectl is configured:
kubectl get pods --all-namespaces
  • In the helm directory, within the indexing repo, find the helm/values.yaml and fill in the following missing values (search for # UPDATE THE VALUE comments):
    • The database host should be the PRIVATE IP printed by the terraform apply command. Alternatively you can find it by running gcloud sql instances describe graph-indexer.
    • The Klaytn network API endpoint should be something you have.
  • Deploy the services to Kubernetes:
kubectl create -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml
helm install graph-indexer . --create-namespace --namespace=graph-indexer
  • Confirm services were deployed:
helm list --all-namespaces
kubectl get pods --all-namespaces
  • Get the external IP of the Ingress controller:
kubectl get all -n ingress-controller
  • Navigate to the http://<EXTERNAL_IP>/subgraphs/graphql url in a browser to confirm it is working correctly
NOTE: : To destroy everything, simply run terraform destroy --auto-approve -var="project=<YOUR_PROJECT_ID>". If you get an error, run the same command again.
NOTE: : You can now return to the root documentation and continue the guide.

(OPTIONAL) Making everything production-ready

  • Terraform uses a local statefile. To make it persistent, you would have to create a GCS Bucket manually following the instructions on this page: https://www.terraform.io/language/settings/backends/gcs
NOTE: : The additional configuratios should go in provider.tf. After updating the terraform configs, you would have to run terraform init to start storing the state remotely.
  • Restrict network access
    • Modify the gke_management_ips variable in the infrastructure/gcp/variables.tf file to only allow access from your network.
    • Modify the nginx.ingress.kubernetes.io/whitelist-source-range variable in the helm/values.yaml file to only allow access from your network.
NOTE: : After updating the configs you would have to run both terraform apply and helm upgrade graph-indexer . --namespace=graph-indexer to apply them.
  • The database credentials are currently stored in plain text.
    • Remove the default values of postgresql_admin_user and postgresql_admin_password from infrastructure/gcp/variables.tf.
    • Define new values in a .tfvars file in the same directory like this:
    postgresql_admin_user = "<your-desired-username>"
    postgresql_admin_password = "<your-desired-password>"
    NOTE: : Do NOT commit .tfvars to source control.
    • Terraform apply the changes: terraform apply --auto-approve
    • Create a Kubernetes secret:
    kubectl create secret generic postgresql.credentials \
    --namespace=graph-indexer \
    --from-literal=username="<your-desired-username>" \
    --from-literal=password="<your-desired-password>"
    • Update the Helm charts to use the new secret:
      • Remove the username and password variables from values.yaml
      • Edit deployment-index-node and deployment-query-node and replace this part:
      - name: postgres_user
      value: {{ index .Values.CustomValues "postgress" "indexer" "username" }}
      - name: postgres_pass
      value: {{ index .Values.CustomValues "postgress" "indexer" "password" }}
      with:
      - name: postgres_user
      valueFrom:
      secretKeyRef:
      name: postgresql.credentials
      key: username
      - name: postgres_pass
      valueFrom:
      secretKeyRef:
      name: postgresql.credentials
      key: password
    • Apply the changes: helm upgrade graph-indexer . --namespace=graph-indexer
  • Configure a DNS entry and set up certificates for the kubernetes nginx-ingress: https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
NOTE: : We already have many components set up, like a Kubernetes cluster, the nginx-ingress controller, services, etc.
  • For monitoring:
    • A prometheus node which scrapes metrics from indexer nodes is available at http://<EXTERNAL_IP>/prometheus/graph. You could configure it as a data source in Grafana Cloud. Follow the instructions in this guide.
    • An alertmanager node is available at http://<EXTERNAL_IP>/alertmanager. You could configure it to send Prometheus alerts to Pagerduty.
      • Create a Pagerduty API key and configure it the alertmanager.yaml file. More information at: https://www.pagerduty.com/docs/guides/prometheus-integration-guide/
      • For creating alerts, use the prometheusRules.yaml snippetfile.
    NOTE: : Consider storing the Pagerduty API key in a Kubernetes secret.
NOTE: : You have to run helm upgrade graph-indexer . --namespace=graph-indexer to apply the changes.
Previous
AWS (EKS)
Next
Bare Metal
Last modified 4mo ago