Google Kubernetes Engine

Prerequisites

1. Setup Kubernetes cluster

Supported version of Kubernetes: 1.17 and later.

You can use an existing cluster or create a new one using managed service Google Kubernetes Engine

  • Make your default storage class underlying provider resizable, so all PVCs for Hyperledger Fabric nodes that will be created by Catalyst Blockchain Manager can be resized when needed.

  • Add zone labels to Kubernetes nodes when a cluster is stretched across multiple availability zones to be able to schedule a Hyperledger Fabric node in a specific zone.

Define your cluster size considering the following minimum requirements and your business needs:

 1 . Minimal requirements for the Catalyst Blockchain Manager Hyperledger Fabric service for one organization — 1 instance with:
** 2 core CPU
** 4GB RAM
** 10GB disk space
2 . Each node (CA, orderer, or peer) that will be deployed consumes additional resources. Minimal requirements for one node:

Node

CPU

Memory Mi

Storage Gi

CA

0.1

128

1

Peer

0.1

128

1

Orderer

0.1

128

Deciding on the size of the cluster, please consider the expected load of the nodes and increase these values accordingly.
3 . Each chaincode runs as a separate pod and consumes additional resources (CPU and RAM).

2. Install Helm to your workstation

Installation manuals: helm.sh/docs/intro/install/ No customization is needed.

Supported version of Helm: 3.*

3. Configure ingress and DNS

Ingress controller

We recommend using the Traefik ingress controller.

Minimal supported version of Traefik: 2.3.

The ingress-controller is needed for traffic routing to expose Hyperledger Fabric nodes (peer, orderer, CA) as well as API and UI of the Catalyst Blockchain Manager Hyperledger Fabric service. All components are exposed through the port :443.

  • Ingress resources for Hyperledger Fabric nodes will be provisioned automatically by the Catalyst Blockchain Manager Hyperledger Fabric service operator upon creation/deletion of a node. These nodes require TLS passthrough enabled because of mutual TLS.

  • Ingress resources for API and UI will be created as a part of the helm package during the installation process. It can be configured in the helm values.

While Hyperledger Fabric nodes have self-signed TLS certificates managed by the Catalyst Blockchain Manager Hyperledger Fabric service operator, API and UI require a trusted TLS certificate.

You can choose any of the following options:

  1. Single load balancer of a cloud.google.com/load-balancing/docs/network [TCP Load Balancer (Level 4)] type which does TLS passthrough. TLS certificate is provisioned by cert-manager or provided in Secret Traefik is responsible for TLS termination for the API and UI.

  2. Two load balancers - The first one is the HTTPS Load Balancer (Level 7) with Managed SSL Certificate that is responsible for TLS termination for API and UI and the second one is TCP Load Balancer (Level 4) with TLS passthrough. In this scenario, additional DNS configuration is needed to route API and UI Traefik through the first LB and Hyperledger Fabric nodes traefik through the second one.

Single load balancer: NLB with TLS passthrough

No additional customization needed.

Two load balancers: HTTPS LB Level 7 + Managed SSL Certificate and TCP LB Level 4

By default when installing Traefik a TCP Load Balancer Level 4 is creted through creation of a Kuberenetes Service of type: LoadBalancer. In order to create an external HTTPS LB you would need to:

  1. *Create Managed SSL Certificate through UI or using networking.gke.io/v1 ManagedCertificate

  2. *Create Service of type ClusterIP with cloud.google.com/neg annotation which will result in creation of an Network Endpoint Group targeting GKE Node protocol

  3. *Create Backend Service that points to the NEG above

  4. *Create HTTPS Load Balancer with Managed SSL Certificate with Frontend on 443 port and the Backend Service

  5. *Create Firewall rule to allow incoming traffic for the LoadBalancer that is being creating

  6. *Configure DNS Routing for the domain so traffic goes to the created load balancer

Create a DNS record

DNS configuration differs depending on your choice:

Option

DNS records to be put

Comments

#1. Single load balancer

A *.example.com → TCP LB address

API, UI and nodes will go through this single record.

#2. Two load balancers

A example.com → HTTPS LB L7 address

A *.example.com → TCP LB L4 address

HTTPS LB L7 handles API and UI and does TLS termination using Managed Certificate. TCP LB L7 handles nodes traffic with TLS passthrough

With the following configuration each component is exposed on:

  • UI : example.com:443

  • API: example.com:443/api

  • Any HL Fabric node: <nodeName>.example.com:443

Create a namespace for the Catalyst Blockchain Manager Hyperledger Fabric service application

4. Create a namespace for the Catalyst Blockchain Manager Hyperledger Fabric service application

kubectl create ns ${ns_name}

where _ ${ns_name} — name of namespace (can be any).

4.1 Get the credentials to the Helm repository in the JFrog artifactory provided by the IntellectEU admin team.

4.2. Add the repo to Helm with the username and password provided:

helm repo add catbp <https://intellecteu.jfrog.io/artifactory/catbp-helm> --username ${ARTIFACTORY_USERNAME} --password ${ARTIFACTORY_PASSWORD}

As a result: "catbp" has been added to your repositories

5. Create an ImagePullSecret to access the Catalyst Blockchain Manager Hyperledger service deployable images

For example, create this Secret, naming it intellecteu-jfrog-access:

kubectl create secret intellecteu-jfrog-access regcred --docker-server=intellecteu-catbp-docker.jfrog.io --docker-username=${your-name} --docker-password=${your-password} --docker-email=${your-email} -n ${ns_name}

where:

  • ${your-name} — Docker username provided by IntellectEU.

  • ${your-password} — Docker password provided by IntellectEU.

  • ${your-email} — your email.

  • ${ns_name} — the namespace created for the Catalyst Blockchain Manager Hyperledger Fabric service on the previous step.

6. Deploy a message broker

Message broker is needed by the Catalyst Blockchain Manager Hyperledger Fabric service to schedule commands, emit events and control workflows.

Currently, only RabbitMQ is supported.

Version: 3.7 and later.

Since Google Cloud does not have native service for AMQP we recommend to use self-deployed solution or use any existing one from the Google Cloud Marketpace.

  • No specific configurations are needed.

  • 1GB RAM is recommended as a minimum setup.

  • Usually, load on the message broker is low so it does not require much resources. Free tier is applicable.

The Catalyst Blockchain Manager Hyperledger Fabric service requires a vhost and a user with full access for the vhost. Single queue will be propagated upon the Catalyst Blockchain Manager Hyperledger Fabric service startup.

Default configuration comes with TLS enabled, even for private VPC, make sure you enable amqp.tls option in helm values.

Backups are not required.

7. Deploy a database

We recommend using Google Cloud SQL managed service.

  • 1 CPU and 3 GB of RAM is recommended as a minimum setup.

  • No specific configurations are needed.

A database is required by the Catalyst Blockchain Manager Hyperledger Fabric service to support internal architecture for workflows as well as store users action logs.

No secure data is stored in the database.

The Catalyst Blockchain Manager Hyperledger Fabric service requires a database and a user with full read/write access to the database. Database tables will be provisioned in default schema on application startup.

In this example we will use PostgreSQL. Schema is ‘public’ by default.

Run these commands to provision a database on the recently deployed server:

CREATE DATABASE "catbp-org1";
CREATE USER "catbp" WITH ENCRYPTED PASSWORD 'catbp';
GRANT ALL PRIVILEGES ON DATABASE "catbp-org1" to "catbp";
Make sure to set up automatic backups so that all action logs of users won’t be lost in case of failure.

Catalyst Blockchain Manager supports cloud.google.com/sql/docs/mysql/sql-proxy [Google Cloud SQL Proxy] authentication mechanism into the database. It’s useful when default basic auth isn’t suitable. To enable Cloud SQL Proxy authentication the following values must be provided in the helm chart.

# -- enables the use of google cloud sdk
gcIAM:
  enabled: true

database:
  # -- database type. `postgres` or `mysql` can be specified here
  type: postgres
  # -- example values for postgres database. change them for your env
  host: "postgresql.postgresql"
  tls: true
  port: "5432"
  username: "test1"
  dbname: "test1"
  # -- ignore password and use AWS IAM authentication into the database
  sqlProxy: true

It’s expected that the Catalyst Blockchain Manager API pod will have sidecar running with google SQL proxy container.

Setup

Configure helm chart values

# -- address where application will be hosted. All created nodes (peers, orderers, cas) will have <NodeName>.proxy.<domainName> address
domainName: example.com
# -- auth config
auth:
  method: basic
  basic:
   username: <change to your console_login>
   password: <change to your console_pwd>
# -- external RabbitMQ Message broker parameters
amqp:
  host: <change to your amqp host>
  vhost: catbp-org1
  username: catbp
  password: catbp
  port: 5671
  tls: true #Depends in the setup
  # -- external database parameters
database:
  dbname: catbp-org1
  host: <change to your db host>
  username: catbp
  password: catbp
  port: 5432
  tls: false #Depends on the setup of Google Cloud SQL
  type: postgres
  # -- ignore password and use SQL Proxy authentication into the database
  sqlProxy: false
# -- enables the use of aws sdk
gcIAM:
  enabled: false
# -- ingressConfig addd Ingress resources for API and UI. By default, traefik is used.
ingressConfig:
  enabled: true
  tls:
    enabled: false # TLS termination is done by HTTPS LB L7 + Managed Certificate
api:
  image:
    repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console
    tag: "2.6"
  imagePullSecrets:
  - name: intellecteu-jfrog-access
operator:
  image:
   repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console
   tag: "2.6"
  imagePullSecrets:
  - name: intellecteu-jfrog-access
ui:
  image:
    repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console-ui
    tag: "2.6"
  imagePullSecrets:
  - name: intellecteu-jfrog-access
In case of using a single load balancer change ingressConfig to:
ingressConfig:
  # -- specify whether to create IngresRoute resource
  enabled: true
  tls:
    enabled: true
    certManager:
      enabled: true
      email: "<change to your EMAIL>"
      server: "https://acme-v02.api.letsencrypt.org/directory"

You can configure other helm chart values if needed.

You can see the full list of values here

=== Install the Catalyst Blockchain Manager Hypeledger Fabric service

helm upgrade --install ${fabric_release_name} catbp/fabric-console --values values.yaml -n ${ns_name} --version 2.5

where:

  • ${fabric_release_name}— name of the Catalyst Blockchain Manager Hypeledger Fabric service release. You can choose any name/alias. It is used to address for updating, deleting the Helm chart.

  • catbp/fabric-console - chart name, where “catbp” is a repository name, “fabric-console” is the chart name.

  • values.yaml— a values file.

  • ${ns_name}—ame of the namespace you’ve created before.

You can check the status of the installation by using these commands:

  • helm ls - check the "status" field of the installed chart.

Status “deployed” should be shown.
  • kubectl get pods — get the status of applications separately.

All pods statuses must be “running.”
  • kubectl describe pod $pod_name — get detailed information about pods.

The following rbac will be created:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: {{ include "fabric-console.fullname" . }}
  labels:
    {{- include "fabric-console.labels" . | nindent 4 }}
rules:
- apiGroups:
  - "*"
  resources:
  - namespaces
  - pods
  - services
  - persistentvolumes
  - persistentvolumeclaims
  - persistentvolumeclaims/finalizers
  - events
  - secrets
  - customresourcedefinitions
  - deployments
  - peersets
  - peersets/finalizers
  - peers
  - peers/status
  - peers/finalizers
  - orderingservices
  - orderingservices/finalizers
  - orderers
  - orderers/status
  - orderers/finalizers
  - chaincodeservices
  - chaincodeservices/status
  - chaincodeservices/finalizers
  - fabriccas
  - fabriccas/status
  - fabriccas/finalizers
  - configmaps
  {{- if .Values.openshiftRoute.enabled }}
  - routes
  - routes/custom-host
  {{- else }}
  - ingressroutetcps
  - ingressroutes
  {{- end }}
  verbs:
  - get
  - watch
  - list
  - create
  - update
  - patch
  - delete
  - create