Google Kubernetes Engine
Prerequisites
1. Setup Kubernetes cluster
Supported version of Kubernetes: 1.17 and later. You can use an existing cluster or create a new one using managed service Google Kubernetes Engine
|
Define your cluster size considering the following minimum requirements and your business needs:
1 . Minimal requirements for the Catalyst Blockchain Manager Hyperledger Fabric service for one organization — 1 instance with: ** 2 core CPU ** 4GB RAM ** 10GB disk space
2 . Each node (CA, orderer, or peer) that will be deployed consumes additional resources. Minimal requirements for one node:
Node |
CPU |
Memory Mi |
Storage Gi |
CA |
0.1 |
128 |
1 |
Peer |
0.1 |
128 |
1 |
Orderer |
0.1 |
128 |
Deciding on the size of the cluster, please consider the expected load of the nodes and increase these values accordingly. |
3 . Each chaincode runs as a separate pod and consumes additional resources (CPU and RAM).
2. Install Helm to your workstation
Installation manuals: helm.sh/docs/intro/install/ No customization is needed.
Supported version of Helm: 3.* |
3. Configure ingress and DNS
Ingress controller
We recommend using the Traefik ingress controller. Minimal supported version of Traefik: 2.3. Installation manuals: github.com/traefik/traefik-helm-chart |
The ingress-controller is needed for traffic routing to expose Hyperledger Fabric nodes (peer, orderer, CA) as well as API and UI of the Catalyst Blockchain Manager Hyperledger Fabric service. All components are exposed through the port :443.
-
Ingress resources for Hyperledger Fabric nodes will be provisioned automatically by the Catalyst Blockchain Manager Hyperledger Fabric service operator upon creation/deletion of a node. These nodes require TLS passthrough enabled because of mutual TLS.
-
Ingress resources for API and UI will be created as a part of the helm package during the installation process. It can be configured in the helm values.
While Hyperledger Fabric nodes have self-signed TLS certificates managed by the Catalyst Blockchain Manager Hyperledger Fabric service operator, API and UI require a trusted TLS certificate.
You can choose any of the following options:
-
Single load balancer of a cloud.google.com/load-balancing/docs/network [TCP Load Balancer (Level 4)] type which does TLS passthrough. TLS certificate is provisioned by cert-manager or provided in Secret Traefik is responsible for TLS termination for the API and UI.
-
Two load balancers - The first one is the HTTPS Load Balancer (Level 7) with Managed SSL Certificate that is responsible for TLS termination for API and UI and the second one is TCP Load Balancer (Level 4) with TLS passthrough. In this scenario, additional DNS configuration is needed to route API and UI Traefik through the first LB and Hyperledger Fabric nodes traefik through the second one.
Single load balancer: NLB with TLS passthrough
No additional customization needed.
Two load balancers: HTTPS LB Level 7 + Managed SSL Certificate and TCP LB Level 4
By default when installing Traefik a TCP Load Balancer Level 4 is creted through creation of a Kuberenetes Service of type: LoadBalancer
.
In order to create an external HTTPS LB you would need to:
-
*Create Managed SSL Certificate through UI or using
networking.gke.io/v1 ManagedCertificate
-
*Create Service of type
ClusterIP
withcloud.google.com/neg
annotation which will result in creation of an Network Endpoint Group targeting GKE Node protocol -
*Create Backend Service that points to the NEG above
-
*Create HTTPS Load Balancer with Managed SSL Certificate with Frontend on 443 port and the Backend Service
-
*Create Firewall rule to allow incoming traffic for the LoadBalancer that is being creating
-
*Configure DNS Routing for the domain so traffic goes to the created load balancer
Create a DNS record
DNS configuration differs depending on your choice:
Option |
DNS records to be put |
Comments |
#1. Single load balancer |
A *.example.com → TCP LB address |
API, UI and nodes will go through this single record. |
#2. Two load balancers |
A example.com → HTTPS LB L7 address A *.example.com → TCP LB L4 address |
HTTPS LB L7 handles API and UI and does TLS termination using Managed Certificate. TCP LB L7 handles nodes traffic with TLS passthrough |
With the following configuration each component is exposed on:
-
UI : example.com:443
-
API: example.com:443/api
-
Any HL Fabric node: <nodeName>.example.com:443
Create a namespace for the Catalyst Blockchain Manager Hyperledger Fabric service application
4. Create a namespace for the Catalyst Blockchain Manager Hyperledger Fabric service application
kubectl create ns ${ns_name}
where _ ${ns_name}
— name of namespace (can be any).
4.1 Get the credentials to the Helm repository in the JFrog artifactory provided by the IntellectEU admin team.
4.2. Add the repo to Helm with the username and password provided:
helm repo add catbp <https://intellecteu.jfrog.io/artifactory/catbp-helm> --username ${ARTIFACTORY_USERNAME} --password ${ARTIFACTORY_PASSWORD}
As a result: "catbp" has been added to your repositories
5. Create an ImagePullSecret to access the Catalyst Blockchain Manager Hyperledger service deployable images
For example, create this Secret, naming it intellecteu-jfrog-access:
kubectl create secret intellecteu-jfrog-access regcred --docker-server=intellecteu-catbp-docker.jfrog.io --docker-username=${your-name} --docker-password=${your-password} --docker-email=${your-email} -n ${ns_name}
where:
-
${your-name}
— Docker username provided by IntellectEU. -
${your-password}
— Docker password provided by IntellectEU. -
${your-email}
— your email. -
${ns_name}
— the namespace created for the Catalyst Blockchain Manager Hyperledger Fabric service on the previous step.
6. Deploy a message broker
Message broker is needed by the Catalyst Blockchain Manager Hyperledger Fabric service to schedule commands, emit events and control workflows.
Currently, only RabbitMQ is supported. Version: 3.7 and later. Since Google Cloud does not have native service for AMQP we recommend to use self-deployed solution or use any existing one from the Google Cloud Marketpace.
|
The Catalyst Blockchain Manager Hyperledger Fabric service requires a vhost and a user with full access for the vhost. Single queue will be propagated upon the Catalyst Blockchain Manager Hyperledger Fabric service startup.
Default configuration comes with TLS enabled, even for private VPC, make sure you enable amqp.tls option in helm values. |
Backups are not required. |
7. Deploy a database
We recommend using Google Cloud SQL managed service.
|
A database is required by the Catalyst Blockchain Manager Hyperledger Fabric service to support internal architecture for workflows as well as store users action logs.
No secure data is stored in the database. |
The Catalyst Blockchain Manager Hyperledger Fabric service requires a database and a user with full read/write access to the database. Database tables will be provisioned in default schema on application startup.
In this example we will use PostgreSQL. Schema is ‘public’ by default.
Run these commands to provision a database on the recently deployed server:
CREATE DATABASE "catbp-org1";
CREATE USER "catbp" WITH ENCRYPTED PASSWORD 'catbp';
GRANT ALL PRIVILEGES ON DATABASE "catbp-org1" to "catbp";
Make sure to set up automatic backups so that all action logs of users won’t be lost in case of failure. |
Catalyst Blockchain Manager supports cloud.google.com/sql/docs/mysql/sql-proxy [Google Cloud SQL Proxy] authentication mechanism into the database. It’s useful when default basic auth isn’t suitable. To enable Cloud SQL Proxy authentication the following values must be provided in the helm chart.
# -- enables the use of google cloud sdk
gcIAM:
enabled: true
database:
# -- database type. `postgres` or `mysql` can be specified here
type: postgres
# -- example values for postgres database. change them for your env
host: "postgresql.postgresql"
tls: true
port: "5432"
username: "test1"
dbname: "test1"
# -- ignore password and use AWS IAM authentication into the database
sqlProxy: true
It’s expected that the Catalyst Blockchain Manager API pod will have sidecar running with google SQL proxy container.
Setup
Configure helm chart values
# -- address where application will be hosted. All created nodes (peers, orderers, cas) will have <NodeName>.proxy.<domainName> address
domainName: example.com
# -- auth config
auth:
method: basic
basic:
username: <change to your console_login>
password: <change to your console_pwd>
# -- external RabbitMQ Message broker parameters
amqp:
host: <change to your amqp host>
vhost: catbp-org1
username: catbp
password: catbp
port: 5671
tls: true #Depends in the setup
# -- external database parameters
database:
dbname: catbp-org1
host: <change to your db host>
username: catbp
password: catbp
port: 5432
tls: false #Depends on the setup of Google Cloud SQL
type: postgres
# -- ignore password and use SQL Proxy authentication into the database
sqlProxy: false
# -- enables the use of aws sdk
gcIAM:
enabled: false
# -- ingressConfig addd Ingress resources for API and UI. By default, traefik is used.
ingressConfig:
enabled: true
tls:
enabled: false # TLS termination is done by HTTPS LB L7 + Managed Certificate
api:
image:
repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console
tag: "2.6"
imagePullSecrets:
- name: intellecteu-jfrog-access
operator:
image:
repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console
tag: "2.6"
imagePullSecrets:
- name: intellecteu-jfrog-access
ui:
image:
repository: intellecteu-catbp-docker.jfrog.io/catbp/fabric-platform/fabric-console-ui
tag: "2.6"
imagePullSecrets:
- name: intellecteu-jfrog-access
In case of using a single load balancer change ingressConfig to: |
ingressConfig:
# -- specify whether to create IngresRoute resource
enabled: true
tls:
enabled: true
certManager:
enabled: true
email: "<change to your EMAIL>"
server: "https://acme-v02.api.letsencrypt.org/directory"
You can configure other helm chart values if needed.
You can see the full list of values here
=== Install the Catalyst Blockchain Manager Hypeledger Fabric service
helm upgrade --install ${fabric_release_name} catbp/fabric-console --values values.yaml -n ${ns_name} --version 2.5
where:
-
${fabric_release_name}
— name of the Catalyst Blockchain Manager Hypeledger Fabric service release. You can choose any name/alias. It is used to address for updating, deleting the Helm chart. -
catbp/fabric-console
- chart name, where “catbp” is a repository name, “fabric-console” is the chart name. -
values.yaml
— a values file. -
${ns_name}
—ame of the namespace you’ve created before.
You can check the status of the installation by using these commands:
-
helm ls
- check the "status" field of the installed chart.
Status “deployed” should be shown. |
-
kubectl get pods
— get the status of applications separately.
All pods statuses must be “running.” |
-
kubectl describe pod $pod_name
— get detailed information about pods.
The following rbac will be created: |
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ include "fabric-console.fullname" . }}
labels:
{{- include "fabric-console.labels" . | nindent 4 }}
rules:
- apiGroups:
- "*"
resources:
- namespaces
- pods
- services
- persistentvolumes
- persistentvolumeclaims
- persistentvolumeclaims/finalizers
- events
- secrets
- customresourcedefinitions
- deployments
- peersets
- peersets/finalizers
- peers
- peers/status
- peers/finalizers
- orderingservices
- orderingservices/finalizers
- orderers
- orderers/status
- orderers/finalizers
- chaincodeservices
- chaincodeservices/status
- chaincodeservices/finalizers
- fabriccas
- fabriccas/status
- fabriccas/finalizers
- configmaps
{{- if .Values.openshiftRoute.enabled }}
- routes
- routes/custom-host
{{- else }}
- ingressroutetcps
- ingressroutes
{{- end }}
verbs:
- get
- watch
- list
- create
- update
- patch
- delete
- create