http://www.clue2solve.io/tanzu/2020/07/14/install-ksm-and-configure-the-cf-marketplace.html
Steps
It's assumed you have already installed KSM into your Kubernetes Cluster
as shown below. If not please refer to the documentation to get this done
first
$ kubectl get all -n ksm NAME READY STATUS RESTARTS AGE pod/ksm-chartmuseum-78d5d5bfb-2ggdg 1/1 Running 0 15d pod/ksm-ksm-broker-6db696894c-blvpp 1/1 Running 0 15d pod/ksm-ksm-broker-6db696894c-mnshg 1/1 Running 0 15d pod/ksm-ksm-daemon-587b6fd549-cc7sv 1/1 Running 1 15d pod/ksm-ksm-daemon-587b6fd549-fgqx5 1/1 Running 1 15d pod/ksm-postgresql-0 1/1 Running 0 15d NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ksm-chartmuseum ClusterIP 10.100.200.107 <none> 8080/TCP 15d service/ksm-ksm-broker LoadBalancer 10.100.200.229 10.195.93.188 80:30086/TCP 15d service/ksm-ksm-daemon LoadBalancer 10.100.200.222 10.195.93.179 80:31410/TCP 15d service/ksm-postgresql ClusterIP 10.100.200.213 <none> 5432/TCP 15d service/ksm-postgresql-headless ClusterIP None <none> 5432/TCP 15d NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ksm-chartmuseum 1/1 1 1 15d deployment.apps/ksm-ksm-broker 2/2 2 2 15d deployment.apps/ksm-ksm-daemon 2/2 2 2 15d NAME DESIRED CURRENT READY AGE replicaset.apps/ksm-chartmuseum-78d5d5bfb 1 1 1 15d replicaset.apps/ksm-ksm-broker-6db696894c 2 2 2 15d replicaset.apps/ksm-ksm-broker-8645dfcf98 0 0 0 15d replicaset.apps/ksm-ksm-daemon-587b6fd549 2 2 2 15d NAME READY AGE statefulset.apps/ksm-postgresql 1/1 15d
1. let's start by getting the Broker IP address which when installed using
LoadBalancer type can be retrieved as shown below.
$ kubectl get service ksm-ksm-broker -n ksm
-o=jsonpath='{@.status.loadBalancer.ingress[0].ip}'
10.195.93.188
2. Upgrade your Helm release by running the following using the IP address
from above
$ export BROKER_IP=$(kubectl get service ksm-ksm-broker -n ksm
-o=jsonpath='{@.status.loadBalancer.ingress[0].ip}')
$ helm upgrade ksm ./ksm -n ksm --reuse-values \
--set
cf.brokerUrl="http://$BROKER_IP" \
--set cf.brokerName=KSM
\
--set
cf.apiAddress="https://api.system.run.haas-210.pez.pivotal.io" \
--set cf.username="admin"
\
--set
cf.password="admin-password"
3. Next we configure the ksm CLI. You can download the CLI from
here
configure-ksm-cli.sh
export KSM_IP=$(kubectl get service ksm-ksm-daemon -n ksm
-o=jsonpath='{@.status.loadBalancer.ingress[0].ip}')
export KSM_TARGET=http://$KSM_IP:$(kubectl get svc ksm-ksm-daemon -n
ksm -o=jsonpath='{@.spec.ports[0].port}')
export KSM_USER=admin
export KSM_PASSWORD=$(kubectl get secret -n ksm ksm-ksm-daemon
-o=jsonpath='{@.data.SECURITY_USER_PASSWORD}' | base64 --decode)
4. Verify ksm CLI is configured correctly
$ ksm version
Client Version [0.10.80]
Server Version [0.10.80]
5. Create a YAML file for the KSM service account and
ClusterRoleBinding using the following YAML:
ksm-sa.yml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: ksm-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: ksm-cluster-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: ksm-admin
namespace: kube-system
Apply as follows
$ kubectl apply -f ksm-sa.yml
6. You need a cluster credential file to register and set default
Kubernetes clusters that is done as follows
cluster-creds.sh
export kube_config="/Users/papicella/.kube/config"
cluster=`grep current $kube_config|sed "s/ //g"|cut -d ":" -f
2`
echo "Using cluster $cluster"
export server=`grep -B 2 "name: $cluster" $kube_config \
|grep server|sed "s/ //g"|sed "s/^[^:]*://g"`
export certificate=`grep -B 2 "name: $cluster" $kube_config \
|grep certificate|sed "s/ //g"|sed "s/.*://"`
export secret_name=$(kubectl get serviceaccount ksm-admin \
--namespace=kube-system -o
jsonpath='{.secrets[0].name}')
export secret_val=$(kubectl --namespace=kube-system get secret
$secret_name \
-o jsonpath='{.data.token}')
export secret_val=$(echo ${secret_val} | base64 --decode)
cat > cluster-creds.yaml << EOF
token: ${secret_val}
server: ${server}
caData: ${certificate}
EOF
echo ""
echo "ready to roll!!!!"
echo ""
Before running this script it's best to make sure you have targeted the
correct K8s cluster you wish to. You can run a command as follows to
verify that
$ kubectl config current-context
tas4k8s
7. Now we have a "cluster-creds.yaml" file we can go ahead and
register the Kubernetes cluster with KSM as follows
$ ksm cluster register ksm-svcs ./cluster-creds.yaml
$ ksm cluster set-default ksm-svcs
Verify as follows:
$ ksm cluster list
CLUSTER NAME IP ADDRESS
DEFAULT
ksm-svcs
https://tas4k8s.run.haas-210.pez.pivotal.io:8443
true
8. Now we can go ahead and create a Marketplace offering for MySQL. To do
that we will use the Bitnami MySQL chart as shown below
$ git clone https://github.com/bitnami/charts.git
$ cd ./charts/bitnami/mysql
** create bind.yaml as follows which is required so our service
binding from Tanzu Application Service will inject the right JSON we
are expecting or requiring at bind time **
$ cat bind.yaml
template: |
local filterfunc(j) = std.length(std.findSubstr("mysql",
j.name)) > 0;
local s1 = std.filter(filterfunc, $.services);
{
hostname:
s1[0].status.loadBalancer.ingress[0].ip,
name: s1[0].name,
jdbcUrl: "jdbc:mysql://" + self.hostname +
"/my_db?user=" + self.username + "&password=" + self.password +
"&useSSL=false",
uri: "mysql://" + self.username + ":" + self.password +
"@" + self.hostname + ":" + self.port + "/my_db?reconnect=true",
password:
$.secrets[0].data['mysql-root-password'],
port: 3306,
username: "root"
}
$ helm package .
# cd ..
$ ksm offer save ./mysql ./mysql/mysql-6.14.7.tgz
Verify MySQL is now part of the offer list as follows
$ ksm offer list MARKETPLACE NAME INCLUDED CHARTS VERSION PLANS rabbitmq rabbitmq 6.18.1 [persistent ephemeral] mysql mysql 6.14.7 [default]
9. Now we need to login as an ADMIN user
Verify you are logged in as admin user using the CF CLI:
$ cf target
api endpoint:
https://api.system.run.haas-210.pez.pivotal.io
api version: 2.151.0
user: admin
org: system
space: development
10. At this point you can see the KSM service broker registered with
TAS4K8s as follows
$ cf service-brokers
Getting service brokers as admin...
name url
KSM http://10.195.93.188
11. Enable access to the MySQL service as follows
$ cf enable-service-access mysql
Verify it's enabled:
$ cf service-access
Getting service access as admin...
broker: KSM
service plan
access orgs
mysql default
all
rabbitmq ephemeral all
rabbitmq persistent all
12. At this point it's best to log out of admin and log back in as a user
that is not admin
$ cf target
api endpoint:
https://api.system.run.haas-210.pez.pivotal.io
api version: 2.151.0
user: pas
org: apples-org
space: development
13. Create a MySQL service as follows. I passing in some JSON to indicate
that my K8s cluster support's a LoadBalancer type so use that as part of the
creation of the service.
$ cf create-service mysql default pas-mysql -c
'{"service":{"type":"LoadBalancer"}}'
14. Check that the service has created correctly it will take a few
minutes
$ cf services
Getting services in org apples-org / space development as
pas...
name service plan
bound apps last
operation broker upgrade
available
pas-mysql mysql default
my-springboot-app create succeeded
KSM no
15. Your service is created in it's own K8s namespace BUT that may not be
the case at some point.
$ kubectl get all -n ksm-2e526124-11a3-4d38-966c-b3ffd45471d7 NAME READY STATUS RESTARTS AGE pod/k-wqo5mubw-mysql-master-0 1/1 Running 0 15d pod/k-wqo5mubw-mysql-slave-0 1/1 Running 0 15d NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/k-wqo5mubw-mysql LoadBalancer 10.100.200.12 10.195.93.192 3306:30563/TCP 15d service/k-wqo5mubw-mysql-slave LoadBalancer 10.100.200.130 10.195.93.191 3306:31982/TCP 15d NAME READY AGE statefulset.apps/k-wqo5mubw-mysql-master 1/1 15d statefulset.apps/k-wqo5mubw-mysql-slave 1/1 15d
16. At this point we can now test our new MySQL service we created and use
a Spring Boot application to test this out with.
The following GitHub repo can be used for that. Ignore the steps to create
a service as you have already done that
Finally to define service plans see the link below
More Information
Container Services Manager(KSM)
Tanzu Application Service for Kubernetes
