Deploy Embedded PDI in a Kubernetes cluster

Suppose you have a Kubernetes cluster up and running. You want to deploy a docker container (https://hub.docker.com/repository/docker/aliakkas/karatepdi). Karatepadi is a Spring Boot Rest application for automating ETL testing using Behaviour Driven Development framework Karate and embedded Pentaho Data Integration engine. You can use it to execute PDI job/transformation or running Karate feature file to produce cucumber reports.

Prerequisites

  • working kubernetes cluster with kubectl configured
  • locally installed kubectl and helm commands
  • RBAC authorisation already setup
  • docker image of karatepdi.1.0.0

Setting up karatepdi

Keep etl resources together by creating a namespace called ‘etl’.

  • create a file called etl/namespaces.yml with the following content:
kind: Namespace
apiVersion: v1
metadata:
  name: etl
  • apply and test the namespace exists
$ kubectl get namespaces -n etl
etl               Active   2d
  • deploy karatepdi

create a file called etl/pdi.yml with the following content:

apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: "pdi"
  namespace: "etl"
  labels:
    app: "pdi"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "pdi"
  template:
    metadata:
      labels:
        app: "pdi"
    spec:
      containers:
      - name: "karatepdi"
        image: "docker.io/aliakkas/karatepdi:1.0.0"
        imagePullPolicy: "IfNotPresent"
        resources: {}
        ports:
          - containerPort: 9999

status: {}
  • test the deployment
$ kubectl create -f ~/apps/docker/pdi.yml
verify that the PODs are running
$ kubectl get pods -n etl
NAME                   READY   STATUS    RESTARTS   AGE
pdi-5bfd55877d-6k2tz   1/1     Running   0          33m
  • Port forward and access karatepdi etl tool
// port forward to access the karatepdi ETL tool
$ kubectl --namespace etl port-forward pdi-5bfd55877d-6k2tz 9999
Forwarding from 127.0.0.1:9999 -> 9999
Forwarding from [::1]:9999 -> 9999

Go to http://localhost:9999