Using Static Provisioning to Migrate a Persistent Volume Between Tanzu Kubernetes Clusters

User Guides for VMware Solutions

Audience

Public

Content Type

User Guides

Source Type

Documentation

Data mobility of Persistent Volumes is a critical element of what makes Kubernetes so attractive: being able to move data and their associated applications to almost any underlying infrastructure whether it be off-premises or on-premises. Within the scope of this guide, another key consideration is being able to move PVs from one Tanzu Kubernetes Cluster to another. For example, moving from Kubernetes release v1.17 to v1.20 in order to take advantage of some of the new features and functions available in the later release, such as online volume expansion.

This section will show how to move a persistent volume from a Tanzu Kubernetes Cluster running at v1.17 to Tanzu Kubernetes Cluster running at v1.20 within the same Tanzu Namespace. In it and as in earlier examples, we assume that we have a single instance of mysql with a persistent volume running via TKC v1.17. We wish to retain that persistent volume and redeploy the mysql instance in a new Tanzu Kubernetes Cluster at v1.20.

We will first need to pull the volumeHandle and storageclass values from the existing mysql PV. This information can be found in a couple of ways: either by describing the PV from within the 'old' TKC context or by switching the the vSphere Namespace within which the TKC resides and describing the Persistent Volume Claim. As shown in these examples, it is also expected that the Persistent Volume reclaim policy has been set to Retain previously.

$ kubectl describe pv

Name:            pvc-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb

Labels:          <none>

Annotations:     pv.kubernetes.io/provisioned-by: csi.vsphere.vmware.com

Finalizers:      [kubernetes.io/pv-protection]

StorageClass:    cns-vvols

Status:          Bound

Claim:           default/pvc-mysql

Reclaim Policy:  Retain

Access Modes:    RWO

VolumeMode:      Filesystem

Capacity:        5Gi

Node Affinity:   <none>

Message:

Source:

    Type:              CSI (a Container Storage Interface (CSI) volume source)

    Driver:            csi.vsphere.vmware.com

    FSType:            ext4

    VolumeHandle:      c88a36fb-6035-44b8-9eca-1b913d72954b-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb

    ReadOnly:          false

    VolumeAttributes:      storage.kubernetes.io/csiProvisionerIdentity=1623435682465-8081-csi.vsphere.vmware.com

                           type=vSphere CNS Block Volume

Events:                <none>

For the 2nd method of finding the volumeHandle and storageclass information, we see that both TKC clusters (tkc-117 and tkc-120) reside inside of the kg-ns1 vSphere namespace, so we will switch to that.

$ kubectl config get-contexts

CURRENT   NAME           CLUSTER        AUTHINFO                                       NAMESPACE

          10.21.132.25   10.21.132.25   wcp:10.21.132.25:administrator@vsphere.local

          kg-ns1         10.21.132.25   wcp:10.21.132.25:administrator@vsphere.local   kg-ns1

*         tkc-117        10.21.132.27   wcp:10.21.132.27:administrator@vsphere.local

          tkc-120        10.21.132.27   wcp:10.21.132.27:administrator@vsphere.local

$ kubectl config use-context kg-ns1

Switched to context "kg-ns1".

The name of the pvc is the volumeHandle value we require and the storageclass is also shown:

$ kubectl get pvc

NAME                                                                        STATUS   VOLUME                                     CAPACITY      STORAGECLASS 

a0394f5b-87d6-4716-be75-9f114498bb68-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c   Bound    pvc-c0a73e9b-77ba-471c-a70f-b945662cf1bb   6Gi           cns-vvols     

c88a36fb-6035-44b8-9eca-1b913d72954b-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb   Bound    pvc-b547a08b-b61d-4c27-8840-da5058eaf8f2   5Gi           cns-vvols

With the needed values for creating the Persistent Volume for the 'new' v1.20TKC cluster acquired, we can now remove the old v1.17 TKC. If the older v1.17 TKC is still needed for other uses; it does not need to be removed, however, the PVC and PV objects need to be deleted from the old v1.17TKC cluster before they can be used in the new v1.20 TKC.

$ kubectl config use-context tkc-117

Switched to context "tkc-117".

$ kubectl delete pvc pvc-mysql

persistentvolumeClaim "pvc-mysql" deleted

$ kubectl delete pv pvc-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb

persistentvolume "pvc-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb" deleted

Moving along with our example we now change contexts to the new v1.20 TKC cluster and create a yaml file to statically provision an existing persistent volume in the new v1.20 TKGS cluster which include the information gathered above from the persistent volume.

$ kubectl config use-context tkc-120

Switched to context "tkc-120".

apiVersion: v1

kind: PersistentVolume (1)

metadata:

  name: pv-mysql-new (2)

  annotations:

    pv.kubernetes.io/provisioned-by: csi.vsphere.vmware.com

spec:

  storageClassName: cns-vvols (3)

  capacity:

    storage: 5Gi (4)

  accessModes:

    - ReadWriteOnce

  persistentVolumeReclaimPolicy: Delete (5)                        
 claimRef:

    namespace: default

    name: pvc-mysql-new (6)

  csi:

    driver: "csi.vsphere.vmware.com"

    volumeAttributes:

      type: "vSphere CNS Block Volume"

    "volumeHandle":"c88a36fb-6035-44b8-9eca-1b913d72954b-9e7822f0-2a81-4e3f-b1a4-986b4e9f98bb" (7)

kind: This where we designate what the YAML file is creating. In this case, we are creating a PV.
name: This is the name we provide our PV.
storageClassName: This points the persistent volume to the StorageClass to use and is the same storage class the persistent volume was associated with.
storage: The amount of capacity we want to give the persistent volume. This needs to match the original persistent volume storage capacity.
persistentVolumeReclaimPolicy: Because the storageclass uses Delete as default, this value needs to be set to delete as well. It can be changed to retain after the persistent volume is created.
persistent volume claim: We need to provide a persistent volume claim name that we will use to match the persistent volume to in the next step.
volumeHandle: The volumehandle is an alpha-numeric code that we found earlier in this section from the old TKC v1.17 cluster.
Once the above YAML file has been applied and the persistent volume has been created, we can now bind it to a PVC. Note that the subtle difference between this PVC is that it is using static provisioning (while the earlier example in this guide was using dynamic provisioning) via calling out the volumeName from the PV we just created above.
```
kind: PersistentVolumeClaim (1)

apiVersion: v1

metadata:

  name: mysql-pvc-new (2)

spec:

  accessModes:

    - ReadWriteOnce

  resources:

    requests:

      storage: 5Gi (3)

  storageClassName: cns-vvols (4)

  volumeName: pv-mysql-new (5)
                  
```
kind: This where we designate what the YAML file is creating. In this case, we are creating a PVC.
name: This is the name we provide our PVC. This value must match the claimRef.Name value (#6) from the above persistent volume YAML.
storage: The amount of capacity we want to give the persistent volume. This needs to match the requested storage amount from the above persistent volume.
storageClassName: This points the persistent volume to the StorageClass to use and is the same storage class the persistent volume was associated with.
volumeName: This is the persistent volume name value (#2) from the above persistent volume YAML file.
After running the above YAML file we see that our static volume has been correct bound to the new Persistent Volume Claim in the v1.20 TKC cluster. It can now be used by a new Mysql instance deployed within the new TKC v1.20.
```
$ kubectl get pvc

NAME               STATUS   VOLUME            CAPACITY   ACCESS MODES   STORAGECLASS

mysql-pvc-new      Bound    pvc-mysql-new     5Gi        RWO            cns-vvols
```
A more in-depth example of this procedure can be viewed in this technical demo video: