A Persistent Volume is assigned a given capacity when it is created as part of the Persistent Volume Claim for dynamic provisioning. Over time, the volume will fill with data and the Kubernetes developer will need to make a decision on if they want to expand the volume to allow for additional capacity. There are two methods to expand a PV: Online Expansion and Offline Expansion.
In vCenter 7.0U2, VMware has added the capability for online volume expansion, meaning that the Persistent Volume Claim does not need to be brought out of service in order to be expanded as was the case in previous versions of Tanzu. This means that a simple patch command can dynamically expand the volume transparently without impacting the running Kubernetes application.
Source: https://docs.vmware.com/en/VMware-vS...2FF46F59B.html
Offline Persistent Volume Claim Expansion
For developers using relatively older versions of Kubernetes, the primary method to expand a Persistent Volume was accomplished through Offline Expansion. Basically this means that a Persistent Volume Claim must be unbound from a pod prior to expansion completing successfully. One way that this annoyance has been addressed by setting up deployments or statefulsets which have multiple persistent volumes sharing data amongst themselves, thereby enabling a single volume to be taken offline, expanded and then brought back into service sequentially until all volumes are at the new desired size.
For this simple example of offline expansion, we have a single replica mysql deployment with an attached PVC:
NAME STATUS VOLUME CAPACITY STORAGE CLASS
pvc-vvols-mysql Bound pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c 5Gi cns-vvolsBy describing the mysql deployment, we can see that the above PVC is bound to it:
$ kubectl describe deployment mysql-deployment
Name: mysql-deployment
Namespace: default
CreationTimestamp: Wed, 09 Jun 2021 14:41:11 -0700
Labels: app=mysql
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=mysql
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=mysql
Containers:
mysql:
Image: mysql:5.7
Port: 3306/TCP
Host Port: 0/TCP
Environment:
MYSQL_ROOT_PASSWORD: password
Mounts:
/var/lib/mysql from mysql-data (rw,path="mysql")
Volumes:
mysql-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: pvc-vvols-mysql
ReadOnly: falseThe below patch command is applied to the PVC to request the additional storage needed:
$ kubectl patch pvc pvc-vvols-mysql -p '{"spec": {"resources": {"requests": {"storage": "6Gi"}}}}'
However, since this version (Supervisor cluster 7.0U1 in this example) does not support online expansion, we see this message when we describe the PVC:
$ kubectl describe pvc pvc-vvols-mysql
Essentially, the FileSystemResizePending condition shown above means that the PVC is waiting to be unmounted from its pod before it can complete the resize operation. One way to accomplish this is simply to delete the mysql deployment, wait, and then recreate it.
kubectl delete deployment mysql-deployment
kubectl apply -f ./mysql-deployment.yaml
Now when we describe the PVC again after waiting a few moments for the mysql deployment to be recreated we can see that it has been successfully expanded under the events section:
An example of Offline Volume Expansion can be viewed in the below technical demo:
Online Persistent Volume Claim Expansion
As the name suggests, Online Volume expansion means that a given PV can have its capacity expanded without first being unattached from a pod or node. The key benefits of this expansion method are that users do not need to take the associated application or deployment offline and the volume expansion completes transparently without performance degradation. This is a much simpler operation than offline volume expansion and we will show the exact same operation as the previous example.
We started with a PVC of 5GB as before:
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY STORAGE CLASS
pvc-vvols-mysql Bound pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c 5Gi cns-vvolsThen run the same patch command against the PVC, increasing the amount of storage it is requesting from 5Gi to 6Gi:
$ kubectl patch pvc pvc-vvols-mysql -p '{"spec": {"resources": {"requests": {"storage": "6Gi"}}}}'
After waiting a few moments we can see that the PVC has increased its capacity to 6Gi:
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS
pvc-vvols-mysql Bound pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c 6Gi RWO cns-vvols Taking a closer look at the PVC via the describe command shows that it indeed increased the PV size while it remained mounted to the mysql-deployment node:
$ kubectl describe pvc
Name: pvc-vvols-mysql
Namespace: default
StorageClass: cns-vvols
Status: Bound
Volume: pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: csi.vsphere.vmware.com
volumehealth.storage.kubernetes.io/health: accessible
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 6Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By: mysql-deployment-5d8574cb78-xhhq5
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ExternalExpanding 52s volume_expand Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC.
Normal Resizing 52s external-resizer csi.vsphere.vmware.com External resizer is resizing volume pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c
Normal FileSystemResizeRequired 51s external-resizer csi.vsphere.vmware.com Require file system resize of volume on node
Normal FileSystemResizeSuccessful 40s kubelet, tkc-120-workers-mbws2-68d7869b97-sdkgh MountVolume.NodeExpandVolume succeeded for volume "pvc-f37c39fd-dbe9-4f27-abe8-bca85bf9e87c"
Below you can find a technical demo video showing a quick example of online volume expansion: