How much space should I reserve for Portworx container images in my local registry?
KB0016941
Pure Product
Portworx
Pure Feature(s)
Kubernetes
Question
How much space should I reserve for Portworx container images in my local registry?
Answer
This is a good question.
You see, Portworx uses images from three different registries: docker.io, quay.io, and registry.k8s.io, to calculate the size that you would require to reserve for all our images we have to be creative.
Here is a shell script to perform this task.
You have to specify your docker hub credentials, as well as the intended Portworx version to check, and the Kubernetes version:
#!/bin/bash
# Docker Hub credentials (optional, for authenticated access)
DOCKER_USERNAME="your_username"
DOCKER_PASSWORD="your_password"
K8S_VERSION="1.29.2"
PX_VERSION="3.1.1"
# URL to download the script, from air-gapped script
# https://docs.portworx.com/portworx-enterprise/install-portworx/kubernetes/airgapped/air-gapped-baremetal
url="https://install.portworx.com/${PX_VERSION}/air-gapped?kbver=${K8S_VERSION}"
total_size_bytes=0
# Function to convert bytes to human-readable format
human_readable_size() {
local size=$1
local units=("B" "KB" "MB" "GB" "TB")
local unit_index=0
while (( size >= 1024 && unit_index < ${#units[@]} - 1 )); do
size=$((size / 1024))
unit_index=$((unit_index + 1))
done
echo "${size}${units[$unit_index]}"
}
# Function to get the size of a Docker Hub container image
get_dockerhub_image_size() {
local image=$1
# Extract the repository and tag from the image
repository=$(echo $image | cut -d'/' -f2- | cut -d':' -f1)
tag=$(echo $image | cut -d':' -f2)
# Authenticate with Docker Hub if necessary
auth_token=$(curl -s -u $DOCKER_USERNAME:$DOCKER_PASSWORD "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$repository:pull" | jq -r .token)
auth_header="Authorization: Bearer $auth_token"
# Fetch the image manifest
manifest=$(curl -s -L -H "Accept: application/vnd.docker.distribution.manifest.v2+json" -H "$auth_header" "https://registry-1.docker.io/v2/$repository/manifests/$tag")
# Check if manifest is null
if [[ -z $manifest || $manifest == "null" ]]; then
echo "Failed to fetch manifest for $image"
return
fi
# Extract the layer sizes
layer_sizes=$(echo $manifest | jq -r '.layers[].size')
# Check if layer_sizes is null
if [[ -z $layer_sizes || $layer_sizes == "null" ]]; then
echo "Failed to fetch layer sizes for $image"
return
fi
local total_size=0
# Loop through each layer size and sum them up
for size in $layer_sizes
do
total_size=$((total_size + size))
done
# Add to the overall total size
total_size_bytes=$((total_size_bytes + total_size))
# Convert size from bytes to a human-readable format
size_readable=$(human_readable_size $total_size)
echo "Size of $image: $size_readable"
}
# Function to get the size of a quay.io container image
get_quay_image_size() {
local image=$1
# Extract the repository and tag from the image
repository=$(echo $image | cut -d'/' -f2- | cut -d':' -f1)
tag=$(echo $image | cut -d':' -f2)
# Fetch the image manifest
manifest=$(curl -s -L -H "Accept: application/vnd.docker.distribution.manifest.v2+json" "https://quay.io/v2/$repository/manifests/$tag")
# Check if manifest is null
if [[ -z $manifest || $manifest == "null" ]]; then
echo "Failed to fetch manifest for $image"
return
fi
# Extract the layer sizes
layer_sizes=$(echo $manifest | jq -r '.layers[].size')
# Check if layer_sizes is null
if [[ -z $layer_sizes || $layer_sizes == "null" ]]; then
echo "Failed to fetch layer sizes for $image"
return
fi
local total_size=0
# Loop through each layer size and sum them up
for size in $layer_sizes
do
total_size=$((total_size + size))
done
# Add to the overall total size
total_size_bytes=$((total_size_bytes + total_size))
# Convert size from bytes to a human-readable format
size_readable=$(human_readable_size $total_size)
echo "Size of $image: $size_readable"
}
# Function to get the size of a registry.k8s.io container image
get_k8s_image_size() {
local image=$1
# Extract the repository and tag from the image
repository=$(echo $image | cut -d'/' -f2- | cut -d':' -f1)
tag=$(echo $image | cut -d':' -f2)
# Fetch the image manifest
manifest=$(curl -s -L -H "Accept: application/vnd.docker.distribution.manifest.v2+json" "https://registry.k8s.io/v2/$repository/manifests/$tag")
# Check if manifest is null
if [[ -z $manifest || $manifest == "null" ]]; then
echo "Failed to fetch manifest for $image"
return
fi
# Extract the layer sizes
layer_sizes=$(echo $manifest | jq -r '.layers[].size')
# Check if layer_sizes is null
if [[ -z $layer_sizes || $layer_sizes == "null" ]]; then
echo "Failed to fetch layer sizes for $image"
return
fi
local total_size=0
# Loop through each layer size and sum them up
for size in $layer_sizes
do
total_size=$((total_size + size))
done
# Add to the overall total size
total_size_bytes=$((total_size_bytes + total_size))
# Convert size from bytes to a human-readable format
size_readable=$(human_readable_size $total_size)
echo "Size of $image: $size_readable"
}
# Main function to get the size of a container image
get_image_size() {
local image=$1
if [[ $image == docker.io/* ]]; then
get_dockerhub_image_size $image
elif [[ $image == quay.io/* ]]; then
get_quay_image_size $image
elif [[ $image == registry.k8s.io/* ]]; then
get_k8s_image_size $image
else
echo "Unsupported registry for image: $image"
fi
}
# Download the script content
script_content=$(curl -s $url)
# Extract the image lines
image_lines=$(echo "$script_content" | grep '^IMAGES=')
# Read the images from the extracted lines and loop through each one
while read -r line; do
image=$(echo $line | cut -d' ' -f2 | tr -d '"')
echo "Checking size for image: $image"
get_image_size $image
echo "--------------------------------"
done <<< "$image_lines"
# Convert total size from bytes to a human-readable format
total_size_readable=$(human_readable_size $total_size_bytes)
echo "Total size of all images: $total_size_readable"
You will get something like this as output:
./checkSize.sh
Checking size for image: docker.io/portworx/px-enterprise:3.1.1
Size of docker.io/portworx/px-enterprise:3.1.1: 1GB
--------------------------------
Checking size for image: docker.io/portworx/oci-monitor:3.1.1
Size of docker.io/portworx/oci-monitor:3.1.1: 128MB
--------------------------------
Checking size for image: docker.io/openstorage/stork:23.11.1
Size of docker.io/openstorage/stork:23.11.1: 1007MB
--------------------------------
Checking size for image: docker.io/openstorage/cmdexecutor:23.11.1
Size of docker.io/openstorage/cmdexecutor:23.11.1: 44MB
--------------------------------
Checking size for image: docker.io/portworx/autopilot:1.3.14
Size of docker.io/portworx/autopilot:1.3.14: 52MB
--------------------------------
Checking size for image: docker.io/purestorage/ccm-go:1.2.2
Size of docker.io/purestorage/ccm-go:1.2.2: 26MB
--------------------------------
Checking size for image: docker.io/purestorage/realtime-metrics:1.0.23
Size of docker.io/purestorage/realtime-metrics:1.0.23: 25MB
--------------------------------
Checking size for image: docker.io/purestorage/telemetry-envoy:1.1.11
Size of docker.io/purestorage/telemetry-envoy:1.1.11: 49MB
--------------------------------
Checking size for image: docker.io/purestorage/log-upload:px-1.1.8
Size of docker.io/purestorage/log-upload:px-1.1.8: 19MB
--------------------------------
Checking size for image: docker.io/portworx/px-operator:23.10.5
Size of docker.io/portworx/px-operator:23.10.5: 69MB
--------------------------------
Checking size for image: registry.k8s.io/pause:3.1
Size of registry.k8s.io/pause:3.1: 306KB
--------------------------------
Checking size for image: registry.k8s.io/kube-controller-manager-amd64:v1.29.2
Size of registry.k8s.io/kube-controller-manager-amd64:v1.29.2: 31MB
--------------------------------
Checking size for image: registry.k8s.io/kube-scheduler-amd64:v1.29.2
Size of registry.k8s.io/kube-scheduler-amd64:v1.29.2: 17MB
--------------------------------
Checking size for image: docker.io/portworx/portworx-dynamic-plugin:1.1.0
Size of docker.io/portworx/portworx-dynamic-plugin:1.1.0: 85MB
--------------------------------
Checking size for image: docker.io/nginxinc/nginx-unprivileged:1.25
jq: error (at <stdin>:1): Cannot iterate over null (null)
Failed to fetch layer sizes for docker.io/nginxinc/nginx-unprivileged:1.25
--------------------------------
Checking size for image: registry.k8s.io/kube-scheduler-amd64:v1.21.4
Size of registry.k8s.io/kube-scheduler-amd64:v1.21.4: 13MB
--------------------------------
Checking size for image: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.0
Size of registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.0: 10MB
--------------------------------
Checking size for image: registry.k8s.io/sig-storage/csi-provisioner:v3.6.1
Size of registry.k8s.io/sig-storage/csi-provisioner:v3.6.1: 27MB
--------------------------------
Checking size for image: docker.io/openstorage/csi-attacher:v1.2.1-1
Size of docker.io/openstorage/csi-attacher:v1.2.1-1: 14MB
--------------------------------
Checking size for image: registry.k8s.io/sig-storage/csi-resizer:v1.9.1
Size of registry.k8s.io/sig-storage/csi-resizer:v1.9.1: 25MB
--------------------------------
Checking size for image: registry.k8s.io/sig-storage/csi-snapshotter:v6.3.1
Size of registry.k8s.io/sig-storage/csi-snapshotter:v6.3.1: 25MB
--------------------------------
Checking size for image: registry.k8s.io/sig-storage/snapshot-controller:v6.3.1
Size of registry.k8s.io/sig-storage/snapshot-controller:v6.3.1: 24MB
--------------------------------
Checking size for image: quay.io/prometheus/prometheus:v2.48.1
Size of quay.io/prometheus/prometheus:v2.48.1: 94MB
--------------------------------
Checking size for image: quay.io/prometheus-operator/prometheus-operator:v0.70.0
Size of quay.io/prometheus-operator/prometheus-operator:v0.70.0: 15MB
--------------------------------
Checking size for image: quay.io/prometheus-operator/prometheus-config-reloader:v0.70.0
Size of quay.io/prometheus-operator/prometheus-config-reloader:v0.70.0: 12MB
--------------------------------
Checking size for image: quay.io/prometheus/alertmanager:v0.26.0
Size of quay.io/prometheus/alertmanager:v0.26.0: 29MB
--------------------------------
Total size of all images: 3GB