Node and Upgrade Operations

Red Hat

Audience
Public
Technology Integrations
Linux
Source Type
Documentation
Node and upgrade operations are among the highest-risk Day 2 activities in an OpenShift environment because they can affect compute availability, storage resilience, and workload continuity at the same time. In a platform running OpenShift Virtualization, Portworx, and FlashArray-backed storage, administrators should never treat node maintenance or upgrades as isolated infrastructure tasks. Every action must account for the workloads on the node, the storage dependencies behind those workloads, and the health of the broader cluster.
  1. Cordon and Drain with Storage Awareness

    Cordon and drain operations are the foundation of safe node maintenance. In OpenShift, cordoning marks a node unschedulable so that no new workloads land there, while draining evicts or migrates existing workloads so the node can be safely serviced. In a mixed VM and storage environment, draining a node without first understanding its storage role can create avoidable disruption.

    This task is used to:

    • prepare a worker node for planned maintenance

    • safely evacuate workloads before reboot or repair

    • protect storage-backed workloads from unnecessary disruption

    • reduce the risk of maintenance-related application impact

    Maintenance workflow for worker nodes that host VM and storage workloads

    1. In the OpenShift web console, go to Compute > Nodes.
    2. Select the worker node that will be maintained.
    3. Open the node details page and review: current workload placement, whether the node is hosting VMs, whether the node is part of the storage-serving set, recent warnings or resource pressure.
    4. Confirm cluster health before beginning maintenance.
    5. Confirm Portworx health before beginning maintenance.
    6. Use the node action menu to Cordon the node.
    7. Verify the node is now marked unschedulable.
    8. Review running workloads on the node.
    9. If virtual machines are running on the node, confirm that migration is supported and healthy before draining.
    10. Use the node action menu to begin the Drain workflow.
    11. Monitor workload eviction, migration, and completion.
    12. Confirm the node is fully drained before host-side maintenance begins.

      Replica placement, disruption, and migration checks before maintenance

      Before draining a node, validate:
      • the cluster is not already degraded

      • Portworx nodes and pools are healthy

      • no storage pool is critically full

      • the workloads on the node can tolerate movement

      • virtual machines that require live migration are eligible to move

      • the node is not hosting a storage role that would make the maintenance unsafe without additional preparation

      For storage-aware maintenance, administrators should also identify whether:

      • the node hosts a Portworx storage service role

      • any critical VM volumes are attached there

      • draining the node would reduce redundancy below an acceptable threshold

      • a path issue already exists elsewhere in the environment

  2. MachineConfig Rolling Updates

    MachineConfig is how OpenShift applies many host-level configuration changes consistently across nodes. In practice, this means worker nodes may reboot or cycle through an update state as configuration is rolled out. In environments using FC, iSCSI, or NVMe/TCP-backed storage, MachineConfig changes must be monitored carefully because they can affect both workload scheduling and storage transport behavior.

    This task is used to:

    • apply host configuration changes consistently

    • roll out transport, kernel, or system configuration updates

    • validate that node configuration remains uniform across the cluster

    • reduce drift between worker nodes

    Monitoring and validating rolling updates

    1. In the OpenShift web console, go to Compute > MachineConfigPools.
    2. Review the worker pool and any custom pools.
    3. Confirm whether a planned update is in progress.
    4. Open the relevant pool and review: updated machine count, ready machine count, degraded status, updating status.
    5. If a rollout is underway, monitor nodes as they cycle through the update.
    6. Go to Compute > Nodes and confirm nodes return to Ready after the update.
    7. Spot check storage-serving nodes first.
    8. After the rollout completes, confirm that: the pool shows fully updated, no node is stuck in an abnormal state, no unexpected degradation is reported.

      Reducing workload impact during configuration rollout

      Before approving or starting a MachineConfig change, administrators should evaluate:

      • whether it affects storage-facing services, transport, or host configuration

      • whether VMs are currently running on the affected nodes

      • whether Portworx is healthy enough to tolerate node rotation

  3. OpenShift and Portworx Upgrade Coordination

    OpenShift and Portworx upgrades must be treated as coordinated platform events, not as unrelated version changes. Even if both products have individually supported upgrade paths, administrators still need to validate compatibility, storage readiness, backend health, and workload impact before beginning.

    This task is used to:

    • upgrade OpenShift safely without breaking storage-backed workloads

    • coordinate Portworx and platform version alignment

    • reduce upgrade-related risk across compute, storage, and virtualization

    • provide clear pre-check, execution, and validation steps

Planning upgrades across OpenShift, Portworx, and storage dependencies

Before any upgrade, administrators should confirm:

  • the target OpenShift version is compatible with the deployed Portworx version

  • the target Portworx version is compatible with the deployed OpenShift version

  • storage-facing worker nodes are healthy

  • no Portworx node, pool, or volume is degraded

  • no FlashArray transport issue exists

  • backup and recovery workflows have been validated

  • no unrelated cluster degradation is already present

Pre-checks, sequencingConfirm cluster operators are healthy.

Before beginning the upgrade:

  • Confirm all relevant nodes are ready.

  • Confirm MachineConfigPools are stable.

  • Confirm Portworx StorageCluster health is good.

  • Confirm storage pools have safe capacity headroom.

  • Confirm no critical VM migration or restore activity is in progress.

  • Confirm FlashArray path health is stable across the transport in use.

  • Confirm the backup posture is current for the workloads in scope.