NVMe-TCP on RHEL/Rocky/AlmaLinux - Network Configuration

Linux

Audience
Public
Product
FlashBlade
FlashArray
Technology Integrations
Linux
Source Type
Documentation

NetworkManager Best Practices

Why NetworkManager:

  • Default in RHEL 8/9
  • Better integration with systemd
  • Dynamic configuration support
  • Consistent across RHEL ecosystem

Disable network-scripts (RHEL 8):

# Network scripts are deprecated
sudo systemctl disable network
sudo systemctl enable NetworkManager
sudo systemctl start NetworkManager

Storage Network Configuration

Dedicated Network Interfaces (Recommended)

Important: Repeat these steps to configure additional storage interfaces. For redundancy at least two are recommended.
# Create connection for storage interface
sudo nmcli connection add type ethernet \
    con-name storage-nvme-1 \
    ifname <INTERFACE_1_NAME> \
    ipv4.method manual \
    ipv4.addresses <INTERFACE_1_IP>/<INTERFACE_1_CIDR> \
    ipv4.never-default yes \
    ipv4.may-fail no \
    802-3-ethernet.mtu 9000 \
    connection.autoconnect yes \
    connection.autoconnect-priority 10

# Optimize for storage
sudo nmcli connection modify storage-nvme-1 \
    ethtool.ring-rx 4096 \
    ethtool.ring-tx 4096 \
    ethtool.coalesce-rx-usecs 50 \
    ethtool.coalesce-tx-usecs 50

# Activate
sudo nmcli connection up storage-nvme-1

Key NetworkManager parameters:

  • ipv4.never-default yes - No default route on storage interface
  • ipv4.may-fail no - Boot waits for this interface
  • connection.autoconnect-priority 10 - Higher priority for storage
  • ethtool.* - NIC tuning parameters

MTU Configuration:

# Set MTU 9000 for jumbo frames
sudo nmcli connection modify storage-nvme-1 802-3-ethernet.mtu 9000

# Verify
nmcli connection show storage-nvme-1 | grep mtu

# Test MTU
ping -M do -s 8972 <storage_portal_ip>
Important:

MTU must be 9000 on:

  • Host interfaces
  • All switches in path
  • Storage array ports