Best Practices Guide for NVMe-TCP on Oracle Linux

Linux

Audience
Public
Product
FlashBlade
FlashArray
Technology Integrations
Linux
Source Type
Documentation

Deployment Topology

Network Architecture

Key Design Principles

Component Best Practice
Switches Dual switches for redundancy
NICs Minimum 2 per host for multipath
Controllers Dual controller array for HA
Paths 2 NICs 4 portals = 8 paths
Protocol NVMe-TCP (port 4420 or 8009)
Multipath Native NVMe multipathing (kernel)

Network Isolation

Network Configuration Recommendations

Setting Recommendation Why
MTU 9000 (Jumbo Frames) Reduces CPU overhead, improves throughput
VLAN Dedicated storage VLAN Isolates storage traffic
Bonding Do NOT bond storage NICs Preserves multipath redundancy
Gateway None on storage interfaces Storage traffic stays on local subnet

ARP Configuration for Same-Subnet Multipath

When using multiple interfaces on the same subnet, configure ARP settings:

# Required for same-subnet multipath
sysctl -w net.ipv4.conf.all.arp_ignore=2
sysctl -w net.ipv4.conf.all.arp_announce=2

See ARP Configuration for detailed explanation.