NVMe-TCP on Debian/Ubuntu - Security

Linux
Audience
Public
Product
FlashBlade
FlashArray
Technology Integrations
Linux
Source Type
Documentation
Warning:

This content is for reference only. Always consult official vendor documentation for your distribution. Test thoroughly in a lab environment before production use. In case of conflicts, vendor documentation takes precedence.

Network Isolation

Dedicated Storage Network:

  • Never route storage traffic through management network

    • Why: Prevents unauthorized access to storage traffic; reduces attack surface; improves performance

  • Use dedicated VLANs or physical networks for storage

    • Why: Isolates storage from other network traffic; prevents VLAN hopping attacks

  • No default gateway on storage interfaces

    • Why: Prevents routing storage traffic outside the storage network; reduces exposure

Configuration: 

# Storage interfaces should NOT have gateway configured
# /etc/network/interfaces
auto ens1f0
iface ens1f0 inet static
    address 10.100.1.101/24
    mtu 9000
    # NO gateway line

Verification: 

# Verify no default route on storage interface
ip route show dev ens1f0
# Should show only local subnet route

Firewall Configuration

Option 1: Disable Filtering on Storage Interfaces (Recommended)

For dedicated, isolated storage networks, disable firewall filtering on storage interfaces to eliminate CPU overhead from packet inspection.

# Add storage interfaces to trusted zone (firewalld)
firewall-cmd --permanent --zone=trusted --add-interface=ens1f0
firewall-cmd --permanent --zone=trusted --add-interface=ens1f1
firewall-cmd --reload

# Or accept all traffic on interfaces (iptables)
iptables -A INPUT -i ens1f0 -j ACCEPT
iptables -A INPUT -i ens1f1 -j ACCEPT

Why disable filtering on storage interfaces:

  • CPU overhead: Firewall packet inspection adds latency and consumes CPU cycles

  • Performance impact: At high IOPS (millions with NVMe-TCP), filtering overhead becomes significant

  • Network isolation: Dedicated storage VLANs provide security at the network layer

  • Simplicity: No port rules to maintain for storage traffic

Option 2: Port Filtering (For Shared or Non-Isolated Networks)

Use port filtering only when storage interfaces share a network with other traffic or when additional host-level security is required by policy.

Warning:

Port filtering adds CPU overhead for every packet. For production storage with high IOPS requirements, use Option 1 with network-level isolation instead.

# Allow only NVMe-TCP traffic on storage interfaces
# Port 4420 = Data port (connections)
# Port 8009 = Discovery port (optional)
iptables -A INPUT -i ens1f0 -p tcp --dport 4420 -j ACCEPT
iptables -A INPUT -i ens1f0 -p tcp --dport 8009 -j ACCEPT
iptables -A INPUT -i ens1f0 -j DROP  # Drop all other traffic

Required Ports (if using port filtering):

  • NVMe-TCP: Port 4420 (data), Port 8009 (discovery)

3. Access Control

Storage Array Configuration:

  • Register only authorized Host NQNs.

  • Implement IP-based ACLs on storage array.

  • Regularly audit authorized hosts.

Verify host identifier: 

# Check host NQN
cat /etc/nvme/hostnqn
# Example: nqn.2014-08.org.nvmexpress:uuid:12345678-1234-1234-1234-123456789abc

Best Practice:

  • Use unique NQN per host (don't clone VMs without regenerating).

  • Document all registered NQNs.

  • Remove decommissioned hosts from storage array.

Host Security

  1. Minimize Attack Surface:

    Disable unnecessary services:

    # List running services
systemctl list-units --type=service --state=running

# Disable unnecessary services
systemctl disable <service_name>
systemctl stop <service_name>

  2. Keep Systems Updated:

    Regular patching:

    # RHEL/Rocky/AlmaLinux
dnf update

# Debian/Ubuntu
apt update && apt upgrade

# SUSE
zypper update

    Best Practices:

    • Patch monthly at minimum.

    • Test patches in non-production first.

    • Prepare rollback plan.

    • Monitor security advisories for NVMe/kernel updates.

  3. Audit and Logging:

    Enable audit logging:

    # Install and enable auditd
systemctl enable --now auditd

# Add audit rules for NVMe devices
auditctl -w /dev/nvme0n1 -p rwa -k nvme_access
auditctl -w /etc/nvme/ -p wa -k nvme_config

Debian/Ubuntu-Specific Security

Unattended upgrades (security updates): 

# Install unattended-upgrades
sudo apt install -y unattended-upgrades apt-listchanges

# Configure
sudo dpkg-reconfigure -plow unattended-upgrades

# Edit configuration
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

# Enable automatic security updates
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
};

# Enable automatic reboot if needed (optional)
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "03:00";

Audit daemon: 

# Install auditd
sudo apt install -y auditd audispd-plugins

# Add rules for storage access
sudo tee -a /etc/audit/rules.d/storage.rules > /dev/null <<EOF
# Monitor NVMe device access
-w /dev/nvme0n1 -p rwa -k nvme_access

# Monitor NVMe configuration changes
-w /etc/nvme/ -p wa -k nvme_config
-w /etc/modprobe.d/nvme-tcp.conf -p wa -k nvme_multipath_config
-w /etc/udev/rules.d/99-nvme-iopolicy.rules -p wa -k nvme_iopolicy_config
EOF

# Reload rules
sudo augenrules --load

# Enable and start auditd
sudo systemctl enable --now auditd

Fail2ban for SSH protection: 

# Install fail2ban
sudo apt install -y fail2ban

# Create local configuration
sudo tee /etc/fail2ban/jail.local > /dev/null <<EOF
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5

[sshd]
enabled = true
port = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
EOF

# Enable and start
sudo systemctl enable --now fail2ban