Recommended DM-Multipath Settings

Sample multipath.conf

The following multipath.conf file has been tested with recent versions of RHEL 8. It provides settings for volumes on FlashArray exposed through either SCSI or NVMe. Before use, verify the configuration with multipath -t. Some settings may be incompatible with older distributions; we list some known incompatibilities and workarounds below.

defaults {
        polling_interval       10
}


devices {
    device {
        vendor                      "NVME"
        product                     "Pure Storage FlashArray"
        path_selector               "queue-length 0"
        path_grouping_policy        group_by_prio
        prio                        ana
        failback                    immediate
        fast_io_fail_tmo            10
        user_friendly_names         no
        no_path_retry               10
        features                    0
        dev_loss_tmo                60
    }
    device {
        vendor                   "PURE"
        product                  "FlashArray"
        path_selector            "service-time 0"
        hardware_handler         "1 alua"
        path_grouping_policy     group_by_prio
        prio                     alua
        failback                 immediate
        path_checker             tur
        fast_io_fail_tmo         10
        user_friendly_names      no
        no_path_retry            0
        features                 0
        dev_loss_tmo             600
    }
}

Setting Compatibility Notes

• Path selectors: as listed in the sample above, Pure recommends the use of queue-length 0 with NVMe and service-time 0 with SCSI, which improve performance in situations where paths have differing latencies by biasing I/Os towards paths that are servicing I/O more quickly. Older kernels (before RHEL 6.2/before SUSE 12) may not support these path selectors and should specify path_selector "round-robin 0" instead.
• Path prioritizers (ALUA for SCSI, and ANA for NVMe) and failback immediate must be enabled on hosts connected to arrays configured in an ActiveCluster.
  • The ANA path prioritizer for NVMe is a relatively recent feature (RHEL 8), and older distributions may not support it. In non-ActiveCluster configurations, it can be safely disabled by removing the line prio ana and replacing path_grouping_policy group_by_prio with path_grouping_policy multibus.
• Please note that fast_io_fail_tmo and dev_loss_tmo do not apply to iSCSI.
• Please note that the above settings can differ based on a use case, for example - if user has RHEL Open Stack Cinder driver configured, the settings can differ, so please, before making recommendations, ask the customer if they have anything specific configured, or it is just a standard Linux host.
• If multipath nodes are not showing up on the host after a rescan, you may need to add find_multipaths yes to the defaults section above. This is the case for some hosts which boot of a local non-multipath disk.
• As per https://access.redhat.com/solutions/3234761, RHV-H multipath configuration must include user_friendly_names no.
• As per https://access.redhat.com/site/solutions/110553, running DM-multipath along with EMC PowerPath is not a supported configuration and may result in kernel panics on the host.
• Consult man 5 multipath.conf and/or the RHEL Documentation before making modifications to the configuration.

Verifying DM-Multipathd Configuration

After creating and connecting some volumes on the FlashArray to the host, run multipath -ll to check the configuration. The below output was obtained by creating two volumes and connecting the first to the host via NVMe, and the second through SCSI.

[root@init116-13 ~]# multipath -ll
eui.00292fd80c2afd4724a9373400011425 dm-4 NVME,Pure Storage FlashArray
size=2.0T features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=50 status=active
  |- 2:2:1:70693 nvme2n1 259:0 active ready running
  |- 3:1:1:70693 nvme3n1 259:1 active ready running
  |- 6:0:1:70693 nvme6n1 259:2 active ready running
  `- 4:3:1:70693 nvme4n1 259:3 active ready running
3624a9370292fd80c2afd473400011424 dm-3 PURE,FlashArray
size=1.0T features='0' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
  |- 6:0:2:1     sdd     8:48  active ready running
  |- 6:0:3:1     sdf     8:80  active ready running
  |- 7:0:3:1     sdl     8:176 active ready running
  `- 7:0:2:1     sdj     8:144 active ready running

Note the policy='queue-length 0' and policy='service-time 0' which indicate the active path selection policies. These should match the path selection policy settings from the configuration file.

To check if path prioritizers are working correctly in an ActiveCluster environment, create a stretched volume and set a preferred array for the host as described in ActiveCluster: Optimizing Host Performance with Array Preferences. The output of multipath -ll should then look similar to the following example.

# multipath -ll
3624a9370292fd80c2afd473400011426 dm-2 PURE,FlashArray
size=3.0T features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 6:0:2:2     sde     8:64  active ready running
| |- 6:0:3:2     sdg     8:96  active ready running
| |- 7:0:2:2     sdk     8:160 active ready running
| `- 7:0:3:2     sdm     8:192 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 6:0:0:1     sdb     8:16  active ready running
  |- 6:0:1:1     sdc     8:32  active ready running
  |- 7:0:0:1     sdh     8:112 active ready running
  `- 7:0:1:1     sdi     8:128 active ready running

Notice the two distinct groups of paths. The paths to the preferred array (SCSI target numbers 2 and 3) have priority 50, while the paths to the non-preferred array (SCSI target numbers 0 and 1) have priority 10.

Excluding Third-Party Vendor LUNs from DM-Multipathd

When systems have co-existing multipathing software, it is a good practice to exclude control from one multipathing software in order to allow control by another multipathing software.

The following is an example of using DM-Multipathd to blacklist LUNs from a third party vendor. The syntax blocks DM-Multipathd from controlling those luns that are "blacklisted".

The following can be added to the 'blacklist' section of the multipath.conf file.

blacklist {
                device {
                             vendor "XYZ.*"
                             product ".*"
                             }

                device {
                             vendor "ABC.*"
                             product ".*"
                             }
                 }