Queue Settings

Pure recommends the use of the none IO elevator in Linux. For RHEL 7 and older releases, none was called noop.

Manually Changing Queue Settings

We recommend that you modify the GRUB boot loader, to automatically set the default IO elevator when the kernel boots.

Note: Alternative methods include the use of udev rules or a script. Alternative elevators might yield benefits for a user's workload, such as mq-deadline or bfq. Use the following steps to test different IO elevators to determine which one works best for your environment.

How to determine the current IO elevator

The current IO scheduler in use can be determined with the following Linux command:

cat /sys/block/sd*/queue/scheduler

How to change the IO elevator with GRUB

The default IO elevator can be set in the Linux GRUB boot loader. Use the following code sample to set the default IO elevator to none.

GRUB_CMDLINE_LINUX="crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swaprhgb quiet numa=off transparent_hugepage=never elevator=none

If you set the IO elevator with GRUB, and then reboot your system, all block devices will default to that elevator. This approach is simple as it applies to all devices, and will likely be supported for future devices. Setting the IO elevator in the GRUB will also make all block devices default to your chosen elevator, unless explicitly over-ridden. This approach is simple as it applies to all devices, including new devices that might be added in the future.

CAUTION:

Whereas none is the ideal elevator setting for Pure flash devices, in environments which combine Pure products with devices from other vendors, it might be necessary to use different settings for the non-Pure devices. In this scenario, using the GRUB approach might not be optimal.

How to manually change the IO elevator

To set IO elevators on a device-by-device basis, use the following command to write the value none to the block device's sysfs settings. Change is immediate with this approach, even without a reboot, for kernel versions 2.6.18 (RHEL5) and higher.

echo none > /sys/block/sdx/queue/scheduler

Logical Unit Numbers (LUNs) are unique identifiers that distinguish storage devices within a Storage Area Network. LUNs address and manage individual storage units or partitions on a storage array, so that they are accessible to servers and hosts. The following example shows a “for loop” that sets all Pure LUNs to the desired none elevator.

for disk in $(lsscsi | grep PURE | awk '{print $6}'); do
    echo none > /sys/block/${disk##/dev/}/queue/scheduler
done

CAUTION:

This method will not maintain queue settings that persist across reboots, unless they are added to a udev rules file, or a startup script.

How to manually change the IO elevator with udev rules

If, after you set the IO scheduler elevator to none, you want to keep that setting persistent across reboots, we recommend that you use the following process to set udev rules.

Step 1: Create the udev rules file.

Create a new file for udev rules that will reset your elevators when you reboot. The following file location paths are shown below for RHEL and Ubuntu respectively.

RHEL: /etc/udev/rules.d/99-pure-storage.rules
Ubuntu: /lib/udev/rules.d/99-pure-storage.rules

Step 2: Reset your elevators when you reboot.

The following entries in your udev rules file will automatically set the elevator to noop or none (depending on version, see below rules) each time the system is rebooted, for the devices that match the rule criteria.

RHEL 9.x, RHEL 8.x, SuSE 15.2+, and Ubuntu

These udev rules apply to the RHEL9.x, RHEL 8.x, SuSE 15.2+, and Ubuntu OSes.

Rule 1: The first rule applies to SCSI devices that show up in the /dev/sd device table.
Rule 2: The second rule applies to multi-path devices that show up in the /dev/dm device table.

# Recommended settings for Pure Storage FlashArray.
# Use none scheduler for high-performance solid-state storage for SCSI devices
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", OPTIONS="nowatch", ATTR{queue/scheduler}="none"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", OPTIONS="nowatch", ATTR{queue/scheduler}="none"

# Reduce CPU overhead due to entropy collection
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", OPTIONS="nowatch", ATTR{queue/add_random}="0"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", OPTIONS="nowatch", ATTR{queue/add_random}="0"

# Spread CPU load by redirecting completions to originating CPU
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", OPTIONS="nowatch", ATTR{queue/rq_affinity}="2"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", OPTIONS="nowatch", ATTR{queue/rq_affinity}="2"

# Set the HBA timeout to 60 seconds
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", OPTIONS="nowatch", ATTR{device/timeout}="60"

# You can have Linux apply the rules by reloading the UDEV rules and then triggering Linux to apply them immediately.

/sbin/udevadm control –-reload-rules

/sbin/udevadm trigger –-type=devices --action=change

RHEL 6.x, 7.x

After applying the following changes, you can expect to see these settings take effect for your sd* devices, but dm-* devices do not reflect these changes directly because dm-* devices inherit such settings from the sd* devices that make up its path.

# Recommended settings for Pure Storage FlashArray.

# Use noop scheduler for high-performance solid-state storage
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="noop"

# Reduce CPU overhead due to entropy collection
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/add_random}="0"

# Spread CPU load by redirecting completions to originating CPU
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/rq_affinity}="2"

# Set the HBA timeout to 60 seconds
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{model}=="FlashArray      ", RUN+="/bin/sh -c 'echo 60 > /sys/$DEVPATH/device/timeout'"

Note:

Please note that 6 spaces are needed after "FlashArray" under "Set the HBA timeout to 60 seconds" above for the rule to take effect.

Note:

In RHEL 6.x, 7.x the following will cause all of the rules in the rules.d directory to be triggered immediately. The customer needs to take extreme caution when running this command because it may crash the host or have other unintended consequences. We recommend the customer reboots when they have a change control windows if at all possible.

[root@host ~]# udevadm control --reload-rules && udevadm trigger

RHEL 5.x

# Recommended settings for Pure Storage FlashArray.
 
# Use noop scheduler for high-performance solid-state storage
ACTION=="add|change", KERNEL=="sd*[!0-9]|", SYSFS{vendor}=="PURE*", RUN+="/bin/sh -c 'echo noop > /sys/$devpath/queue/scheduler'"

Note:

It is expected behavior that you only see the settings take effect for the sd* devices. The dm-* devices will not reflect the change directly but will inherit it from the sd* devices that make up its path.