This example walks you through connecting Everpure Cloud Dedicated volumes to an AWS Linux 2 AMI. Some steps might be repeated from steps seen earlier in this guide.
The following instructions cover the 'basics' in terms of setting up Linux Host iSCSI connectivity to Everpure Cloud Dedicated. More in-depth best practices for Linux can be found in this Linux Recommended Settings.
Similar setup instructions for Ubuntu 18.04 can be found at this link.
The following steps include:
- Configuring the Linux host for iSCSI and MPIO with Everpure Cloud Dedicated
- Host and volume creation on Everpure Cloud Dedicated
- Connecting and mounting Everpure Cloud Dedicated volumes to Linux host
On the AWS Linux 2 AMI Host
1. Log in to the Amazon Linux 2 EC2 instance.
2. Install iscsi-initiator-utils onto Linux host.
yum -y install iscsi-initiator-utils
3. Install lsscsi.
yum -y install lsscsi
4. Install the device-mapper-multipath package.
yum -y install device-mapper-multipath
5. Start iSCSI daemon service.
service iscsid start
6. This step increases the total bandwidth performance by allowing the host to create additional iSCSI sessions per iSCSI connection. You can add up to 16 iSCSI sessions to each controller, but it is recommended to vary and test the number of sessions against your unique workloads.
This command below will edit the /etc/iscsi/iscsid.conf file and change the node.session.nr_sessions field to 16. See Networking Best Practices for more detailed information about iSCSI sessions.
Run command below.
sed -i 's/^\(node\.session\.nr_sessions\s*=\s*\).*$/\116/' /etc/iscsi/iscsid.conf
7. Remove 51-ec2-hvm-devices.rules file.
rm /etc/udev/rules.d/51-ec2-hvm-devices.rules
The step above is only required with Amazon Linux 2 AMI.
8. (Optional) This step may help with performance between the host and Everpure Cloud Dedicated volumes.
Create a new udev rules file called 99-pure-storage.rules for Everpure and copy the contents into the file as shown in the following example.
RHEL 6.x, 7.x:vim /etc/udev/rules.d/99-pure-storage.rules
Ubuntu:vim /lib/udev/rules.d/99-pure-storage.rules
# Recommended settings for Pure Storage FlashArray.
# Use none scheduler for high-performance solid-state storage
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="noop"
# Reduce CPU overhead due to entropy collection
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/add_random}="0"
# Spread CPU load by redirecting completions to originating CPU
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/rq_affinity}="2"
# Set the HBA timeout to 300 seconds
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{model}=="FlashArray ", RUN+="/bin/sh -c 'echo 300 > /sys/$DEVPATH/device/timeout'"
Please note that 6 spaces are needed after "FlashArray" under "Set the HBA timeout to 300 seconds" above for the rule to take effect.
In RHEL 8.x and SuSE 15.2 and higher 'noop' no longer exists and has been replaced by 'none'.
# Recommended settings for Pure Storage FlashArray.
# Use none scheduler for high-performance solid-state storage for SCSI devices
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="none"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", ATTR{queue/scheduler}="none"
# Reduce CPU overhead due to entropy collection
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/add_random}="0"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", ATTR{queue/add_random}="0"
# Spread CPU load by redirecting completions to originating CPU
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/rq_affinity}="2"
ACTION=="add|change", KERNEL=="dm-[0-9]*", SUBSYSTEM=="block", ENV{DM_NAME}=="3624a937*", ATTR{queue/rq_affinity}="2"
# Set the HBA timeout to 300 seconds
ACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{device/timeout}="300"
9. Enable default multipath configuration file and start the multipath daemon.
mpathconf --enable --with_multipathd y
10. Replace the content of the multipath.conf file with the following configuration for Everpure.
vim /etc/multipath.conf
-
polling_interval 10 -
vendor "PURE" -
path_selector "queue-length 0" -
path_grouping_policy group_by_prio -
path_checker tur -
fast_io_fail_tmo 10 -
no_path_retry queue -
hardware_handler “1 alua” -
prio alua -
failback immediate
See your OS documentation for /etc/multipath.conf attribute descriptions.
Example of /etc/multipath.conf:
defaults {
polling_interval 10
user_friendly_names no
find_multipaths yes
}
devices {
device {
vendor "PURE"
product "FlashArray"
path_selector "queue-length 0"
path_grouping_policy group_by_prio
path_checker tur
fast_io_fail_tmo 10
no_path_retry queue
hardware_handler "1 alua"
prio alua
failback immediate
}
}
11. Restart multipathd service to get the multipath.conf changes to take effect.
service multipathd restart
12. Retrieve the Linux initiator IQN.
cat /etc/iscsi/initiatorname.iscsi
Example:
[ec2-user@ip-10-0-1-235 ~]$ cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:361dfc3de38713. Reboot the Linux host.
sudo reboot
On the Everpure Cloud Dedicated Instance
14. Using the CLI (ssh), log into the Everpure Cloud Dedicated instance using the management IP. See Viewing Everpure Cloud Dedicated Network Interfaces section for the location of your management IP addresses. Note that all of the following steps can be accomplished via the Everpure Cloud Dedicated GUI as well.
- Default username: pureuser
- Default password: pureuser
ubuntu@ip-10-0-0-107:~$ ssh pureuser@10.0.1.61
pureuser@10.0.1.61's password:
Mon Sep 09 11:40:25 2019
Welcome pureuser. This is Purity Version 5.3.0.beta10 on FlashArray MPIOConfig
http://www.purestorage.com/
pureuser@CBS>
15. Create a host on Everpure Cloud Dedicated.
purehost create --iqnlist <IQN number> <hostname>
where
<IQN number> is the initiator IQN number gathered in step 12.
<hostname> is the desired Linux hostname.
Example:
pureuser@CBS> purehost create --iqnlist iqn.1994-05.com.redhat:361dfc3de387 Linux2AMI
Name WWN IQN NQN Host Group
Linux2AMI - iqn.1994-05.com.redhat:361dfc3de387 - -
16. Create one or more volumes on Everpure Cloud Dedicated.
purevol create <volume name> --size <size>
where
<volume name> is the desired volume name.
<size> is the desired volume size (GB or TB suffix).
This example shows the creation of a 2 TB volume:
pureuser@CBS> purevol create vol1 --size 2TB
Name Size Source Created Serial
vol1 2T - 2019-09-09 11:41:55 PDT 2B60622E2B014A2200011010
17. Connect host to volumes.
purevol connect <volume name> --host <host name>
where
<volume name> is the name of the volume.
<host name> is the name of the host.
Example:
pureuser@CBS> purevol connect vol1 --host Linux2AMI
Name Host Group Host LUN
vol1 - Linux2AMI 1
18. Collect the IP addresses of each controller and the IQN number for Everpure Cloud Dedicated. The IQN is identical for both iSCSI interfaces.
pureport list
Example
pureuser@CBS> pureport list
Name WWN Portal IQN NQN Failover
CT0.ETH2 - 10.0.1.202:3260 iqn.2010-06.com.purestorage:flasharray.666667d86130ec06 - -
CT1.ETH2 - 10.0.1.110:3260 iqn.2010-06.com.purestorage:flasharray.666667d86130ec06 - -
iSCSI Login
On the AMI Linux Host
19. Create a new iSCSI interface on the Linux host initiator.
iscsiadm -m iface -I iscsi0 -o new
Example:
[ec2-user@ip-10-0-1-235 ~]$ sudo iscsiadm -m iface -I iscsi0 -o new
New interface iscsi0 added
20. Discover target iSCSI portals using iSCSI interface IP.
iscsiadm -m discovery -t st -p <CBS iSCSI IP>:3260
where
<Everpure Cloud Dedicated iSCSI IP> is the iSCSI IP address of the Cloud Block controller 1 or controller 2, collected in step 18. You only need to enter one iSCSI IP address.
Example: It will discover iSCSI IP's from both Everpure Cloud Dedicated controllers.
[ec2-user@ip-10-0-1-235 ~]$ iscsiadm -m discovery -t st -p 10.0.1.202:3260
10.0.1.202:3260,1 iqn.2010-06.com.purestorage:flasharray.666667d86130ec06
10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.666667d86130ec06
21. Connect the Linux host to the Everpure Cloud Dedicated instance.
iscsiadm -m node --login
Example: You will notice there will be 32 logins (16 iSCSi session login per Everpure Cloud Dedicated controller).
[ec2-user@ip-10-0-1-235 ~]$ iscsiadm -m node --login
Logging in to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.202:3260] (multiple)
Logging in to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.202:3260] (multiple)
.
.
.
Logging in to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.110:3260] (multiple)
Logging in to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.110:3260] (multiple)
Login to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.202:3260] successful.
Login to [iface: iscsi0, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.202:3260] successful.
.
.
.
Login to [iface: iscsi2, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.110:3260] successful.
Login to [iface: iscsi3, target: iqn.2010-06.com.purestorage:flasharray.666667d86130ec06, portal: 10.0.1.110:3260] successful.
22. Add automatic iSCSI login on boot.
iscsiadm -m node -L automatic
23. Confirm the number of iSCSI sessions. There should be 32 entries (16 iSCSI sessions per Everpure Cloud Dedicated controller)
iscsiadm --mode session
Example: There should be 32 entries (16 iSCSI sessions per Everpure Cloud Dedicated controller).
[ec2-user@ip-10-0-1-235 ~]$ iscsiadm --mode session
tcp: [1] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [10] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [11] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [12] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [13] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
.
.
.
tcp: [31] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [32] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [7] 110.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [8] 110.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
tcp: [9] 10.0.1.110:3260,1 iqn.2010-06.com.purestorage:flasharray.8650085ea65b9fa (non-flash)
24. Confirm that each volume has 32 entries, each representing a virtual device path.
lsscsi -d
Example: There should be 32 entries per Everpure Cloud Dedicated volume connected to this Linux host.
[ec2-user@ip-10-0-1-235 ~]$ lsscsi -d
[2:0:0:1] disk PURE FlashArray 8888 /dev/sdb [8:16]
[3:0:0:1] disk PURE FlashArray 8888 /dev/sda [8:0]
[4:0:0:1] disk PURE FlashArray 8888 /dev/sdc [8:32]
[5:0:0:1] disk PURE FlashArray 8888 /dev/sdd [8:48]
[6:0:0:1] disk PURE FlashArray 8888 /dev/sde [8:64]
[7:0:0:1] disk PURE FlashArray 8888 /dev/sdf [8:80]
[8:0:0:1] disk PURE FlashArray 8888 /dev/sdg [8:96]
[9:0:0:1] disk PURE FlashArray 8888 /dev/sdh [8:112]
.
.
.
[30:0:0:1] disk PURE FlashArray 8888 /dev/sdar[66:176]
[31:0:0:1] disk PURE FlashArray 8888 /dev/sdbe[67:128]
[32:0:0:1] disk PURE FlashArray 8888 /dev/sdbj[67:208]
25. Run the multipath command below to confirm each Everpure Cloud Dedicated volume has multiple paths. A multipathed Everpure Cloud Dedicated volume should be represented by a device-mapped ID, as seen in the example below. Verify the paths are divided into two priority groups, as seen in the following example.
multipath -ll
Example: Each Everpure Cloud Dedicated volumes are represented by a device-mapped IDs.
[ec2-user@ip-10-0-1-235 ~]$ multipath -ll
3624a93702b60622e2b014a2200011011 dm-0 PURE ,FlashArray
size=2.0T features='0' hwhandler='1 alua' wp=rw
|-+- policy='queue-length 0' prio=50 status=active
| |- 2:0:0:2 sdb 8:16 active ready running
| |- 3:0:0:2 sdf 8:80 active ready running
| |- 4:0:0:2 sdl 8:176 active ready running
| `- 5:0:0:2 sdk 8:160 active ready running
.
.
.
.
`-+- policy='queue-length 0' prio=10 status=enabled
|- 6:0:0:2 sdd 8:48 active ready running
|- 7:0:0:2 sdh 8:112 active ready running
|- 8:0:0:2 sdp 8:240 active ready running
`- 9:0:0:2 sdo 8:224 active ready running
.
.
.
26. Create mount points on the initiator.
mkdir /mnt/store0
27. Create the desired file system on each Everpure Cloud Dedicated volume using the device-mapped IDs, and then mount each volume to the mount point.
mkfs.ext4 /dev/mapper/<device-mapped ID>
where
<device-mapped ID> is the device-mapped ID from step 25.
The following example uses filesystem ext4 for the Everpure Cloud Dedicated volume
dm-0
[ec2-user@ip-10-0-1-235 ~]$ mkfs.ext4 /dev/mapper/3624a93702b60622e2b014a2200011011
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=1024 blocks
134217728 inodes, 536870912 blocks
26843545 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
16384 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
28. Mount Everpure Cloud Dedicated volume onto mount point.
mount /dev/mapper/<device-mapped ID> <mount point>
where
<device-mapped ID> is the device-mapped ID collected from step 25.
<mount point> is the mount point created in step 26.
[ec2-user@ip-10-0-1-235 ~]$ mount /dev/mapper/3624a93702b60622e2b014a2200011011 /mnt/store0
To make iSCSI device mount persistent across reboots, you will need to add an entry in /etc/fstab.
In this case we recommend to use _netdev and nofail options in /etc/fstab which will prevent the OS boot failure in case a iSCSI block device doesn't even exists, network isn't initialized yet etc.
Example of /etc/fstab:
# <file system> <mount point> <type> <options> <dump> <pass>
# iSCSI LUN mounted to /mnt/store0
/dev/mapper/3624a93702b60622e2b014a2200011011 /mnt/store0 ext4 defaults,_netdev,nofail, 0 229. The mount points now report 2TB, and block storage can be consumed.
[ec2-user@ip-10-0-1-235 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/3624a93702b60622e2b014a2200011010 2.0T 81M 1.9T 1% /mnt/store0
On Everpure Cloud Dedicated
- (Optional Check) I/O should only flow to the primary controller instance. Run I/O on your Linux host and confirm on your Everpure Cloud Dedicated instance with the following command:
purehost monitor --balance --interval 3
Example:
pureuser@CBS> purehost monitor --balance --interval 3
Name Time Initiator WWN Initiator IQN Initiator NQN Target Target WWN Failover I/O Count I/O Relative to Max
Linux2AMI 2019-08-26 10:31:32 PDT - iqn.1994-05.com.redhat:b9ddc64322ef - (primary) - - 1626 100%
iqn.1994-05.com.redhat:b9ddc64322ef (secondary) - 0 0%
Adding Additional Volumes
On Everpure Cloud Dedicated
You can add more volume using CLI or GUI.