Configuring SCSI Device Binding with udev Rules in CentOS 7
When working with disk devices in Linux environments, device names can change unexpectedly, which may break existing configurations. To address this issue, we can use scsi_id with udev rules to create persistent device bindings that remain stable regardless of device name chenges.
Understanding the Problem
Traditional disk naming (like /dev/sdb, /dev/sdc) can change when device are added or removed from the system. This can cause issues with applications that rely on specific device names. The scsi_id utility provides a stable identifier for each disk based on its SCSI properties, ensuring consistent device naming.
CentOS 6 and Earlier Syntax
While direct disk usage is possible, following established naming conventions is recommended. Here's an example of a CentOS 6 udev rule:
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --device=/dev/$name", RESULT=="36000c29b263ed2452f80e9848bdf2fa5", NAME="asm-2g-2fa5-grid1", OWNER="grid", GROUP="asmadmin", MODE="0660"
CentOS 7 and Later Syntax
CentOS 7 introduced changes to udev syntax. The naming convention follows a pattern: asm-lunsize-id_last_four_digits-diskgroupname+number. This approach helps administrators quickly identify disks during operations like adding or removing storage.
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7bcdd0000000f", SYMLINK+="asm-400g-000f-data1", OWNER="grid", GROUP="asmadmin", MODE="0660"
Retrieving SCSI Identifiers in RHEL 7
The lsscsi command can display SCSI device information including their unique identifiers:
lsscsi --scsi_id
[0:2:0:0] disk AVAGO MR-SAS3316 4.74 /dev/sda 36f80f41fe9c3f0002305604d0f7eea16
[0:2:1:0] disk AVAGO MR-SAS3316 4.74 /dev/sdb 36f80f41fe9c3f000230560520fce45e6
lsscsi --scsi_id -g
[0:2:0:0] disk AVAGO MR-SAS3316 4.74 /dev/sda 36f80f41fe9c3f0002305604d0f7eea16 /dev/sg0
[0:2:1:0] disk AVAGO MR-SAS3316 4.74 /dev/sdb 36f80f41fe9c3f000230560520fce45e6 /dev/sg1
Automating Rule Creation
For environments with multiple disks, manual creating rules can be time-consuming and error-prone. Here are automation scripts for different CentOS versions:
CentOS 6 Automation Script
for disk in b c d e
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$disk`\", NAME=\"asm-disk$disk\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
done
CentOS 7 Automation Script
for disk in b c d
do
echo "KERNEL==\"sd*\", SUBSYSTEM==\"block\", PROGRAM==\"/lib/udev/scsi_id -g -u -d /dev/\$name\", RESULT==\"`/lib/udev/scsi_id -g -u -d /dev/sd$disk`\", SYMLINK+=\"asm-5g-xxxx-grid1\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
done
Executing the Script
The script generates udev rules that can be saved to a file and applied:
sh /u01/asmdisk.sh
# Sample output:
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7bcdd0000000f", SYMLINK+="asm-5g-xxxx-grid1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7be3900000010", SYMLINK+="asm-5g-xxxx-grid1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7bec100000011", SYMLINK+="asm-5g-xxxx-grid1", OWNER="grid", GROUP="asmadmin", MODE="0660"
Configuring udev Rules File
Create or edit the udev rules file with the generated rules:
vi /etc/udev/rules.d/99-oracle-asmdevices.rules
# Example content:
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7bcdd0000000f", SYMLINK+="asm-400g-000f-data1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7be3900000010", SYMLINK+="asm-400g-0010-data2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name", RESULT=="36acb3b510041191b0de7bec100000011", SYMLINK+="asm-400g-0011-data3", OWNER="grid", GROUP="asmadmin", MODE="0660"
Note that the scsi_id output matches the LUN WWN from storage multipathing, with the exception of an additional '3' prefix.
Applying the New Rules
After creating the rules file, apply them using these commands:
udevadm control --reload
udevadm trigger
/sbin/udevadm trigger --type=devices --action=change
/sbin/udevadm control --reload
systemctl restart systemd-udev-trigger.service
Verifying the Configuration
Check that the symbolic links have been created correctly:
ls -l /dev/asm*
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-100g-0039-arch1 -> sdm
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-000f-data1 -> sdb
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0010-data2 -> sdc
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0011-data3 -> sdd
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0012-data4 -> sde
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0013-data5 -> sdf
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0014-data6 -> sdg
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0015-data7 -> sdh
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-400g-0016-data8 -> sdi
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00da-grid1 -> sdj
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00db-grid2 -> sdk
lrwxrwxrwx. 1 root root 3 Sep 8 10:19 /dev/asm-5g-00dc-grid3 -> sdl
Verify the underlying block devices:
ls -l /dev/sd*
brw-rw----. 1 root disk 8, 0 Sep 8 10:10 /dev/sda
brw-rw----. 1 root disk 8, 1 Sep 8 10:10 /dev/sda1
brw-rw----. 1 root disk 8, 2 Sep 8 10:10 /dev/sda2
brw-rw----. 1 grid asmadmin 8, 16 Sep 8 16:02 /dev/sdb
brw-rw----. 1 grid asmadmin 8, 32 Sep 8 10:19 /dev/sdc
brw-rw----. 1 grid asmadmin 8, 48 Sep 8 10:19 /dev/sdd
brw-rw----. 1 grid asmadmin 8, 64 Sep 8 10:19 /dev/sde
brw-rw----. 1 grid asmadmin 8, 80 Sep 8 10:19 /dev/sdf
brw-rw----. 1 grid asmadmin 8, 96 Sep 8 10:19 /dev/sdg
brw-rw----. 1 grid asmadmin 8, 112 Sep 8 10:19 /dev/sdh
brw-rw----. 1 grid asmadmin 8, 128 Sep 8 10:19 /dev/sdi
brw-rw----. 1 grid asmadmin 8, 144 Sep 8 16:02 /dev/sdj
brw-rw----. 1 grid asmadmin 8, 160 Sep 8 16:02 /dev/sdk
brw-rw----. 1 grid asmadmin 8, 176 Sep 8 16:02 /dev/sdl
brw-rw----. 1 grid asmadmin 8, 192 Sep 8 16:02 /dev/sdm
Final Verification with Oracle ASM
After configuring the udev rules, you can create ASM disk groups using the persistent device names:
[grid@xxdb01 ~]$ asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 4194304 102400 102276 0 102276 0 N ARCH/
MOUNTED EXTERN N 512 4096 4194304 3276800 3276620 0 3276620 0 N DATA/
MOUNTED NORMAL N 512 4096 4194304 15360 14320 5120 4600 0 Y GRID/